about summary refs log tree commit diff
path: root/tvix/eval/tests/nix_oracle.rs
AgeCommit message (Collapse)AuthorFilesLines
2024-07-06 r/8351 refactor(tvix/eval): Builderize EvaluationAspen Smith1-3/+9
Make constructing of a new Evaluation use the builder pattern rather than setting public mutable fields. This is currently a pure refactor (no functionality has changed) but has a few advantages: - We've encapsulated the internals of the fields in Evaluation, meaning we can change them without too much breakage of clients - We have type safety that prevents us from ever changing the fields of an Evaluation after it's built (which matters more in a world where we reuse Evaluations). More importantly, this paves the road for doing different things with the construction of an Evaluation - notably, sharing certain things like the GlobalsMap across subsequent evaluations in eg the REPL. Fixes: b/262 Change-Id: I4a27116faac14cdd144fc7c992d14ae095a1aca4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11956 Tested-by: BuildkiteCI Autosubmit: aspen <root@gws.fyi> Reviewed-by: flokli <flokli@flokli.de>
2024-05-30 r/8181 fix(tvix/eval/tests/nix_oracle): tests are impure-onlyFlorian Klink1-0/+2
This uses StdIO, which is only available when the `impure` feature is enabled. Change-Id: I039b1f45f6619dd099fa943e58322ff521482dfa Reviewed-on: https://cl.tvl.fyi/c/depot/+/11724 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-04-30 r/8038 test(tvix): Fix tvix tests on macOSAlice Carroll1-1/+8
Prior to this, some tests would not build or would fail in an obscure way. Change-Id: I68587cc7592492ebfd71ca02fc7ccc9ff7c0196f Reviewed-on: https://cl.tvl.fyi/c/depot/+/11544 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-01-18 r/7409 refactor(tvix/eval): move Evaluation::{default,new_pure}() againFlorian Klink1-1/+1
Have a Evaluation::new() function that's used to set up the Evaluation struct initially - which is also used by both new_pure and new_impure internally. It's generic over the exact type of IO, making it easier to instantiate Evaluation with non-tvix-eval EvalIO implementations, that might not be in a Box. Change-Id: Ibf728da24aca59639c5b6df58d00ae98c99a63f5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10640 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-12-31 r/7289 refactor(tvix/eval): remove code and location from structFlorian Klink1-2/+2
Instead, it's passed in the evaluate/compile_only functions, which feels more naturally. It lets us set up the Evaluation struct long before we actually feed it with data to evaluate. Now that Evaluation::new() would be accepting an empty list of arguments, we can simply implement Default, making things a bit more idiomatic. Change-Id: I4369658634909a0c504fdffa18242a130daa0239 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10475 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de>
2023-10-08 r/6739 refactor(tvix/eval/tests): migrate to tempfileFlorian Klink1-2/+1
tempdir pulls in remove_dir_all 0.5.3 with https://rustsec.org/advisories/RUSTSEC-2023-0018.html, and we use tempfile everywhere else too, so let's just migrate to that. Change-Id: I735ade7b65e12fc26e3d43ca95fcfa07fcc64642 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9565 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-09-09 r/6574 fix(tvix/eval): make compare_strict_eval_tests use StrictAdam Joseph1-1/+1
Previously, compare_strict_eval_tests() was using Strictness::Lazy. This appears to be a minor oversight from 0ab6494286e77abcdc0bba1c92386fc7e654ca12. This commit corrects that, by changing Strictness::Lazy to Strictness::Strict. Change-Id: Ia2389a5d30481cd322ed55230731340b795e5d87 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9282 Tested-by: BuildkiteCI Autosubmit: Adam Joseph <adam@westernsemico.com> Reviewed-by: sterni <sternenseemann@systemli.org>
2023-06-14 r/6297 fix(tvix/eval): don't thunk home relative pathssterni1-0/+3
C++ Nix resolves home relative paths at [parse] time. This is not an option for us, since it prevents being able to separate the compilation and execution phase later (e.g. precompiled nix expressions). However, a practical consequence of this is that paths expressions are always literals (strict) and never thunks. [parse]: https://github.com/NixOS/nix/blob/7066d21a0ddb421967980094222c4bc1f5a0f45a/src/libexpr/parser.y#L518-L527 Change-Id: Ie4b9dc68f62c86d6c7fd5f1c9460c850d97ed1ca Reviewed-on: https://cl.tvl.fyi/c/depot/+/7041 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-07 r/6244 fix(tvix/eval): use normal thunking behavior for default in formalssterni1-0/+5
When comparing to C++ Nix, we notice that the thunking of default expressions in function formals corresponds to their normal thunking, e.g. literals are not thunked. This means that we can just invoke compile() without much of a care and trust that it will sort it out correctly. If function formals blow up as a result of this, it likely indicates that the expression is treated incorrectly by compile(), not compile_param_pattern(). Change-Id: I64acbff2f251423eb72ce43e56a0603379305e1d Reviewed-on: https://cl.tvl.fyi/c/depot/+/8704 Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-29 r/6219 test(tvix/eval): check thunking behavior of basic exprssterni1-0/+55
nix_oracle.rs now gives us the possibility to check this by stuffing the expressions in a list. In fact, the incorrect behavior fixed in - cl/8656 - cl/8655 - cl/8662 was discovered using this test suite. Change-Id: Id0ab01ee6be0b875791214e0a72a2ac941c46c96 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8658 Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-29 r/6218 refactor(tvix/eval/nix_oracle): allow specifying eval strictnesssterni1-10/+28
This will be useful for comparing thunking behavior to C++ Nix. I considered adding this capability to the tvix_tests/nix_tests infrastructure, but as it would require changing the test file naming scheme to do it in a clean way, I've postponed it–it's nice that our tests are compatible with C++ Nix's test suite. Change-Id: I60bcdd98ed25140e716f0858f8dc28f21ab957aa Reviewed-on: https://cl.tvl.fyi/c/depot/+/8657 Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-03-22 r/6037 feat(tvix/eval): add Evaluation::strict to toggle top-level deepseqVincent Ambo1-0/+1
This makes it possible for callers to control whether they can receive partially evaluated values from an evaluation or not. We're actually flipping the default behaviour to non-strict top-level evaluation, which means that callers have to set `strict = true` on the Evaluation to get the previous behaviour. Change-Id: Ic048e9ba09c88866d4c3177d5fa07db11c4eb20e Reviewed-on: https://cl.tvl.fyi/c/depot/+/8325 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-12-21 r/5467 feat(tvix/eval): use `EvalIO::import_path` when coercing pathsVincent Ambo1-1/+4
This "ties the knot" of importing files into a store when referring to them through path literals, e.g. inside of strings. I'm not yet sure if this interface is sufficient for builtins.path (which we haven't implemented at all yet), but it's enough to wire up eval & store initially. In the default implementations nothing interesting happens in this function at all. Change-Id: Ie01ff4161617d1e743a68dbd1a5e54c1b40c0990 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7582 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-21 r/5439 refactor(tvix/eval): use new public API in test codeVincent Ambo1-2/+4
This removes internal uses of the previous crate::eval module, which is being removed. Change-Id: I5fb3c53460a9c5381853d0258f9ed074ab23c630 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7543 Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: grfn <grfn@gws.fyi>
2022-10-10 r/5085 feat(tvix/eval): Allow adding strings to pathsGriffin Smith1-1/+10
Implement adding paths and strings via OpAdd. Since the nix rules are quite obscure, I'm electing to test this one with an oracle test to avoid the danger of getting the actual asserted result wrong. Change-Id: Icdcca3690ca2e8459e386c1f29cc48eaaa39e9a3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6914 Autosubmit: grfn <grfn@gws.fyi> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-09-18 r/4911 refactor(tvix/eval): use Clap for arg+env parsingGriffin Smith1-1/+3
Refactor the environment variable and argument parsing for the tvix repl to use Clap instead of doing things ad-hoc, and thread through options obtained from environment variables via explicit arguments rather than obtaining them from the environment as they're needed. This makes adding more flags more sustainable, and also makes the binary fully self-documenting, including supported env vars, via `--help`. Change-Id: Ib1f6a0cd20056e8c9196760ff755fa5729667760 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6653 Autosubmit: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-17 r/4886 test(tvix/eval): Add the start of a nix oracle test suiteGriffin Smith1-0/+67
Add the start of a test suite that compares tvix eval results against nix, using the string repr of the value as the comparison. This shells out to a nix-instantiate binary, which is configurable as an environment variable, to eval - there's some extra machinery there to setup a new nix store as a tempdir to allow running this test inside the nix build for tvix-eval itself. Currently this has a macro that'll allow writing lots and lots of hardcoded tests, but going forward I'm also going to be looking into adding proptest-based generation of expressions to compare. Change-Id: I9f4895fab1e668ed2b7dfd6f92f8c80de1bbb16b Reviewed-on: https://cl.tvl.fyi/c/depot/+/6307 Autosubmit: grfn <grfn@gws.fyi> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI