about summary refs log tree commit diff
path: root/tvix/eval/default.nix
diff options
context:
space:
mode:
authorGriffin Smith <root@gws.fyi>2022-09-03T15·42-0400
committerclbot <clbot@tvl.fyi>2022-09-17T17·23+0000
commit67e796b2e128f7fcbf7e41e6403b0fad455e6f9c (patch)
tree1bdb023b5c04a685f235243da22228c1d04a874e /tvix/eval/default.nix
parentef80d00b06f3d90e37a94d65ce8c56062b19a43a (diff)
test(tvix/eval): Add the start of a nix oracle test suite r/4886
Add the start of a test suite that compares tvix eval results against
nix, using the string repr of the value as the comparison. This shells
out to a nix-instantiate binary, which is configurable as an environment
variable, to eval - there's some extra machinery there to setup a new
nix store as a tempdir to allow running this test inside the nix build
for tvix-eval itself.

Currently this has a macro that'll allow writing lots and lots of
hardcoded tests, but going forward I'm also going to be looking into
adding proptest-based generation of expressions to compare.

Change-Id: I9f4895fab1e668ed2b7dfd6f92f8c80de1bbb16b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/6307
Autosubmit: grfn <grfn@gws.fyi>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Diffstat (limited to 'tvix/eval/default.nix')
-rw-r--r--tvix/eval/default.nix4
1 files changed, 4 insertions, 0 deletions
diff --git a/tvix/eval/default.nix b/tvix/eval/default.nix
index a3b4a01ee01e..aa8d8a4af414 100644
--- a/tvix/eval/default.nix
+++ b/tvix/eval/default.nix
@@ -9,6 +9,10 @@ depot.third_party.naersk.buildPackage {
   doCheck = true;
   cargoBuildOptions = opts: opts ++ [ "--all-targets" ];
 
+  # Tell the test suite where to find upstream nix, to compare eval results
+  # against
+  NIX_INSTANTIATE_BINARY_PATH = "${pkgs.nix}/bin/nix-instantiate";
+
   meta.ci.targets = builtins.attrNames self.passthru;
 
   passthru.cpp-nix-run-lang-tests = pkgs.stdenv.mkDerivation {