From 67e796b2e128f7fcbf7e41e6403b0fad455e6f9c Mon Sep 17 00:00:00 2001 From: Griffin Smith Date: Sat, 3 Sep 2022 11:42:27 -0400 Subject: test(tvix/eval): Add the start of a nix oracle test suite Add the start of a test suite that compares tvix eval results against nix, using the string repr of the value as the comparison. This shells out to a nix-instantiate binary, which is configurable as an environment variable, to eval - there's some extra machinery there to setup a new nix store as a tempdir to allow running this test inside the nix build for tvix-eval itself. Currently this has a macro that'll allow writing lots and lots of hardcoded tests, but going forward I'm also going to be looking into adding proptest-based generation of expressions to compare. Change-Id: I9f4895fab1e668ed2b7dfd6f92f8c80de1bbb16b Reviewed-on: https://cl.tvl.fyi/c/depot/+/6307 Autosubmit: grfn Reviewed-by: tazjin Tested-by: BuildkiteCI --- tvix/eval/default.nix | 4 ++++ 1 file changed, 4 insertions(+) (limited to 'tvix/eval/default.nix') diff --git a/tvix/eval/default.nix b/tvix/eval/default.nix index a3b4a01ee01e..aa8d8a4af414 100644 --- a/tvix/eval/default.nix +++ b/tvix/eval/default.nix @@ -9,6 +9,10 @@ depot.third_party.naersk.buildPackage { doCheck = true; cargoBuildOptions = opts: opts ++ [ "--all-targets" ]; + # Tell the test suite where to find upstream nix, to compare eval results + # against + NIX_INSTANTIATE_BINARY_PATH = "${pkgs.nix}/bin/nix-instantiate"; + meta.ci.targets = builtins.attrNames self.passthru; passthru.cpp-nix-run-lang-tests = pkgs.stdenv.mkDerivation { -- cgit 1.4.1