about summary refs log tree commit diff
path: root/tvix/shell.nix
diff options
context:
space:
mode:
authorFlorian Klink <flokli@flokli.de>2023-12-09T16·45+0200
committerclbot <clbot@tvl.fyi>2023-12-12T10·27+0000
commit8018313b6880d9fae71ba189a476502b68a26d25 (patch)
treea32663b7c6a7bc5938755c95684d7593a6ee12a0 /tvix/shell.nix
parent6c87ab960dcfdaa9727d692dabd4196851ecef95 (diff)
feat(tvix/glue): add nixpkgs eval benchmark r/7161
This adds a criterion.rs-based testbench into tvix-glue.

It can be invoked by running `cargo bench` from inside the `tvix-glue`
crate.
`target/criterion/report/index.html` contains nice graphs.
It's able to diff against the previous run, so you can invoke `cargo
bench` before and after a certain change to reason about the impact in
evaluation performance.

Currently, we need to create a bunch of Evaluator resources inside the
benchmark loop itself, which is a bit annoying, as it leaks into the
things we benchmark.

This should become better with b/262.

Fixes b/322.

Change-Id: I91656a308887baa1d459ed54d58baae919a4aaf2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10245
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Diffstat (limited to 'tvix/shell.nix')
-rw-r--r--tvix/shell.nix9
1 files changed, 9 insertions, 0 deletions
diff --git a/tvix/shell.nix b/tvix/shell.nix
index 4859b2b1aa2a..9a14554ac6dc 100644
--- a/tvix/shell.nix
+++ b/tvix/shell.nix
@@ -35,4 +35,13 @@ pkgs.mkShell {
     pkgs.libiconv
     pkgs.buildPackages.darwin.apple_sdk.frameworks.Security
   ];
+
+  # Set TVIX_BENCH_NIX_PATH to a somewhat pinned nixpkgs path.
+  # This is for invoking `cargo bench` imperatively on the developer machine.
+  # For tvix benchmarking across longer periods of time (by CI), we probably
+  # should also benchmark with a more static nixpkgs checkout, so nixpkgs
+  # refactorings are not observed as eval perf changes.
+  shellHook = ''
+    export TVIX_BENCH_NIX_PATH=nixpkgs=${pkgs.path}
+  '';
 }