From 8018313b6880d9fae71ba189a476502b68a26d25 Mon Sep 17 00:00:00 2001 From: Florian Klink Date: Sat, 9 Dec 2023 18:45:39 +0200 Subject: feat(tvix/glue): add nixpkgs eval benchmark This adds a criterion.rs-based testbench into tvix-glue. It can be invoked by running `cargo bench` from inside the `tvix-glue` crate. `target/criterion/report/index.html` contains nice graphs. It's able to diff against the previous run, so you can invoke `cargo bench` before and after a certain change to reason about the impact in evaluation performance. Currently, we need to create a bunch of Evaluator resources inside the benchmark loop itself, which is a bit annoying, as it leaks into the things we benchmark. This should become better with b/262. Fixes b/322. Change-Id: I91656a308887baa1d459ed54d58baae919a4aaf2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10245 Autosubmit: flokli Tested-by: BuildkiteCI Reviewed-by: tazjin --- tvix/shell.nix | 9 +++++++++ 1 file changed, 9 insertions(+) (limited to 'tvix/shell.nix') diff --git a/tvix/shell.nix b/tvix/shell.nix index 4859b2b1aa..9a14554ac6 100644 --- a/tvix/shell.nix +++ b/tvix/shell.nix @@ -35,4 +35,13 @@ pkgs.mkShell { pkgs.libiconv pkgs.buildPackages.darwin.apple_sdk.frameworks.Security ]; + + # Set TVIX_BENCH_NIX_PATH to a somewhat pinned nixpkgs path. + # This is for invoking `cargo bench` imperatively on the developer machine. + # For tvix benchmarking across longer periods of time (by CI), we probably + # should also benchmark with a more static nixpkgs checkout, so nixpkgs + # refactorings are not observed as eval perf changes. + shellHook = '' + export TVIX_BENCH_NIX_PATH=nixpkgs=${pkgs.path} + ''; } -- cgit 1.4.1