about summary refs log tree commit diff
path: root/tvix
diff options
context:
space:
mode:
authorAspen Smith <grfn@gws.fyi>2023-12-11T03·51-0500
committerclbot <clbot@tvl.fyi>2023-12-11T03·59+0000
commitb2c8c1ef946d8f60ad686c00f9b07540b5f12283 (patch)
treef7e20159580d4c9fced87425f548314b98a77f53 /tvix
parent19fd6ffae53277067df2b38a7757c225427ce970 (diff)
feat(tvix): Benchmark nixpkgs eval with hyperfine r/7144
Add some hyperfine benchmarks to Tvix's windtunnel benchmark script for
evaluating the outPath of hello and a cross-compiled hello.

Change-Id: I9d76e5ce0a3fd7d9c125c36c5fced675b660a8a8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10248
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Autosubmit: grfn <grfn@gws.fyi>
Diffstat (limited to 'tvix')
-rwxr-xr-xtvix/scripts/bench.sh12
-rw-r--r--tvix/shell.nix1
2 files changed, 11 insertions, 2 deletions
diff --git a/tvix/scripts/bench.sh b/tvix/scripts/bench.sh
index 5598087a0cf7..d88638701a88 100755
--- a/tvix/scripts/bench.sh
+++ b/tvix/scripts/bench.sh
@@ -1,4 +1,5 @@
-#!/usr/bin/env bash
+#!/usr/bin/env nix-shell
+#!nix-shell -i bash ../.. -A tvix.shell
 
 # Benchmark script that runs inside the Windtunnel build agent
 
@@ -6,5 +7,12 @@ set -euo pipefail
 
 echo "Running benchmarks for tvix/eval..."
 cd "$(dirname "$(dirname "$0")")/eval"
-docker run --rm -v "$(pwd):/app" -w /app rust cargo bench
+cargo bench
 windtunnel-cli report -f criterion-rust .
+
+echo "Running tvix macrobenchmarks..."
+cargo build --release --bin tvix
+hyperfine --export-json ./results.json \
+    -n 'eval-nixpkgs-hello' "target/release/tvix -E '(import ../../nixpkgs {}).hello.outPath'" \
+    -n 'eval-nixpkgs-cross-hello' "target/release/tvix -E '(import ../../nixpkgs {}).pkgsCross.aarch64-multiplatform.hello.outPath'"
+windtunnel-cli report -f hyperfine-json ./results.json
diff --git a/tvix/shell.nix b/tvix/shell.nix
index b2bd16c6da5d..4859b2b1aa2a 100644
--- a/tvix/shell.nix
+++ b/tvix/shell.nix
@@ -22,6 +22,7 @@ pkgs.mkShell {
     pkgs.evans
     pkgs.fuse
     pkgs.go
+    pkgs.hyperfine
     pkgs.nix_2_3 # b/313
     pkgs.pkg-config
     pkgs.rust-analyzer