about summary refs log tree commit diff
path: root/tvix/cli
AgeCommit message (Collapse)AuthorFilesLines
2024-06-15 r/8280 feat(tvix/cli,store): add tracy featureFlorian Klink1-0/+4
If compiled with this features, this emits packets compatible with the [Tracy](https://github.com/wolfpld/tracy) format. Change-Id: I330f5d85ab290abe51f2df38dc55464f3ccfc6cd Reviewed-on: https://cl.tvl.fyi/c/depot/+/11815 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-06-14 r/8271 feat(tvix/tracing): correctly close otlp on exitSimon Hauser1-1/+4
Provide a new interface for forcing a flush of otlp traces and use this interface to shutdown otlp prior to exiting tvix-store, either if the tool was stopped with a SIGTERM or ended regularly. This also fixes an issue where traces were not even exported if for example we just imported 10 paths and never even emitted more than 256 traces. The implementation uses a mpsc channel so a flush can be done without having to wait for it to complete. If you want to wait for a flush to complete you can provide a oneshot channel which will receive a message once flushing is complete. Because of a otlp bug `force_flush` as well as `shutdown_tracer_provider` need to be executed using `spawn_blocking` otherwise the function will deadlock. See https://github.com/open-telemetry/opentelemetry-rust/issues/1395#issuecomment-1953280335 Change-Id: I0a828391adfb1f72dc8305f62ced8cba0515847c Reviewed-on: https://cl.tvl.fyi/c/depot/+/11803 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: Simon Hauser <simon.hauser@helsinki-systems.de>
2024-06-13 r/8261 feat(tvix/cli): add toplevel progress spanFlorian Klink2-1/+10
This gives some better feedback something is going on, gives a (named) root for all the progress children we're drawing, and also counts the time we are in eval. Change-Id: Ibe81dcebf0a2b59bb0680da62e206eb5270f9d3c Reviewed-on: https://cl.tvl.fyi/c/depot/+/11798 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: aspen <root@gws.fyi>
2024-06-10 r/8242 feat(tvix/tracing): introduce common tvix-tracing crateSimon Hauser2-25/+9
Introduce a new common crate that contains tracing boilerplate which then can be used in the cli, tvix-store and tvix-build crates. It has otlp as an optional feature, which is currently only used by tvix-store. Change-Id: I41468ac4d9c65174515d721513b96fea463d6ed2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11758 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Simon Hauser <simon.hauser@helsinki-systems.de>
2024-06-06 r/8222 feat(tvix): Switch to jemallocAspen Smith2-0/+10
Switch tvix to using jemalloc as the default global allocator on supported (eg, non-msvc) platforms. This gives a pretty reasonable performance boost basically for free: int time: [24.361 µs 24.386 µs 24.418 µs] change: [-19.355% -18.859% -18.527%] (p = 0.00 < 0.05) Performance has improved. merge small attrs time: [37.201 µs 37.328 µs 37.442 µs] change: [-24.609% -24.266% -23.982%] (p = 0.00 < 0.05) Performance has improved. merge large attrs with small attrs time: [20.030 ms 20.135 ms 20.251 ms] change: [-4.2811% -3.2549% -2.3807%] (p = 0.00 < 0.05) Performance has improved. hello outpath time: [967.91 ms 974.07 ms 983.33 ms] change: [-5.5008% -4.4552% -3.4099%] (p = 0.00 < 0.05) Performance has improved. Change-Id: I6c6e6e3295ec2fca01ea28dc37bcb201cd811767 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10851 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org> Autosubmit: aspen <root@gws.fyi> Reviewed-by: flokli <flokli@flokli.de>
2024-06-01 r/8191 feat(tvix/repl): Add a command to recursively printAspen Smith2-1/+16
Add a command, :p, to evaluate an expression and recursively print the result, as if `--strict` had been passed on the command line. Demonstration of this working: ❯ cargo r --bin tvix Finished dev [unoptimized + debuginfo] target(s) in 0.27s Running `target/debug/tvix` tvix-repl> { x = (x: x) 1; } => { x = <CODE>; } :: set tvix-repl> :p { x = (x: x) 1; } => { x = 1; } :: set Change-Id: I1a81d7481160c30d2a4483c6308e25fa45f2dfdf Reviewed-on: https://cl.tvl.fyi/c/depot/+/11738 Autosubmit: aspen <root@gws.fyi> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-06-01 r/8190 feat(tvix/repl): Add a help commandAspen Smith1-6/+29
Add a command to display help for the REPL, which can be either :? or :h. Change-Id: Ifdfd8c31130ca5afcde05a4c4276b768eb54c06f Reviewed-on: https://cl.tvl.fyi/c/depot/+/11737 Autosubmit: aspen <root@gws.fyi> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-06-01 r/8189 feat(tvix/repl): Implement :q, to quitAspen Smith1-0/+4
Change-Id: Ib3b314b21f4d8c30bfd674e79771179d51d4f2e9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11736 Autosubmit: aspen <root@gws.fyi> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-06-01 r/8188 refactor(tvix/repl): Abstract out REPL command handlingAspen Smith1-10/+26
Prepare for introducing additional REPL commands by splitting out the *parsing* of the repl commands, which returns a new ReplCommand type, from the actual *handling* of the commands. Change-Id: If81a53c1e2d90204d26ce3bb2ea9eebf7bb3fd51 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11734 Autosubmit: aspen <root@gws.fyi> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-06-01 r/8187 refactor(tvix): Break out REPL into its own moduleAspen Smith2-99/+122
In preparation for adding some new functionality to the tvix REPL, break it out into its own module. Change-Id: I4fb78320e92562e3474a3724536cb22c1d893e57 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11733 Tested-by: BuildkiteCI Autosubmit: aspen <root@gws.fyi> Reviewed-by: flokli <flokli@flokli.de>
2024-06-01 r/8186 feat(tvix/repl): Support multiline inputAspen Smith2-11/+97
Transparently support multiline input in the Tvix REPL, by handling the UnexpectedEOF error returned by the parser and using it to progressively build up an input expr over multiple iterations of the REPL's outer loop. This works quite nicely: ❯ cargo r --bin tvix Compiling tvix-cli v0.1.0 (/home/aspen/code/depot/tvix/cli) Finished dev [unoptimized + debuginfo] target(s) in 1.72s Running `target/debug/tvix` tvix-repl> { foo > = > 1; > } => { foo = 1; } :: set <press up arrow> tvix-repl> { foo = 1; } => { foo = 1; } :: set Change-Id: Ib0ed4766b13e8231d696cdc27281ac158e20a777 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11732 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: aspen <root@gws.fyi> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-05-16 r/8152 chore(tvix): drop json feature for tracing-subscriberFlorian Klink1-1/+1
Drop the (unused) feature flags here, and get rid of some crate dependencies. Change-Id: Id64852b498725467d56abb12b548301cfba6a760 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11678 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-05-11 r/8103 refactor(tvix/store): drop calculate_nar from PathInfoServiceFlorian Klink1-15/+17
This shouldn't be part of the PathInfoService trait. Pretty much none of the PathInfoServices do implement it, and requiring them to implement it means they also cannot make use of this calculation already being done by other PathInfoServices. Move it out into its own NarCalculationService trait, defined somewhere at tvix_store::nar, and have everyone who wants to trigger nar calculation use nar_calculation_service directly, which now is an additional field in TvixStoreIO for example. It being moved outside the PathInfoService trait doesn't prohibit specific implementations to implement it (like the GRPC client for the `PathInfoService` does. This is currently wired together in a bit of a hacky fashion - as of now, everything uses the naive implementation that traverses blob and directoryservice, rather than composing it properly. I want to leave that up to a later CL, dealing with other parts of store composition too. Change-Id: I18d07ea4301d4a07651b8218bc5fe95e4e307208 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11619 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-05-09 r/8096 feat(tvix/cli): Use the same TvixStoreIO for the full runtimeAspen Smith1-16/+26
This specifically allows subsequent fetch calls in the REPL to avoid fetching the same path twice (because they share the same PathInfoService). Change-Id: Ieda089080174913a8c014f13d6852cac599a7e17 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11484 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: aspen <root@gws.fyi>
2024-04-26 r/8019 feat(tvix/cli): change log format to compact, add RUST_LOG supportFlorian Klink1-1/+8
This brings feature parity with the tvix-store CLI command, at least as far as logging (not tracing) is concerned. Alternative to cl/11482, RUST_LOG seems to be the more canonical way to influence this, and is consistent with how it's done in tvix-store. Change-Id: I923a0b0ae55dc49af7efdacdcf5b1f24e561b3c2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11527 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
2024-04-22 r/7989 refactor(tvix/cli/default.nix): make eval tests and benches derivationsFlorian Klink1-49/+45
This makes the eval tests and benchmarks standalone Nix derivations and readTree targets: ``` nix-build -A tvix.cli.eval-nixpkgs-cross-hello-outpath nix-build -A tvix.cli.benchmark-nixpkgs-attrnames ``` Even without doing any fetches, We need to set `SSL_CERT_FILE`, so reqwest is able to load its CA roots. Change-Id: Ib45282d01044165c7816391adbeeb26334f8e924 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11493 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-04-19 r/7976 chore(tvix/cli): remove unused dependencyFlorian Klink1-3/+0
Change-Id: I6a6cabde22d28bb9afc391b9bcd896a999090366 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11475 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-03-20 r/7752 fix(tvix): allow trace level log messages in debug buildsFlorian Klink1-1/+1
We had max_level_debug, not max_level_trace, so it was not possible to have `RUST_LOG` emit trace-level values (at least for our crates). release_max_level_info still controls the level for release builds, so that's not affected. Change-Id: I1f127ab63f13b1622d6616e06759a9414d9bb201 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11208 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de>
2024-02-21 r/7588 revert(tvix): don't use Rc::clone explicitlyaspen1-2/+2
This reverts commit d9565a4d0af3bffd735a77aa6f1fd0ec0e03b14a. Reason for revert: this was intentional - putting Rc::clone instead of .clone is a common Rust idiom, and makes it explicit that we're cloning a shared reference, not an underlying resource Change-Id: I41a5f323ee35d7025dc7bb02f7d5d05d0051798d Reviewed-on: https://cl.tvl.fyi/c/depot/+/10995 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-02-20 r/7578 feat(tvix/eval): implement `builtins.filterSource`Ryan Lahfa1-1/+3
We add a new set of builtins called `import_builtins`, which will contain import-related builtins, such as `builtins.path` and `builtins.filterSource`. Both can import paths into the store, with various knobs to alter the result, e.g. filtering, renaming, expected hashes. We introduce `filtered_ingest` which will drive the filtered ingestion via the Nix function via the generator machinery, and then we register the root node to the path info service inside the store. `builtins.filterSource` is very simple, `builtins.path` is a more complicated model requiring the same logic albeit more sophisticated with name customization, file ingestion method and expected SHA-256. Change-Id: I1083f37808b35f7b37818c8ffb9543d9682b2de2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10654 Autosubmit: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-02-20 r/7576 refactor(tvix): don't use Rc::clone explicitlyFlorian Klink1-1/+1
All these functions have an Rc of Rc<TvixStoreIO> or Rc<dyn StoreIO>, so we can call io.clone() directly. Change-Id: I96a5bcb9af4aca93892cb72bcfaf14540da52381 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10989 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2024-02-20 r/7572 refactor(tvix/eval): use internal SourceCode field in error printersVincent Ambo1-2/+2
Makes use of the SourceCode field now being stored directly in errors (see parent CL). With this change, the default `Display` implementation can now format errors correctly, and there is no need to keep a `SourceCode` around just for error formatting. Updates dependent crates (CLI, serde, tvixbolt) to use this correctly. Change-Id: Iddc5d7a6b4bab391f30a999e4c68aca34304c059 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10987 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-02-19 r/7557 feat(tvix/glue): Init fetcher builtinsAspen Smith1-1/+3
Initialize a new empty builtins module `fetcher_builtins`, which will contain the builtins which fetch URLs from the internet: * fetchurl * fetchGit * fetchTarball * fetchTree (maybe? this is experimental) These builtins are all implemented in CPP nix at: https://github.com/NixOS/nix/blob/2.20.2/src/libexpr/primops/fetchTree.cc These builtins are added to the evaluation context using a similar mechanism to the derivation builtins, and have been added everywhere derivation builtins were previously being added. Change-Id: I133b91cc9560f23028621414537f712e7bd8a825 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10974 Autosubmit: aspen <root@gws.fyi> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-02-18 r/7548 fix(tvix/cli): set max level to info on release modeFlorian Klink1-1/+1
Log levels can be statically disabled at compile time via Cargo features. Do this for everything more verbose than info in release mode. Change-Id: I57d4282b4e19a8d44f68da2f15c7534dd3e08977 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10955 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2024-02-17 r/7543 feat(tvix/cli): set up tracing[-subscriber]Florian Klink2-1/+20
No otlp yet, this simply gives us structured log output and a cli argument for the log level. Change-Id: Ifaa60bae419640e92baebb6ee59eedd775c769c2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10853 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-02-17 r/7533 feat(tvix/cli): support configuring BuildServiceFlorian Klink1-3/+20
Allow using a BUILD_SERVICE_ADDR env var, or cmdline args to configure it. Still default to the dummy implementation. Change-Id: I68f34f7b09eabef2b0491103857bbc798398ebfc Reviewed-on: https://cl.tvl.fyi/c/depot/+/10846 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-31 r/7460 fix(tvix): Represent strings as byte arraysAspen Smith1-1/+1
C++ nix uses C-style zero-terminated char pointers to represent strings internally - however, up to this point, tvix has used Rust `String` and `str` for string values. Since those are required to be valid utf-8, we haven't been able to properly represent all the string values that Nix supports. To fix that, this change converts the internal representation of the NixString struct from `Box<str>` to `BString`, from the `bstr` crate - this is a wrapper around a `Vec<u8>` with extra functions for treating that byte vector as a "morally string-like" value, which is basically exactly what we need. Since this changes a pretty fundamental assumption about a pretty core type, there are a *lot* of changes in a lot of places to make this work, but I've tried to keep the general philosophy and intent of most of the code in most places intact. Most notably, there's nothing that's been done to make the derivation stuff in //tvix/glue work with non-utf8 strings everywhere, instead opting to just convert to String/str when passing things into that - there *might* be something to be done there, but I don't know what the rules should be and I don't want to figure them out in this change. To deal with OS-native paths in a way that also works in WASM for tvixbolt, this also adds a dependency on the "os_str_bytes" crate. Fixes: b/189 Fixes: b/337 Change-Id: I5e6eb29c62f47dd91af954f5e12bfc3d186f5526 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10200 Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de> Reviewed-by: sterni <sternenseemann@systemli.org> Autosubmit: aspen <root@gws.fyi> Tested-by: BuildkiteCI
2024-01-31 r/7459 feat(tvix/eval/observer): Allow capturing timing of eventsAspen Smith1-0/+8
Add a new --trace-runtime-timing flag (probably a better bikeshed for this) that enables capturing the time, relative to the last event, of each event recorded with the tracing observer. This probably isn't *super* useful yet, but I'd like to start here in adding new profiling tools to the VM, specifically based on the runtime observer Change-Id: Id7f12077291c39bf3eef42ab6744bfba53687a65 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10713 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2024-01-18 r/7411 refactor(tvix/glue): add BuildService to TvixStoreIOFlorian Klink2-0/+4
TvixStoreIO triggers builds whenever IO into a not-yet-built store path is requested, if it knows how to build that path. Change-Id: If30e9db6be2f2a30cbc9d0576f357f3ecfa0d35a Reviewed-on: https://cl.tvl.fyi/c/depot/+/10645 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-01-18 r/7410 feat(tvix/glue): use TvixStoreIO as derivation builtin stateRyan Lahfa1-18/+20
We propagate a `TvixStoreIO` as the `state` of our derivation-specific builtins in the glue crate. The evaluators `io_handle` itself is using a Rc<dyn EvalIO>. An earlier version of TvixStoreIO was also introducing generics over the different internal services themselves, but we opted for instead hardcoding this to Arc<dyn …> for the sake of less macro voodoo. Change-Id: I535c476f06b840858fa3070c4a237ece47f7a15b Reviewed-on: https://cl.tvl.fyi/c/depot/+/10636 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-01-16 r/7393 fix(tvix/cli/ci): prevent unnecessary dep on compared out/drv pathsterni1-1/+1
Change-Id: Ieab16fd5fad67a4ed0124f118aef8abbefd51cdb Reviewed-on: https://cl.tvl.fyi/c/depot/+/10646 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org>
2024-01-05 r/7352 refactor(tvix/glue): simplify TvixStoreIO furtherFlorian Klink1-1/+1
We don't need to spawn in all these places, we can just block_on directly, this is all IO bound. This also means, we don't need to clone any of the service handles (except preserving clone-ability of the BlobService). Change-Id: I7d90f4d6a263a98491caa071ada538a5197a5472 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10540 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-01-05 r/7350 chore(tvix): bump test-case dep to 3.3.1Florian Klink1-1/+1
Change-Id: I643548d95a5fab84563c7cbe51ca2ce640c186a9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10537 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-01-03 r/7342 feat(tvix/cli): test `firefox` `(drv|out)Path`Ryan Lahfa1-0/+2
With context strings, we are now able to reproduce up to firefox. Until a more problematic example can be found, we should at least lean on this particular example. Change-Id: Ibeaf799d26e1f160b6c93f8ccd978702a2a7e3e1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10460 Reviewed-by: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2024-01-03 r/7341 feat(tvix/cli): context-aware raw printingRyan Lahfa1-1/+1
Raw printing will transform the result into a string and print it. In case of a contextful string, this will fail by default, as expected. Change-Id: I5e564329e7b001adc57a77a9153b4425cb332bb7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10457 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2024-01-03 r/7338 feat(tvix/glue): contextful derivationRyan Lahfa1-9/+6
We calculate the input context by performing the union of context over all input of the derivation. Then, we just pass the rest of it to the remaining machinery. Finally, we re-emit an `outPath` and a `drvPath` containing the expected contexts. Change-Id: I74905fb258b5bee8b08d1208c9eb87f51b92a890 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10436 Autosubmit: raitobezarius <tvl@lahfa.xyz> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-12-31 r/7295 refactor(tvix/store): move construct_services helper hereFlorian Klink1-32/+1
This takes three URLs, and constructs Arc'ed {Blob,Directory,PathInfo}Service, allowing to remove some of the boilerplate. Change-Id: I40e7c2b551442ef2acdc543dfc87ab97e7c742bb Reviewed-on: https://cl.tvl.fyi/c/depot/+/10484 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-12-31 r/7294 refactor(tvix/castore/directorysvc): return Box, not ArcFlorian Klink1-1/+4
While we currently mostly use it in an Arc, as we need to clone it inside PathInfoService, there might be other usecases not requiring it to be Clone. Change-Id: Ia05bb370340792a048e2036be30e285ef1e63870 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10483 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-12-31 r/7293 refactor(tvix/castore/blobsvc): return Box, not ArcFlorian Klink1-1/+3
While we currently mostly use it in an Arc, as we need to clone it inside PathInfoService, there might be other usecases not requiring it to be Clone. Change-Id: I7bd337cd2e4c2d4154b385461eefa62c9b78345d Reviewed-on: https://cl.tvl.fyi/c/depot/+/10482 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-12-31 r/7292 feat(tvix/cli): allow configuring different servicesFlorian Klink1-14/+53
At some point, tvix-cli needs to talk to the outside world to persist things into a real store. Introduce the same CLI options to configure {Blob,Directory,PathInfo}Service URLs. We need to be a bit careful with how we set up stores, and make this separate from setting up TvixStoreIO, as it's holding a Rc<RefCell<KnonPath>> which not Send. At some point, we might make this a Arc<RwLock<_>> later anyways, and then this can be simplified a bit, but for now, this is sufficient. Change-Id: I87d84ca3a10ce947e194ff985073791469773f35 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10474 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-12-31 r/7289 refactor(tvix/eval): remove code and location from structFlorian Klink1-4/+4
Instead, it's passed in the evaluate/compile_only functions, which feels more naturally. It lets us set up the Evaluation struct long before we actually feed it with data to evaluate. Now that Evaluation::new() would be accepting an empty list of arguments, we can simply implement Default, making things a bit more idiomatic. Change-Id: I4369658634909a0c504fdffa18242a130daa0239 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10475 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de>
2023-12-12 r/7171 feat(tvix/cli): add benchmark for bf286a54bcAdam Joseph1-0/+2
cl/7558 used this expression as a benchmark to justify the introduction of LightSpan::Delayed: builtins.length (builtins.attrNames (import ${pkgs.path} {})) Let's add it as a benchmark case so it can be referenced easily. Benchmark: {"nixpkgs-attrnames":{"kbytes":"233824","system":"0.32","user":"2.02"}} Change-Id: Idb6c69ddd284605dd3b5fd9ac5c79a69b9a470b7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10253 Autosubmit: Adam Joseph <adam@westernsemico.com> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-12-12 r/7163 feat(tvix/cli): add macrobenchmarkAdam Joseph1-1/+61
This commit adds a simple MVP benchmark, built on our nix infrastructure instead of cargo. It simply runs `tvix-eval` inside of GNU time, and prints the three essential statistics in a short JSON blob. You can run the benchmark with a simple `nix run`, like: nix run -f . tvix.cli.benchmark-hello nix run -f . tvix.cli.benchmark-firefox nix run -f . tvix.cli.benchmark-cross-firefox Currently these blobs are stored only in the CI logs, which I'm sure get garbage-collected at some point. We should be putting them in the git trailers, but that can wait for a future CL. I tried using `cargo bench` for this but found it incredibly frustrating. Maybe I'm doing it wrong. It seems to be designed for microbenchmarks only, and very hard to control. It kept building all sorts of unnecessary stuff (like the tests), and unlike crate2nix it was doing all the builds on only a single machine instead of using more than one machine. Worse, for that single machine it kept picking my laptop instead of my fast servers! It seems excessively cargo-flavored for such a straightforward task. Benchmark: {"hello.outPath":{"kbytes":"244736","system":"0.36","user":"2.76"}} Benchmark: {"firefox.outPath":{"kbytes":"1506736","system":"2.38","user":"32.01"}} Benchmark: {"pkgsCross.aarch64-multiplatform.firefox.outPath":{"kbytes":"11334548","system":"10.70","user":"107.07"}} Change-Id: I85bc046ec551360284d7ecfc81a03914f0085909 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10216 Autosubmit: Adam Joseph <adam@westernsemico.com> Reviewed-by: grfn <grfn@gws.fyi> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-11-04 r/6939 refactor(tvix/glue): move builtins into separate directoryFlorian Klink1-1/+1
Change-Id: I25b7197458dbfbde8623545dc0a0286eb2744f10 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9911 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-11-04 r/6938 refactor(tvix/glue): move nix_path handling to helper in tvix-glueFlorian Klink1-9/+2
Change-Id: I2327560c4cf0d3f90e253e3c2f47cb29c762461e Reviewed-on: https://cl.tvl.fyi/c/depot/+/9910 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-11-04 r/6936 refactor(tvix): move tvix glue code into glue crateFlorian Klink11-1475/+8
There's various bits and pieces in tvix-cli that use both the store and evaluator, as well as nix-compat. For example, builtins.derivation, as well as the reference scanning implementation. This "glue code" currently isn't accessible from anywhere else, but it'd be very useful if it were. Move it out into a `glue` crate, and make `tvix-cli` a consumer of it. All the KnownPaths setup and passing around, as well as NIX_PATH handling is also something that should probably be moved into the glue crate as well, but that's something left for a future CL. Change-Id: I080ed3d1825ab23790666486840f301f00856277 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9908 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-10-27 r/6888 refactor(nix-compat/store_path): make digest and name privateedef1-4/+4
Change-Id: I62cbe883afcf3dd0c8d4de0e3b845069eb750c97 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9855 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-10-23 r/6876 fix(tvix/cli/derivation): fix populate_output_configurationFlorian Klink1-302/+81
The `if let` wasn't matching `outputHashAlgo` being unset, and didn't populate it in that case. Port the remaining commented-out testcases over to nix-lang based tests. Change-Id: I140b5643b9ed9d29f9522ec65d98d0b12262d728 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9825 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-10-23 r/6875 refactor(tvix/cli): rename `a` to `algo_str`Florian Klink1-2/+3
Change-Id: I65d307126647841b50db008c21c155441a6b6cb5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9818 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-10-23 r/6874 test(tvix/cli/derivation): add test cases for output path calcFlorian Klink2-0/+42
Get some test coverage with some calls to builtins.derivation. The expected output paths were calculated with Nix. Change-Id: I2ce205ea5244e8ef939d9cacb033283fc6f15d17 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9817 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>