about summary refs log tree commit diff
path: root/tvix
AgeCommit message (Collapse)AuthorFilesLines
2024-06-12 r/8252 feat(tvix/glue/fetchers): add NAR fetching infrastructureFlorian Klink1-5/+134
The magic <nix/fetchurl.nix> derivation can cause two other types of fetch to happen, one that unpacks NAR files, and another one that puts a file as an executable at the store path root. This adds the necessary enum type and path calculation logic for it to the fetcher code. It also adds code to do the actual NAR fetching. The executable case is still stubbed out. Change-Id: I79103fd58c7e22ad7fde34efa5e2d89cad7d5a0e Reviewed-on: https://cl.tvl.fyi/c/depot/+/11790 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-06-12 r/8251 refactor(tvix/glue/fetchers): move store_path() tests hereFlorian Klink2-42/+40
Move the part asking a fetch for its store_path() to the place where this function is defined, and add some more test cases. Change-Id: I96f326d0d56aa5835f23274b8cd1b1afe3724153 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11789 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-06-12 r/8250 refactor(tvix/glue/fetchers): use named field for structsFlorian Klink3-30/+76
This allows giving more self-speaking names, as well as documenting each field individually. Change-Id: Ide164d684b7f819aac279cc8e657c02fc24d093f Reviewed-on: https://cl.tvl.fyi/c/depot/+/11786 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-06-12 r/8249 test(tvix/glue/fetchurl): add output path tests for fetchurlFlorian Klink2-1/+41
These are all derived from the FOD representation of the ATerm serialization, so no new code necessary to calculate these hashes correctly. Change-Id: I8c5dacd2a8de3d5042c9dd56910511c19e1b9b31 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11788 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-06-11 r/8248 refactor(tvix/store/nar/import): add ingest_nar_and_hashFlorian Klink3-23/+46
This wraps ingest_nar, but also keeps track of the number of bytes read, and calculates the sha256 digest of it. Make use of it in the NixHTTPPathInfoService, where this code is coming from. Change-Id: I9c54e93d3ec8ed9ede87aed43e04d114fb06897b Reviewed-on: https://cl.tvl.fyi/c/depot/+/11787 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-06-11 r/8247 fix(tvix/eval): handle builtins.split matching the empty stringbinarycat4-1/+15
This prevents the following statements from looping endlessly: ``` builtins.split "(.*)" "" builtins.split "([abc]*)" "abc" builtins.split "(.*)" "abc" builtins.split ".*" "" ``` Cover these (and some more examples) in the test suite. Co-Authored-By: Florian Klink <flokli@flokli.de> Change-Id: Ibd339f971e0f4e3e5c229816e2be5a8e3836fec9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11743 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2024-06-11 r/8246 docs(tvix/docs/TODO): document ChunkService split ideaFlorian Klink1-1/+16
Change-Id: Ie9c88b0d14902c642e2d3d6603265688eef0e10d Reviewed-on: https://cl.tvl.fyi/c/depot/+/11755 Reviewed-by: yuka <yuka@yuka.dev> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-06-10 r/8242 feat(tvix/tracing): introduce common tvix-tracing crateSimon Hauser15-203/+271
Introduce a new common crate that contains tracing boilerplate which then can be used in the cli, tvix-store and tvix-build crates. It has otlp as an optional feature, which is currently only used by tvix-store. Change-Id: I41468ac4d9c65174515d721513b96fea463d6ed2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11758 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Simon Hauser <simon.hauser@helsinki-systems.de>
2024-06-10 r/8241 docs(tvix/docs/TODO): document attempt to add more granular errorsFlorian Klink1-0/+8
Change-Id: I46f99d15a4ce9c98dcdad822d3032bda3b09c12f Reviewed-on: https://cl.tvl.fyi/c/depot/+/11754 Reviewed-by: flokli <flokli@flokli.de> Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Simon Hauser <simon.hauser@helsinki-systems.de> Tested-by: BuildkiteCI
2024-06-10 r/8239 docs(tvix/docs/TODO): document crate2nix for WASM attemptsFlorian Klink1-0/+26
I tried moving web/tvixbolt to tvix/tvixbolt, and adding it to the cargo workspace. I then made `crates =` in `default.nix` a function accepting `pkgs` (so we can pass in another nixpkgs for some invocations), and then constructed another nixpkgs and crates instance like this: ``` pkgs-wasm = (import pkgs.path { localSystem = localSystem; crossSystem = { system = localSystem; rustc.config = "wasm32-unknown-unknown"; }; } crates-wasm = (crates pkgs-wasm); tvixbolt-test = crates-wasm.workspaceMembers.tvixbolt.build; ``` … leading to the architecture build failures described in the TODO. Change-Id: I32112d75f8c098d9810ca52b2d07cd76fae8d8d7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11777 Reviewed-by: flokli <flokli@flokli.de> Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-06-08 r/8227 docs(tvix): fix some typos across various documentsBen Webb5-8/+8
Fix some typos found while reading various documents, mostly those relating to the castore. Here is a summary of the edits. - fix broken link between documents in the store and castore directories - clarify expression in castore's data model document that indicates that the *name* of each child node of a directory must be unique across all three lists of children - add missing closing parenthesis in castore's data model document - replace "how" with "what" in the phrase "unclear how a ... would even look like" in castore's why-not-git-trees document - remove unnecessary articles in castore's blobstore chunking document - add missing "y" to "optionall" in eval's compilation of bindings document Change-Id: I1997ea91bb4e9c40abcd81e0cde9405968580ba6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11763 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2024-06-08 r/8225 chore(3p/sources): Bump channels & overlayssterni7-7/+7
* agenix has not been updated (https://github.com/ryantm/agenix/pull/241). * wasm-bindgen bumped to 0.2.92 in Rust WASM projects * 3p/lisp: port lispPackages from stable channel The Lisp package set we are using (`pkgs.lispPackages`) is the "old old" package set, whereas we were supposed to have been using `pkgs.lispPackages_new` (which is the "old new" package set). Either way we missed that train, and now there's a "new new" package set, but with a twist: Lisp packages in nixpkgs are now tied to their compilers, so the most generic way to access them seems to be from `pkgs.sbclPackages`. Switching to the packages from the "new new" package set doesn't work: Lots of stuff stops building if we just switch the sources over, and not everything is trivially fixable. For now we stay on the lispPackages from the stable channel. We need to look into the migration later. Or rewrite panettone. * tvix: update generated protobuf files * 3p/nixpkgs: pick trunk from stable channel; newer versions try to read files and do network I/O during build, but don't print enough details in error messages to figure out why. * 3p/overlays: remove tdlib override (nixpkgs is currently new enough) * 3p/overlays: override telega.el sources while updates are lagging in nixpkgs * users/flokli/ipu6-softisp: update firmware paths, which NixOS now stores zstd-compressed. Change-Id: I5a7a6c8b5d0688461bca92b9e6d654356d3a1cf1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11711 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su>
2024-06-06 r/8222 feat(tvix): Switch to jemallocAspen Smith8-0/+132
Switch tvix to using jemalloc as the default global allocator on supported (eg, non-msvc) platforms. This gives a pretty reasonable performance boost basically for free: int time: [24.361 µs 24.386 µs 24.418 µs] change: [-19.355% -18.859% -18.527%] (p = 0.00 < 0.05) Performance has improved. merge small attrs time: [37.201 µs 37.328 µs 37.442 µs] change: [-24.609% -24.266% -23.982%] (p = 0.00 < 0.05) Performance has improved. merge large attrs with small attrs time: [20.030 ms 20.135 ms 20.251 ms] change: [-4.2811% -3.2549% -2.3807%] (p = 0.00 < 0.05) Performance has improved. hello outpath time: [967.91 ms 974.07 ms 983.33 ms] change: [-5.5008% -4.4552% -3.4099%] (p = 0.00 < 0.05) Performance has improved. Change-Id: I6c6e6e3295ec2fca01ea28dc37bcb201cd811767 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10851 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org> Autosubmit: aspen <root@gws.fyi> Reviewed-by: flokli <flokli@flokli.de>
2024-06-06 r/8221 fix(tvix/glue): drop tracing-indicatif from depsFlorian Klink3-6/+0
We don't use this in tvix-glue currently. A previous version of cl/11747 did, and I forgot to remove it afterwards. Change-Id: I6191ffc60450f3e79f22a3aca55246e3956ee9cc Reviewed-on: https://cl.tvl.fyi/c/depot/+/11756 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Reviewed-by: Simon Hauser <simon.hauser@helsinki-systems.de> Autosubmit: flokli <flokli@flokli.de>
2024-06-06 r/8220 feat(tvix/store/bin): add progress bar infrastructureFlorian Klink8-19/+452
This adds the tracing-indicatif crate, and configures it as a layer in our tracing_subscriber pipeline to emit progress for every span that's configured so. It also moves from using std::io::stderr to write logs to using their writer, to avoid clobbering output. Progress bar styles are defined in a lazy_static, moving this into a general tracing is left for later. This adds some usage of this to the `imports` and `copy` commands. The output can still be improved a bit - we should probably split each task up into a smaller (instrumented) helper functions, so we can create a progress bar for each task. Change-Id: I59a1915aa4e0caa89c911632dec59c4cbeba1b89 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11747 Reviewed-by: flokli <flokli@flokli.de> Reviewed-by: Simon Hauser <simon.hauser@helsinki-systems.de> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-06-05 r/8219 fix(tvix/crate2nix-check): make drv less likely to be cachedIlan Joselevich1-7/+7
The derivation name for the check will now be calculated from the hash of all Cargo related files (including in subdirs), this makes it less likely for the drv to be cached and for CI to miss an outdated Cargo.lock. Change-Id: I900e9355be3f8a9d6f01162e8ef0da4d8901af30 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11753 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-06-05 r/8218 fix(tvix): make rstest pick up new test case filesIlan Joselevich3-1/+17
Context: https://github.com/la10736/rstest/issues/256 Cargo will now cause a rebuild whenever a new test case file is added. Previsouly running `cargo test` after adding a new test case resulted in that case not being picked up and ignored. Change-Id: Ibfc420b5bfe3f3ee41d3ebd3fb9d248819fa6ed9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11751 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
2024-06-05 r/8217 feat(tvix/glue): Implement builtins.storePathAspen Smith7-2/+62
This one's relatively simple - we just check if the store path exists, and if it does we make a new contextful string containing the store path as its only context element. Automatic testing seems tricky for this (I think?) so I tested it manually: tvix-repl> builtins.storePath /nix/store/yn46i4xx5alh7gs6fpkxk430i34rp2q9-hello-2.12.1 => "/nix/store/yn46i4xx5alh7gs6fpkxk430i34rp2q9-hello-2.12.1" :: string Change-Id: I8a0d9726e4102ab872c53c2419679c2c855a5a18 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11696 Tested-by: BuildkiteCI Autosubmit: aspen <root@gws.fyi> Reviewed-by: flokli <flokli@flokli.de>
2024-06-05 r/8216 docs(tvix/docs/TODO): expand Store Composition config sectionFlorian Klink1-3/+27
Write down some of the thoughts after brainstorming with yuka. Change-Id: I01c94474dc643b8c4993db80e50d3ec65f5c17f4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11749 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: yuka <yuka@yuka.dev>
2024-06-05 r/8215 fix(tvix): update Cargo.nix as it misses a default featureIlan Joselevich1-0/+1
Change-Id: I30d91a9aca9f5546ae5e1fa5c55ee389695ae085 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11752 Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-06-05 r/8214 docs(tvix/docs/TODO): extend O11Y sectionFlorian Klink1-6/+33
Expand on tvix-tracing crate strategy, add some more context regarding OTLP and span propagation. Change-Id: Ice55c116c20aaf60531100465192ce11969551ac Reviewed-on: https://cl.tvl.fyi/c/depot/+/11750 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Simon Hauser <simon.hauser@helsinki-systems.de> Reviewed-by: flokli <flokli@flokli.de>
2024-06-03 r/8211 feat(tvix): add CI targets for checking crate features powersetIlan Joselevich10-47/+112
Closes: https://b.tvl.fyi/issues/401 With this change all crate features (and their combinations) will be built and tested in CI. From now on, when adding/removing a Cargo feature for a crate, you will want to add it to the features power set that gets tested in CI. For each crate there's a default.nix with a `mkFeaturePowerset` invocation, modify the list to include/remove the feature. Note that you don't want to add "collection" features, such as `fs` for tvix-[ca]store or `default`. Change-Id: I966dde1413d057770787da3296cce9c1924570e0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11717 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-06-03 r/8210 fix(tvix/nix-compat): require async and async,wire for some testsIlan Joselevich2-1/+3
Makes the following build: `cargo test --no-default-features --features async` `cargo test --no-default-features --features wire` Change-Id: I47ba0c944f08895f67ed3b861706ef2e4ba384b3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11739 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-06-01 r/8191 feat(tvix/repl): Add a command to recursively printAspen Smith2-1/+16
Add a command, :p, to evaluate an expression and recursively print the result, as if `--strict` had been passed on the command line. Demonstration of this working: ❯ cargo r --bin tvix Finished dev [unoptimized + debuginfo] target(s) in 0.27s Running `target/debug/tvix` tvix-repl> { x = (x: x) 1; } => { x = <CODE>; } :: set tvix-repl> :p { x = (x: x) 1; } => { x = 1; } :: set Change-Id: I1a81d7481160c30d2a4483c6308e25fa45f2dfdf Reviewed-on: https://cl.tvl.fyi/c/depot/+/11738 Autosubmit: aspen <root@gws.fyi> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-06-01 r/8190 feat(tvix/repl): Add a help commandAspen Smith1-6/+29
Add a command to display help for the REPL, which can be either :? or :h. Change-Id: Ifdfd8c31130ca5afcde05a4c4276b768eb54c06f Reviewed-on: https://cl.tvl.fyi/c/depot/+/11737 Autosubmit: aspen <root@gws.fyi> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-06-01 r/8189 feat(tvix/repl): Implement :q, to quitAspen Smith1-0/+4
Change-Id: Ib3b314b21f4d8c30bfd674e79771179d51d4f2e9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11736 Autosubmit: aspen <root@gws.fyi> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-06-01 r/8188 refactor(tvix/repl): Abstract out REPL command handlingAspen Smith1-10/+26
Prepare for introducing additional REPL commands by splitting out the *parsing* of the repl commands, which returns a new ReplCommand type, from the actual *handling* of the commands. Change-Id: If81a53c1e2d90204d26ce3bb2ea9eebf7bb3fd51 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11734 Autosubmit: aspen <root@gws.fyi> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-06-01 r/8187 refactor(tvix): Break out REPL into its own moduleAspen Smith2-99/+122
In preparation for adding some new functionality to the tvix REPL, break it out into its own module. Change-Id: I4fb78320e92562e3474a3724536cb22c1d893e57 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11733 Tested-by: BuildkiteCI Autosubmit: aspen <root@gws.fyi> Reviewed-by: flokli <flokli@flokli.de>
2024-06-01 r/8186 feat(tvix/repl): Support multiline inputAspen Smith4-11/+102
Transparently support multiline input in the Tvix REPL, by handling the UnexpectedEOF error returned by the parser and using it to progressively build up an input expr over multiple iterations of the REPL's outer loop. This works quite nicely: ❯ cargo r --bin tvix Compiling tvix-cli v0.1.0 (/home/aspen/code/depot/tvix/cli) Finished dev [unoptimized + debuginfo] target(s) in 1.72s Running `target/debug/tvix` tvix-repl> { foo > = > 1; > } => { foo = 1; } :: set <press up arrow> tvix-repl> { foo = 1; } => { foo = 1; } :: set Change-Id: Ib0ed4766b13e8231d696cdc27281ac158e20a777 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11732 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: aspen <root@gws.fyi> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-05-30 r/8184 fix(tvix/eval): nix_tests.rs's eval_test requires impure flagIlan Joselevich1-0/+1
This fixes the following command: 'cargo test --no-default-features --features nix_tests' Change-Id: I9883c39e1e428c72a0e7e0b75a73c8ed734abd3b Reviewed-on: https://cl.tvl.fyi/c/depot/+/11740 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-05-30 r/8183 fix(tvix/eval): proptests require arbitrary featureFlorian Klink2-2/+2
`cargo test --no-default-features` fails, if we don't conditionalize this on the `arbitrary` feature too. Change-Id: I81a277810119fed0cfc37c942c422f731aa14b2e Reviewed-on: https://cl.tvl.fyi/c/depot/+/11726 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-05-30 r/8182 fix(tvix/eval/tests/one_offs): test_source_builtin can be pureFlorian Klink1-1/+1
`Evaluation::new_impure()` would require the test to be impure, but there's nothing in this test specifically requiring us to make use of impure features. Change-Id: Idb24981195d1a94f51053ae04403eb5f0e27f3d9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11725 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-05-30 r/8181 fix(tvix/eval/tests/nix_oracle): tests are impure-onlyFlorian Klink1-0/+2
This uses StdIO, which is only available when the `impure` feature is enabled. Change-Id: I039b1f45f6619dd099fa943e58322ff521482dfa Reviewed-on: https://cl.tvl.fyi/c/depot/+/11724 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-05-30 r/8180 refactor(tvix/eval): move nix_tests to separate moduleFlorian Klink2-198/+207
Gate this behind the nix_tests feature flag (which wasn't applied consistently). Also, at least right now all of these use StdIO internally, either by creating it on their own, or through the `eval_test` helper. Once we have some proper classification system for tests we can probably split this up further, but for now only run them if the impure feature is enabled. Change-Id: I3236cf09b3391585df99073367c4e4832c5e7898 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11723 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-05-30 r/8179 fix(tvix/eval/vm/generators): allow unused rq_{path_exists,read_dir}Florian Klink1-0/+2
These VM requests are only emitted by code gated behind the impure feature. To prevent warning from popping up when building without default features, allow these to be unused if `impure` is not enabled. Change-Id: Id871a5215e9a0f09aa78edecdd111369ee7ffe34 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11722 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-05-26 r/8176 fix(tvix/eval/nix_search_path): gate tests on impure featureFlorian Klink1-0/+2
These use StdIO, which is only available if the impure feature is enabled. Change-Id: I18d8e191a7eba6ba5bd59f43631973eaa796c7bb Reviewed-on: https://cl.tvl.fyi/c/depot/+/11721 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su>
2024-05-26 r/8175 fix(tvix/eval/io): OsStringExt and std::fs::File import are impure-onlyFlorian Klink1-2/+4
These are only needed when building with the impure feature enabled. This removes some warnings when building with --no-default-features. Change-Id: I3139d9133d4846aeb1b1b5f3830c0d078d047292 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11720 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2024-05-26 r/8174 fix(tvix/nix-compat): async nar reader requires wire featureFlorian Klink2-2/+2
It uses it internally. Change-Id: I8cb8fc9f567260d57f3a203407333d83beddf537 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11719 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2024-05-26 r/8173 fix(tvix/build): tonic-reflection feature needs castore with it tooFlorian Klink2-2/+2
If building tvix-build with the tonic-reflection feature, it needs to import `tvix_castore::proto::FILE_DESCRIPTOR_SET`, which is only available if tvix-castore is built with the `tonic-reflection` feature. Change-Id: I355b4c5b4c1333d5cc56335de47ad5d2f1db6337 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11716 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de>
2024-05-26 r/8172 refactor(tvix/castore): move src/fs/test into fuse modFlorian Klink3-8/+9
These tests only interact with the FUSE layer, and import super::fuse to do its work. However, this only works if the `fuse` feature is enabled, which we don't do if we enable the `virtiofs` feature only, causing the tests to fail: ``` ❯ cargo test --no-default-features --features virtiofs Compiling tvix-castore v0.1.0 (/home/flokli/dev/nixos/code.tvl.fyi-submit2/tvix/castore) error[E0432]: unresolved import `super::fuse` --> castore/src/fs/tests.rs:14:13 | 14 | use super::{fuse::FuseDaemon, TvixStoreFs}; | ^^^^ could not find `fuse` in `super` ``` We move src/fs/tests.rs to src/fs/fuse/tests.rs (and src/fs/fuse.rs to src/fs/fuse/mod.rs) to better structure this, which will automatically cause both tests and code to only be built if we have the `fuse` feature enabled. Change-Id: I8fbbad3e4457e326bdfd171aa5c43d25d3187b5b Reviewed-on: https://cl.tvl.fyi/c/depot/+/11715 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2024-05-26 r/8169 feat(tvix): filter src via lib.filesetIlan Joselevich1-0/+60
Previously changing any file (including default.nix or README.md) would cause Nix to compute a different derivation path, resulting in needing to (very often redundantly) rebuild the crate/package. With this change you can expect less rebuilds. Crate2nix currently does its own insufficient src filtering and it does not expose an API to override the filtering, so instead I created a function that we can use to override the src to have stricter filtering. I implemented the filtering for all of the workspace members, if you want to modify any of them, please read the lib.fileset docs https://nixos.org/manual/nixpkgs/unstable/#sec-functions-library-fileset Change-Id: I7ab5a0064a76938d14f7f65801be9f3a5c3bad04 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11714 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-05-25 r/8168 fix(tvix/eval): drop superfluous string context checkedef1-4/+1
cl/11712 simultaneously introduced this check and made it unnecessary, since NixString::context should never return `Some` for empty contexts now. Change-Id: I41a655ff33910e8326cbb7d7526eb91bd19e9585 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11713 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-05-25 r/8167 fix(tvix/eval): disallow empty but allocated string contextsedef1-3/+25
Both `Some(NixContext::new())` and `None` represent empty contexts, but the former trips up `NixString::has_context`, and seems likely to trip up other things. We could hide the difference in the accessors, but we don't really *want* the distinction to exist, since heap-allocating a null value is pretty much always a mistake. Change-Id: Ie84d26fb0d4b59e68354891ba13bde3bae40ab6e Reviewed-on: https://cl.tvl.fyi/c/depot/+/11712 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-05-23 r/8166 docs(tvix/docs/TODO): add item for string context reworkFlorian Klink1-0/+9
Change-Id: I4592490a75fb05093b3fd8649db3b57bff748c0d Reviewed-on: https://cl.tvl.fyi/c/depot/+/11707 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI
2024-05-23 r/8165 docs(tvix/TODO): toXML string context and self-closing tags is doneFlorian Klink1-3/+0
Change-Id: I83f6b1863a023ea0bf8518c67eb51f740c3ea89a Reviewed-on: https://cl.tvl.fyi/c/depot/+/11706 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Alyssa Ross <hi@alyssa.is>
2024-05-23 r/8164 feat(tvix/eval): rm NixContext::join, add take_context & IntoIteratorFlorian Klink6-60/+57
In places where we want to extend context with that from another NixString, use take_context() to split it off, then call .extend(), making use of IntoIterator to avoid a bunch of clones. Change-Id: I2460141a3ed776c64c36132b2203b6a1d710b922 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11705 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: edef <edef@edef.eu>
2024-05-23 r/8163 feat(tvix/eval): teach builtins.toXML contextFlorian Klink4-6/+53
XmlEmitter gains a NixContext field, and `write_typed_value` extends it with all context elements present in the passed value. Once all serialization is done, a into_context() function returns the collected context, so we can construct a NixString with context. Tests for this live in tvix-glue, as we use builtins.derivation, which is not present in the tvix-eval crate. Fixes b/398. Change-Id: I85feaaa17b753885f8a017a54e419ec4e602af21 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11704 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Alyssa Ross <hi@alyssa.is>
2024-05-23 r/8162 feat(tvix/eval): add NixContext::extendFlorian Klink1-1/+9
This is a slightly less annoying version of `join`, which does not consume self. It's more consistent with HashSet::extend(). Change-Id: Ifd0872da36fe8e7b2aa6948674cb8e4023abe9d7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11703 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: edef <edef@edef.eu>
2024-05-23 r/8161 fix(tvix/eval): NixString are bytesFlorian Klink1-1/+1
This is not necessarily valid UTF-8. Change-Id: I72f3157240772eb9c558e5699b4785e44d256fd4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11702 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2024-05-22 r/8160 refactor(tvix/eval): rewrite xml emitter to be simple-stupidProfpatsch9-89/+205
In order to be compatible with the nix XML generator, it’s easier to generate the XML directly, instead of going through a library which we have to bend to do what we need. Removes dependency on `xml-rs`, which came with a full XML parser that we didn’t use. Only takes a tiny bit of code for the XML escaping, somewhat simplified. I add a little escaping value, to make sure we have the same behaviour as nix proper. Interestingly enough, we never need to escape XML attribute names, because the `builtins.toXML` format encodes user-defined values as attribute keys only. So we only escape attribute values. Fixes: https://b.tvl.fyi/issues/399 Change-Id: If4d407d324864b3bb9aa3160e2ec6889f7727127 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11697 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Profpatsch <mail@profpatsch.de>