about summary refs log tree commit diff
AgeCommit message (Collapse)AuthorFilesLines
2024-08-19 r/8525 fix(tazjin/german-string): add maximum length checkVincent Ambo1-0/+4
Change-Id: I4fd9b2a31749d65632b662807c4e703a9df9ec9e Reviewed-on: https://cl.tvl.fyi/c/depot/+/12237 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2024-08-19 r/8524 feat(tazjin/german-string): add data accessors for &str and &[u8]Vincent Ambo1-1/+67
Change-Id: I992e625861f79ef6d9993e8caee4e02d3fc5557e Reviewed-on: https://cl.tvl.fyi/c/depot/+/12236 Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: tazjin <tazjin@tvl.su>
2024-08-19 r/8523 feat(tazjin/german-string): add Drop impl for transient stringsVincent Ambo1-0/+17
All of these strings are currently transient (the storage class is not yet represented anywhere), and the ones that are heap allocated need to be deallocated when the transient string dies. Change-Id: Iba0ca926df5db7594f304c5d5318db9d69c6f42c Reviewed-on: https://cl.tvl.fyi/c/depot/+/12235 Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2024-08-19 r/8522 feat(tazjin/german-string): init initial representationVincent Ambo4-0/+82
This adds an initial implementation of the so-called "German Strings" in Rust. https://cedardb.com/blog/german_strings/ This implementation is *far from* complete, the only thing that can be done right now is construct a string, and even that I'm not fully happy with. Change-Id: I2697932a0ef373be76ffd14d59677493a5783b58 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12234 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2024-08-19 r/8521 refactor(tvix/eval): remove use of imbl::OrdMapVincent Ambo8-451/+95
Removes imbl::OrdMap in favour of an Rc over the standard library's BTreeMap, which allows us to drop the imbl dependency completely. In my local tests this is actually slightly faster for `hello` and `firefox`. Change-Id: Ic9597ead4e98bf9530f290c6a94a3c5c3efd0acc Reviewed-on: https://cl.tvl.fyi/c/depot/+/12201 Reviewed-by: aspen <root@gws.fyi> Tested-by: BuildkiteCI
2024-08-19 r/8520 refactor(tvix/eval): remove use of imbl::VectorVincent Ambo4-47/+46
This vector type has served us well for now, but it contains internal refcounts which are incompatible with upcoming changes related to garbage collection. The performance impact of this change within all benchmarks I ran was within the margin of error: [nix-shell:/tmp/perf]$ hyperfine "./before -E '(import <nixpkgs> {}).firefox.outPath' --log-level ERROR --no-warnings" Benchmark 1: ./u64 -E '(import <nixpkgs> {}).firefox.outPath' --log-level ERROR --no-warnings Time (mean ± σ): 7.528 s ± 0.272 s [User: 6.578 s, System: 0.631 s] Range (min … max): 7.160 s … 8.012 s 10 runs nix-shell:/tmp/perf]$ hyperfine "./std-vec -E '(import <nixpkgs> {}).firefox.outPath' --log-level ERROR --no-warnings" Benchmark 1: ./std-vec -E '(import <nixpkgs> {}).firefox.outPath' --log-level ERROR --no-warnings Time (mean ± σ): 7.515 s ± 0.178 s [User: 6.508 s, System: 0.652 s] Range (min … max): 7.276 s … 7.861 s 10 runs Change-Id: Ib95f871956e336a1e5771f6293583854b1efb276 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12197 Reviewed-by: aspen <root@gws.fyi> Tested-by: BuildkiteCI
2024-08-19 r/8519 refactor(tvix/eval): ensure VM operations fit in a single byteVincent Ambo14-611/+821
This replaces the OpCode enum with a new Op enum which is guaranteed to fit in a single byte. Instead of carrying enum variants with data, every variant that has runtime data encodes it into the `Vec<u8>` that a `Chunk` now carries. This has several advantages: * Less stack space is required at runtime, and fewer allocations are required while compiling. * The OpCode doesn't need to carry "weird" special-cased data variants anymore. * It is faster (albeit, not by much). On my laptop, results consistently look approximately like this: Benchmark 1: ./before -E '(import <nixpkgs> {}).firefox.outPath' --log-level ERROR --no-warnings Time (mean ± σ): 8.224 s ± 0.272 s [User: 7.149 s, System: 0.688 s] Range (min … max): 7.759 s … 8.583 s 10 runs Benchmark 2: ./after -E '(import <nixpkgs> {}).firefox.outPath' --log-level ERROR --no-warnings Time (mean ± σ): 8.000 s ± 0.198 s [User: 7.036 s, System: 0.633 s] Range (min … max): 7.718 s … 8.334 s 10 runs See notes below for why the performance impact might be less than expected. * It is faster while at the same time dropping some optimisations we previously performed. This has several disadvantages: * The code is closer to how one would write it in C or Go. * Bit shifting! * There is (for now) slightly more code than before. On performance I have the following thoughts at the moment: In order to prepare for adding GC, there's a couple of places in Tvix where I'd like to fence off certain kinds of complexity (such as mutating bytecode, which, for various reaons, also has to be part of data that is subject to GC). With this change, we can drop optimisations like retroactively modifying existing bytecode and *still* achieve better performance than before. I believe that this is currently worth it to pave the way for changes that are more significant for performance. In general this also opens other avenues of optimisation: For example, we can profile which argument sizes actually exist and remove the copy overhead of varint decoding (which does show up in profiles) by using more adequately sized types for, e.g., constant indices. Known regressions: * Op::Constant is no longer printing its values in disassembly (this can be fixed, I just didn't get around to it, will do separately). Change-Id: Id9b3a4254623a45de03069dbdb70b8349e976743 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12191 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-08-19 r/8518 docs(tvix/glue/tvix_io): update doc-commentsFlorian Klink1-4/+2
Since we switched from reference scanning to string context, this only handles the `__corepkgs__` hack. Update the docstrings. Change-Id: Ie857c8c99ae1cdb4697323ec738f88be0580df3e Reviewed-on: https://cl.tvl.fyi/c/depot/+/12246 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2024-08-19 r/8517 feat(ops/pipelines): support buildkite retriesFlorian Klink1-4/+6
cl/12228 did enable automatic retries for some flaky tests, which generally did work, as can be seen in https://buildkite.com/tvl/depot/builds/35893 However, ":duck:" still reports as failing, because we check the number of steps to be nonzero, which is not the case if retries have happened. We cannot check for the overall status of the build, as it's still "RUNNING", but instead of counting all failed steps so far, we can query all failed jobs and then filter out the ones that were already retried. Change-Id: Ib9d27587c8a8ba7970850812c4302fecdc4482e7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12233 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2024-08-18 r/8516 docs(views): update where `josh-filter` is coming fromFlorian Klink1-3/+2
We started using josh from nixpkgs since cl/11457, but forgot to update this documentation. Change-Id: I6e07227bcd3e955076d46146024edd89b69f08f7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12244 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com> Tested-by: BuildkiteCI
2024-08-18 r/8515 refactor(tvix/eval): Pull context out into its own moduleAspen Smith2-156/+165
I'm gonna be doing some poking around in the internals of Context, so in preparation this pulls it out into its own module. Change-Id: I72ea7df80b5f36f838934ee07bdba66874c334c9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12189 Autosubmit: aspen <root@gws.fyi> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-08-18 r/8514 refactor(tvix/castore): have SymlinkTarget-specific errorsFlorian Klink5-26/+172
Don't use ValidateNodeError, but SymlinkTargetError. Also, add checks for too long symlink targets. Change-Id: I4b533325d494232ff9d0b3f4f695f5a1a0a36199 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12230 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: edef <edef@edef.eu> Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com> Tested-by: BuildkiteCI
2024-08-18 r/8513 refactor(tvix/castore): have PathComponent-specific errorsFlorian Klink6-76/+246
Don't use DirectoryError, but PathComponentError. Also add checks for too long path components. Change-Id: Ia9deb9dd0351138baadb2e9c9454c3e019d5a45e Reviewed-on: https://cl.tvl.fyi/c/depot/+/12229 Tested-by: BuildkiteCI Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com> Autosubmit: flokli <flokli@flokli.de> Reviewed-by: edef <edef@edef.eu>
2024-08-18 r/8512 feat(tvix/castore/proto): add owned conv to castore::DirectoryFlorian Klink2-65/+84
Replace the hand-rolled code comparing names with a try_fold. Also, make it slightly stricter here, detecting duplicates in the same fields earlier. Change-Id: I9c560838ece88c3b8b339249a8ecbf3b05969538 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12226 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI
2024-08-18 r/8511 feat(tvix/castore): add Directory::try_from_iter()Florian Klink4-160/+201
This provides a batched variant to construct a Directory, which reuses the previously calculated size. Checking and inserting code is factored out into a check_insert_node function, taking the current size as a parameter and returning the new size. Change-Id: Ia6c2970a0c12181b7c40e63cf7ce8c93298ea37c Reviewed-on: https://cl.tvl.fyi/c/depot/+/12225 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI
2024-08-17 r/8510 fix(tvix/boot): workaround test flakinessFlorian Klink1-0/+8
The boot tests are sometimes flaky, and we don't want them to periodically fail other build. Have Buildkite auto-retry them 2 times on failure. Logs for individual failures are still available, so it won't hinder flakiness debuggability. See https://buildkite.com/docs/pipelines/command-step#retry-attributes for a documentation of a parameter, and cl/8983 for the introduction of that feature to //nix/buildkite. Change-Id: I1c0d25fa1d0ca940b3bdcd145ede87154b0c28eb Reviewed-on: https://cl.tvl.fyi/c/depot/+/12228 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com> Tested-by: BuildkiteCI
2024-08-17 r/8509 chore(tvix/glue): drop some explicit allow(clippy::mutable_key_type)Florian Klink2-3/+0
This is covered by clippy.toml these days. Change-Id: I2330af5781844d5f9d975793d770efcea48d371b Reviewed-on: https://cl.tvl.fyi/c/depot/+/12223 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-08-17 r/8508 refactor(tvix/castore): add `name` back to edge weightsFlorian Klink1-11/+22
Make this a proper struct with named fields. We apparently never access B3Digest in there, so it can be removed. Change-Id: Ifc07310393e1afb0a835778eae137a19b54070b9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12224 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-08-17 r/8507 refactor(tvix/castore): add into_nodes(), implement consuming proto convFlorian Klink5-29/+24
Provide a into_nodes() function on a Directory, which consumes self and returns owned PathComponent and Node. Use it to provide a proper conversion from Directory to the proto variant that doesn't clone. There's no need for the one taking only &Directory, we don't use it anywhere, and once someone needs that they might as well clone Directory before converting it. Update all other users of the `.nodes()` function to use `.into_nodes()` where applicable, and avoid some more cloning there. Change-Id: Id4577b9eb173c012e225337458898d3937112bcb Reviewed-on: https://cl.tvl.fyi/c/depot/+/12218 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-08-17 r/8506 refactor(tvix/castore): add PathComponent type for checked componentsFlorian Klink25-165/+282
This encodes a verified component on the type level. Internally, it contains a bytes::Bytes. The castore Path/PathBuf component() and file_name() methods now return this type, the old ones returning bytes were renamed to component_bytes() and component_file_name() respectively. We can drop the directory_reject_invalid_name test - it's not possible anymore to pass an invalid name to Directories::add. Invalid names in the Directory proto are still being tested to be rejected in the validate_invalid_names tests. Change-Id: Ide4d16415dfd50b7e2d7e0c36d42a3bbeeb9b6c5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12217 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-08-17 r/8505 refactor(tvix/castore): drop {Directory,File,Symlink}NodeFlorian Klink27-463/+557
Add a `SymlinkTarget` type to represent validated symlink targets. With this, no invalid states are representable, so we can make `Node` be just an enum of all three kind of types, and allow access to these fields directly. Change-Id: I20bdd480c8d5e64a827649f303c97023b7e390f2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12216 Reviewed-by: benjaminedwardwebb <benjaminedwardwebb@gmail.com> Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-08-17 r/8504 refactor(tvix/castore): remove `name` from NodesFlorian Klink46-1002/+785
Nodes only have names if they're contained inside a Directory, or if they're a root node and have something else possibly giving them a name externally. This removes all `name` fields in the three different Nodes, and instead maintains it inside a BTreeMap inside the Directory. It also removes the NamedNode trait (they don't have a get_name()), as well as Node::rename(self, name), and all [Partial]Ord implementations for Node (as they don't have names to use for sorting). The `nodes()`, `directories()`, `files()` iterators inside a `Directory` now return a tuple of Name and Node, as does the RootNodesProvider. The different {Directory,File,Symlink}Node struct constructors got simpler, and the {Directory,File}Node ones became infallible - as there's no more possibility to represent invalid state. The proto structs stayed the same - there's now from_name_and_node and into_name_and_node to convert back and forth between the two `Node` structs. Some further cleanups: The error types for Node validation were renamed. Everything related to names is now in the DirectoryError (not yet happy about the naming) There's some leftover cleanups to do: - There should be a from_(sorted_)iter and into_iter in Directory, so we can construct and deconstruct in one go. That should also enable us to implement conversions from and to the proto representation that moves, rather than clones. - The BuildRequest and PathInfo structs are still proto-based, so we still do a bunch of conversions back and forth there (and have some ugly expect there). There's not much point for error handling here, this will be moved to stricter types in a followup CL. Change-Id: I7369a8e3a426f44419c349077cb4fcab2044ebb6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12205 Tested-by: BuildkiteCI Reviewed-by: yuka <yuka@yuka.dev> Autosubmit: flokli <flokli@flokli.de> Reviewed-by: benjaminedwardwebb <benjaminedwardwebb@gmail.com> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-08-16 r/8503 feat(tvix/cli): Extend nixpkgs eval test to include NixOS installerIlan Joselevich1-23/+39
The tvix-cli tests now include evaluating the NixOS GNOME installer ISO image and making sure that its drvPath and outPath matches the one evaluated by Nix. (This required extending the helper function a bit and adding docs). NixOS docs generation is disabled for now, see comments in diff. Change-Id: Ia510f209b1ec3ef9a823f1e5ac0ef2f5f193976f Reviewed-on: https://cl.tvl.fyi/c/depot/+/12177 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-08-16 r/8502 feat(tvix/glue): Add refscanner pattern and AsyncReadBrian Olsen4-22/+249
This splits the existing ReferenceScanner into a ReferenceScanner and ReferencePattern as well as adds an AsyncRead implementation that can do a scan while you read from it. The reason to split the scanner in two is that generating the pattern is expensive and when ingesting build results with multiple outputs you want to do several independant scans that look for the same pattern. The reader is for scanning files without having to load the entire file into memory. Change-Id: I993f5a32308c12d9035840f8e04fe82e8dc1d962 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12052 Autosubmit: Brian Olsen <me@griff.name> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-08-16 r/8501 docs(tvix/build/protos): clarify content-addressingFlorian Klink2-4/+4
This comment is a bit misleading - everything that's not in this field is referenced in a content-addressed fashion (by their digest). Change-Id: I5097131530fd188173393063643c057f588ea2c4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12219 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-08-16 r/8500 chore(tvix): bump object_storeFlorian Klink2-14/+44
Fix for RUSTSEC-2024-0358. Change-Id: Id9e8c6ae67845d95a52c884035fcea02d2d71505 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12214 Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-08-16 r/8499 chore(tvix/tools/crunch-v2): bump curve25519-dalekFlorian Klink2-30/+4
Fix for RUSTSEC-2024-0344. Change-Id: I11651ea56e52988ee7cef2e02d0cbf61062f5c7b Reviewed-on: https://cl.tvl.fyi/c/depot/+/12213 Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-08-16 r/8498 chore(tvix/tools/narinfo2parquet): bump curve25519-dalekFlorian Klink2-30/+4
Fix for RUSTSEC-2024-0344. Change-Id: Iad02bcf312af8b6bf450ab3c4c6de31d9c02ab87 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12212 Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-08-16 r/8497 chore(tvix/tools/weave): bump curve25519-dalekFlorian Klink2-30/+4
Fix for RUSTSEC-2024-0344. Change-Id: I414c1424941af1352f4d3b4e5908083ab0cf4e9d Reviewed-on: https://cl.tvl.fyi/c/depot/+/12211 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com> Tested-by: BuildkiteCI
2024-08-16 r/8496 chore(tvix/nix-compat): bump curve25519-dalekFlorian Klink2-30/+238
Fix for RUSTSEC-2024-0344. Change-Id: I9e9cac5f0dac9e5b2e5a934c99762d982136c0e4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12210 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com> Tested-by: BuildkiteCI
2024-08-16 r/8495 chore(tvix/tools/crunch-v2): update Cargo.{lock,nix}Florian Klink2-46/+245
These were out of date. Change-Id: Ideaf888c2851bb9ec36ae01b11d93165b62cd7a5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12209 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
2024-08-15 r/8494 chore(tvix): Retain original passthru in Rust buildsBrian Olsen8-13/+13
When using the runTests feature of crate2nix the derivation that runs the tests is put into passthru.test but all default.nix files for Rust crates in Tvix threw that away. This commit retains passthru so that you can get access to the test derivation. Change-Id: I8b7b7db57a49069348f08c12c00a3b1a41a0c05b Reviewed-on: https://cl.tvl.fyi/c/depot/+/12215 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-08-15 r/8493 chore(tvix/tools/narinfo2parquet): update Cargo.{lock,nix}Florian Klink2-39/+209
These were out of date. Change-Id: I89df37f088ad6c53b676d965f0dc1023f40481f4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12208 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
2024-08-15 r/8492 chore(tvix/tools/turbofetch): update Cargo.nixFlorian Klink1-39/+100
This was out of date. Change-Id: Icea6ee865d389d56bc46941f6049721db293ef7e Reviewed-on: https://cl.tvl.fyi/c/depot/+/12207 Tested-by: BuildkiteCI Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com> Autosubmit: flokli <flokli@flokli.de>
2024-08-15 r/8491 chore(tvix/tools/wave): update Cargo.lock and Cargo.nixFlorian Klink2-39/+209
This apparently was out of date. Change-Id: I9768d5e62f200169d7815ad85aa0f12eadc56a8e Reviewed-on: https://cl.tvl.fyi/c/depot/+/12206 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
2024-08-15 r/8490 fix(aspen/system): Remove duplicate emailAspen Smith1-1/+1
Change-Id: I7b30ca4940bcc65488b2fbb190b3f51dc4d76c15 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12204 Autosubmit: aspen <root@gws.fyi> Tested-by: BuildkiteCI Reviewed-by: aspen <root@gws.fyi>
2024-08-14 r/8489 chore(tvix/verify-lang-tests): update latest Nix versionsterni1-1/+1
Change-Id: I656c183d970806565d45ffd31f99f73c804dda1d Reviewed-on: https://cl.tvl.fyi/c/depot/+/12203 Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
2024-08-14 r/8488 chore(3p/sources): Bump channels & overlayssterni2-17/+17
* //users/sterni/machines/ingeborg: adjust to yet another API change in the fcgiwrap module Change-Id: Ic601bb7161887dec5cfbe68205be816cf9b92d17 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12202 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: sterni <sternenseemann@systemli.org>
2024-08-13 r/8487 docs(tvix/TODO): castore Directory struct is doneFlorian Klink1-16/+8
There's some followup work on making tvix-build use the stricter structs natively, but that should probably be combined with the overall trait changes there. Add an item there so we won't forget, but this isn't really castore territory anymore. Change-Id: I6b1f9fa02d5c87c821d4ddc4b1dcc6a98c4eeaa5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12200 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: yuka <yuka@yuka.dev> Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
2024-08-13 r/8486 refactor(tvix/castore): move *Node and Directory to crate rootFlorian Klink42-622/+620
*Node and Directory are types of the tvix-castore model, not the tvix DirectoryService model. A DirectoryService only happens to send Directories. Move types into individual files in a nodes/ subdirectory, as it's gotten too cluttered in a single file, and (re-)export all types from the crate root. This has the effect that we now cannot poke at private fields directly from other files inside `crate::directoryservice` (as it's not all in the same file anymore), but that's a good thing, it now forces us to go through the proper accessors. For the same reasons, we currently also need to introduce the `rename` functions on each *Node directly. A followup is gonna move the names out of the individual enum kinds, so we can better represent "unnamed nodes". Change-Id: Icdb34dcfe454c41c94f2396e8e99973d27db8418 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12199 Reviewed-by: yuka <yuka@yuka.dev> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-08-13 r/8485 docs(tvix/TODO): DirectoryService with bucketed directories is doneFlorian Klink1-6/+0
We have this (`ObjectStoreDirectoryService`), as well as some store composition. Change-Id: I876f85da1d5b183d80b148c378f825033ec080e4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12198 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com> Tested-by: BuildkiteCI
2024-08-13 r/8484 refactor(tvix/castore): use Directory struct separate from proto oneYureka53-1365/+1417
This uses our own data type to deal with Directories in the castore model. It makes some undesired states unrepresentable, removing the need for conversions and checking in various places: - In the protobuf, blake3 digests could have a wrong length, as proto doesn't know fixed-size fields. We now use `B3Digest`, which makes cloning cheaper, and removes the need to do size-checking everywhere. - In the protobuf, we had three different lists for `files`, `symlinks` and `directories`. This was mostly a protobuf size optimization, but made interacting with them a bit awkward. This has now been replaced with a list of enums, and convenience iterators to get various nodes, and add new ones. Change-Id: I7b92691bb06d77ff3f58a5ccea94a22c16f84f04 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12057 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-08-12 r/8483 docs(tvix/store/utils): add docstrings for ServiceUrls*Florian Klink1-2/+15
Describe what these structs are used for, and for each of it, explain which usecases it's used for. Change-Id: I8b7857bc68ec2b37df9f5163e06d028a64a12c79 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12195 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
2024-08-12 r/8482 feat(tvix/cli): Add derivation file dumping functionalityIlan Joselevich4-2/+49
Provides a derivation file dumping functionality for tvix-cli that can be used when passing the --drv-dumpdir CLI arg to tvix-cli. This will dump all the known derivation files into the specified directory, making it easier to debug derivation divergences between Tvix generated drvs and the drvs generated by Nix. Supersedes: https://cl.tvl.fyi/c/depot/+/11265 Change-Id: I0e10b26eba22032b84ac543af0d4150ad87aed3e Reviewed-on: https://cl.tvl.fyi/c/depot/+/12192 Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-08-12 r/8481 fix(tvix/cli): add toplevel docstring to Args structFlorian Klink1-0/+8
Otherwise the flattened `ServiceUrlsMemory` docstring becomes the main binary description, which doesn't make an awful lot of sense. The help now describes what this CLI does currently (an interface to use the different Tvix component. Also, describe what it does not, so people running it without looking too much other documentation might not get confused about why this doesn't do anything in `/nix/store`, or doesn't provide the same CLI surface as Nix. Change-Id: Ia4838b444f03a10821801a6171d3e956b3cdfdaf Reviewed-on: https://cl.tvl.fyi/c/depot/+/12194 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-08-11 r/8480 chore(tvix/eval): Update module comment for value::stringAspen Smith1-3/+2
This sentence is a little stale; let's just link to NixString directly for the authoritative source of truth. Change-Id: I64e065c4148d29702b09820a0e7724a65fae7c67 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12181 Autosubmit: aspen <root@gws.fyi> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-08-11 r/8479 feat(tvix/value): Print derivation values speciallyAspen Smith1-0/+12
Just like tvix-repl does (except we don't force values when printing them, so... not entirely like tvix-repl does). But it's something. Change-Id: I2e69b08d7d82b0b2d337f1d4c5d87ed28475fa84 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12180 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: aspen <root@gws.fyi>
2024-08-11 r/8478 feat(ops/users): Add elle to usersElle Najt1-0/+5
Change-Id: I18492d6e6167f3c010e8f66670a127807ac7d99c Reviewed-on: https://cl.tvl.fyi/c/depot/+/12183 Reviewed-by: aspen <root@gws.fyi> Autosubmit: aspen <root@gws.fyi> Tested-by: BuildkiteCI
2024-08-11 r/8477 chore(mailmap): Add my name and emailAspen Smith1-0/+1
Change-Id: I109da892dc01368ceb38d34e9095559895cbd60c Reviewed-on: https://cl.tvl.fyi/c/depot/+/12182 Tested-by: BuildkiteCI Reviewed-by: aspen <root@gws.fyi> Autosubmit: aspen <root@gws.fyi>
2024-08-11 r/8476 fix(tvix/cli): always configure nix pathAspen Smith2-2/+15
Configure the nix path even if globals is already set. Change-Id: I6598c92ab40ff952f73da04d9e7d3aeb13c16b53 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12176 Tested-by: BuildkiteCI Autosubmit: aspen <root@gws.fyi> Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>