about summary refs log tree commit diff
path: root/tvix
AgeCommit message (Collapse)AuthorFilesLines
2024-09-26 r/8716 feat(tvix/store): Add a signing PathInfoServicesinavir6-0/+225
- Add a new PathInfoService implementation that wraps transparently around another except that it dynamically signs all the incoming path-infos with the provided signer. - Add a ServiceBuilder for this PathInfoService that provides a SigningPathInfoService with a keyfile signer Change-Id: I845ddfdf01d14c503c796b2b80c720dab98be091 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12032 Reviewed-by: flokli <flokli@flokli.de> Autosubmit: sinavir <tvix@sinavir.fr> Tested-by: BuildkiteCI
2024-09-25 r/8715 feat(tvix/store): seekable nar rendererYureka4-0/+535
Co-authored-by: edef <edef@edef.eu> Change-Id: I233206e8aae35504ca0519ac88178dfc5596bedb Reviewed-on: https://cl.tvl.fyi/c/depot/+/12439 Reviewed-by: flokli <flokli@flokli.de> Autosubmit: yuka <yuka@yuka.dev> Tested-by: BuildkiteCI
2024-09-25 r/8714 refactor(tvix/store/tests): combine tests into one parametrized functionYureka2-179/+103
Change-Id: I9ff43b29be68b9840c58286da96fa52927691804 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12507 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-09-25 r/8713 feat(tvix/eval): Use thiserror for ErrorKind and CatchableErrorKindIlan Joselevich4-242/+100
thiserror is much more easier to maintain than manually implementing Error and Display. Change-Id: Ibf13e2d8a96fba69c8acb362b7515274a593dfd6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12452 Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com> Tested-by: BuildkiteCI
2024-09-24 r/8711 fix(tvix/nar-bridge): Remove name check for root node in nar generationsinavir4-16/+27
Nar-bridge tried to parse the name of the protobuf node encoded in the URL into a PathComponent but this name was empty, leading to an error when the user tried to retrieve the nar file. This was an oversight from the conversion to stricter types (some of the CLs in the serious containing cl/12217). We need a version converting a protobuf without a name to our stricter types, but an empty PathComponent cannot be constructed. So we need a into_name_and_node() version that returns the name as Bytes, not PathComponent. Change-Id: I2996cdd2e0107133e502748947298f512f1cc521 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12504 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-09-23 r/8710 feat(tvix/nix-compat/nar/writer/sync): add file_manual_writeYureka1-0/+68
This is useful for building other NAR writers which use custom (async or optimized) I/O to write the blob parts of the NAR. Change-Id: I447c09914fb0c99044e2fa910d4213660dc51c64 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12437 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-09-19 r/8704 fix(tvix/castore/import): check small blobs firstFlorian Klink1-0/+13
ConcurrentBlobUploader buffers small blobs in memory, and then uploads them to the BlobService in the background. In these cases, we know the hash of the whole blob, so we could check if it exists first before, uploading it. We were however not, and this caused rate limiting issues in GCS, as it has an update limit of one write per second on the same key, which we ran into especially frequently with the empty blob. This reduces the amount of writes of the same blob considerably. In the future, we might be able to drop this, as our chunked blob uploading protocol gets smarter and covers these cases. Change-Id: Icf482df815812f80a0b65cec0426f8e686308abb Reviewed-on: https://cl.tvl.fyi/c/depot/+/12497 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-09-08 r/8666 docs(tvix/nix-compat/wire/bytes/reader): None case doesn't existedef1-2/+0
We always read the length before returning a BytesReader, so len() cannot be called before the length is read. Change-Id: Ifa1cbc2c0923dd24d59e0af5e135ab8ed7314aa9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12453 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-09-07 r/8663 feat(tvix/nix-compat/nar/writer/sync): trait objects -> genericsYureka1-13/+10
This will later be used to allow the user to access the original writer. Change-Id: I1734600c5d580d18b16727f892cd2f6335cc9459 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12442 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-09-07 r/8662 feat(tvix/castore): expose ValidatedDirectoryGraph typeYureka1-1/+1
Change-Id: Id994258e74f75df8790159ed3180ce360874d0de Reviewed-on: https://cl.tvl.fyi/c/depot/+/12438 Reviewed-by: benjaminedwardwebb <benjaminedwardwebb@gmail.com> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-09-05 r/8653 chore(3p/sources): bump channels & overlays (2024-09-01)Vincent Ambo5-11/+12
Included changes: * users/aspen: explicitly use open-source nvidia driver This now has to be specified explicitly, otherwise evaluation fails with an error. * users/aspen: nixfmt -> nixfmt-classic * users/aspen: fixes for renamed packages & options * users/tazjin: fixes for renamed packages & options * 3p/overlays: remove cbtemulator patch (merged upstream) * tvix/shell: remove unnecessary patches (merged upstream) * 3p/rust-crates: mark libgit2_sys as broken * users/Profpatsch: mark git-db as broken * 3p/overlays: pick `mypaint` from stable channel * tvix: fix comments that clippy doesn't like anymore * tvix/glue: disable a misfiring clippy lint (applying its suggestion breaks code below) Change-Id: I6d3fc027694bbe7425a2d25dc53d65467a44f3b0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12403 Tested-by: BuildkiteCI Reviewed-by: aspen <root@gws.fyi> Reviewed-by: Profpatsch <mail@profpatsch.de> Autosubmit: tazjin <tazjin@tvl.su>
2024-09-03 r/8645 docs(tvix/contributing): remove reference to b/201Florian Klink1-6/+3
This should be fixed with the switch to a "GitHub App" for authentication. Change-Id: I0bd8b4b9c9f53a754a12ce07f193d19f61f56014 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12414 Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-08-30 r/8619 fix(tvix): Follow-up fixing no-default-features and commentsIlan Joselevich3-8/+4
In https://cl.tvl.fyi/c/depot/+/12389 I accidentally deleted some comments. Also fixed some useless no-default-features with futures crate, making it match the previous feature combination. Change-Id: I72bb2cfb88719ff2f8812d90193de2bd49149cce Reviewed-on: https://cl.tvl.fyi/c/depot/+/12395 Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-08-30 r/8613 fix(tvix/utils): Add missing src filtering for nix-compat-derive[-tests]Ilan Joselevich1-1/+10
Change-Id: I2beed2cdcb5423d3594562e0011b1cb889add07f Reviewed-on: https://cl.tvl.fyi/c/depot/+/12390 Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com> Tested-by: BuildkiteCI
2024-08-30 r/8611 docs(tvix): update account creation instructionsFlorian Klink1-1/+2
This is tvix/docs, so asking to be a member of #tvl here is confusing. Being a member of #tvix-dev is sufficient, we can still mention #tvl though. Change-Id: If9fbf7d098c0a8d77a45a95e321fc0a553f4c63b Reviewed-on: https://cl.tvl.fyi/c/depot/+/12396 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: yuka <yuka@yuka.dev> Tested-by: BuildkiteCI
2024-08-29 r/8610 chore(tvix): Migrate members to inherit deps from workspaceIlan Joselevich16-375/+451
From now on we will add the dependencies and their version in the root Cargo.toml and in order to enable the dependency for a workspace member we set `workspace = true` in the member's Cargo.toml. Change-Id: I9738c1cf99810b7ace87ca712c3ea965ba846e25 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12389 Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-08-29 r/8609 chore(tvix): Bump versions of all compatible deps via cargo upgradeIlan Joselevich15-688/+1206
This updates all the dependencies and their "minimum" versions in Cargo.{lock,toml} to the latest compatible version using `cargo-edit`'s `cargo upgrade` command that will eventually be merged into `cargo update`. Change-Id: Iccb2aa4a1c84a0465222244a0bd0cafe2a82e781 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12388 Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com> Tested-by: BuildkiteCI
2024-08-28 r/8608 chore(tvix/[ca]store): bump bigtable_rsFlorian Klink6-1938/+318
This bumps bigtable_rs to https://github.com/liufuyang/bigtable_rs/pull/86, allowing us to drop our second set of prost/tonic/http/axum crates. Change-Id: I70f9150289c3e8611ebe8a7d99490e3dfd085a6e Reviewed-on: https://cl.tvl.fyi/c/depot/+/12384 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de>
2024-08-28 r/8607 chore(tvix/[ca]store): bump hyper-util to 0.1.7Florian Klink4-11/+11
Change-Id: Ia5eaf7f4614701ad4dd06114476a73e1a041d830 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12383 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-08-28 r/8606 chore(tvix): bump tonic[-build] from 0.12.1 to 0.12.2Florian Klink2-46/+47
Change-Id: If7d1ae4491d3c70f323cf872d1a70afe6238f78a Reviewed-on: https://cl.tvl.fyi/c/depot/+/12382 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
2024-08-28 r/8605 feat(tvix/eval): allow external users to construct native thunksVincent Ambo1-0/+6
This interface is the only way to construct lazy builtins outside of eval, which is actually a useful feature to have. Change-Id: I386323af20aa3134bb4f669fa66fbb21e9b05fd4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12386 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: Yury Shvedov <yury.shvedov@kaspersky.com>
2024-08-28 r/8604 refactor(tvix/eval): remove useless impl blockVincent Ambo1-8/+5
There's no need to duplicate this. Change-Id: If3d930211a1d625d6c7ef129b05034e7a915da83 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12385 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: Yury Shvedov <yury.shvedov@kaspersky.com>
2024-08-28 r/8602 fix(tvix/nix-compat-derive): Get rid of external feature flagBrian Olsen5-56/+3
The external feature flag was there because I couldn't find a way to refer to crate and nix-compat with the same name so that the generated code could be the same. In essence `use nix_compat::nix_daemon::de::NixDeserialize` is an error when used inside nix_compat crate. So my best fix was the external feature flag until I found the solution used here which also removes the flag completely. Change-Id: Ia3e89c6c350c3fb22ca87f974a39c21542aae152 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12376 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Brian Olsen <me@griff.name>
2024-08-28 r/8601 fix(tvix/nix-compat-derive): Fix doctest for Rust older than 1.80Brian Olsen1-1/+1
Exclusive range patterns were stabilized in Rust version 1.80 but Tvix still uses Rust 1.79 and so would fail this one test when you ran doctests from `mg shell //tvix:shell`. It was not caught by CI because that does not currently run doctests. The fix is just to use an inclusive pattern instead. Fixes: b/417 Change-Id: Ifea7a3b84bb8f6f8c76e277979833713bdf51f46 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12375 Autosubmit: Brian Olsen <me@griff.name> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-08-27 r/8593 fix(tvix/store): restore v1alpha reflection endpointFlorian Klink7-17/+31
tonic-reflection 0.12.x moved from the v1alpha to v1 of the reflection protocol. However, most clients, like Postman, Kreya and evans don't support that one yet. Bump tonic-reflection to 0.12.2, which re-introduces v1alpha support alongside the v1 version of it, registering both services. This fixes the example documented in tvix/store/README.md, it was previously failing as evans couldn't find the v1alpha reflection service. See https://github.com/hyperium/tonic/pull/1888 for details. Change-Id: I55438877317f82dc39face13afeb9594cda07a4e Reviewed-on: https://cl.tvl.fyi/c/depot/+/12353 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
2024-08-25 r/8587 fix(tvix/nix-compat): Feature flag code only used in asyncBrian Olsen1-2/+7
When only the wire feature was enabled two methods were unused and so would faild `cargo check --no-default-features --features wire`. This feature flags those two methods on async feature since that is the only place they are used. Change-Id: I6ec18a670e3c6e3ecee8d1417c99f1a5084e0ae7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12346 Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Brian Olsen <me@griff.name> Tested-by: BuildkiteCI
2024-08-25 r/8586 feat(tvix/nix-compat-derive): Add deriver for NixDeserializeBrian Olsen48-3/+2376
This adds a nix-compat-derive derive crate that implements a deriver for NixDeserialize implementations. This is to reduce the amount of code needed to implement deserialization for all the types used by the Nix daemon protocol. Change-Id: I484724b550e8a1d5e9adad9555d9dc1374ae95c2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12022 Autosubmit: Brian Olsen <me@griff.name> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-08-25 r/8585 feat(nix-compat): Add NixDeserialize and NixRead traitsBrian Olsen20-160/+1486
Add a trait for deserializing a type from a daemon worker connection. This adds the NixDeserialize trait which is kind of like the serde Deserialize trait in that individual types are meant to implement it and it can potentially be derived in the future. The NixDeserialize trait takes something that implements NixRead as input so that you can among other things mock the reader. Change-Id: Ibb59e3562dfc822652f7d18039f00a1c0d422997 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11990 Autosubmit: Brian Olsen <me@griff.name> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-08-24 r/8578 chore(tvix/tools/turbofetch): bump depsFlorian Klink2-34/+53
h2: - https://rustsec.org/advisories/RUSTSEC-2024-0332.html - https://rustsec.org/advisories/RUSTSEC-2024-0003.html mio: - https://rustsec.org/advisories/RUSTSEC-2024-0019.html shlex: - https://rustsec.org/advisories/RUSTSEC-2024-0006.html zerocopy: - https://rustsec.org/advisories/RUSTSEC-2024-0006.html Change-Id: I7ce3cab410b9c2b6c28474ca81c3fa5b5283f3fd Reviewed-on: https://cl.tvl.fyi/c/depot/+/12341 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-08-24 r/8577 chore(tvix/tools/crunch-v2): bump depsFlorian Klink2-20/+24
h2: - https://rustsec.org/advisories/RUSTSEC-2024-0332.html - https://rustsec.org/advisories/RUSTSEC-2024-0003.html mio: - https://rustsec.org/advisories/RUSTSEC-2024-0019.html shlex: - https://rustsec.org/advisories/RUSTSEC-2024-0006.html zerocopy: - https://rustsec.org/advisories/RUSTSEC-2024-0006.html Change-Id: Ia5f6f96378d5f7e77e178dc054f47bda8b766a30 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12340 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-08-24 r/8576 chore(tvix/tools/weave): bump mioFlorian Klink2-4/+4
https: //rustsec.org/advisories/RUSTSEC-2024-0019.html Change-Id: I03543105b35da6407a4930893b1514b379a89fbc Reviewed-on: https://cl.tvl.fyi/c/depot/+/12339 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-08-23 r/8571 feat(tvix/tools/weave): Add crate2nix-checkIlan Joselevich1-1/+3
Change-Id: I4e02f56678e6efa6e2656b2f7398b6ad92581c57 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12303 Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-08-23 r/8570 feat(tvix/tools/turbofetch): Add crate2nix-checkIlan Joselevich1-1/+3
Change-Id: Ie615d824032c85af6f8f40419bc22d7e3f38bb4c Reviewed-on: https://cl.tvl.fyi/c/depot/+/12302 Tested-by: BuildkiteCI Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com> Reviewed-by: flokli <flokli@flokli.de>
2024-08-23 r/8569 feat(tvix/tools/narinfo2parquet): Add crate2nix-checkIlan Joselevich1-1/+3
Change-Id: I22bc6e7711e5e2c0a76d4eea4dd6edaf724e4cef Reviewed-on: https://cl.tvl.fyi/c/depot/+/12301 Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-08-23 r/8568 feat(tvix/tools/crunch-v2): Add crate2nix-checkIlan Joselevich1-1/+3
Change-Id: I922caae2ec3f87a7f3a5dcc8e9d366f0e71ce900 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12300 Tested-by: BuildkiteCI Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com> Reviewed-by: flokli <flokli@flokli.de>
2024-08-23 r/8565 feat(tvix/utils): Add mkCrate2nixCheckIlan Joselevich2-56/+36
This adds a function which can be used across the monorepo to create a an extra CI step that checks whether the Cargo.nix file is up-to-date. Change-Id: Idb8298b29ddc2ca5dff1facb1b9ed86a236ee66d Reviewed-on: https://cl.tvl.fyi/c/depot/+/12227 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-08-23 r/8561 feat(tvix/nar-bridge): send content-type headersFlorian Klink3-9/+18
This prevents browsers from treating NARInfo and nix-cache-info paths as a separate "Download", but just show it in plaintext. Change-Id: If99abe20ef1d24e4fa86c055160861ca47aa81ce Reviewed-on: https://cl.tvl.fyi/c/depot/+/12267 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-08-23 r/8560 feat(nix-compat): add constants for mime typesFlorian Klink1-0/+7
This adds the MIME types Nix uses when uploading files to a S3 bucket / HTTP endpoint. It ignores them when retrieving. Change-Id: I75fa96d5e53349f0470bbe8cbba93f961fbd5f05 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12266 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-08-21 r/8548 fix(tvix/castore): u32 -> u64 in DirectoryError::SizeOverflow messageBen Webb1-1/+1
Fix a discrepancy in the error message for DirectoryError::SizeOverflow. The message indicates that the SizeOverflow error occurs when total size exceeds u32::MAX, but that's not true. All size fields within the castore's internal Directory ADT are u64, and the SizeOverflow error is only returned after a call to the checked_add implementation on u64. See tvix/castore/nodes/directory.rs +111 and tvix/castore/nodes/directory.rs +88 as of this commit. Change-Id: I74d161ea8927362e1cb601ba163489aa96fb91b1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12259 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-08-21 r/8547 feat(nix-compat/nix_http): init parse_nar[info]_strFlorian Klink13-104/+433
This moves the URL component parsing code we had in nar-bridge to nix-compat. We change the function signature to return an Option, not a Result<_, StatusCode>. This allows returning more appropriate error codes, as we can ok_or(…) at the callsite, which we now do: on an upload to an invalid path, we now return "unauthorized", while on a GET/HEAD, we return "not found". This also adds support to parse compression suffixes. While not supported in nar-bridge, other users of nix-compat might very well want to parse these paths. Also fix the error message when parsing NAR urls, it mentioned 32, not 52, which is a copypasta error from the narinfo URL parsing code. Change-Id: Id1be9a8044814b54ce68b125c52dfe933c9c4f74 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12260 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-08-20 r/8545 refactor(nix-compat/store_path): make StorePath generic on SFlorian Klink15-294/+280
Similar to how cl/12253 already did this for `Signature`, we apply the same logic to `StorePath`. `StorePathRef<'a>'` is now a `StorePath<&'a str>`, and there's less redundant code for the two different implementation. `.as_ref()` returns a `StorePathRef<'_>`, `.to_owned()` gives a `StorePath<String>` (for now). I briefly thought about only publicly exporting `StorePath<String>` as `StorePath`, but the diff is not too large and this will make it easier to gradually introduce more flexibility in which store paths to accept. Also, remove some silliness in `StorePath::from_absolute_path_full`, which now doesn't allocate anymore. Change-Id: Ife8843857a1a0a3a99177ca997649fd45b8198e6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12258 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-08-20 r/8544 refactor(nix-compat/aterm): update function names a bitFlorian Klink3-26/+26
Don't call functions bstr or str when they return BString or String, it's confusing. Rename them to `string` and `bytes`. We might be able to generalize over this being BString or Vec<u8> later. Change-Id: I8198551ed3ba1cfc479bf7e3cbbc13a426faf4c0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12257 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-08-19 r/8541 feat(tvix/nix-compat): entry locator in listing structuresRyan Lahfa2-1/+113
This implements a simple DFS locator in listing structures. It is interoperable with the Rust standard library paths, we also build our own errors to restrict path values to reasonable secure defaults, e.g. relative paths with no `..` component. Tests are added for this new feature for a positive and a negative check. In addition, a path validation test was added. The Windows-style prefix is gated on the Windows platform as UNIX does not parse `C:\\` as a `Component::Prefix(_)` but as a `Component::Normal(_)`. Change-Id: Iae2a80bebd8138e41af94aa7d09f2842c3c5a786 Signed-off-by: Ryan Lahfa <tvl@lahfa.xyz> Reviewed-on: https://cl.tvl.fyi/c/depot/+/12255 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-08-19 r/8540 feat(nix-compat/narinfo/signature): generalize name fieldFlorian Klink9-45/+117
Requiring `name` to be a `&str` means it'll get annoying to pass around `Signature`, but being able to pass them around in an owned fashion is kinda a requirement for a stronger typed `PathInfo` struct, where we want to have full ownership. Rework the `Signature` struct to become generic over the type of the `name` field. This means, it becomes possible to have owned versions of it. We don't want to impose `String` or `SmolStr` for example, but want to leave it up to the nix-compat user to decide. Provide a type alias for the existing `&str` variant (`SignatureRef`), and use it where we previously used the non-generic `Signature` one. Add some tests to ensure it's possible to *use* `Signature` with both `String` and `SmolStr` (but only pull in `smol_str` as dev dependency for the tests). Also, add some more docstrings, these were a bit sparse. Change-Id: I3f75691498c6bda9cd072d2d9dac83c4f6c57287 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12253 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-08-19 r/8539 feat(tvix/nix-compat): add a basic listing deserializerRyan Lahfa4-0/+77
.ls files are useful to seek in a NAR without parsing it entirely. The responsibility of validating the files is on the caller. Change-Id: I5d1da28b5479c38f20ca5babe60e362a2217c9ea Signed-off-by: Ryan Lahfa <tvl@lahfa.xyz> Reviewed-on: https://cl.tvl.fyi/c/depot/+/12196 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-08-19 r/8538 feat(tvix): drop usage of sparseTree in favor of lib.sourceByRegexIlan Joselevich6-41/+25
We can avoid depending on things outside //tvix by just using a similar util from nixpkgs. Change-Id: I9ea3e1f0a8a059ea10caaec173569ba9f316aec6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12247 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-08-19 r/8537 feat(tvix/boot/tests): use pkgs.nixos instead of nixosForIlan Joselevich1-4/+5
Using pkgs.nixos directly allows us to create a smaller nixos closure for the tests and also not depend on things in depot.ops which can be beneficial for extending the tvix josh workspace. Change-Id: Ic6ad2122733418114b43aa692d6e42ac1e308eeb Reviewed-on: https://cl.tvl.fyi/c/depot/+/12251 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
2024-08-19 r/8536 refactor(nix-compat/narinfo/signature): use ed25519::SignatureBytesFlorian Klink1-4/+5
It's a `[u8; SIGNATURE_LENGTH]` type alias, and conveys what we're accepting or returning a bit nicer. Change-Id: I974cd97d56d383e51417eb0f26e1431a05711922 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12252 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-08-19 r/8527 feat(tvix/boot): Use buildGoModule instead of buildGo for tvix-initIlan Joselevich1-5/+8
//tvix depending less on other parts of depot is prefered as it will help with extending the josh workspace of tvix. Change-Id: Ifcac3af1782dfd82e7543cb4c3ae57fbd186edff Reviewed-on: https://cl.tvl.fyi/c/depot/+/12250 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-08-19 r/8521 refactor(tvix/eval): remove use of imbl::OrdMapVincent Ambo8-451/+95
Removes imbl::OrdMap in favour of an Rc over the standard library's BTreeMap, which allows us to drop the imbl dependency completely. In my local tests this is actually slightly faster for `hello` and `firefox`. Change-Id: Ic9597ead4e98bf9530f290c6a94a3c5c3efd0acc Reviewed-on: https://cl.tvl.fyi/c/depot/+/12201 Reviewed-by: aspen <root@gws.fyi> Tested-by: BuildkiteCI