about summary refs log tree commit diff
path: root/tvix/store
AgeCommit message (Collapse)AuthorFilesLines
2024-04-25 r/8008 fix(tvix): typoedef1-1/+1
Change-Id: Ibe4741b8086e9da442232c14cdb337556704cef6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11514 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-04-20 r/7988 feat(tvix/store/bin): upload paths concurrently in `copy`Florian Klink1-15/+53
We can speedup uploads further by not uploading all store paths sequentially, but in parallel. We still don't respect the reference graph, so nothing changed here. Change-Id: I40edb5725fe7a15170f5b9f8f4c8bec2130ecca3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11492 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-04-20 r/7981 refactor(tvix/castore/import): make module, split off fs and errorFlorian Klink2-4/+4
Move error types and filesystem-specific functions to a separate file, and keep the fs:: namespace in public exports. Change-Id: I5e9e83ad78d9aea38553fafc293d3e4f8c31a8c1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11486 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de>
2024-04-19 r/7979 refactor(tvix/castore): generalize store ingestion streamsConnor Brewster2-4/+7
Previously the store ingestion code was coupled to `walkdir::DirEntry`s produced by the `walkdir` crate which made it impossible to reuse ingesting from other sources like tarballs or NARs. This introduces a `IngestionEntry` which carries enough information for store ingestion and a future for computing the Blake3 digest of files. This allows the producer to perform file uploads in a way that makes sense for the source, ie. the filesystem upload could concurrently upload multiple files at the same time, while the NAR ingestor will need to ingest the entire blob before yielding the next blob in the stream. In the future we can buffer small blobs and upload them concurrently, but the full blob still needs to be read from the NAR before advancing. Change-Id: I6d144063e2ba5b05e765bac1f27d41b3c8e7b283 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11462 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-04-19 r/7966 chore(tvix/store): remove test-case dependencyFlorian Klink1-1/+0
Change-Id: I62c3b93af438fd6397ded85dd4fe5acc01e0093a Reviewed-on: https://cl.tvl.fyi/c/depot/+/11465 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-04-19 r/7965 chore(tvix/store): migrate import.rs and tests/pathinfo.rs to rstestFlorian Klink5-87/+72
Also, rename the DUMMY_NAME constant in the fixtures to DUMMY_PATH, which aligns more with the ToString representation and from_bytes conversions we have on StorePath[Ref]. Change-Id: I39763c9dfa84c5d86f2fd0171b3a4d36fd72f267 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11464 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-04-19 r/7964 chore(tvix/store): migrate from test_case to rstestFlorian Klink1-43/+37
Change-Id: Id673cd9c574b00a739de78aab2ee86f9d9b1ed40 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11463 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-04-15 r/7912 feat(tvix/castore/fs): support extended attributesFlorian Klink2-0/+14
This exposes `user.tvix.castore.{blob,directory}.digest` xattr keys for files and directories: ``` ❯ getfattr -d /tmp/tvix/06jrrv6wwp0nc1m7fr5bgdw012rfzfx2-nano-7.2-info getfattr: Removing leading '/' from absolute path names user.tvix.castore.directory.digest="b3:SuYDcUM9RpWcnA40tYB1BtYpR0xw72v3ymhKDQbBfe4=" ❯ getfattr -d /tmp/tvix/156a89x10c3kaby9rgf3fi4k0p6r9wl1-etc-shells getfattr: Removing leading '/' from absolute path names user.tvix.castore.blob.digest="b3:pZkwZoHN+/VQ8wkaX0wYVXZ0tV/HhtKlSqiaWDK7uRs=" ``` It's currently mostly used for debugging, though it might be useful for tvix-castore-aware syncing programs using the filesystem too. Change-Id: I26ac3cb9fe51ffbf7f880519f26741549cb5ab6a Reviewed-on: https://cl.tvl.fyi/c/depot/+/11422 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Reviewed-by: Brian Olsen <me@griff.name>
2024-04-13 r/7906 feat(tvix/*store/bigtable): limit retries connecting to cbtemulatorFlorian Klink1-1/+3
This kept retrying indefinitely if the socket didn't appear. Change-Id: I4d4ef61df73cef6abda698501432f370abc8a82c Reviewed-on: https://cl.tvl.fyi/c/depot/+/11406 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-04-13 r/7900 feat(tvix/store/bin): add "copy" commandFlorian Klink2-0/+94
This allows copying individual store paths from the local /nix/store into tvix-store. As we don't support getting this information from Nix yet, we currently expect metadata to be provided externally: Nix' `exportReferencesGraph` feature contains pretty much all data we need for this. Expect a list of this information at a well-known key (`closure`, similar to nixpkgs' `pkgs/build-support/binary-cache/ default.nix`). We currently simply upload all store paths sequentially, without any parallelism or awareness in how the reference graph looks like. As long as the connected stores don't enforce this, this is fine, at least for now. Change-Id: Ib83c998465adddfdb110db994843c44e26b3d3d8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11397 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-04-13 r/7898 refactor(tvix/nix-compat): move nar writer to tokioFlorian Klink1-5/+3
There's little reason to keep the nar writer using Async{Read,Write} traits from futures, while everything else async in tvix (and nix-compat) uses tokio. Change-Id: I8cd1efcd0dd5bb76471de997603c7b701a5095de Reviewed-on: https://cl.tvl.fyi/c/depot/+/11391 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz> Reviewed-by: Brian Olsen <me@griff.name>
2024-04-13 r/7892 docs(tvix/nix-compat): rename as_narinfo() to to_narinfo()Florian Klink2-5/+13
This actually does a bit of allocation. There's two Vecs, one for references and one for signatures. We can get rid of the Vec at nar_hash. Change-Id: Ie025309b6678f83f5b961d49ff75dcfc7da145a1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11395 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-04-12 r/7887 feat(tvix/store/pathinfo/grpc): instrument functionsFlorian Klink1-0/+6
Change-Id: Idb7d7144be1917fbaf83e9fd76c5b2ebb3df98d2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11400 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-04-12 r/7885 chore(tvix): bump rstest to 0.19.0Florian Klink1-1/+1
Change-Id: Ib2f5e84fdb8be1210b3507da67d4fe84f061651e Reviewed-on: https://cl.tvl.fyi/c/depot/+/11387 Tested-by: BuildkiteCI Reviewed-by: Brian Olsen <me@griff.name> Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-04-09 r/7884 feat(tvix/store): add bigtable pathinfoservice backendFlorian Klink6-6/+471
Put behind the "cloud" backend, like in the `tvix-castore` crate. Change-Id: Ib38d198baf11ab2a4b6dc405121676147c424611 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11362 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-04-05 r/7856 feat(tvix/store/proto): avoid clone in PathInfo::validate()Florian Klink2-18/+13
Have this return a StorePathRef<'_>, rather than a StorePath, and leave it up to the caller to possibly convert it to a owned StorePath. This avoids some allocations, if we only want to validate. Change-Id: I5cf8e246fe02bd4e631f46a5cb86d3f77a728a0d Reviewed-on: https://cl.tvl.fyi/c/depot/+/11361 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-04-01 r/7840 feat(tvix/eval): implement `builtins.path`Ryan Lahfa2-29/+32
Now, it supports almost everything except `recursive = false;`, i.e. `flat`-ingestion because we have no knob exposed in the tvix store import side to do it. This has been tested to work. Change-Id: I2e9da10ceccdfbf45b43c532077ed45d6306aa98 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10597 Tested-by: BuildkiteCI Autosubmit: raitobezarius <tvl@lahfa.xyz> Reviewed-by: flokli <flokli@flokli.de>
2024-04-01 r/7839 refactor(tvix/store): generalize `PathInfo` constructorsRyan Lahfa1-9/+32
Instead of enforcing NAR SHA256 all the time, we generalize the `PathInfo` constructor to take a `CAHash` argument which can drive whether we are having a flat, NAR or text scheme. With this, it is now possible to implement flat schemes in our evaluation builtins, e.g. `builtins.path`. Change-Id: I15bfee0ef4f0f428bfbd2f30c57c012cdcf6a976 Signed-off-by: Ryan Lahfa <tvl@lahfa.xyz> Reviewed-on: https://cl.tvl.fyi/c/depot/+/11286 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-03-28 r/7794 refactor(tvix/store): drop proto/tests/{utils,grpc_pathinfoservice}.rsFlorian Klink4-93/+0
As for grpc_pathinfoservice, we test this in the rstest-based tests, with more implementations. tests/utils.rs is unused now. Change-Id: I62b192170d181e4fb432bf7dfff2057cb048b52c Reviewed-on: https://cl.tvl.fyi/c/depot/+/11279 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-03-28 r/7793 refactor(tvix/store/pathinfo/from_addr): stop using gen_*_serviceFlorian Klink1-7/+10
Remove usage of the gen_{blob,directory}_service() helper functions from utils. We populate Memory{Blob,Directory}Services here directly, as test_case and rstest doesn't compose well. Change-Id: I0fb48aadb8c818f508b18ceb83c85eb91359442a Reviewed-on: https://cl.tvl.fyi/c/depot/+/11278 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-03-28 r/7792 refactor(tvix/pathinfo/grpc): use rstest fixturesFlorian Klink1-6/+12
Change-Id: Ib114a4d141ca829520aed34600678d613994f875 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11277 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-03-28 r/7791 refactor(tvix/store/tests/nar_renderer): use rstest fixturesFlorian Klink1-70/+67
Change-Id: I975729cb97d69d080fb63fbb8f81b4ac46cbfb7f Reviewed-on: https://cl.tvl.fyi/c/depot/+/11276 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-03-28 r/7790 refactor(tvix/store/nar/renderer): don't require AsRefFlorian Klink1-8/+6
Previous CLs ensured BlobService and DirectoryService is implemented on AsRef<dyn BlobService> and AsRef<dyn DirectoryService> respectively, so there's no need to require AsRef in here. Change-Id: I1e8b93188ca784e1b9a592f3d35fcf6b9df75c75 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11275 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-03-28 r/7789 refactor(tvix/store/nar/import): use rstest fixtures from castoreFlorian Klink1-14/+18
This stops using the gen_{blob,directory}_service() functions from tvix_castore::utils in favor of the rstest-based fixtures. Change-Id: I4c2fff2926f34af63d3bd5c29c8e939dbdf7e062 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11274 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-03-28 r/7788 feat(tvix/store): add rstest-based PathInfoService testsFlorian Klink6-2/+204
This introduces rstest-based tests. We also add fixtures for creating some BlobService / DirectoryService out of thin air. To test a PathInfoService, we don't really care too much about its internal storage - ensuring they work is up to the castore tests. Change-Id: Ia62af076ef9c9fbfcf8b020a781454ad299d972e Reviewed-on: https://cl.tvl.fyi/c/depot/+/11272 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-03-28 r/7787 refactor(tvix/store/tests/fixtures): use [u8; 20]Florian Klink4-10/+9
This makes the fixture more use-able when interacting with the trait, the Bytes are only useful for the gRPC version. Change-Id: Iaaea1adc6df18491f236a28c4343f5b4ee5fcfd3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11271 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-03-27 r/7786 feat(tvix/store): AsRef<dyn PathInfoService> impl PathInfoServiceFlorian Klink1-0/+25
Change-Id: I7fc06ae97a50d04b8c36292b3457c112242a7a70 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11270 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-03-20 r/7749 refactor(tvix/castore): introduce "cloud" feature flagFlorian Klink1-1/+2
This controls whether tvix-castore has support for various cloud backends or not. Use this to control the set of feature flags for the object_store backend, and only enable the aws, azure and gcp ones if it's set. In the future this can be used to enable/disable other cloud backends too. Without feature flags, `object_store` already supports the `InMemory` and `LocalFilesystem` backends, and we also want to unconditionally enable the `http` one. Make sure at least the construction of these services is covered in the tests. Similarly, the tvix-store crate, which provides the tvix-store CLI has a `cloud` feature flag too (defaulting to enabled). Change-Id: I9fb9c87b740e7dc83f8ff7a0862905d036d513f2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11204 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-03-20 r/7747 refactor(tvix/store/pathinfo/from_addr): use match guardsFlorian Klink1-59/+62
This will allow feature-flagging some of the backends. Change-Id: Ie92914c3e2ad870eee87e73b3b5abe605fb56fe7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11202 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-03-18 r/7733 refactor(tvix/store/import): use B3Digest in log_nodeFlorian Klink1-5/+3
Change-Id: I2347bbae8e7d4e19eeed4a3fb13729d0a94feedd Reviewed-on: https://cl.tvl.fyi/c/depot/+/11195 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-03-18 r/7732 refactor(tvix/store/pathinfo/nix_http): NixHash to display NARHashFlorian Klink1-3/+5
This will give us the sha256: prefix, and hashes we're more used to, in that context. Change-Id: I72e42fe685e365ba9baa7cd81001387d239fa7c8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11194 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-03-18 r/7731 refactor(tvix/store/nar): use B3Digest in RenderErrorFlorian Klink1-5/+4
Ensure consistent formatting of blake3 digests. It looks like we don't actually construct these error types anywhere, so no code needs to be refactored to it currently, but still good to be consistent. Change-Id: I49dc8a7f3cb4245ac06b9a6a44b72060434a3d32 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11193 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-03-18 r/7723 feat(tvix/store): support RUST_LOG env varFlorian Klink2-6/+24
This allows selectively increasing the log level for only parts of the stack. For example, the following RUST_LOG env var enables "tracing" level logging for `tvix_store` and `tvix_castore`, while keeping it at "info" for the rest of the stack: export RUST_LOG='info,tvix_store=trace,tvix_castore=trace' It only affects logs, not traces (if enabled). Change-Id: Ib936bd132a405f216e75c843db83fbd71d20a18a Reviewed-on: https://cl.tvl.fyi/c/depot/+/11182 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-03-16 r/7705 chore(tvix): bump tonic to 0.11.0Florian Klink1-4/+4
This bumps tonic and surrounding crates to 0.11.x. We added support for tonic 0.11.x into tokio-listener (https://github.com/vi/tokio-listener/pull/4), so that's bumped as well. Change-Id: Icfade5894403228299836fefb21b2f9ae59dbebb Reviewed-on: https://cl.tvl.fyi/c/depot/+/11156 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-03-16 r/7701 fix(tvix): don't emit rerun-if-changedFlorian Klink1-2/+2
`build.rs` emits rerun-if-changed statements for all proto files, as well as all include paths we pass it. Unfortunately, due to protobufs include path rules, we need to specify the path to the depot root itself as an include path, at least when building impurely with `cargo`. This causes cargo to essentially always rebuild, as it also puts its own temporary files in there. Unfortunately, tonic-build does not chase down to individual .proto files that are included. Disable emitting these `rerun-if-changed` statements for now. This could cause cargo to not rebuild protos every time, causing stale data until the next local `cargo clean`, but considering the protos change not that frequently, and it'll immediately surface if trying to build via Nix (either locally or in CI), it's a good-enough compromise. Change-Id: Ifd279a2216222ef3fc0e70c5a2fe6f87997f562e Reviewed-on: https://cl.tvl.fyi/c/depot/+/11157 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2024-03-15 r/7700 docs(tvix): fix some docstringsFlorian Klink1-1/+1
Change-Id: Ife599387d0472cd746b992bd6755a2fb6a0e0dc4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11158 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-03-11 r/7683 feat(tvix/store/bin): allow disabling OTLP at runtimeFlorian Klink1-32/+39
This was only possible by disabling without the otlp feature flag so far. Introduce the same --otlp=false mechanism that nar-bridge also supports to be able to turn it off at runtime. Change-Id: Ib22a364c35056ca9d8e327c0e2a79970a4cf4b2b Reviewed-on: https://cl.tvl.fyi/c/depot/+/11135 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-03-09 r/7661 fix(tvix/store/grpc/pathinfo): skip_all fields, handle errorsFlorian Klink1-15/+20
request only contains the outer metadata wrapping, and that's not too interesting: > Request { metadata: MetadataMap { headers: {"content-type": > "application/grpc", "user-agent": "grpc-go/1.60.1", "te": "trailers", > "grpc-accept-encoding": "gzip"} }, message: Streaming, extensions: > Extensions } Drop these fields for now, and rely on the underlying implementations to add instrumentation for the application-specific fields. Also, ensure we handle all error cases properly, and log them. We don't use `err` from instrument, as that'd also log an error on `Status::not_found`. Change-Id: Id1b983cb8b059c148c8a376f8802a1d28c59ba97 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11103 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-03-03 r/7649 refactor(tvix/store/protos): more granular nix-compat conversionsFlorian Klink1-17/+18
implement From<&nix_compat::nixhash::CAHash> for nar_info::ca::Hash individually, and make From<&nix_compat::nixhash::CAHash> for nar_info::Ca a small wrapper that uses it, as well as the .hash().digest_as_bytes() for the digest. Change-Id: I7e9b6edd1e3f149eb270faf2928cd846d74e77ad Reviewed-on: https://cl.tvl.fyi/c/depot/+/11071 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Brian Olsen <me@griff.name> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-03-03 r/7648 refactor(tvix/*/from_addr): improve test debuggabilityFlorian Klink1-2/+6
If there's an unexpected test failure, print it out, rather than just saying something is false even though it should be true. Use .expect() for this, which displays the error if it failed. We can't use expect_err(), as our stores are not display'able, so use an assertion with a message there. Change-Id: I2d88861d979d107edc0717fbdb3cdac9a6bfc5e4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11091 Tested-by: BuildkiteCI Reviewed-by: Brian Olsen <me@griff.name> Reviewed-by: flokli <flokli@flokli.de>
2024-03-03 r/7642 feat(tvix/store): use reqwests' rustls-native-roots featureFlorian Klink2-1/+5
This makes reqwest honor `SSL_CERT_FILE` - previously it was using the chain bundled in webpki-roots. `object_store` pulls in `reqwest` with this feature, and the cargo solver will enable that feature globally as soon as we pull it in, as it assumes features are additive. This requires setting `SSL_CERT_FILE` when running tests, otherwise they'll fail with the unhelpful "NotFound" error. This was quite some fun to debug, why adding `object_store` to tvix-castore suddenly made tvix-store tests fail! Change-Id: I64fc82b4d994715480efdb1ffecb279716456ab9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11090 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-02-20 r/7566 refactor(tvix/store/proto): use bstr::ByteSliceFlorian Klink2-1/+3
String::from_utf8_lossy simply discards invalid bytes, while bstr replaces them with their replacement character. Change-Id: Ib78ff36ca5faacc1ad60bc4ddde7b62773848c07 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10981 Tested-by: BuildkiteCI Reviewed-by: Peter Kolloch <info@eigenvalue.net> Reviewed-by: flokli <flokli@flokli.de>
2024-02-20 r/7564 fix(tvix/store): Improve error message for InvalidNodeNameAspen Smith1-1/+1
Rather than debug-formatting a Vec<u8>, which gives just a sequence of numbers (not very helpful!) use String::from_utf8_lossy to give a much more human-readable error message Change-Id: I6cae795ebe2ab9a656c087b82a143411e6c7c81a Reviewed-on: https://cl.tvl.fyi/c/depot/+/10980 Autosubmit: aspen <root@gws.fyi> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-02-17 r/7535 feat(tvix/castore/fs): make allow_other configurableFlorian Klink1-1/+8
Also add a cli argument to the `tvix-store` binary. Change-Id: Id07d7fedb60d6060543b195f3a810a46137f9ad5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10945 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su>
2024-02-05 r/7475 docs(tvix/store/pathinfosvc): document nix+http[s]Florian Klink1-0/+5
Change-Id: I5f4e93aceec373eb3377e3f8fbb55a5142cc1363 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10740 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-01-21 r/7435 chore(tvix/store): Use BoxStream type aliasConnor Brewster8-43/+27
The BoxStream type alias is a more concise and easier to read than the full `Pin<Box<dyn Stream<Item = ...> + Send + ...>>` type. Change-Id: I5b7bccfd066ded5557e01f7895f4cf5c4a33bd44 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10677 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: Connor Brewster <cbrewster@hey.com>
2024-01-20 r/7433 feat(tvix/store): enable `name` customization in the storeRyan Lahfa2-15/+18
Sometimes, Nix lets someone customize the `name` in the store for a path, this is the case for `builtins.path` which takes a `name` argument, we leave it to the caller to choose the name, which can be the basename by default of the path. Change-Id: Icdbf71d1d8f2dca5716b99d20aac885aab905b80 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10653 Tested-by: BuildkiteCI Autosubmit: raitobezarius <tvl@lahfa.xyz> Reviewed-by: flokli <flokli@flokli.de>
2024-01-20 r/7432 refactor(tvix/store): `import_path` → `import_path_as_nar_ca`Ryan Lahfa4-117/+160
Add multiple additional helpers such as: - `path_to_name`: derive the basename of a given path - `derive_nar_ca_path_info`: derive the `PathInfo` for a content addressed NAR which isolates further the tree walking feature and the ingestion feature. Additionally, we don't `expect` anymore and propagate properly ingestion errors up. Change-Id: I60edb5b633911c58ade7e19f5002e6f75f90e262 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10574 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: raitobezarius <tvl@lahfa.xyz>
2024-01-12 r/7371 refactor(tvix/store/pathinfoservice): make more genericFlorian Klink6-54/+51
We don't need Arcs in most of the cases, we're fine with some container. Change-Id: Ic4f8acb5b9d93e2b0923bb607463fb91e9d0e4fe Reviewed-on: https://cl.tvl.fyi/c/depot/+/10606 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-12 r/7370 refactor(tvix/store/nar/renderer): don't require Arc, Clone or SyncFlorian Klink1-28/+45
To render NARs, we're fine with a simple AsRef to a BlobService and DirectoryService. We just need to have the function pass back the references, so we can reuse it after the recursion. Change-Id: I8a1b899134ddda26cf14aa829a08383986101850 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10605 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-11-16T16·03+0000