about summary refs log tree commit diff
path: root/tvix/castore
AgeCommit message (Collapse)AuthorFilesLines
2024-03-17 r/7719 feat(tvix/castore): impl Debug for B3DigestFlorian Klink1-1/+7
Use the same format as Display, b3: followed by the base64 representation. This makes the debug implementation of everything containing a b3 digest much nicer to read. Change-Id: I3ca3154d0b6fb07781c8f9c83ece3ff1a6957902 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11181 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-03-16 r/7705 chore(tvix): bump tonic to 0.11.0Florian Klink1-3/+3
This bumps tonic and surrounding crates to 0.11.x. We added support for tonic 0.11.x into tokio-listener (https://github.com/vi/tokio-listener/pull/4), so that's bumped as well. Change-Id: Icfade5894403228299836fefb21b2f9ae59dbebb Reviewed-on: https://cl.tvl.fyi/c/depot/+/11156 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-03-16 r/7701 fix(tvix): don't emit rerun-if-changedFlorian Klink1-2/+2
`build.rs` emits rerun-if-changed statements for all proto files, as well as all include paths we pass it. Unfortunately, due to protobufs include path rules, we need to specify the path to the depot root itself as an include path, at least when building impurely with `cargo`. This causes cargo to essentially always rebuild, as it also puts its own temporary files in there. Unfortunately, tonic-build does not chase down to individual .proto files that are included. Disable emitting these `rerun-if-changed` statements for now. This could cause cargo to not rebuild protos every time, causing stale data until the next local `cargo clean`, but considering the protos change not that frequently, and it'll immediately surface if trying to build via Nix (either locally or in CI), it's a good-enough compromise. Change-Id: Ifd279a2216222ef3fc0e70c5a2fe6f87997f562e Reviewed-on: https://cl.tvl.fyi/c/depot/+/11157 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2024-03-11 r/7685 feat(tvix/castore/blobsvc/from_addr): support object_storeFlorian Klink1-1/+21
The object_store crate supports a ton of different stores, with different schemes. For now, use a objectstore+ scheme prefix to enable these. Change-Id: I946f76e32a0fb0867ef59060217894cda5b959b9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11080 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de>
2024-03-11 r/7684 feat(tvix/castore/blobsvc): add object storage implementationFlorian Klink4-0/+564
This uses the `object_store` crate to expose a tvix-castore BlobService backed by object storage. It's using FastCDC to chunk blobs into smaller chunks when writing to it. These are exposed at the .chunks() method. Change-Id: I2858c403d4d6490cdca73ebef03c26290b2b3c8e Reviewed-on: https://cl.tvl.fyi/c/depot/+/11076 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI Reviewed-by: Brian Olsen <me@griff.name>
2024-03-09 r/7660 fix(tvix/castore/grpc/directory): skip_all fields in instrumentFlorian Klink1-15/+21
This only contains the outer metadata wrapping, and that's not too interesting: > Request { metadata: MetadataMap { headers: {"content-type": > "application/grpc", "user-agent": "grpc-go/1.60.1", "te": "trailers", > "grpc-accept-encoding": "gzip"} }, message: Streaming, extensions: > Extensions } Drop these fields for now, and rely on the underlying implementations to add instrumentation for the application-specific fields. Clean up the error logging a bit. Change-Id: Ife1090ed411766a61e1fa60fd4c9570f38de1e98 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11102 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-03-09 r/7659 fix(tvix/castore/grpc/blob): skip_all fields in instrumentFlorian Klink1-5/+11
This only contains the outer metadata wrapping, and that's not too interesting: > Request { metadata: MetadataMap { headers: {"content-type": > "application/grpc", "user-agent": "grpc-go/1.60.1", "te": "trailers", > "grpc-accept-encoding": "gzip"} }, message: Streaming, extensions: > Extensions } Drop these fields for now, and rely on the underlying implementations to add instrumentation for the application-specific fields. Log errors in some places where we didn't so far. Change-Id: Ia68d6c526987d3716be62a0809195401cf28512b Reviewed-on: https://cl.tvl.fyi/c/depot/+/11101 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-03-03 r/7650 fix(tvix/castore): also set SSL_CERT_FILE for tests thereFlorian Klink1-0/+3
For everything using reqwest here during test cases, we also need to set SSL_CERT_FILE. Change-Id: If8aeda65f3d75cb9ac5c9bc64e37a0cb7dffc17c Reviewed-on: https://cl.tvl.fyi/c/depot/+/11092 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-03-03 r/7648 refactor(tvix/*/from_addr): improve test debuggabilityFlorian Klink2-4/+12
If there's an unexpected test failure, print it out, rather than just saying something is false even though it should be true. Use .expect() for this, which displays the error if it failed. We can't use expect_err(), as our stores are not display'able, so use an assertion with a message there. Change-Id: I2d88861d979d107edc0717fbdb3cdac9a6bfc5e4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11091 Tested-by: BuildkiteCI Reviewed-by: Brian Olsen <me@griff.name> Reviewed-by: flokli <flokli@flokli.de>
2024-03-03 r/7641 feat(tvix/castore): add HashingReader, B3HashingReaderFlorian Klink2-0/+90
HashingReader wraps an existing AsyncRead, and allows querying for the digest of all data read "through" it. The hash function is configurable by type parameter, and we define B3HashingReader. Change-Id: Ic08142077566fc08836662218f5ec8c3aff80be5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11087 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-03-03 r/7640 feat(tvix/castore/digests): impl From digest::Output<_> for B3DigestFlorian Klink2-3/+11
This allows calling .into() to get a B3Digest. Change-Id: I6e63b496413cd00d84acfcd15c7de0f64c79721f Reviewed-on: https://cl.tvl.fyi/c/depot/+/11086 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-03-03 r/7631 refactor(tvix/castore/blobsvc/chunked_reader): refactor, documentFlorian Klink1-131/+139
The public-consumable thing here is ChunkedReader, not ChunkedBlob. ChunkedBlob is a helper that can be used to get a new AsyncRead, but not AsyncSeek. It is used internally by ChunkedReader whenever the client seeks. Make this more obvious, by extending the documentation, and putting ChunkedReader at the top of this file. Also make ChunkedBlob and its methods private, and give ChunkedReader a more useful constructor (from_chunks, instead of from_chunked_blob). Change-Id: I2399867591df923faa73927b924e7c116ad98dc0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11079 Tested-by: BuildkiteCI Reviewed-by: Brian Olsen <me@griff.name> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-03-02 r/7627 feat(tvix/castore/blobsvc): BlobReader for more trivial typesFlorian Klink1-0/+2
Change-Id: I80e4f26c41a504fa4c6a013c2a1e76de613ba294 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11078 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-03-02 r/7626 fix(tvix/castore/blobwriter): don't require Sync + 'staticFlorian Klink1-1/+1
There's no reason for these two. Change-Id: Ie6f238bbb0b17971c9877b11b61ea7ebca573c13 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11075 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-02-20 r/7570 docs(tvix/castore/directorysvc): K/V is not necessarily flatFlorian Klink2-0/+10
Some implementations of DirectoryService might not allow retrieval of intermediate Directory nodes, that are not at the "root". Think about an object store implementation. The client is doing a get_recursive anyways to reduce the number of roundtrips. By documenting the fact we don't need to support looking up intermediate Directory messages, we can just batch all directories into the same object, keyed by the root. Change-Id: I019d720186d03c4125cec9191e93d20586a20963 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10988 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2024-02-19 r/7558 feat(tvix/castore): Compile fix for DarwinPeter Kolloch1-0/+3
Towards https://b.tvl.fyi/issues/264 Change-Id: If8fa912ae3fb2987b761f649ab738529ebf3b2e8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10970 Autosubmit: Peter Kolloch <info@eigenvalue.net> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-02-18 r/7547 fix(tvix/castore): don't emit ret as INFOFlorian Klink1-1/+2
This otherwise gets a bit spammy. Change-Id: I288350a600d79a394c239f253424ad55bc3cefc5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10954 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2024-02-17 r/7535 feat(tvix/castore/fs): make allow_other configurableFlorian Klink2-3/+8
Also add a cli argument to the `tvix-store` binary. Change-Id: Id07d7fedb60d6060543b195f3a810a46137f9ad5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10945 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su>
2024-02-10 r/7494 feat(tvix/castore/blobsvc): add Chunked{Blob,Reader}Florian Klink2-0/+489
These provide seekable access into a Blob for which we have more granular chunking information. There's no support for verified streaming in here yet, this simply produces a stream of readers for each chunk, skipping irrelevant chunks and data from the first chunk at the beginning. A seek simply does produce a new reader using the same process. Change-Id: I37f76b752adce027586770475435f3990a6dee0b Reviewed-on: https://cl.tvl.fyi/c/depot/+/10731 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-02-06 r/7479 docs(tvix/castore/blobstore): reorganize docsFlorian Klink3-121/+251
docs/verified-streaming.md explained how CDC and verified streaming can work together, but didn't really highlight enough how chunking in general also helps with seeking. In addition, a lot of the thoughts w.r.t. the BlobStore protocol, both gRPC and Rust traits, as well as why there's no support for seeking directly in gRPC, as well as how clients should behave w.r.t. chunked fetching was missing, or mixed together with the verified streaming bits. While there is no verified streaming version yet, a chunked one is coming soon, and documenting this a bit better is gonna make it easier to understand, as well as provide some lookout on where this is heading. Change-Id: Ib11b8ccf2ef82f9f3a43b36103df0ad64a9b68ce Reviewed-on: https://cl.tvl.fyi/c/depot/+/10733 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-02-02 r/7471 fix(tvix/castore/grpc/svc_wrapper): expose chunks() over gRPCFlorian Klink1-3/+6
The Stat() method was just always signalling no granular chunks are available. However, as we now have a .chunks() method, we can expose it over gRPC. Change-Id: I74f0890ae083f301bb0cec62f1ea4a95463ac590 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10736 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-02-02 r/7470 feat(tvix/castore/blobsvc): validate StatBlobResponseFlorian Klink2-0/+29
All chunks must have valid blake3 digests. It is allowed to send an empty list, if no more granular chunking is available. Change-Id: I7ecb53579cdf40fd938bb68a85685751b4d3626f Reviewed-on: https://cl.tvl.fyi/c/depot/+/10726 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de>
2024-02-02 r/7469 refactor(tvix/castore/grpc/blobsvc): inline stream_mapperFlorian Klink1-12/+3
This can be written without the additional function. Change-Id: Ib11c5d5254d3e44c8fa9661414835b0622eb1ac4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10735 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-02-02 r/7468 docs(tvix/castore/blobsvc): fix doc comments on traitFlorian Klink1-10/+14
The readers implement AsyncRead/AsyncSeek, not their sync counterparts. Also update expectations around chunks. Change-Id: Ic266688039d80d16d33f651b96ce2bcdedecfa00 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10734 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-02-02 r/7466 feat(tvix/castore/docs/verified-streaming): clarify replyFlorian Klink1-1/+1
"given chunksize" is misleading here. It's up to the backend to decide if it does chunking at all, and how it chunks. Change-Id: I4f130ca9ac34db79f18ef1d6475295806ac7f9a4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10728 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-02-02 r/7465 refactor(tvix/castore/blobsvc/combinator): compact trait boundsFlorian Klink1-1/+2
BlobService already implies Send and Sync, we don't need to explicitly list it here. Change-Id: I58a4c5912be61a60acd961565979aa01d94ee0f7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10727 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-01-22 r/7437 feat(tvix/castore): `process_entry` cannot process unsupported nodesRyan Lahfa1-1/+13
In the past, we had a `todo!` on unsupported node types, this returns a proper error that can be caught by the caller. Change-Id: Icba4c1dab33c0d670a97f162c9b358d1ed5855cb Reviewed-on: https://cl.tvl.fyi/c/depot/+/10675 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-01-21 r/7435 chore(tvix/store): Use BoxStream type aliasConnor Brewster7-24/+17
The BoxStream type alias is a more concise and easier to read than the full `Pin<Box<dyn Stream<Item = ...> + Send + ...>>` type. Change-Id: I5b7bccfd066ded5557e01f7895f4cf5c4a33bd44 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10677 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: Connor Brewster <cbrewster@hey.com>
2024-01-20 r/7431 refactor(tvix/castore): break down `ingest_path`Ryan Lahfa1-70/+187
In one function that does the heavy lifting: `ingest_entries`, and three additional helpers: - `walk_path_for_ingestion` which perform the tree walking in a very naive way and can be replaced by the user - `leveled_entries_to_stream` which transforms a list of a list of entries ordered by their depth in the tree to a stream of entries in the bottom to top order (Merkle-compatible order I will say in the future). - `ingest_path` which calls the previous functions. Change-Id: I724b972d3c5bffc033f03363255eae448f017cef Reviewed-on: https://cl.tvl.fyi/c/depot/+/10573 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: raitobezarius <tvl@lahfa.xyz>
2024-01-20 r/7430 feat(tvix/castore): ingestion does DFS and invert itRyan Lahfa1-49/+69
To make use of the filtering feature, we need to revert the internal walker to a real DFS. We will therefore just invert the whole tree by storing all of its contents in a level-keyed vector. This is horribly expensive in memory, this is a compromise between CPU and memory, here is the fundamental reason for why: When you encounter a directory, it's either a leaf or not, i.e. it contains subdirectories or not. To know this fact, you can: - wait until you notice subdirectories under it, i.e. you need to store any intermediate nodes you see in the meantime -> memory penalty. - getdents or readdir on it to determine *NOW* its subdirectories -> CPU penalty and I/O penalty. This is an implementation of the first proposal, we pay memory. In practice, we are paying O(#nb of nodes) in memory. There's a smarter albeit much more complicated algorithm that pays only O(\sum_i #siblings(p_i)) nodes where (p_1, ..., p_n) is the path to a leaf. which means for: A / \ B C / / \ D E F We would never store D, E, F but only E, F at a given time. But we would still store B, C no matter what. Change-Id: I456ed1c3f0db493e018ba1182665d84bebe29c11 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10567 Tested-by: BuildkiteCI Autosubmit: raitobezarius <tvl@lahfa.xyz> Reviewed-by: flokli <flokli@flokli.de>
2024-01-19 r/7422 chore(3p/sources): Bump channels & overlayssterni1-3/+1
- Adjust to ecl 23.9.9 release - Regenerate go protos after protoc-gen-go update - Drop dhall fork which hasn't kept up with 1.42.* - Address new clippy warnings: - Variant naming of Error::ValidationError - Simplify .try_into().unwrap() - Drop unnecessary identity function - Test module must be last in file - Drop unused `pub use` - Update agenix to 0.15.0. Current master has a installCheckPhase that doesn't work with C++ Nix 2.3.*: https://github.com/ryantm/agenix/commit/a23aa271bec82d3e962bafb994595c1c4a62b133#commitcomment-137185861 Change-Id: Ic29eef20d6fd1362ce1031364a5ca6b4edf195bd Reviewed-on: https://cl.tvl.fyi/c/depot/+/10615 Reviewed-by: aspen <root@gws.fyi> Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org>
2024-01-18 r/7412 feat(tvix/castore): convert import error to `std::io::Error`Ryan Lahfa1-0/+6
So that we can just `map_err` easily in functions returning `std::io::Error` but calling functions returning `castore::import::Error`. Change-Id: Id181b95e8431c69e95f3a8cd569ca10306656e1d Reviewed-on: https://cl.tvl.fyi/c/depot/+/10572 Autosubmit: raitobezarius <tvl@lahfa.xyz> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-01-15 r/7380 feat(tvix/castore): implement Ord for node::NodeFlorian Klink1-0/+14
This allows assembling BTreeSets of node::Node. Change-Id: I97b83be5ffc3e891307a8ef2b5fc31e38b747a62 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10625 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de>
2024-01-09 r/7366 feat(tvix/castore): implement CombinedBlobServiceFlorian Klink2-0/+137
First attempt on composition of BlobServices. Change-Id: I6e70248007edfd322a503fd40c1c4b4300cbc30c Reviewed-on: https://cl.tvl.fyi/c/depot/+/10587 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de>
2024-01-09 r/7365 feat(tvix/castore/blobsvc): add chunks methodFlorian Klink2-2/+47
This adds support to retrieve a list of chunks for a given blob to the BlobService interface. While theoretically all chunk-awareness could be kept private inside each BlobService reader, we'd not be able to resolve individual chunks from different Blobservices - and due to this, not able to substitute chunks we already have in a more local store. This function allows asking a BlobService for the list of chunks, leaving any actual fetching up to the caller (be it through individual calls to open_read), or asking another store for it. Change-Id: I1d33c591195ed494be3aec71a8c804743cbe0dca Reviewed-on: https://cl.tvl.fyi/c/depot/+/10586 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-01-09 r/7364 refactor(tvix/castore): do clone inside a scopeFlorian Klink1-6/+5
Make it clear this is only used inside the scope. Change-Id: Ie94f88d7f0fb58cd4bf9c2f1176000b272e6f2e6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10585 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-09 r/7363 feat(tvix/castore/grpc): instrument some more functionsFlorian Klink1-0/+2
Change-Id: Icedb148c88c5f4a3b2242ed12df1dd8692af94fd Reviewed-on: https://cl.tvl.fyi/c/depot/+/10584 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-09 r/7361 refactor(tvix/castore/directorysvc): AsRef traverse_toFlorian Klink1-14/+12
Change-Id: I641bd4ab3de591a013f03137f1e16295946315f3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10579 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-01-09 r/7360 refactor(tvix/castore/blobservice/grpc): remove fn pointer hackFlorian Klink1-15/+8
It looks like the workaround isn't necessary anymore. Change-Id: Ifbcef1d631b3f369cac3db25a2c793480043f697 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10583 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-09 r/7359 refactor(tvix): use AsRef<dyn …> instead of Deref<Target= …>Florian Klink5-36/+29
Removes some more needs for Arcs. Change-Id: I9a9f4b81641c271de260e9ffa98313a32944d760 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10578 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-09 r/7358 feat(tvix/castore/blobsvc/grpc): rm VecDec, fix docstringFlorian Klink1-21/+9
The docstrings were not updated once we made the BlobService trait async. There's no more need to turn things into a sync reader. Also, rearrange the stream manipulation a bit, and remove the need to create a new VecDeque for each element in the stream. bytes::Bytes implements the Buf trait. Fixes b/289. Change-Id: Id2bbedca5876b462e630c144b74cc289c3916c4d Reviewed-on: https://cl.tvl.fyi/c/depot/+/10582 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-06 r/7354 chore(tvix/castore): fix the docstring for `process_entry`Ryan Lahfa1-15/+15
It was a `//` not a `///`. Change-Id: Iee3e8c116d73b5dd8a41c027153714415a66695f Reviewed-on: https://cl.tvl.fyi/c/depot/+/10566 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-01-05 r/7350 chore(tvix): bump test-case dep to 3.3.1Florian Klink1-1/+1
Change-Id: I643548d95a5fab84563c7cbe51ca2ce640c186a9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10537 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-01-05 r/7347 fix(tvix/castore): validate Option<Node>Florian Klink1-1/+15
Extend our validation function to also check for the None case. Change-Id: Ib75f880646d7fb3d66588f1988e61ec18be816a2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10534 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-01-05 r/7346 refactor(tvix/castore): relax trait bounds for DSFlorian Klink2-5/+6
Make this an `AsRef<dyn DirectoryService>`. This helps dropping some Clone requirements. Unfortunately, we can't thread this through to TvixStoreIO just yet. Change-Id: I3f07eb28d6c793d3313fe21506ada84d5a8aa3ac Reviewed-on: https://cl.tvl.fyi/c/depot/+/10533 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-01 r/7307 refactor(tvix/castore/tests): let gen_*_service return BoxesFlorian Klink3-15/+22
Only convert to and reuse an Arc<…> where needed. Change-Id: I2c1bc69cca5a4a3ebd3bdb33d6e28e1f5fb86cb9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10514 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-01 r/7305 refactor(tvix/castore/blobsvc/grpc/wrapper): don't require Arc<_>Florian Klink3-14/+13
Change-Id: I9655f5588c7dc98427de6af47d74b4ab7ce22071 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10516 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-01 r/7304 refactor(tvix/castore/directorysvc/grpc/wrapper): no Arc<_>Florian Klink3-54/+48
We can also drop the Clone requirement. Because the trait is async since some time, there's no need to clone before moving into an async closure, allowing us to simplify the code a bit. Change-Id: I9b0a0e10077d8c548d218207b908bfd92c5b8de0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10515 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de>
2024-01-01 r/7303 refactor(tvix/castore): make directorysvc more genericFlorian Klink1-4/+7
This works on Box<dyn DirectoryService> too. Change-Id: Ib869f0f4d963ef4dbaeab22db03ff6afb71ede04 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10513 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-01-01 r/7300 feat(tvix/castore/import): generalize ingest_pathFlorian Klink1-8/+16
We don't actually care if it's an Arc<dyn BlobService>, or something else, as long as we can Deref to a BlobService and clone. Change-Id: I0852aaf723f51c5e6b820be8db1199d17309ab08 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10510 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-11-07T03·53+0000