about summary refs log tree commit diff
path: root/tvix/castore/src
AgeCommit message (Collapse)AuthorFilesLines
2024-02-02 r/7471 fix(tvix/castore/grpc/svc_wrapper): expose chunks() over gRPCFlorian Klink1-3/+6
The Stat() method was just always signalling no granular chunks are available. However, as we now have a .chunks() method, we can expose it over gRPC. Change-Id: I74f0890ae083f301bb0cec62f1ea4a95463ac590 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10736 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-02-02 r/7470 feat(tvix/castore/blobsvc): validate StatBlobResponseFlorian Klink2-0/+29
All chunks must have valid blake3 digests. It is allowed to send an empty list, if no more granular chunking is available. Change-Id: I7ecb53579cdf40fd938bb68a85685751b4d3626f Reviewed-on: https://cl.tvl.fyi/c/depot/+/10726 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de>
2024-02-02 r/7469 refactor(tvix/castore/grpc/blobsvc): inline stream_mapperFlorian Klink1-12/+3
This can be written without the additional function. Change-Id: Ib11c5d5254d3e44c8fa9661414835b0622eb1ac4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10735 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-02-02 r/7468 docs(tvix/castore/blobsvc): fix doc comments on traitFlorian Klink1-10/+14
The readers implement AsyncRead/AsyncSeek, not their sync counterparts. Also update expectations around chunks. Change-Id: Ic266688039d80d16d33f651b96ce2bcdedecfa00 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10734 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-02-02 r/7465 refactor(tvix/castore/blobsvc/combinator): compact trait boundsFlorian Klink1-1/+2
BlobService already implies Send and Sync, we don't need to explicitly list it here. Change-Id: I58a4c5912be61a60acd961565979aa01d94ee0f7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10727 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-01-22 r/7437 feat(tvix/castore): `process_entry` cannot process unsupported nodesRyan Lahfa1-1/+13
In the past, we had a `todo!` on unsupported node types, this returns a proper error that can be caught by the caller. Change-Id: Icba4c1dab33c0d670a97f162c9b358d1ed5855cb Reviewed-on: https://cl.tvl.fyi/c/depot/+/10675 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-01-21 r/7435 chore(tvix/store): Use BoxStream type aliasConnor Brewster7-24/+17
The BoxStream type alias is a more concise and easier to read than the full `Pin<Box<dyn Stream<Item = ...> + Send + ...>>` type. Change-Id: I5b7bccfd066ded5557e01f7895f4cf5c4a33bd44 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10677 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: Connor Brewster <cbrewster@hey.com>
2024-01-20 r/7431 refactor(tvix/castore): break down `ingest_path`Ryan Lahfa1-70/+187
In one function that does the heavy lifting: `ingest_entries`, and three additional helpers: - `walk_path_for_ingestion` which perform the tree walking in a very naive way and can be replaced by the user - `leveled_entries_to_stream` which transforms a list of a list of entries ordered by their depth in the tree to a stream of entries in the bottom to top order (Merkle-compatible order I will say in the future). - `ingest_path` which calls the previous functions. Change-Id: I724b972d3c5bffc033f03363255eae448f017cef Reviewed-on: https://cl.tvl.fyi/c/depot/+/10573 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: raitobezarius <tvl@lahfa.xyz>
2024-01-20 r/7430 feat(tvix/castore): ingestion does DFS and invert itRyan Lahfa1-49/+69
To make use of the filtering feature, we need to revert the internal walker to a real DFS. We will therefore just invert the whole tree by storing all of its contents in a level-keyed vector. This is horribly expensive in memory, this is a compromise between CPU and memory, here is the fundamental reason for why: When you encounter a directory, it's either a leaf or not, i.e. it contains subdirectories or not. To know this fact, you can: - wait until you notice subdirectories under it, i.e. you need to store any intermediate nodes you see in the meantime -> memory penalty. - getdents or readdir on it to determine *NOW* its subdirectories -> CPU penalty and I/O penalty. This is an implementation of the first proposal, we pay memory. In practice, we are paying O(#nb of nodes) in memory. There's a smarter albeit much more complicated algorithm that pays only O(\sum_i #siblings(p_i)) nodes where (p_1, ..., p_n) is the path to a leaf. which means for: A / \ B C / / \ D E F We would never store D, E, F but only E, F at a given time. But we would still store B, C no matter what. Change-Id: I456ed1c3f0db493e018ba1182665d84bebe29c11 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10567 Tested-by: BuildkiteCI Autosubmit: raitobezarius <tvl@lahfa.xyz> Reviewed-by: flokli <flokli@flokli.de>
2024-01-19 r/7422 chore(3p/sources): Bump channels & overlayssterni1-3/+1
- Adjust to ecl 23.9.9 release - Regenerate go protos after protoc-gen-go update - Drop dhall fork which hasn't kept up with 1.42.* - Address new clippy warnings: - Variant naming of Error::ValidationError - Simplify .try_into().unwrap() - Drop unnecessary identity function - Test module must be last in file - Drop unused `pub use` - Update agenix to 0.15.0. Current master has a installCheckPhase that doesn't work with C++ Nix 2.3.*: https://github.com/ryantm/agenix/commit/a23aa271bec82d3e962bafb994595c1c4a62b133#commitcomment-137185861 Change-Id: Ic29eef20d6fd1362ce1031364a5ca6b4edf195bd Reviewed-on: https://cl.tvl.fyi/c/depot/+/10615 Reviewed-by: aspen <root@gws.fyi> Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org>
2024-01-18 r/7412 feat(tvix/castore): convert import error to `std::io::Error`Ryan Lahfa1-0/+6
So that we can just `map_err` easily in functions returning `std::io::Error` but calling functions returning `castore::import::Error`. Change-Id: Id181b95e8431c69e95f3a8cd569ca10306656e1d Reviewed-on: https://cl.tvl.fyi/c/depot/+/10572 Autosubmit: raitobezarius <tvl@lahfa.xyz> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-01-15 r/7380 feat(tvix/castore): implement Ord for node::NodeFlorian Klink1-0/+14
This allows assembling BTreeSets of node::Node. Change-Id: I97b83be5ffc3e891307a8ef2b5fc31e38b747a62 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10625 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de>
2024-01-09 r/7366 feat(tvix/castore): implement CombinedBlobServiceFlorian Klink2-0/+137
First attempt on composition of BlobServices. Change-Id: I6e70248007edfd322a503fd40c1c4b4300cbc30c Reviewed-on: https://cl.tvl.fyi/c/depot/+/10587 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de>
2024-01-09 r/7365 feat(tvix/castore/blobsvc): add chunks methodFlorian Klink2-2/+47
This adds support to retrieve a list of chunks for a given blob to the BlobService interface. While theoretically all chunk-awareness could be kept private inside each BlobService reader, we'd not be able to resolve individual chunks from different Blobservices - and due to this, not able to substitute chunks we already have in a more local store. This function allows asking a BlobService for the list of chunks, leaving any actual fetching up to the caller (be it through individual calls to open_read), or asking another store for it. Change-Id: I1d33c591195ed494be3aec71a8c804743cbe0dca Reviewed-on: https://cl.tvl.fyi/c/depot/+/10586 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-01-09 r/7364 refactor(tvix/castore): do clone inside a scopeFlorian Klink1-6/+5
Make it clear this is only used inside the scope. Change-Id: Ie94f88d7f0fb58cd4bf9c2f1176000b272e6f2e6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10585 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-09 r/7363 feat(tvix/castore/grpc): instrument some more functionsFlorian Klink1-0/+2
Change-Id: Icedb148c88c5f4a3b2242ed12df1dd8692af94fd Reviewed-on: https://cl.tvl.fyi/c/depot/+/10584 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-09 r/7361 refactor(tvix/castore/directorysvc): AsRef traverse_toFlorian Klink1-14/+12
Change-Id: I641bd4ab3de591a013f03137f1e16295946315f3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10579 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-01-09 r/7360 refactor(tvix/castore/blobservice/grpc): remove fn pointer hackFlorian Klink1-15/+8
It looks like the workaround isn't necessary anymore. Change-Id: Ifbcef1d631b3f369cac3db25a2c793480043f697 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10583 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-09 r/7359 refactor(tvix): use AsRef<dyn …> instead of Deref<Target= …>Florian Klink5-36/+29
Removes some more needs for Arcs. Change-Id: I9a9f4b81641c271de260e9ffa98313a32944d760 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10578 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-09 r/7358 feat(tvix/castore/blobsvc/grpc): rm VecDec, fix docstringFlorian Klink1-21/+9
The docstrings were not updated once we made the BlobService trait async. There's no more need to turn things into a sync reader. Also, rearrange the stream manipulation a bit, and remove the need to create a new VecDeque for each element in the stream. bytes::Bytes implements the Buf trait. Fixes b/289. Change-Id: Id2bbedca5876b462e630c144b74cc289c3916c4d Reviewed-on: https://cl.tvl.fyi/c/depot/+/10582 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-06 r/7354 chore(tvix/castore): fix the docstring for `process_entry`Ryan Lahfa1-15/+15
It was a `//` not a `///`. Change-Id: Iee3e8c116d73b5dd8a41c027153714415a66695f Reviewed-on: https://cl.tvl.fyi/c/depot/+/10566 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-01-05 r/7347 fix(tvix/castore): validate Option<Node>Florian Klink1-1/+15
Extend our validation function to also check for the None case. Change-Id: Ib75f880646d7fb3d66588f1988e61ec18be816a2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10534 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-01-05 r/7346 refactor(tvix/castore): relax trait bounds for DSFlorian Klink2-5/+6
Make this an `AsRef<dyn DirectoryService>`. This helps dropping some Clone requirements. Unfortunately, we can't thread this through to TvixStoreIO just yet. Change-Id: I3f07eb28d6c793d3313fe21506ada84d5a8aa3ac Reviewed-on: https://cl.tvl.fyi/c/depot/+/10533 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-01 r/7307 refactor(tvix/castore/tests): let gen_*_service return BoxesFlorian Klink3-15/+22
Only convert to and reuse an Arc<…> where needed. Change-Id: I2c1bc69cca5a4a3ebd3bdb33d6e28e1f5fb86cb9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10514 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-01 r/7305 refactor(tvix/castore/blobsvc/grpc/wrapper): don't require Arc<_>Florian Klink3-14/+13
Change-Id: I9655f5588c7dc98427de6af47d74b4ab7ce22071 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10516 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-01 r/7304 refactor(tvix/castore/directorysvc/grpc/wrapper): no Arc<_>Florian Klink3-54/+48
We can also drop the Clone requirement. Because the trait is async since some time, there's no need to clone before moving into an async closure, allowing us to simplify the code a bit. Change-Id: I9b0a0e10077d8c548d218207b908bfd92c5b8de0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10515 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de>
2024-01-01 r/7303 refactor(tvix/castore): make directorysvc more genericFlorian Klink1-4/+7
This works on Box<dyn DirectoryService> too. Change-Id: Ib869f0f4d963ef4dbaeab22db03ff6afb71ede04 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10513 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-01-01 r/7300 feat(tvix/castore/import): generalize ingest_pathFlorian Klink1-8/+16
We don't actually care if it's an Arc<dyn BlobService>, or something else, as long as we can Deref to a BlobService and clone. Change-Id: I0852aaf723f51c5e6b820be8db1199d17309ab08 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10510 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-12-31 r/7294 refactor(tvix/castore/directorysvc): return Box, not ArcFlorian Klink1-6/+5
While we currently mostly use it in an Arc, as we need to clone it inside PathInfoService, there might be other usecases not requiring it to be Clone. Change-Id: Ia05bb370340792a048e2036be30e285ef1e63870 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10483 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-12-31 r/7293 refactor(tvix/castore/blobsvc): return Box, not ArcFlorian Klink1-7/+6
While we currently mostly use it in an Arc, as we need to clone it inside PathInfoService, there might be other usecases not requiring it to be Clone. Change-Id: I7bd337cd2e4c2d4154b385461eefa62c9b78345d Reviewed-on: https://cl.tvl.fyi/c/depot/+/10482 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-12-29 r/7270 feat(tvix/castore/fs): borrow some matchesFlorian Klink1-4/+4
We only do things with the reference, so we don't need to locally borrow it. Change-Id: I6073f7ec7aff717ae3069e28a00b1cb408a50ceb Reviewed-on: https://cl.tvl.fyi/c/depot/+/10455 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-12-29 r/7267 feat(tvix/castore/fs): instrument FuseDaemon functionsFlorian Klink1-2/+4
Change-Id: I696b7ab6b4c08004db147c0fda7312bbebaa0eec Reviewed-on: https://cl.tvl.fyi/c/depot/+/10451 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-12-24 r/7259 refactor(tvix/castore/fs/tests): drop unused argsFlorian Klink1-39/+20
There's no need to pass in an unused directory service into the populate_blob_* method, and considering we have one or two invocation of each of these, we don't really gain much from having all these functions follow the same structure, at least for now. Also, update some function names to better describe what they're doing. Change-Id: I92f680745c157fb0a602b07342f8838bfad23ecd Reviewed-on: https://cl.tvl.fyi/c/depot/+/10411 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-12-24 r/7258 refactor(tvix/castore): add RootNode impl for BTreeMap, mv fs testsFlorian Klink3-2/+1183
cl/10378 did already move store/fs to castore/fs, but we kept the tests in tvix-store, as they were populating a PathInfoService to make nodes appear in the mount root. Update these tests to now just insert root nodes into a BTreeMap, and ensure we can use that as a RootNodes too. Change-Id: Iad7d1ee4f9423eb6e3a1da33f433842c9ae0de1f Reviewed-on: https://cl.tvl.fyi/c/depot/+/10410 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-12-22 r/7256 chore(tvix): move store/fs to castore/fsFlorian Klink8-0/+1316
With the recent introduction of the RootNodes trait, there's nothing in the fs module pulling in tvix-store dependencies, so it can live in tvix-castore. This allows other crates to make use of TvixStoreFS, without having to pull in tvix-store. For example, a tvix-build using a fuse mountpoint at /nix/store doesn't need a PathInfoService to hold the root nodes that should be present, but just a list. tvix-store now has a pathinfoservice/fs module, which contains the necessary glue logic to implement the RootNodes trait for a PathInfoService. To satisfy Rust orphan rules for trait implementations, we had to add a small wrapper struct. It's mostly hidden away by the make_fs helper function returning a TvixStoreFs. It can't be entirely private, as its still leaking into the concrete type of TvixStoreFS. tvix-store still has `fuse` and `virtiofs` features, but they now simply enable these features in the `tvix-castore` crate they depend on. The tests for the fuse functionality stay in tvix-store for now, as they populate the root nodes through a PathInfoService. Once above mentioned "list of root nodes" implementation exists, we might want to shuffle this around one more time. Fixes b/341. Change-Id: I989f664827a5a361b23b34368d242d10c157c756 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10378 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2023-12-21 r/7247 docs(tvix/castore): fix referenceFlorian Klink1-3/+4
Change-Id: I00b1d56d58c4d3779b57ab0056cff1c7e6053b9b Reviewed-on: https://cl.tvl.fyi/c/depot/+/10401 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI
2023-12-17 r/7228 feat(tvix/castore): simple filesystem blob serviceRyan Lahfa3-1/+208
The simple filesystem `BlobService` enable a user to write blob store on an existing filesystem using a prefix-style layout in the provided root directory, e.g. the two first bytes of the blake3 hashes are used as directories prefixes. Change-Id: I3451a688a6f39027b9c6517d853b95a87adb3a52 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10071 Autosubmit: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-12-16 r/7227 refactor(tvix/castore): drop is_closed() from impl DirectoryPutterFlorian Klink3-20/+33
This is only used in the gRPC version (GRPCPutter), during the test automation. So define it as a method there, behind #[cfg(test)], and remove from the trait. Change-Id: Idf170884e3a10be0e96c75d946d9c431171e5e88 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10340 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-12-13 r/7216 docs(tvix/castore/directorysvc): update commentFlorian Klink1-3/+3
This comment didn't make a lot of sense before. Change-Id: Ie057a133ca4b1a099ed3c885e32316b0d87c5eb0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10339 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-12-13 r/7215 docs(tvix/castore): document expectations about DirectoryServiceFlorian Klink1-1/+7
Namely, all trait implementations should reject invalid data being fed, and detect invalid data being returned. b/355 tracks writing some more tests for this, to ensure we're compliant with this. Change-Id: I3b05752932837ce208785efb21ffc21508b4b33a Reviewed-on: https://cl.tvl.fyi/c/depot/+/10338 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi> Autosubmit: flokli <flokli@flokli.de>
2023-12-13 r/7214 docs(tvix/castore): fix docstringsFlorian Klink1-2/+2
There's been some copypasta errors. Change-Id: I8fcad6cfc951ead6c789e0dce823c798adbfcf97 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10337 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2023-12-12 r/7209 fix(tvix/castore/import): don't unwrap entryFlorian Klink1-2/+8
If the path specified doesn't exist, construct a proper error instead of panicking. Part of b/344. Change-Id: Id5c6a91248b0a387f3e8f138f8e686e402009e8f Reviewed-on: https://cl.tvl.fyi/c/depot/+/10330 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-12-12 r/7208 feat(tvix/castore/import): log returned errorsFlorian Klink1-1/+1
This will emit a log event / trace in case this function returns an error-y type. Change-Id: I48db6807f3e42304357c422a2b6e177cb8b95228 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10329 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-12-12 r/7207 refactor(tvix/castore/blobservice): use io::Result in traitFlorian Klink5-40/+39
For all these calls, the caller has enough context about what it did, so it should be fine to use io::Result here. We pretty much only constructed crate::Error::StorageError before anyways, so this conveys *more* information. Change-Id: I5cabb3769c9c2314bab926d34dda748fda9d3ccc Reviewed-on: https://cl.tvl.fyi/c/depot/+/10328 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-12-11 r/7159 refactor(tvix/*store/sled): make ::new() more genericFlorian Klink4-7/+6
We don't really require the Path to be a PathBuf, we don't even require it to be a Path, we only need it to be AsRef<Path>>. This removes some conversion in the from_addr cases, which can just reuse `url.path()` (a `&str`). Change-Id: I38d536dbaf0b44421e41f211a9ad2b13605179e9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10258 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-12-11 r/7158 chore(tvix): drop sled compression supportFlorian Klink3-3/+7
It's been a while since the last sled release, and that one binds to a pretty old version of zstd, requiring workarounds like cl/10090. Upstream sled main branch currently has zstd halfway patched out (it's a no-op, but the feature flag and options are still there), and it's in that state for a year. Rather than maintaining our own fork of sled, let's just stop using the compression feature in sled, dropping the version pin to zstd that way, removing the need for cl/10090. This doesn't mean we won't reintroduce per-blob compression - but we probably just won't let sled take care of the compression, but do it ourselves - which is necessary for more chunked blob storage anyways. Even though we do drop the feature flag, we still need to explicitly use use_compression(false). Change-Id: I0e4892d29e41c76653272dc1a3625180da6fee12 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10257 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-12-09 r/7140 refactor(tvix/castore): address clippyFlorian Klink1-10/+7
We match to destructure a single pattern. Change-Id: I564a3510b4860e90b3315a9639effc48ee88b483 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10233 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-11-24 r/7053 fix(tvix/castore): correctly flag unreachable codesterni1-1/+1
Change-Id: Id09afa4b77c3c70fb5695f253f6df4aa88b61e19 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10113 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-11-19 r/7037 refactor(tvix/castore/blobservice): rm AsyncBufRead from BlobReaderFlorian Klink1-4/+1
There's no need to already require this to be buffered here. Change-Id: Ib9a11b194e0754d87ab8d2ef0b8cb0f4edc01229 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10074 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-11-18 r/7027 feat(tvix/castore): fix tracing instrument in `MemoryBlobService`Ryan Lahfa1-2/+3
Change-Id: Iedba57e8b3e1a44f14f5baa1e981275d4b02eb56 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10070 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-11-04T15·24+0000