about summary refs log tree commit diff
path: root/tvix/store
AgeCommit message (Collapse)AuthorFilesLines
2023-12-29 r/7268 refactor(tvix/castore/bin/tvix-store) don't import std::ioFlorian Klink1-5/+4
Make it more clear this is not tokio::io. Change-Id: Ic2fa56f0baf1c200b6631098d556388a19629a45 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10452 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-12-29 r/7266 feat(tvix/store): emit more structured log messagesFlorian Klink1-3/+3
Change-Id: If4748cb800276fc42779ba8973d1eef72df0503e Reviewed-on: https://cl.tvl.fyi/c/depot/+/10450 Reviewed-by: edef <edef@edef.eu> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-12-24 r/7261 chore(tvix): switch to upstream futures 0.3.30Florian Klink1-1/+1
The bugs have been fixed, https://github.com/rust-lang/futures-rs/pull/2801 and https://github.com/rust-lang/futures-rs/pull/2812 were merged and ended up in that release. Change-Id: Iefd990d2d1719b884504093343e54e9c5258e2e2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10414 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-12-24 r/7258 refactor(tvix/castore): add RootNode impl for BTreeMap, mv fs testsFlorian Klink3-1177/+0
cl/10378 did already move store/fs to castore/fs, but we kept the tests in tvix-store, as they were populating a PathInfoService to make nodes appear in the mount root. Update these tests to now just insert root nodes into a BTreeMap, and ensure we can use that as a RootNodes too. Change-Id: Iad7d1ee4f9423eb6e3a1da33f433842c9ae0de1f Reviewed-on: https://cl.tvl.fyi/c/depot/+/10410 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-12-22 r/7256 chore(tvix): move store/fs to castore/fsFlorian Klink13-1422/+111
With the recent introduction of the RootNodes trait, there's nothing in the fs module pulling in tvix-store dependencies, so it can live in tvix-castore. This allows other crates to make use of TvixStoreFS, without having to pull in tvix-store. For example, a tvix-build using a fuse mountpoint at /nix/store doesn't need a PathInfoService to hold the root nodes that should be present, but just a list. tvix-store now has a pathinfoservice/fs module, which contains the necessary glue logic to implement the RootNodes trait for a PathInfoService. To satisfy Rust orphan rules for trait implementations, we had to add a small wrapper struct. It's mostly hidden away by the make_fs helper function returning a TvixStoreFs. It can't be entirely private, as its still leaking into the concrete type of TvixStoreFS. tvix-store still has `fuse` and `virtiofs` features, but they now simply enable these features in the `tvix-castore` crate they depend on. The tests for the fuse functionality stay in tvix-store for now, as they populate the root nodes through a PathInfoService. Once above mentioned "list of root nodes" implementation exists, we might want to shuffle this around one more time. Fixes b/341. Change-Id: I989f664827a5a361b23b34368d242d10c157c756 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10378 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2023-12-22 r/7255 refactor(tvix/store): remove Arc<> from PathInfoService::from_addrFlorian Klink5-24/+37
This makes PathInfoService::from_addr return a Box<dyn PathInfoService>, rather than an Arc<dyn …>, and leaves it up to the consumers to rewrap it into an Arc where needed. This allows us to drop the Arc for the tvix-store daemon subcommand. Change-Id: Ic83aa2ade6c51912281bd17c7eef7252e152b2d1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10409 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2023-12-22 r/7254 refactor(tvix/store/bin): move service.clone right before async moveFlorian Klink1-61/+65
Change-Id: I7a2b951d9c9251a053a0de40f31836bda03a922d Reviewed-on: https://cl.tvl.fyi/c/depot/+/10408 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: sterni <sternenseemann@systemli.org>
2023-12-22 r/7253 refactor(tvix/store/bin): remove unneeded cloneFlorian Klink1-4/+2
Change-Id: Ib94376db3a57853d980112919c2d6fcc0f566883 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10407 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de>
2023-12-21 r/7246 docs(tvix/store): fix referenceFlorian Klink1-1/+1
Change-Id: If1970cb8eb9a21c30011c9303f2fcbff646ad12b Reviewed-on: https://cl.tvl.fyi/c/depot/+/10400 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: edef <edef@edef.eu>
2023-12-21 r/7245 docs(tvix/store): fix referenceFlorian Klink1-2/+2
Change-Id: Ifb77097755f25aa6536c60365941507a592cac3e Reviewed-on: https://cl.tvl.fyi/c/depot/+/10399 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: edef <edef@edef.eu>
2023-12-18 r/7229 refactor(tvix/store): remove more clones on pathinfoserviceFlorian Klink1-4/+3
These are still leftovers from before the traits being async, where we had to clone before moving into an async closure. Change-Id: I1b3937edf61ce3e23bb07803306622c37a3572c6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10381 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de>
2023-12-16 r/7226 docs(tvix/store/fs): fix commentFlorian Klink1-5/+7
There's nothing store-path specific here anymore, it's just a name in the mountpoint root. Change-Id: I0f8004491baa03ba560d390053a42678ee81154a Reviewed-on: https://cl.tvl.fyi/c/depot/+/10377 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-12-16 r/7224 chore(tvix/store): switch back to fuse-backend-rs upstreamFlorian Klink1-3/+1
The fix mentioned in Cargo.toml did land in https://github.com/cloud-hypervisor/fuse-backend-rs/pull/162. Change-Id: I4451f910806f05400f7e0f83581ae786f170b9e2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10375 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-12-16 r/7223 refactor(tvix/store/fs): no explicitly required Arc'ed Blob/DirSvcFlorian Klink1-7/+12
Change-Id: Ie55026668cd4a6117e7b07174f5ac6638f93d194 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10374 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-12-14 r/7222 refactor(tvix/store/fs): Decouple TvixStoreFs from Nix conceptsConnor Brewster2-76/+119
To support tvix builds, we need to be able to use the `TvixStoreFs` to materialize the sandbox's /nix/store filesystem with just the list of inputs needed for the build. Currently we'd need to seed an in-memory `PathInfoService`, which includes more functionality than what is required for `TvixStoreFs`. Additionally, the `PathInfoService` is specific to Nix. By decoupling `TvixStoreFs` and `PathInfoService`, we allow for usage of `TvixStoreFs` with `tvix-castore` without needing a `PathInfoService`. This introduces a new `RootNodes` trait which provides a way for the filesystem to look up CA nodes via their basename in the root directory of the filesystem. We then implement `RootNodes` for any `PathInfoService`. Additionally, the filesystem root inode tracker now stores basenames rather than `StorePath`s since `StorePath`s are specific to Nix. As a followup we can rename `TvixStoreFs` to `TvixCaStoreFs` and move it to the `castore` crate (or its own crate). b/341 Change-Id: I928372955017c23b1bf2b37190cbc508a4ed10d5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10363 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-12-12 r/7207 refactor(tvix/castore/blobservice): use io::Result in traitFlorian Klink2-16/+13
For all these calls, the caller has enough context about what it did, so it should be fine to use io::Result here. We pretty much only constructed crate::Error::StorageError before anyways, so this conveys *more* information. Change-Id: I5cabb3769c9c2314bab926d34dda748fda9d3ccc Reviewed-on: https://cl.tvl.fyi/c/depot/+/10328 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-12-11 r/7159 refactor(tvix/*store/sled): make ::new() more genericFlorian Klink2-4/+4
We don't really require the Path to be a PathBuf, we don't even require it to be a Path, we only need it to be AsRef<Path>>. This removes some conversion in the from_addr cases, which can just reuse `url.path()` (a `&str`). Change-Id: I38d536dbaf0b44421e41f211a9ad2b13605179e9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10258 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-12-11 r/7158 chore(tvix): drop sled compression supportFlorian Klink3-3/+5
It's been a while since the last sled release, and that one binds to a pretty old version of zstd, requiring workarounds like cl/10090. Upstream sled main branch currently has zstd halfway patched out (it's a no-op, but the feature flag and options are still there), and it's in that state for a year. Rather than maintaining our own fork of sled, let's just stop using the compression feature in sled, dropping the version pin to zstd that way, removing the need for cl/10090. This doesn't mean we won't reintroduce per-blob compression - but we probably just won't let sled take care of the compression, but do it ourselves - which is necessary for more chunked blob storage anyways. Even though we do drop the feature flag, we still need to explicitly use use_compression(false). Change-Id: I0e4892d29e41c76653272dc1a3625180da6fee12 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10257 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-12-11 r/7155 refactor(tvix/*/protos): separate lint targetFlorian Klink1-16/+36
Break up the go-bindings derivation. Keep "protos" containing all proto files (well, and the buf config), and use it for a check phase running linter and formatter, as well as the existing "go-bindings" attribute Change-Id: I52cb9d08570bb76452acb831eb711c5b6c0eacfb Reviewed-on: https://cl.tvl.fyi/c/depot/+/10239 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-12-11 r/7150 chore(tvix/store/protof): buf formatFlorian Klink3-138/+138
Change-Id: I29b0fb62aa849ee01860dc7fe3dbba9b941ca172 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10238 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Adam Joseph <adam@westernsemico.com>
2023-12-09 r/7138 refactor(store/fs): address clippyFlorian Klink1-1/+1
Change-Id: I28d60263ef672942940656caa8e0f9d20cc6b7d7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10231 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-12-09 r/7129 chore(store/src/tests): make clippy shut upAdam Joseph1-12/+6
Change-Id: I6dfceaa32a8c01f8395b7889ae19847fabf95ed3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10214 Tested-by: BuildkiteCI Autosubmit: Adam Joseph <adam@westernsemico.com> Reviewed-by: flokli <flokli@flokli.de>
2023-11-28 r/7086 feat(tvix/store/pathinfosvc/nix_http): allow configuring pubkeysFlorian Klink1-6/+28
This allows setitng the trusted-public-keys URL parameter to a (whitespace-separated) list of public keys. NARInfo files retrieved need to contain a valid signature. Change-Id: Ifd6580b723cbae3182e9cadfa54f1ca2b41d6599 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10153 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-11-28 r/7085 feat(tvix/store/pathinfosvc): add signature verificationFlorian Klink1-1/+33
Introduce an Option<Vec<narinfo::PubKey>>, configurable with a `set_public_keys` method. If set, this configures NixHTTPPathInfoService to validate signatures. Change-Id: I157c5e13c41fc9bfd40b0655381fb4cf33900868 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10152 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-11-28 r/7081 feat(tvix/store/pathinfosvc/nix_http): check Nar{Size,Hash} matchesFlorian Klink1-13/+84
Ensure the initially communicated NarHash/NarSize from the NarInfo matches what we read, and don't return a PathInfo message if there's a mismatch. Also move the buffering layer around a bit. Change-Id: I68c60ecfaf0f9cd5edacea648437ecb0c9729251 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10148 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-11-28 r/7080 fix(tvix/store/fs): don't panic on PathInfoService errorFlorian Klink1-11/+9
An error in the PathInfoService request can appear in case the underlying request returns an error. We shouldn't panic and bork the fuse mount, but instead return an IO error. Change-Id: I2daeae629e1627d06adcd7b82ddb76c50c602212 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10154 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-11-22 r/7051 feat(tvix/store): add as_narinfo() for PathInfoFlorian Klink2-2/+106
This allows seeing a PathInfo as a nix_compat::narinfo::NarInfo<'_>. It doesn't allocate any new data, but the NarInfo<'_> view allows us to access things like signature verification, or rendering out (alternations of this) as strings. Change-Id: Id0d8d7feeb626ee02c3d8a4932f24ace77022619 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10108 Reviewed-by: edef <edef@edef.eu> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-11-22 r/7049 refactor(tvix/store): impl From<&nar_info::Ca> for nixhash::CAHashFlorian Klink1-0/+62
Change-Id: I637a4cff5a5ca29c4d86e0b76a2f20f8741f5628 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10107 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI
2023-11-22 r/7048 refactor(tvix/store): impl From<nixhash::CAHash> for nar_info::CaFlorian Klink1-16/+25
Change-Id: Iaa68044d3b469f15a932aa3b59548505eaa6b8bb Reviewed-on: https://cl.tvl.fyi/c/depot/+/10106 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI
2023-11-22 r/7047 refactor(tvix/nix-compat): move from_name_and_digest to StorePathRefFlorian Klink1-5/+3
We can simply use .to_owned() on that thing afterwards if we want to construct an owned StorePath. Change-Id: I0f3e2e4434b99ee522f2a7dbfa391e13a987479c Reviewed-on: https://cl.tvl.fyi/c/depot/+/10105 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI
2023-11-22 r/7046 feat(nix-compat/narinfo): drop .drv from Narinfo.deriver fieldFlorian Klink1-4/+1
We always know this needs to end with a .drv, and fail parsing if it doesn't, so there's no need to hang onto these 4 bytes. This will make it much easier to synthesize a NarInfo<'_> later on from a PathInfo proto, because we don't have to make this ".drv" appear out of thin air. Change-Id: Id95e7fd937d7c9a420a39b5a4bab73985640ca3b Reviewed-on: https://cl.tvl.fyi/c/depot/+/10084 Tested-by: BuildkiteCI Reviewed-by: edef <edef@edef.eu> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de>
2023-11-19 r/7038 feat(tvix/store/pathinfoservice): implement NixHTTPPathInfoServiceFlorian Klink4-7/+242
NixHTTPPathInfoService acts as a bridge in between the Nix HTTP Binary cache protocol provided by Nix binary caches such as cache.nixos.org, and the Tvix Store Model. It implements the [PathInfoService] trait in an interesting way: Every [PathInfoService::get] fetches the .narinfo and referred NAR file, inserting components into a [BlobService] and [DirectoryService], then returning a [PathInfo] struct with the root. Due to this being quite a costly operation, clients are expected to layer this service with store composition, so they're only ingested once. The client is expected to be (indirectly) using the same [BlobService] and [DirectoryService], so able to fetch referred Directories and Blobs. [PathInfoService::put] and [PathInfoService::nar] are not implemented and return an error if called. This behaves very similar to the nar-bridge-pathinfo code in nar-bridge, except it's now in Rust. Change-Id: Ia03d4fed9d0657965d100299af97cd917a03f2f0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10069 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-11-19 r/7035 feat(nix-compat/nar/reader): provide passthrough buffered I/Oedef1-5/+5
Allow taking advantage of the buffer of the underlying reader to avoid unnecessary copies of file data. We can't easily implement the methods of BufRead directly, since we have some extra I/O to perform in the final consume() invocation. That could be resolved at the cost of additional bookkeeping, but this will suffice for now. Change-Id: I8100cf0abd79e7469670b8596bd989be5db44a91 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10089 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-11-19 r/7034 fix(nix-compat/nar/reader): require BufReadedef1-2/+2
We rely on being able to make small reads cheaply, so this was already an implicit practical requirement. Requiring it explicitly removes a performance footgun, and makes further optimisations possible. Change-Id: I7f65880a41b1d6b5e6bf2e52dfe47d4c49b34bcd Reviewed-on: https://cl.tvl.fyi/c/depot/+/10088 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-11-18 r/7030 refactor(tvix/store/fs): simpllify readFlorian Klink1-19/+3
We can just use take(size) to restrict reading to that as a max. Change-Id: I0fbda74e4fb98ffeababae86a325233416029acf Reviewed-on: https://cl.tvl.fyi/c/depot/+/10072 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-11-18 r/7029 feat(tvix/store): From<&nix_compat::...::NarInfo<'_>> for PathInfoFlorian Klink2-2/+176
This allows converting from the NarInfo falling out of the NarInfo parser (which is a bit annoying to handle due to lifetimes) to the PathInfo proto struct. The narinfo field, containing most of the data from the original NARInfo file, as well as the references (bytes) are populated. The node field is not populated, because it requires ingesting the NAR itself to describe the root node. Change-Id: I9c04dd6ad4cae556b455188a4255e34b4f6443c5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10067 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-11-15 r/7022 refactor(tvix/castore/tonic): make async, support wait-connect=?Florian Klink3-25/+26
This moves the sync `channel::from_url` to a async `tonic::channel_from_url`. It now allows connecting non-lazily if `wait- connect=1` is set in the URL params. Also, make the pingpong tests for blobsvc and directorysvc use the wait- connect=1 codepath. Change-Id: Ibeea33117c8121814627e7f6aba0e943ae2e92ca Reviewed-on: https://cl.tvl.fyi/c/depot/+/10030 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-11-15 r/7019 refactor(tvix/store/pathinfosvc/from_addr): use test_caseFlorian Klink1-93/+46
All we do is constructing some strings, and checking if from_addr succeeds or not. This can be written in a much more concise way using test_case. Use lazy_static to provide temporary directories. Also add some more grpc-related test cases. Change-Id: Ia310dd01f617f7628f1e7e21304ac70da2ab3534 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10027 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2023-11-15 r/7018 refactor(tvix/store/pathinfosvc): inline SledPathInfoSvc::from_urlFlorian Klink2-136/+81
Change-Id: I0d905228df086a422bb30322add7236ca41e807b Reviewed-on: https://cl.tvl.fyi/c/depot/+/10026 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-11-15 r/7017 refactor(tvix/store/pathinfosvc): inline GRPCPathInfoSvc::from_urlFlorian Klink2-28/+27
Change-Id: Ib53b5525ae13c276e61b7f564673b7c6144ffc0e Reviewed-on: https://cl.tvl.fyi/c/depot/+/10025 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-11-15 r/7016 feat(tvix/castore/src/channel): move from_url testsFlorian Klink1-73/+1
These gRPC PathInfoService tests were actually not too useful in here, what we're mostly testing is the channel construction, so move it to there. Change-Id: Ic8c07558a1b28b46f863d5c39bcaa3a79cea007a Reviewed-on: https://cl.tvl.fyi/c/depot/+/10024 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2023-11-15 r/7015 refactor(tvix/store/pathinfosvc): inline MemoryPathInfoSvc::from_urlFlorian Klink2-88/+47
Change-Id: If27eb518d372f4004b7b38fc765a42957f2a6b50 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10023 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-11-15 r/7014 refactor(tvix/store): remove from_url from PathInfoService traitFlorian Klink4-31/+13
We don't gain much from making this part of the trait, it's still up to `tvix_store::pathinfoservice::from_addr` to do most of the construction. Move it out of the trait and into the specific *Service impls directly. This allows further refactorings in followup CLs. Change-Id: I99b93ef4acd83637a2f4888a1e586f1ca96390dc Reviewed-on: https://cl.tvl.fyi/c/depot/+/10022 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-11-07 r/6978 refactor(tvix/store/fs): mv sparse -> populated directoriesFlorian Klink3-317/+54
Do this upgrade whenever someone is actually interested in the children of a directory, but that directory doesn't contain a more detailed listing. This is much more predictable, and removes a bunch of confusing code from the inode tracker itself. Change-Id: Ib3a13694d6d5d22887d2d04ae429592137f39cb4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9982 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-11-07 r/6977 refactor(tvix/store/fs): simplify name_in_root_to_ino_and_dataFlorian Klink1-82/+72
Have it return libc::ENOENT errors rather than an Option<…>. Also avoid having to traverse inode_data multiple times, by synthesizing the Arc<…> on our own in the insert case. In that case, the data is quite small, so cloning it is faster than traversing a second time. Change-Id: I7ab14bac8bb23859ed8d166a12070d4f4749b6d4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9981 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-11-07 r/6976 refactor(tvix/store/fs): move code to get_directory_children helperFlorian Klink1-122/+76
As already established in the two previous CLs, these two pieces of code where doing the same. Move to a get_directory_children helper. Change-Id: Id6876f0c34f3f40a31a22d59a2cdbfef39e2d8de Reviewed-on: https://cl.tvl.fyi/c/depot/+/9980 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2023-11-07 r/6975 refactor(tvix/store/fs): reduce write lock, return childrenFlorian Klink1-47/+56
Very similar to the previous CL Change-Id: I0df07ddca742b7b9485d48771c8d295dc3aa7136 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9979 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-11-07 r/6974 refactor(tvix/src/fs): reduce write lock, avoid inode_tracker lookupFlorian Klink1-39/+50
Code after this big match block only cares about parent_digest and children, so there's no need to do another inode_tracker.get in there. This also allows removing another if let block, right after, as we don't need to destructure parent_data anymore. Change-Id: I68fbbe3304194670caee5a453722369afa4e77ea Reviewed-on: https://cl.tvl.fyi/c/depot/+/9978 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-11-07 r/6973 refactor(tvix/store/fs): move inode for store_path lookup to helperFlorian Klink1-19/+13
This makes it much harder to keep the read lock around for too long, and the code a bit easier to understand. Change-Id: I7d99c85cadd433cad444b8edd34e2c43d7eaf5a8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9977 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-11-07 r/6972 fix(tvix/castore): B3Digest::{to_vec -> as_slice}edef1-4/+4
Not a single call site actually makes use of the Vec. Change-Id: I6cf31073c9f443d1702a21937a0c3938c2c643b8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9988 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-11-01T23·27+0000