about summary refs log tree commit diff
path: root/tvix/castore/src/import.rs (follow)
AgeCommit message (Collapse)AuthorFilesLines
2024-01-05 r/7346 refactor(tvix/castore): relax trait bounds for DSFlorian Klink1-2/+2
Make this an `AsRef<dyn DirectoryService>`. This helps dropping some Clone requirements. Unfortunately, we can't thread this through to TvixStoreIO just yet. Change-Id: I3f07eb28d6c793d3313fe21506ada84d5a8aa3ac Reviewed-on: https://cl.tvl.fyi/c/depot/+/10533 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-01 r/7300 feat(tvix/castore/import): generalize ingest_pathFlorian Klink1-8/+16
We don't actually care if it's an Arc<dyn BlobService>, or something else, as long as we can Deref to a BlobService and clone. Change-Id: I0852aaf723f51c5e6b820be8db1199d17309ab08 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10510 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-12-12 r/7209 fix(tvix/castore/import): don't unwrap entryFlorian Klink1-2/+8
If the path specified doesn't exist, construct a proper error instead of panicking. Part of b/344. Change-Id: Id5c6a91248b0a387f3e8f138f8e686e402009e8f Reviewed-on: https://cl.tvl.fyi/c/depot/+/10330 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-12-12 r/7208 feat(tvix/castore/import): log returned errorsFlorian Klink1-1/+1
This will emit a log event / trace in case this function returns an error-y type. Change-Id: I48db6807f3e42304357c422a2b6e177cb8b95228 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10329 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-12-12 r/7207 refactor(tvix/castore/blobservice): use io::Result in traitFlorian Klink1-1/+4
For all these calls, the caller has enough context about what it did, so it should be fine to use io::Result here. We pretty much only constructed crate::Error::StorageError before anyways, so this conveys *more* information. Change-Id: I5cabb3769c9c2314bab926d34dda748fda9d3ccc Reviewed-on: https://cl.tvl.fyi/c/depot/+/10328 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-11-24 r/7053 fix(tvix/castore): correctly flag unreachable codesterni1-1/+1
Change-Id: Id09afa4b77c3c70fb5695f253f6df4aa88b61e19 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10113 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-11-05 r/6946 feat(tvix/castore): bump [Directory,File]Node size to u64Florian Klink1-1/+1
Having more than 4GiB files is quite possible (think about the NixOS graphical installer, and an uncompressed iso of it). No wire format changes. Change-Id: Ia78a07e4c554e91b93c5b9f8533266e4bd7f22b6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9950 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-10-17 r/6842 fix(tvix/castore): Fix race when ingesting into castoreConnor Brewster1-0/+4
After finishing the ingestion, the directory putter was not being closed. This caused a race where the root directory node was accessed before the directory node had been flushed to the server. This patch makes it so we close the putter before returning the root node which should ensure that the root node exists on the directory service server before the `ingest_path` function returns. Fixes b/326 Change-Id: Id16cf46bc48962121dde76d3c9c23a845d87d0f1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9761 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-10-13 r/6796 docs(tvix/castore): point out use of contents_firstLinus Heckemann1-0/+5
Change-Id: I7620d2abe01675ea7028a478d4f8447e36d5768b Reviewed-on: https://cl.tvl.fyi/c/depot/+/9605 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-10-08 r/6729 docs(tvix/castore): remove TODOFlorian Klink1-1/+0
This probably was about passing around directory_putter at some point, which we do, so whatever this meant, it's not actionable anymore. Change-Id: I1b4e0cdd2119bf2b2a9cf06d186a3b476b0ff367 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9573 Reviewed-by: Linus Heckemann <git@sphalerite.org> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-10-04 r/6689 fix(tvix/castore): explicitly name lifetimes in process_entryedef1-3/+3
Otherwise this produces absolutely inscrutable errors: note: hidden type `[async fn body@castore/src/import.rs:63:1: 63:94]` captures lifetime '_#24r Change-Id: If5d9626c9edf400de5bcec038bcaa5a3117561f0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9506 Tested-by: BuildkiteCI Autosubmit: edef <edef@edef.eu> Reviewed-by: flokli <flokli@flokli.de>
2023-09-22 r/6629 refactor(tvix): move castore into tvix-castore crateFlorian Klink1-0/+205
This splits the pure content-addressed layers from tvix-store into a `castore` crate, and only leaves PathInfo related things, as well as the CLI entrypoint in the tvix-store crate. Notable changes: - `fixtures` and `utils` had to be moved out of the `test` cfg, so they can be imported from tvix-store. - Some ad-hoc fixtures in the test were moved to proper fixtures in the same step. - The protos are now created by a (more static) recipe in the protos/ directory. The (now two) golang targets are commented out, as it's not possible to update them properly in the same CL. This will be done by a followup CL once this is merged (and whitby deployed) Bug: https://b.tvl.fyi/issues/301 Change-Id: I8d675d4bf1fb697eb7d479747c1b1e3635718107 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9370 Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>