about summary refs log tree commit diff
path: root/tvix/store/src
AgeCommit message (Collapse)AuthorFilesLines
2024-03-18 r/7723 feat(tvix/store): support RUST_LOG env varFlorian Klink1-5/+23
This allows selectively increasing the log level for only parts of the stack. For example, the following RUST_LOG env var enables "tracing" level logging for `tvix_store` and `tvix_castore`, while keeping it at "info" for the rest of the stack: export RUST_LOG='info,tvix_store=trace,tvix_castore=trace' It only affects logs, not traces (if enabled). Change-Id: Ib936bd132a405f216e75c843db83fbd71d20a18a Reviewed-on: https://cl.tvl.fyi/c/depot/+/11182 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-03-15 r/7700 docs(tvix): fix some docstringsFlorian Klink1-1/+1
Change-Id: Ife599387d0472cd746b992bd6755a2fb6a0e0dc4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11158 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-03-11 r/7683 feat(tvix/store/bin): allow disabling OTLP at runtimeFlorian Klink1-32/+39
This was only possible by disabling without the otlp feature flag so far. Introduce the same --otlp=false mechanism that nar-bridge also supports to be able to turn it off at runtime. Change-Id: Ib22a364c35056ca9d8e327c0e2a79970a4cf4b2b Reviewed-on: https://cl.tvl.fyi/c/depot/+/11135 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-03-09 r/7661 fix(tvix/store/grpc/pathinfo): skip_all fields, handle errorsFlorian Klink1-15/+20
request only contains the outer metadata wrapping, and that's not too interesting: > Request { metadata: MetadataMap { headers: {"content-type": > "application/grpc", "user-agent": "grpc-go/1.60.1", "te": "trailers", > "grpc-accept-encoding": "gzip"} }, message: Streaming, extensions: > Extensions } Drop these fields for now, and rely on the underlying implementations to add instrumentation for the application-specific fields. Also, ensure we handle all error cases properly, and log them. We don't use `err` from instrument, as that'd also log an error on `Status::not_found`. Change-Id: Id1b983cb8b059c148c8a376f8802a1d28c59ba97 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11103 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-03-03 r/7649 refactor(tvix/store/protos): more granular nix-compat conversionsFlorian Klink1-17/+18
implement From<&nix_compat::nixhash::CAHash> for nar_info::ca::Hash individually, and make From<&nix_compat::nixhash::CAHash> for nar_info::Ca a small wrapper that uses it, as well as the .hash().digest_as_bytes() for the digest. Change-Id: I7e9b6edd1e3f149eb270faf2928cd846d74e77ad Reviewed-on: https://cl.tvl.fyi/c/depot/+/11071 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Brian Olsen <me@griff.name> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-03-03 r/7648 refactor(tvix/*/from_addr): improve test debuggabilityFlorian Klink1-2/+6
If there's an unexpected test failure, print it out, rather than just saying something is false even though it should be true. Use .expect() for this, which displays the error if it failed. We can't use expect_err(), as our stores are not display'able, so use an assertion with a message there. Change-Id: I2d88861d979d107edc0717fbdb3cdac9a6bfc5e4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11091 Tested-by: BuildkiteCI Reviewed-by: Brian Olsen <me@griff.name> Reviewed-by: flokli <flokli@flokli.de>
2024-02-20 r/7566 refactor(tvix/store/proto): use bstr::ByteSliceFlorian Klink1-1/+2
String::from_utf8_lossy simply discards invalid bytes, while bstr replaces them with their replacement character. Change-Id: Ib78ff36ca5faacc1ad60bc4ddde7b62773848c07 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10981 Tested-by: BuildkiteCI Reviewed-by: Peter Kolloch <info@eigenvalue.net> Reviewed-by: flokli <flokli@flokli.de>
2024-02-20 r/7564 fix(tvix/store): Improve error message for InvalidNodeNameAspen Smith1-1/+1
Rather than debug-formatting a Vec<u8>, which gives just a sequence of numbers (not very helpful!) use String::from_utf8_lossy to give a much more human-readable error message Change-Id: I6cae795ebe2ab9a656c087b82a143411e6c7c81a Reviewed-on: https://cl.tvl.fyi/c/depot/+/10980 Autosubmit: aspen <root@gws.fyi> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-02-17 r/7535 feat(tvix/castore/fs): make allow_other configurableFlorian Klink1-1/+8
Also add a cli argument to the `tvix-store` binary. Change-Id: Id07d7fedb60d6060543b195f3a810a46137f9ad5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10945 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su>
2024-02-05 r/7475 docs(tvix/store/pathinfosvc): document nix+http[s]Florian Klink1-0/+5
Change-Id: I5f4e93aceec373eb3377e3f8fbb55a5142cc1363 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10740 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-01-21 r/7435 chore(tvix/store): Use BoxStream type aliasConnor Brewster8-43/+27
The BoxStream type alias is a more concise and easier to read than the full `Pin<Box<dyn Stream<Item = ...> + Send + ...>>` type. Change-Id: I5b7bccfd066ded5557e01f7895f4cf5c4a33bd44 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10677 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: Connor Brewster <cbrewster@hey.com>
2024-01-20 r/7433 feat(tvix/store): enable `name` customization in the storeRyan Lahfa2-15/+18
Sometimes, Nix lets someone customize the `name` in the store for a path, this is the case for `builtins.path` which takes a `name` argument, we leave it to the caller to choose the name, which can be the basename by default of the path. Change-Id: Icdbf71d1d8f2dca5716b99d20aac885aab905b80 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10653 Tested-by: BuildkiteCI Autosubmit: raitobezarius <tvl@lahfa.xyz> Reviewed-by: flokli <flokli@flokli.de>
2024-01-20 r/7432 refactor(tvix/store): `import_path` → `import_path_as_nar_ca`Ryan Lahfa4-117/+160
Add multiple additional helpers such as: - `path_to_name`: derive the basename of a given path - `derive_nar_ca_path_info`: derive the `PathInfo` for a content addressed NAR which isolates further the tree walking feature and the ingestion feature. Additionally, we don't `expect` anymore and propagate properly ingestion errors up. Change-Id: I60edb5b633911c58ade7e19f5002e6f75f90e262 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10574 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: raitobezarius <tvl@lahfa.xyz>
2024-01-12 r/7371 refactor(tvix/store/pathinfoservice): make more genericFlorian Klink6-54/+51
We don't need Arcs in most of the cases, we're fine with some container. Change-Id: Ic4f8acb5b9d93e2b0923bb607463fb91e9d0e4fe Reviewed-on: https://cl.tvl.fyi/c/depot/+/10606 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-12 r/7370 refactor(tvix/store/nar/renderer): don't require Arc, Clone or SyncFlorian Klink1-28/+45
To render NARs, we're fine with a simple AsRef to a BlobService and DirectoryService. We just need to have the function pass back the references, so we can reuse it after the recursion. Change-Id: I8a1b899134ddda26cf14aa829a08383986101850 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10605 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-01-09 r/7362 refactor(tvix/store/nar/import): use AsRefFlorian Klink1-25/+26
We need to be a bit careful and pass the BlobService around (similar to how we already do with the directory_putter), but that allows getting rid of a bunch of annoying trait bounds. We also stop spawning additional tasks where we can just use block_on. Change-Id: If36de0ee947d2c779d20a384308241d2262d4764 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10580 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-01-09 r/7359 refactor(tvix): use AsRef<dyn …> instead of Deref<Target= …>Florian Klink2-17/+15
Removes some more needs for Arcs. Change-Id: I9a9f4b81641c271de260e9ffa98313a32944d760 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10578 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-05 r/7348 feat(tvix/store/pathinfosvc): call validateFlorian Klink1-1/+9
Change-Id: I557ed67a04112ba3e363303637d1988309250a93 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10535 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-05 r/7346 refactor(tvix/castore): relax trait bounds for DSFlorian Klink1-4/+5
Make this an `AsRef<dyn DirectoryService>`. This helps dropping some Clone requirements. Unfortunately, we can't thread this through to TvixStoreIO just yet. Change-Id: I3f07eb28d6c793d3313fe21506ada84d5a8aa3ac Reviewed-on: https://cl.tvl.fyi/c/depot/+/10533 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-03 r/7315 fix(tvix/store): set service.nameFlorian Klink1-24/+44
While this can be influenced with `OTEL_SERVICE_NAME` or `OTEL_RESOURCE_ATTRIBUTES`, "unknown_service" is an annoying name. Let's pick tvix.store as a default. Change-Id: I9b5c45703d9546775bb9266b29ab54daf7c54a9f Reviewed-on: https://cl.tvl.fyi/c/depot/+/10524 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2024-01-01 r/7308 feat(tvix/store): add opentelemetryFlorian Klink1-5/+36
This is behind the otlp feature flag (currently enabled by default). By default, this will try to push traces to a OTLP collector running at https://localhost:4317. You can make one available by running: ``` docker run -d --name jaeger \ -e COLLECTOR_ZIPKIN_HOST_PORT=:9411 \ -e COLLECTOR_OTLP_ENABLED=true \ -p 6831:6831/udp \ -p 6832:6832/udp \ -p 5778:5778 \ -p 16686:16686 \ -p 4317:4317 \ -p 4318:4318 \ -p 14250:14250 \ -p 14268:14268 \ -p 14269:14269 \ -p 9411:9411 --rm \ jaegertracing/all-in-one:1.42 ``` Started like that, jaeger brings a web interface at http://localhost:16686/search As documented in https://docs.rs/opentelemetry-otlp/latest/opentelemetry_otlp/, you can point this to another location by setting `OTEL_EXPORTER_OTLP_ENDPOINT`. Change-Id: Id1dca367d70027b2ea98bb70bcf99a68363ec2be Reviewed-on: https://cl.tvl.fyi/c/depot/+/8194 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de> Reviewed-by: aaqaishtyaq <aaqaishtyaq@gmail.com>
2024-01-01 r/7307 refactor(tvix/castore/tests): let gen_*_service return BoxesFlorian Klink5-22/+41
Only convert to and reuse an Arc<…> where needed. Change-Id: I2c1bc69cca5a4a3ebd3bdb33d6e28e1f5fb86cb9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10514 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-01 r/7306 refactor(tvix/store/nar/import): be a bit more genericFlorian Klink1-29/+40
Change-Id: If9a536949f36f428abea1a893f937fe7063e2f41 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10517 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-01 r/7305 refactor(tvix/castore/blobsvc/grpc/wrapper): don't require Arc<_>Florian Klink1-1/+1
Change-Id: I9655f5588c7dc98427de6af47d74b4ab7ce22071 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10516 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-01 r/7304 refactor(tvix/castore/directorysvc/grpc/wrapper): no Arc<_>Florian Klink1-1/+1
We can also drop the Clone requirement. Because the trait is async since some time, there's no need to clone before moving into an async closure, allowing us to simplify the code a bit. Change-Id: I9b0a0e10077d8c548d218207b908bfd92c5b8de0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10515 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de>
2024-01-01 r/7301 feat(tvix/store/bin): factor out importFlorian Klink2-113/+132
While at it, make it a bit more generic. Change-Id: Ic4caefda93aca3ffb656a09f8b4d648b41415532 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10511 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-01-01 r/7299 feat(nix-compat/store_path): build_ca_path may failFlorian Klink1-1/+9
Change-Id: Ia74ee870f38b7966501458bace541092256c3213 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10509 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-01-01 r/7298 refactor(tvix/store/bin): don't reconstruct StorePath twiceFlorian Klink1-8/+1
We already have that data in output_path. Change-Id: Iadf615551f0b44d539098446ff5926994deb606d Reviewed-on: https://cl.tvl.fyi/c/depot/+/10508 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-12-31 r/7297 fix(tvix/store/bin): don't unwrap in case of invalid pathsFlorian Klink1-8/+12
Instead, return an error, and move the entire check before starting to ingest the data underneath. Change-Id: Idcfba115cb7d599f5fc72a156aaad9d4d4714fcf Reviewed-on: https://cl.tvl.fyi/c/depot/+/10507 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-12-31 r/7296 refactor(tvix/store/bin): condense subscriber setup a bitFlorian Klink1-12/+8
We can use cli.json.then_some(…) to create a Some(…), allowing us to omit the else { None } lines. Change-Id: I6c8142a08d8cb88d6c8302e5ca7570698fcf2aa3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10505 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-12-31 r/7295 refactor(tvix/store): move construct_services helper hereFlorian Klink3-62/+67
This takes three URLs, and constructs Arc'ed {Blob,Directory,PathInfo}Service, allowing to remove some of the boilerplate. Change-Id: I40e7c2b551442ef2acdc543dfc87ab97e7c742bb Reviewed-on: https://cl.tvl.fyi/c/depot/+/10484 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-12-31 r/7294 refactor(tvix/castore/directorysvc): return Box, not ArcFlorian Klink1-4/+17
While we currently mostly use it in an Arc, as we need to clone it inside PathInfoService, there might be other usecases not requiring it to be Clone. Change-Id: Ia05bb370340792a048e2036be30e285ef1e63870 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10483 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-12-31 r/7293 refactor(tvix/castore/blobsvc): return Box, not ArcFlorian Klink1-4/+9
While we currently mostly use it in an Arc, as we need to clone it inside PathInfoService, there might be other usecases not requiring it to be Clone. Change-Id: I7bd337cd2e4c2d4154b385461eefa62c9b78345d Reviewed-on: https://cl.tvl.fyi/c/depot/+/10482 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-12-29 r/7268 refactor(tvix/castore/bin/tvix-store) don't import std::ioFlorian Klink1-5/+4
Make it more clear this is not tokio::io. Change-Id: Ic2fa56f0baf1c200b6631098d556388a19629a45 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10452 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-12-29 r/7266 feat(tvix/store): emit more structured log messagesFlorian Klink1-3/+3
Change-Id: If4748cb800276fc42779ba8973d1eef72df0503e Reviewed-on: https://cl.tvl.fyi/c/depot/+/10450 Reviewed-by: edef <edef@edef.eu> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-12-24 r/7258 refactor(tvix/castore): add RootNode impl for BTreeMap, mv fs testsFlorian Klink2-1174/+0
cl/10378 did already move store/fs to castore/fs, but we kept the tests in tvix-store, as they were populating a PathInfoService to make nodes appear in the mount root. Update these tests to now just insert root nodes into a BTreeMap, and ensure we can use that as a RootNodes too. Change-Id: Iad7d1ee4f9423eb6e3a1da33f433842c9ae0de1f Reviewed-on: https://cl.tvl.fyi/c/depot/+/10410 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-12-22 r/7256 chore(tvix): move store/fs to castore/fsFlorian Klink12-1376/+106
With the recent introduction of the RootNodes trait, there's nothing in the fs module pulling in tvix-store dependencies, so it can live in tvix-castore. This allows other crates to make use of TvixStoreFS, without having to pull in tvix-store. For example, a tvix-build using a fuse mountpoint at /nix/store doesn't need a PathInfoService to hold the root nodes that should be present, but just a list. tvix-store now has a pathinfoservice/fs module, which contains the necessary glue logic to implement the RootNodes trait for a PathInfoService. To satisfy Rust orphan rules for trait implementations, we had to add a small wrapper struct. It's mostly hidden away by the make_fs helper function returning a TvixStoreFs. It can't be entirely private, as its still leaking into the concrete type of TvixStoreFS. tvix-store still has `fuse` and `virtiofs` features, but they now simply enable these features in the `tvix-castore` crate they depend on. The tests for the fuse functionality stay in tvix-store for now, as they populate the root nodes through a PathInfoService. Once above mentioned "list of root nodes" implementation exists, we might want to shuffle this around one more time. Fixes b/341. Change-Id: I989f664827a5a361b23b34368d242d10c157c756 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10378 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2023-12-22 r/7255 refactor(tvix/store): remove Arc<> from PathInfoService::from_addrFlorian Klink5-24/+37
This makes PathInfoService::from_addr return a Box<dyn PathInfoService>, rather than an Arc<dyn …>, and leaves it up to the consumers to rewrap it into an Arc where needed. This allows us to drop the Arc for the tvix-store daemon subcommand. Change-Id: Ic83aa2ade6c51912281bd17c7eef7252e152b2d1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10409 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2023-12-22 r/7254 refactor(tvix/store/bin): move service.clone right before async moveFlorian Klink1-61/+65
Change-Id: I7a2b951d9c9251a053a0de40f31836bda03a922d Reviewed-on: https://cl.tvl.fyi/c/depot/+/10408 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: sterni <sternenseemann@systemli.org>
2023-12-22 r/7253 refactor(tvix/store/bin): remove unneeded cloneFlorian Klink1-4/+2
Change-Id: Ib94376db3a57853d980112919c2d6fcc0f566883 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10407 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de>
2023-12-21 r/7246 docs(tvix/store): fix referenceFlorian Klink1-1/+1
Change-Id: If1970cb8eb9a21c30011c9303f2fcbff646ad12b Reviewed-on: https://cl.tvl.fyi/c/depot/+/10400 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: edef <edef@edef.eu>
2023-12-21 r/7245 docs(tvix/store): fix referenceFlorian Klink1-2/+2
Change-Id: Ifb77097755f25aa6536c60365941507a592cac3e Reviewed-on: https://cl.tvl.fyi/c/depot/+/10399 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: edef <edef@edef.eu>
2023-12-18 r/7229 refactor(tvix/store): remove more clones on pathinfoserviceFlorian Klink1-4/+3
These are still leftovers from before the traits being async, where we had to clone before moving into an async closure. Change-Id: I1b3937edf61ce3e23bb07803306622c37a3572c6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10381 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de>
2023-12-16 r/7226 docs(tvix/store/fs): fix commentFlorian Klink1-5/+7
There's nothing store-path specific here anymore, it's just a name in the mountpoint root. Change-Id: I0f8004491baa03ba560d390053a42678ee81154a Reviewed-on: https://cl.tvl.fyi/c/depot/+/10377 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-12-16 r/7223 refactor(tvix/store/fs): no explicitly required Arc'ed Blob/DirSvcFlorian Klink1-7/+12
Change-Id: Ie55026668cd4a6117e7b07174f5ac6638f93d194 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10374 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-12-14 r/7222 refactor(tvix/store/fs): Decouple TvixStoreFs from Nix conceptsConnor Brewster2-76/+119
To support tvix builds, we need to be able to use the `TvixStoreFs` to materialize the sandbox's /nix/store filesystem with just the list of inputs needed for the build. Currently we'd need to seed an in-memory `PathInfoService`, which includes more functionality than what is required for `TvixStoreFs`. Additionally, the `PathInfoService` is specific to Nix. By decoupling `TvixStoreFs` and `PathInfoService`, we allow for usage of `TvixStoreFs` with `tvix-castore` without needing a `PathInfoService`. This introduces a new `RootNodes` trait which provides a way for the filesystem to look up CA nodes via their basename in the root directory of the filesystem. We then implement `RootNodes` for any `PathInfoService`. Additionally, the filesystem root inode tracker now stores basenames rather than `StorePath`s since `StorePath`s are specific to Nix. As a followup we can rename `TvixStoreFs` to `TvixCaStoreFs` and move it to the `castore` crate (or its own crate). b/341 Change-Id: I928372955017c23b1bf2b37190cbc508a4ed10d5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10363 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-12-12 r/7207 refactor(tvix/castore/blobservice): use io::Result in traitFlorian Klink2-16/+13
For all these calls, the caller has enough context about what it did, so it should be fine to use io::Result here. We pretty much only constructed crate::Error::StorageError before anyways, so this conveys *more* information. Change-Id: I5cabb3769c9c2314bab926d34dda748fda9d3ccc Reviewed-on: https://cl.tvl.fyi/c/depot/+/10328 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-12-11 r/7159 refactor(tvix/*store/sled): make ::new() more genericFlorian Klink2-4/+4
We don't really require the Path to be a PathBuf, we don't even require it to be a Path, we only need it to be AsRef<Path>>. This removes some conversion in the from_addr cases, which can just reuse `url.path()` (a `&str`). Change-Id: I38d536dbaf0b44421e41f211a9ad2b13605179e9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10258 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-12-11 r/7158 chore(tvix): drop sled compression supportFlorian Klink2-2/+4
It's been a while since the last sled release, and that one binds to a pretty old version of zstd, requiring workarounds like cl/10090. Upstream sled main branch currently has zstd halfway patched out (it's a no-op, but the feature flag and options are still there), and it's in that state for a year. Rather than maintaining our own fork of sled, let's just stop using the compression feature in sled, dropping the version pin to zstd that way, removing the need for cl/10090. This doesn't mean we won't reintroduce per-blob compression - but we probably just won't let sled take care of the compression, but do it ourselves - which is necessary for more chunked blob storage anyways. Even though we do drop the feature flag, we still need to explicitly use use_compression(false). Change-Id: I0e4892d29e41c76653272dc1a3625180da6fee12 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10257 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-12-09 r/7138 refactor(store/fs): address clippyFlorian Klink1-1/+1
Change-Id: I28d60263ef672942940656caa8e0f9d20cc6b7d7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10231 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-11-19T19·27+0000