about summary refs log tree commit diff
path: root/tvix/store
AgeCommit message (Collapse)AuthorFilesLines
2024-03-28 r/7787 refactor(tvix/store/tests/fixtures): use [u8; 20]Florian Klink4-10/+9
This makes the fixture more use-able when interacting with the trait, the Bytes are only useful for the gRPC version. Change-Id: Iaaea1adc6df18491f236a28c4343f5b4ee5fcfd3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11271 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-03-27 r/7786 feat(tvix/store): AsRef<dyn PathInfoService> impl PathInfoServiceFlorian Klink1-0/+25
Change-Id: I7fc06ae97a50d04b8c36292b3457c112242a7a70 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11270 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-03-20 r/7749 refactor(tvix/castore): introduce "cloud" feature flagFlorian Klink1-1/+2
This controls whether tvix-castore has support for various cloud backends or not. Use this to control the set of feature flags for the object_store backend, and only enable the aws, azure and gcp ones if it's set. In the future this can be used to enable/disable other cloud backends too. Without feature flags, `object_store` already supports the `InMemory` and `LocalFilesystem` backends, and we also want to unconditionally enable the `http` one. Make sure at least the construction of these services is covered in the tests. Similarly, the tvix-store crate, which provides the tvix-store CLI has a `cloud` feature flag too (defaulting to enabled). Change-Id: I9fb9c87b740e7dc83f8ff7a0862905d036d513f2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11204 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-03-20 r/7747 refactor(tvix/store/pathinfo/from_addr): use match guardsFlorian Klink1-59/+62
This will allow feature-flagging some of the backends. Change-Id: Ie92914c3e2ad870eee87e73b3b5abe605fb56fe7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11202 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-03-18 r/7733 refactor(tvix/store/import): use B3Digest in log_nodeFlorian Klink1-5/+3
Change-Id: I2347bbae8e7d4e19eeed4a3fb13729d0a94feedd Reviewed-on: https://cl.tvl.fyi/c/depot/+/11195 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-03-18 r/7732 refactor(tvix/store/pathinfo/nix_http): NixHash to display NARHashFlorian Klink1-3/+5
This will give us the sha256: prefix, and hashes we're more used to, in that context. Change-Id: I72e42fe685e365ba9baa7cd81001387d239fa7c8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11194 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-03-18 r/7731 refactor(tvix/store/nar): use B3Digest in RenderErrorFlorian Klink1-5/+4
Ensure consistent formatting of blake3 digests. It looks like we don't actually construct these error types anywhere, so no code needs to be refactored to it currently, but still good to be consistent. Change-Id: I49dc8a7f3cb4245ac06b9a6a44b72060434a3d32 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11193 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-03-18 r/7723 feat(tvix/store): support RUST_LOG env varFlorian Klink2-6/+24
This allows selectively increasing the log level for only parts of the stack. For example, the following RUST_LOG env var enables "tracing" level logging for `tvix_store` and `tvix_castore`, while keeping it at "info" for the rest of the stack: export RUST_LOG='info,tvix_store=trace,tvix_castore=trace' It only affects logs, not traces (if enabled). Change-Id: Ib936bd132a405f216e75c843db83fbd71d20a18a Reviewed-on: https://cl.tvl.fyi/c/depot/+/11182 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-03-16 r/7705 chore(tvix): bump tonic to 0.11.0Florian Klink1-4/+4
This bumps tonic and surrounding crates to 0.11.x. We added support for tonic 0.11.x into tokio-listener (https://github.com/vi/tokio-listener/pull/4), so that's bumped as well. Change-Id: Icfade5894403228299836fefb21b2f9ae59dbebb Reviewed-on: https://cl.tvl.fyi/c/depot/+/11156 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-03-16 r/7701 fix(tvix): don't emit rerun-if-changedFlorian Klink1-2/+2
`build.rs` emits rerun-if-changed statements for all proto files, as well as all include paths we pass it. Unfortunately, due to protobufs include path rules, we need to specify the path to the depot root itself as an include path, at least when building impurely with `cargo`. This causes cargo to essentially always rebuild, as it also puts its own temporary files in there. Unfortunately, tonic-build does not chase down to individual .proto files that are included. Disable emitting these `rerun-if-changed` statements for now. This could cause cargo to not rebuild protos every time, causing stale data until the next local `cargo clean`, but considering the protos change not that frequently, and it'll immediately surface if trying to build via Nix (either locally or in CI), it's a good-enough compromise. Change-Id: Ifd279a2216222ef3fc0e70c5a2fe6f87997f562e Reviewed-on: https://cl.tvl.fyi/c/depot/+/11157 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2024-03-15 r/7700 docs(tvix): fix some docstringsFlorian Klink1-1/+1
Change-Id: Ife599387d0472cd746b992bd6755a2fb6a0e0dc4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11158 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-03-11 r/7683 feat(tvix/store/bin): allow disabling OTLP at runtimeFlorian Klink1-32/+39
This was only possible by disabling without the otlp feature flag so far. Introduce the same --otlp=false mechanism that nar-bridge also supports to be able to turn it off at runtime. Change-Id: Ib22a364c35056ca9d8e327c0e2a79970a4cf4b2b Reviewed-on: https://cl.tvl.fyi/c/depot/+/11135 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-03-09 r/7661 fix(tvix/store/grpc/pathinfo): skip_all fields, handle errorsFlorian Klink1-15/+20
request only contains the outer metadata wrapping, and that's not too interesting: > Request { metadata: MetadataMap { headers: {"content-type": > "application/grpc", "user-agent": "grpc-go/1.60.1", "te": "trailers", > "grpc-accept-encoding": "gzip"} }, message: Streaming, extensions: > Extensions } Drop these fields for now, and rely on the underlying implementations to add instrumentation for the application-specific fields. Also, ensure we handle all error cases properly, and log them. We don't use `err` from instrument, as that'd also log an error on `Status::not_found`. Change-Id: Id1b983cb8b059c148c8a376f8802a1d28c59ba97 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11103 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-03-03 r/7649 refactor(tvix/store/protos): more granular nix-compat conversionsFlorian Klink1-17/+18
implement From<&nix_compat::nixhash::CAHash> for nar_info::ca::Hash individually, and make From<&nix_compat::nixhash::CAHash> for nar_info::Ca a small wrapper that uses it, as well as the .hash().digest_as_bytes() for the digest. Change-Id: I7e9b6edd1e3f149eb270faf2928cd846d74e77ad Reviewed-on: https://cl.tvl.fyi/c/depot/+/11071 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Brian Olsen <me@griff.name> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-03-03 r/7648 refactor(tvix/*/from_addr): improve test debuggabilityFlorian Klink1-2/+6
If there's an unexpected test failure, print it out, rather than just saying something is false even though it should be true. Use .expect() for this, which displays the error if it failed. We can't use expect_err(), as our stores are not display'able, so use an assertion with a message there. Change-Id: I2d88861d979d107edc0717fbdb3cdac9a6bfc5e4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11091 Tested-by: BuildkiteCI Reviewed-by: Brian Olsen <me@griff.name> Reviewed-by: flokli <flokli@flokli.de>
2024-03-03 r/7642 feat(tvix/store): use reqwests' rustls-native-roots featureFlorian Klink2-1/+5
This makes reqwest honor `SSL_CERT_FILE` - previously it was using the chain bundled in webpki-roots. `object_store` pulls in `reqwest` with this feature, and the cargo solver will enable that feature globally as soon as we pull it in, as it assumes features are additive. This requires setting `SSL_CERT_FILE` when running tests, otherwise they'll fail with the unhelpful "NotFound" error. This was quite some fun to debug, why adding `object_store` to tvix-castore suddenly made tvix-store tests fail! Change-Id: I64fc82b4d994715480efdb1ffecb279716456ab9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11090 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-02-20 r/7566 refactor(tvix/store/proto): use bstr::ByteSliceFlorian Klink2-1/+3
String::from_utf8_lossy simply discards invalid bytes, while bstr replaces them with their replacement character. Change-Id: Ib78ff36ca5faacc1ad60bc4ddde7b62773848c07 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10981 Tested-by: BuildkiteCI Reviewed-by: Peter Kolloch <info@eigenvalue.net> Reviewed-by: flokli <flokli@flokli.de>
2024-02-20 r/7564 fix(tvix/store): Improve error message for InvalidNodeNameAspen Smith1-1/+1
Rather than debug-formatting a Vec<u8>, which gives just a sequence of numbers (not very helpful!) use String::from_utf8_lossy to give a much more human-readable error message Change-Id: I6cae795ebe2ab9a656c087b82a143411e6c7c81a Reviewed-on: https://cl.tvl.fyi/c/depot/+/10980 Autosubmit: aspen <root@gws.fyi> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-02-17 r/7535 feat(tvix/castore/fs): make allow_other configurableFlorian Klink1-1/+8
Also add a cli argument to the `tvix-store` binary. Change-Id: Id07d7fedb60d6060543b195f3a810a46137f9ad5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10945 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su>
2024-02-05 r/7475 docs(tvix/store/pathinfosvc): document nix+http[s]Florian Klink1-0/+5
Change-Id: I5f4e93aceec373eb3377e3f8fbb55a5142cc1363 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10740 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-01-21 r/7435 chore(tvix/store): Use BoxStream type aliasConnor Brewster8-43/+27
The BoxStream type alias is a more concise and easier to read than the full `Pin<Box<dyn Stream<Item = ...> + Send + ...>>` type. Change-Id: I5b7bccfd066ded5557e01f7895f4cf5c4a33bd44 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10677 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: Connor Brewster <cbrewster@hey.com>
2024-01-20 r/7433 feat(tvix/store): enable `name` customization in the storeRyan Lahfa2-15/+18
Sometimes, Nix lets someone customize the `name` in the store for a path, this is the case for `builtins.path` which takes a `name` argument, we leave it to the caller to choose the name, which can be the basename by default of the path. Change-Id: Icdbf71d1d8f2dca5716b99d20aac885aab905b80 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10653 Tested-by: BuildkiteCI Autosubmit: raitobezarius <tvl@lahfa.xyz> Reviewed-by: flokli <flokli@flokli.de>
2024-01-20 r/7432 refactor(tvix/store): `import_path` → `import_path_as_nar_ca`Ryan Lahfa4-117/+160
Add multiple additional helpers such as: - `path_to_name`: derive the basename of a given path - `derive_nar_ca_path_info`: derive the `PathInfo` for a content addressed NAR which isolates further the tree walking feature and the ingestion feature. Additionally, we don't `expect` anymore and propagate properly ingestion errors up. Change-Id: I60edb5b633911c58ade7e19f5002e6f75f90e262 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10574 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: raitobezarius <tvl@lahfa.xyz>
2024-01-12 r/7371 refactor(tvix/store/pathinfoservice): make more genericFlorian Klink6-54/+51
We don't need Arcs in most of the cases, we're fine with some container. Change-Id: Ic4f8acb5b9d93e2b0923bb607463fb91e9d0e4fe Reviewed-on: https://cl.tvl.fyi/c/depot/+/10606 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-12 r/7370 refactor(tvix/store/nar/renderer): don't require Arc, Clone or SyncFlorian Klink1-28/+45
To render NARs, we're fine with a simple AsRef to a BlobService and DirectoryService. We just need to have the function pass back the references, so we can reuse it after the recursion. Change-Id: I8a1b899134ddda26cf14aa829a08383986101850 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10605 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-01-09 r/7362 refactor(tvix/store/nar/import): use AsRefFlorian Klink1-25/+26
We need to be a bit careful and pass the BlobService around (similar to how we already do with the directory_putter), but that allows getting rid of a bunch of annoying trait bounds. We also stop spawning additional tasks where we can just use block_on. Change-Id: If36de0ee947d2c779d20a384308241d2262d4764 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10580 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-01-09 r/7359 refactor(tvix): use AsRef<dyn …> instead of Deref<Target= …>Florian Klink2-17/+15
Removes some more needs for Arcs. Change-Id: I9a9f4b81641c271de260e9ffa98313a32944d760 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10578 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-05 r/7350 chore(tvix): bump test-case dep to 3.3.1Florian Klink1-1/+1
Change-Id: I643548d95a5fab84563c7cbe51ca2ce640c186a9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10537 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-01-05 r/7348 feat(tvix/store/pathinfosvc): call validateFlorian Klink1-1/+9
Change-Id: I557ed67a04112ba3e363303637d1988309250a93 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10535 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-05 r/7346 refactor(tvix/castore): relax trait bounds for DSFlorian Klink1-4/+5
Make this an `AsRef<dyn DirectoryService>`. This helps dropping some Clone requirements. Unfortunately, we can't thread this through to TvixStoreIO just yet. Change-Id: I3f07eb28d6c793d3313fe21506ada84d5a8aa3ac Reviewed-on: https://cl.tvl.fyi/c/depot/+/10533 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-03 r/7315 fix(tvix/store): set service.nameFlorian Klink1-24/+44
While this can be influenced with `OTEL_SERVICE_NAME` or `OTEL_RESOURCE_ATTRIBUTES`, "unknown_service" is an annoying name. Let's pick tvix.store as a default. Change-Id: I9b5c45703d9546775bb9266b29ab54daf7c54a9f Reviewed-on: https://cl.tvl.fyi/c/depot/+/10524 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2024-01-01 r/7308 feat(tvix/store): add opentelemetryFlorian Klink2-7/+43
This is behind the otlp feature flag (currently enabled by default). By default, this will try to push traces to a OTLP collector running at https://localhost:4317. You can make one available by running: ``` docker run -d --name jaeger \ -e COLLECTOR_ZIPKIN_HOST_PORT=:9411 \ -e COLLECTOR_OTLP_ENABLED=true \ -p 6831:6831/udp \ -p 6832:6832/udp \ -p 5778:5778 \ -p 16686:16686 \ -p 4317:4317 \ -p 4318:4318 \ -p 14250:14250 \ -p 14268:14268 \ -p 14269:14269 \ -p 9411:9411 --rm \ jaegertracing/all-in-one:1.42 ``` Started like that, jaeger brings a web interface at http://localhost:16686/search As documented in https://docs.rs/opentelemetry-otlp/latest/opentelemetry_otlp/, you can point this to another location by setting `OTEL_EXPORTER_OTLP_ENDPOINT`. Change-Id: Id1dca367d70027b2ea98bb70bcf99a68363ec2be Reviewed-on: https://cl.tvl.fyi/c/depot/+/8194 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de> Reviewed-by: aaqaishtyaq <aaqaishtyaq@gmail.com>
2024-01-01 r/7307 refactor(tvix/castore/tests): let gen_*_service return BoxesFlorian Klink5-22/+41
Only convert to and reuse an Arc<…> where needed. Change-Id: I2c1bc69cca5a4a3ebd3bdb33d6e28e1f5fb86cb9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10514 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-01 r/7306 refactor(tvix/store/nar/import): be a bit more genericFlorian Klink1-29/+40
Change-Id: If9a536949f36f428abea1a893f937fe7063e2f41 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10517 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-01 r/7305 refactor(tvix/castore/blobsvc/grpc/wrapper): don't require Arc<_>Florian Klink1-1/+1
Change-Id: I9655f5588c7dc98427de6af47d74b4ab7ce22071 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10516 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-01 r/7304 refactor(tvix/castore/directorysvc/grpc/wrapper): no Arc<_>Florian Klink1-1/+1
We can also drop the Clone requirement. Because the trait is async since some time, there's no need to clone before moving into an async closure, allowing us to simplify the code a bit. Change-Id: I9b0a0e10077d8c548d218207b908bfd92c5b8de0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10515 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de>
2024-01-01 r/7301 feat(tvix/store/bin): factor out importFlorian Klink2-113/+132
While at it, make it a bit more generic. Change-Id: Ic4caefda93aca3ffb656a09f8b4d648b41415532 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10511 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-01-01 r/7299 feat(nix-compat/store_path): build_ca_path may failFlorian Klink1-1/+9
Change-Id: Ia74ee870f38b7966501458bace541092256c3213 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10509 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-01-01 r/7298 refactor(tvix/store/bin): don't reconstruct StorePath twiceFlorian Klink1-8/+1
We already have that data in output_path. Change-Id: Iadf615551f0b44d539098446ff5926994deb606d Reviewed-on: https://cl.tvl.fyi/c/depot/+/10508 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-12-31 r/7297 fix(tvix/store/bin): don't unwrap in case of invalid pathsFlorian Klink1-8/+12
Instead, return an error, and move the entire check before starting to ingest the data underneath. Change-Id: Idcfba115cb7d599f5fc72a156aaad9d4d4714fcf Reviewed-on: https://cl.tvl.fyi/c/depot/+/10507 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-12-31 r/7296 refactor(tvix/store/bin): condense subscriber setup a bitFlorian Klink1-12/+8
We can use cli.json.then_some(…) to create a Some(…), allowing us to omit the else { None } lines. Change-Id: I6c8142a08d8cb88d6c8302e5ca7570698fcf2aa3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10505 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-12-31 r/7295 refactor(tvix/store): move construct_services helper hereFlorian Klink3-62/+67
This takes three URLs, and constructs Arc'ed {Blob,Directory,PathInfo}Service, allowing to remove some of the boilerplate. Change-Id: I40e7c2b551442ef2acdc543dfc87ab97e7c742bb Reviewed-on: https://cl.tvl.fyi/c/depot/+/10484 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-12-31 r/7294 refactor(tvix/castore/directorysvc): return Box, not ArcFlorian Klink1-4/+17
While we currently mostly use it in an Arc, as we need to clone it inside PathInfoService, there might be other usecases not requiring it to be Clone. Change-Id: Ia05bb370340792a048e2036be30e285ef1e63870 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10483 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-12-31 r/7293 refactor(tvix/castore/blobsvc): return Box, not ArcFlorian Klink1-4/+9
While we currently mostly use it in an Arc, as we need to clone it inside PathInfoService, there might be other usecases not requiring it to be Clone. Change-Id: I7bd337cd2e4c2d4154b385461eefa62c9b78345d Reviewed-on: https://cl.tvl.fyi/c/depot/+/10482 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-12-29 r/7268 refactor(tvix/castore/bin/tvix-store) don't import std::ioFlorian Klink1-5/+4
Make it more clear this is not tokio::io. Change-Id: Ic2fa56f0baf1c200b6631098d556388a19629a45 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10452 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-12-29 r/7266 feat(tvix/store): emit more structured log messagesFlorian Klink1-3/+3
Change-Id: If4748cb800276fc42779ba8973d1eef72df0503e Reviewed-on: https://cl.tvl.fyi/c/depot/+/10450 Reviewed-by: edef <edef@edef.eu> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-12-24 r/7261 chore(tvix): switch to upstream futures 0.3.30Florian Klink1-1/+1
The bugs have been fixed, https://github.com/rust-lang/futures-rs/pull/2801 and https://github.com/rust-lang/futures-rs/pull/2812 were merged and ended up in that release. Change-Id: Iefd990d2d1719b884504093343e54e9c5258e2e2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10414 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-12-24 r/7258 refactor(tvix/castore): add RootNode impl for BTreeMap, mv fs testsFlorian Klink3-1177/+0
cl/10378 did already move store/fs to castore/fs, but we kept the tests in tvix-store, as they were populating a PathInfoService to make nodes appear in the mount root. Update these tests to now just insert root nodes into a BTreeMap, and ensure we can use that as a RootNodes too. Change-Id: Iad7d1ee4f9423eb6e3a1da33f433842c9ae0de1f Reviewed-on: https://cl.tvl.fyi/c/depot/+/10410 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-12-22 r/7256 chore(tvix): move store/fs to castore/fsFlorian Klink13-1422/+111
With the recent introduction of the RootNodes trait, there's nothing in the fs module pulling in tvix-store dependencies, so it can live in tvix-castore. This allows other crates to make use of TvixStoreFS, without having to pull in tvix-store. For example, a tvix-build using a fuse mountpoint at /nix/store doesn't need a PathInfoService to hold the root nodes that should be present, but just a list. tvix-store now has a pathinfoservice/fs module, which contains the necessary glue logic to implement the RootNodes trait for a PathInfoService. To satisfy Rust orphan rules for trait implementations, we had to add a small wrapper struct. It's mostly hidden away by the make_fs helper function returning a TvixStoreFs. It can't be entirely private, as its still leaking into the concrete type of TvixStoreFS. tvix-store still has `fuse` and `virtiofs` features, but they now simply enable these features in the `tvix-castore` crate they depend on. The tests for the fuse functionality stay in tvix-store for now, as they populate the root nodes through a PathInfoService. Once above mentioned "list of root nodes" implementation exists, we might want to shuffle this around one more time. Fixes b/341. Change-Id: I989f664827a5a361b23b34368d242d10c157c756 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10378 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2023-12-22 r/7255 refactor(tvix/store): remove Arc<> from PathInfoService::from_addrFlorian Klink5-24/+37
This makes PathInfoService::from_addr return a Box<dyn PathInfoService>, rather than an Arc<dyn …>, and leaves it up to the consumers to rewrap it into an Arc where needed. This allows us to drop the Arc for the tvix-store daemon subcommand. Change-Id: Ic83aa2ade6c51912281bd17c7eef7252e152b2d1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10409 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-11-30T15·09+0000