about summary refs log tree commit diff
path: root/tvix/castore/src/directoryservice/memory.rs
AgeCommit message (Collapse)AuthorFilesLines
2024-11-08 r/8896 feat(tvix): add instance_name to instrumentation of *ServicesBob van der Linden1-6/+10
Currently it is not possible to distinguish between tracing of the same *Service type whenever there are multiple of them. Now the instance_name of ServiceBuilder is passed into the *Service and used in the existing instrument as the `instance_name` field. Places that did not already have a instance_name in its context use `"default"`. In tests I used `"test"`. Change-Id: Ia20bf2a7bb849a781e370d087ba7ddb3be79f654 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12739 Tested-by: BuildkiteCI Autosubmit: Bob van der Linden <bobvanderlinden@gmail.com> Reviewed-by: flokli <flokli@flokli.de>
2024-08-13 r/8484 refactor(tvix/castore): use Directory struct separate from proto oneYureka1-24/+10
This uses our own data type to deal with Directories in the castore model. It makes some undesired states unrepresentable, removing the need for conversions and checking in various places: - In the protobuf, blake3 digests could have a wrong length, as proto doesn't know fixed-size fields. We now use `B3Digest`, which makes cloning cheaper, and removes the need to do size-checking everywhere. - In the protobuf, we had three different lists for `files`, `symlinks` and `directories`. This was mostly a protobuf size optimization, but made interacting with them a bit awkward. This has now been replaced with a list of enums, and convenience iterators to get various nodes, and add new ones. Change-Id: I7b92691bb06d77ff3f58a5ccea94a22c16f84f04 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12057 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-07-19 r/8372 feat(tvix): different service types in one compositionYureka1-1/+1
This will be necessary for the PathInfoService composition, as some PathInfoService implementations require a BlobService & DirectoryService to ingest into. Using the Extend trait for creating compositions allows extending the same composition with configs of various types e.g. BlobStore, DirectoryStore Generics are moved from the Composition struct to the functions.The storage of the InstantiatonStates uses the TypeId in the key and a Box<dyn Any> in the value, which is downcasted to InstantiatonState<T>. Change-Id: I2af11f26c535029adfb1c62905e0e7c4aaed7b51 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11980 Reviewed-by: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Autosubmit: yuka <yuka@yuka.dev>
2024-07-18 r/8368 refactor(tvix): use composition & registry for from_addrYureka1-0/+11
Change-Id: I3c94ecb5958294b5973c6fcdf5ee9c0d37fa54ad Reviewed-on: https://cl.tvl.fyi/c/depot/+/11976 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: yuka <yuka@yuka.dev>
2024-07-18 r/8365 feat(tvix/castore): add composition moduleYureka1-0/+17
Change-Id: I0868f3278db85ae5fe030089ee9033837bc08748 Signed-off-by: Yureka <tvl@yuka.dev> Reviewed-on: https://cl.tvl.fyi/c/depot/+/11853 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-05-13 r/8142 feat(tvix/castore/directory/get_recursive): add 'static to BoxStreamFlorian Klink1-1/+1
Necessary to directly use this in the GRPC DirectoryService wrapper directly. Change-Id: Ic6a0038a40dc30071d145af5035345fcd93288ae Reviewed-on: https://cl.tvl.fyi/c/depot/+/11634 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-05-11 r/8114 refactor(tvix/castore/directoryservice/memory): use tokio RwLockFlorian Klink1-3/+4
We don't want to block here, and this also means there's no poisoning to deal with. Change-Id: Ic375571970c48beace0005ae2c012135086a4d67 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11613 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-03-24 r/7773 refactor(tvix/castore/directory): have SimplePutter use ValidatorFlorian Klink1-2/+2
This simplifies a bunch of code, and gets rid of some TODOs. Also, move it out of castore/utils, and into its own file. Change-Id: Ie63e05a6cdfb2a73e878cf7107f9172aed1cdf13 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11224 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de>
2024-01-21 r/7435 chore(tvix/store): Use BoxStream type aliasConnor Brewster1-3/+2
The BoxStream type alias is a more concise and easier to read than the full `Pin<Box<dyn Stream<Item = ...> + Send + ...>>` type. Change-Id: I5b7bccfd066ded5557e01f7895f4cf5c4a33bd44 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10677 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: Connor Brewster <cbrewster@hey.com>
2023-11-15 r/7021 refactor(tvix/castore): remove DirectoryService::from_urlFlorian Klink1-62/+0
Make directoryservice::from_addr use the more specific constructors. Change-Id: I9fee2afed77692505988d631d9fe246d9843d25a Reviewed-on: https://cl.tvl.fyi/c/depot/+/10029 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-09-22 r/6629 refactor(tvix): move castore into tvix-castore crateFlorian Klink1-0/+149
This splits the pure content-addressed layers from tvix-store into a `castore` crate, and only leaves PathInfo related things, as well as the CLI entrypoint in the tvix-store crate. Notable changes: - `fixtures` and `utils` had to be moved out of the `test` cfg, so they can be imported from tvix-store. - Some ad-hoc fixtures in the test were moved to proper fixtures in the same step. - The protos are now created by a (more static) recipe in the protos/ directory. The (now two) golang targets are commented out, as it's not possible to update them properly in the same CL. This will be done by a followup CL once this is merged (and whitby deployed) Bug: https://b.tvl.fyi/issues/301 Change-Id: I8d675d4bf1fb697eb7d479747c1b1e3635718107 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9370 Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>