about summary refs log tree commit diff
path: root/tvix/castore/src/proto/mod.rs
AgeCommit message (Collapse)AuthorFilesLines
2024-08-13 r/8486 refactor(tvix/castore): move *Node and Directory to crate rootFlorian Klink1-65/+51
*Node and Directory are types of the tvix-castore model, not the tvix DirectoryService model. A DirectoryService only happens to send Directories. Move types into individual files in a nodes/ subdirectory, as it's gotten too cluttered in a single file, and (re-)export all types from the crate root. This has the effect that we now cannot poke at private fields directly from other files inside `crate::directoryservice` (as it's not all in the same file anymore), but that's a good thing, it now forces us to go through the proper accessors. For the same reasons, we currently also need to introduce the `rename` functions on each *Node directly. A followup is gonna move the names out of the individual enum kinds, so we can better represent "unnamed nodes". Change-Id: Icdb34dcfe454c41c94f2396e8e99973d27db8418 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12199 Reviewed-by: yuka <yuka@yuka.dev> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-08-13 r/8484 refactor(tvix/castore): use Directory struct separate from proto oneYureka1-369/+190
This uses our own data type to deal with Directories in the castore model. It makes some undesired states unrepresentable, removing the need for conversions and checking in various places: - In the protobuf, blake3 digests could have a wrong length, as proto doesn't know fixed-size fields. We now use `B3Digest`, which makes cloning cheaper, and removes the need to do size-checking everywhere. - In the protobuf, we had three different lists for `files`, `symlinks` and `directories`. This was mostly a protobuf size optimization, but made interacting with them a bit awkward. This has now been replaced with a list of enums, and convenience iterators to get various nodes, and add new ones. Change-Id: I7b92691bb06d77ff3f58a5ccea94a22c16f84f04 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12057 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-07-20 r/8375 feat(tvix/castore/protos): return root node after validationFlorian Klink1-2/+4
This allows avoiding a `.node.unwrap()`` after validation. Change-Id: Ieef1ffebab16cdca94c979ca6831a7ab4f6007da Reviewed-on: https://cl.tvl.fyi/c/depot/+/11989 Reviewed-by: Brian Olsen <me@griff.name> Tested-by: BuildkiteCI
2024-05-01 r/8056 feat(tvix/castore/path): use proto::validate_node_nameedef1-1/+1
Use the shared code for validating node names, since that is what path components represent. Change-Id: I12109c1306b224718faa66cf1f2874c78c1436a7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11566 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-04-19 r/7978 feat(tvix/castore): add convenience `add` method to DirectoryConnor Brewster1-0/+86
This adds `Directory::add` which is a convenience helper for adding nodes into a `Directory` while preserving sorted order. This implements `Ord` and `PartialOrd` for `FileNode`, `SymlinkNode`, and `DirectoryNode` so `binary_search` can be used. Change-Id: I94b86bdef5d0da55aa352e098988b9704cafca19 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11481 Autosubmit: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-03-20 r/7757 feat(tvix/castore): derive Eq and Hash automaticallyFlorian Klink1-3/+1
This allows these messages to be put in HashSets. Change-Id: Ia58094cafe53eb624578821d3d8d969c5d21a1d7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11219 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de>
2024-02-02 r/7470 feat(tvix/castore/blobsvc): validate StatBlobResponseFlorian Klink1-0/+25
All chunks must have valid blake3 digests. It is allowed to send an empty list, if no more granular chunking is available. Change-Id: I7ecb53579cdf40fd938bb68a85685751b4d3626f Reviewed-on: https://cl.tvl.fyi/c/depot/+/10726 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de>
2024-01-15 r/7380 feat(tvix/castore): implement Ord for node::NodeFlorian Klink1-0/+14
This allows assembling BTreeSets of node::Node. Change-Id: I97b83be5ffc3e891307a8ef2b5fc31e38b747a62 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10625 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de>
2024-01-05 r/7347 fix(tvix/castore): validate Option<Node>Florian Klink1-1/+15
Extend our validation function to also check for the None case. Change-Id: Ib75f880646d7fb3d66588f1988e61ec18be816a2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10534 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-11-05 r/6946 feat(tvix/castore): bump [Directory,File]Node size to u64Florian Klink1-6/+6
Having more than 4GiB files is quite possible (think about the NixOS graphical installer, and an uncompressed iso of it). No wire format changes. Change-Id: Ia78a07e4c554e91b93c5b9f8533266e4bd7f22b6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9950 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-10-27 r/6894 feat(tvix/castore): carry name in ValidateNodeError::InvalidNameedef1-9/+9
Change-Id: Ica288e94f3f6025d98ef7d56dc5d6f874ec921b7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9861 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-10-27 r/6893 refactor(tvix): turn nullary enum variants into unit variantsedef1-2/+2
Change-Id: Iad4f2cb4aa92b5bb29ead6050348a8cd3e7b8632 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9860 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-10-16 r/6824 refactor(tvix/castore): clippyFlorian Klink1-1/+1
Change-Id: I8a792d6b238190a78390c5d8f638669a2681f70f Reviewed-on: https://cl.tvl.fyi/c/depot/+/9746 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-10-14 r/6798 refactor(tvix/castore): simplify node validation checksFlorian Klink1-9/+8
We can just check the digest length to be correct, and move the symlink target checks to a single line. Change-Id: I41d2e3a50e7990ef6c04f02acd754b1e17b43e77 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9717 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-10-12 r/6794 refactor(tvix/castore): factor out node checksFlorian Klink1-26/+64
Implement `validate()` on `node::Node`, and call it from PathInfo's validate() too. Node-related errors are moved to a ValidateNodeError error type. This additionally adds some more validations for symlink targets (they must not be empty, and not contain null bytes). Change-Id: Ib9b89f1c9c795e868a1533281239bc8a36d97c5d Reviewed-on: https://cl.tvl.fyi/c/depot/+/9715 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: edef <edef@edef.eu>
2023-10-10 r/6778 fix(tvix/castore): use bstr for formatting names in errorsedef1-4/+4
Much friendlier than either Base64 or raw byte slices. Change-Id: I9b4cdd57c83ddc76c0be8103da4320207657a72b Reviewed-on: https://cl.tvl.fyi/c/depot/+/9622 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-10-10 r/6777 fix(tvix/castore): handle Directory::size overflow explicitlyedef1-6/+24
We use checked arithmetic for computing the total size, and verify that size is in-bounds in Directory::validate. If an out-of-bounds size makes it to the "unchecked" size method, we either panic (in debug mode), or silently saturate to u32::MAX. No new panic sites are added, since overflows in debug mode already panic at the language level. Change-Id: I95b8c066a42614fa447f08b4f8fe74e16fbe8bf9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9616 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-09-26 r/6660 chore(tvix/[ca]store): allow building without tonic-reflectionFlorian Klink1-1/+1
- rename the feature to `tonic-reflection` in both tvix-store and tvix- castore. - set it to disabled in tvix-castore by default - enable it in tvix-store, and pull in tvix-castore with the feature enabled. Change-Id: Ie22833d85569502cae55812f6eeb17a9c15b9e2c Reviewed-on: https://cl.tvl.fyi/c/depot/+/9472 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-09-22 r/6629 refactor(tvix): move castore into tvix-castore crateFlorian Klink1-0/+279
This splits the pure content-addressed layers from tvix-store into a `castore` crate, and only leaves PathInfo related things, as well as the CLI entrypoint in the tvix-store crate. Notable changes: - `fixtures` and `utils` had to be moved out of the `test` cfg, so they can be imported from tvix-store. - Some ad-hoc fixtures in the test were moved to proper fixtures in the same step. - The protos are now created by a (more static) recipe in the protos/ directory. The (now two) golang targets are commented out, as it's not possible to update them properly in the same CL. This will be done by a followup CL once this is merged (and whitby deployed) Bug: https://b.tvl.fyi/issues/301 Change-Id: I8d675d4bf1fb697eb7d479747c1b1e3635718107 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9370 Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>