about summary refs log tree commit diff
path: root/tvix/castore/protos
AgeCommit message (Collapse)AuthorFilesLines
2024-08-19 r/8538 feat(tvix): drop usage of sparseTree in favor of lib.sourceByRegexIlan Joselevich1-12/+6
We can avoid depending on things outside //tvix by just using a similar util from nixpkgs. Change-Id: I9ea3e1f0a8a059ea10caaec173569ba9f316aec6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12247 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-02-20 r/7570 docs(tvix/castore/directorysvc): K/V is not necessarily flatFlorian Klink1-0/+5
Some implementations of DirectoryService might not allow retrieval of intermediate Directory nodes, that are not at the "root". Think about an object store implementation. The client is doing a get_recursive anyways to reduce the number of roundtrips. By documenting the fact we don't need to support looking up intermediate Directory messages, we can just batch all directories into the same object, keyed by the root. Change-Id: I019d720186d03c4125cec9191e93d20586a20963 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10988 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-12-21 r/7252 docs(tvix/castore/protos): remove referenceFlorian Klink1-1/+1
This is not gonna end up as a interlinked docstring. Change-Id: I2b0ca106aa75bae0156c0b411da5931da60c725d Reviewed-on: https://cl.tvl.fyi/c/depot/+/10406 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI
2023-12-11 r/7155 refactor(tvix/*/protos): separate lint targetFlorian Klink1-15/+35
Break up the go-bindings derivation. Keep "protos" containing all proto files (well, and the buf config), and use it for a check phase running linter and formatter, as well as the existing "go-bindings" attribute Change-Id: I52cb9d08570bb76452acb831eb711c5b6c0eacfb Reviewed-on: https://cl.tvl.fyi/c/depot/+/10239 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-12-11 r/7149 chore(tvix/castore/protof): buf formatFlorian Klink4-89/+89
Change-Id: Idf11de78b0d6eca69fda34a89f2c57a00ed89ad5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10237 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Adam Joseph <adam@westernsemico.com>
2023-11-05 r/6954 feat(tvix/castore): extend blobstore protos for verified streamingFlorian Klink1-12/+45
This pdates the proto docstrings a bit, especially w.r.t. verified streaming. It also adds send_chunks, send_bao fields to StatBlobRequest (renamed from BlobMeta) Change-Id: I590cc8646d86b73bca9f38a9b6d9ea15e4df5cb6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9951 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-11-05 r/6946 feat(tvix/castore): bump [Directory,File]Node size to u64Florian Klink1-2/+2
Having more than 4GiB files is quite possible (think about the NixOS graphical installer, and an uncompressed iso of it). No wire format changes. Change-Id: Ia78a07e4c554e91b93c5b9f8533266e4bd7f22b6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9950 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-10-17 r/6847 chore(tvix/castore-go): rename go moduleFlorian Klink3-3/+3
`code.tvl.fyi/tvix/castore/protos` now points to a directory that only contains the `.proto` files, while all golang tooling and .pb.go files live in tvix/castore-go. As discussed in https://cl.tvl.fyi/c/depot/+/9787/comment/fc5d155c_1bd38e3a/, the amount of people currently using this is still small, so rename the go.mod now, while it doesn't yet hurt. Change-Id: Ib3c6a2dac2923b3806ebb05be00af66d0da9f698 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9791 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-10-17 r/6843 chore(tvix): move castore golang bindings to tvix/castore-goFlorian Klink11-2448/+33
Have `tvix/castore/protos` only contain the protos, no go noise. Make the `.pb.go` file generation a pure Nix build at `//tvix/castore/protos:go-bindings`, and have a script at `//tvix:castore-go-generate` (TBD) that copies the results to `tvix/castore-go`. `//tvix:castore-go`, with sources in `tvix/castore-go` now contains the tooling around the generated bindings, and the generated bindings themselves (So go mod replace workflows still work). An additional CI step is added from there to ensure idempotenty of the .pb.go files. The code.tvl.fyi webserver config is updated to the new source code path. I'm still unsure if we want to also update the go.mod name. While being a backwards-incompatible change, it'll probbaly make it easier where to find these files, and the amount of external consumers is still low enough. Part of b/323. Change-Id: I2edadd118c22ec08e57c693f6cc2ef3261c62489 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9787 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2023-10-14 r/6799 feat(tvix/castore/protos): add more granular validation methodsFlorian Klink2-29/+104
Similar to cl/9715, this makes the validation checks more granular, introducing a Validate on all *Node. A check for symlink targets is added too. Once merged, it can also be used from tvix/store/protos. Change-Id: I0909a89fadcd74b74ef0c9a8a1f22658fccc83b0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9716 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2023-10-09 r/6761 chore(tvix/*store): use google.golang.org/protobufFlorian Klink2-6/+12
The other one is deprecated, according to https://pkg.go.dev/github.com/golang/protobuf/proto. Change-Id: I502d5bfa7431df0dea3d564f1e0ecfea14c90e10 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9597 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Brian McGee <brian@bmcgee.ie>
2023-10-09 r/6760 feat(tvix/castore): add RenamedNode helperFlorian Klink1-0/+38
This takes a castorev1pb.Node and updates its name, making sure to copy it. Change-Id: I57bf2b2b4a74496a1a697bcbce72bf67a12e69fe Reviewed-on: https://cl.tvl.fyi/c/depot/+/9594 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Brian McGee <brian@bmcgee.ie>
2023-09-22 r/6629 refactor(tvix): move castore into tvix-castore crateFlorian Klink13-0/+2522
This splits the pure content-addressed layers from tvix-store into a `castore` crate, and only leaves PathInfo related things, as well as the CLI entrypoint in the tvix-store crate. Notable changes: - `fixtures` and `utils` had to be moved out of the `test` cfg, so they can be imported from tvix-store. - Some ad-hoc fixtures in the test were moved to proper fixtures in the same step. - The protos are now created by a (more static) recipe in the protos/ directory. The (now two) golang targets are commented out, as it's not possible to update them properly in the same CL. This will be done by a followup CL once this is merged (and whitby deployed) Bug: https://b.tvl.fyi/issues/301 Change-Id: I8d675d4bf1fb697eb7d479747c1b1e3635718107 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9370 Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>