about summary refs log tree commit diff
path: root/tvix/store (follow)
AgeCommit message (Collapse)AuthorFilesLines
2022-12-26 r/5495 feat(tvix/store/protos): add go_package optionFlorian Klink5-0/+10
Change-Id: I0898b8a0a78e704219da38e5acaabef1e640d4e4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7321 Reviewed-by: Adam Joseph <adam@westernsemico.com> Tested-by: BuildkiteCI
2022-12-26 r/5494 docs(tvix/store): add README, document services and store modelFlorian Klink4-2/+391
These are intended to help digest the protocol definitions for tvix- store, and how they tie into the whole concept. Co-Authored-By: Vincent Ambo <mail@tazj.in> Change-Id: Ic1ba3ba41ef599209453f15d0ac2e07a6144bcca Reviewed-on: https://cl.tvl.fyi/c/depot/+/7439 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-12-26 r/5492 feat(tvix/store): use tonic-build for protosFlorian Klink3-17/+22
While prost-build already exposes protobuf message types as structs, we actually need tonic-build too, to be able to get traits for all the RPC services defined in the proto files. Change-Id: I7f4c08454bf0d280d577975c7cdae13ccc2d933b Reviewed-on: https://cl.tvl.fyi/c/depot/+/7320 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-12-22 r/5471 feat(tvix/proto): add rpc_directory.protoFlorian Klink3-0/+48
This provides an interface to retrieve and upload single Directory messages, or a DAG of them. Change-Id: Id9e7084bd697d843649a122da2c992a3e36d808c Reviewed-on: https://cl.tvl.fyi/c/depot/+/7137 Tested-by: BuildkiteCI Reviewed-by: Adam Joseph <adam@westernsemico.com>
2022-12-22 r/5470 feat(tvix/proto): add rpc_pathinfoFlorian Klink3-0/+44
This defines a service that can be used to upload and retrieve metadata of nix paths. Change-Id: Id86eb531ce4ae316adb15934b0d1386a14ba2132 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7136 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: Adam Joseph <adam@westernsemico.com>
2022-12-15 r/5422 feat(tvix/proto): add rpc_blobstoreFlorian Klink3-0/+34
This defines a service that can be used to get and put content-addressed chunks of data. Change-Id: I36cf2278ed1daf71848c04fdfd14450b2268c5de Reviewed-on: https://cl.tvl.fyi/c/depot/+/7135 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-12-15 r/5420 refactor(tvix): build Rust projects using crate2nixVincent Ambo1-9/+17
Introduces granular dependency builds using crate2nix, bootstrapped off the generated configuration from the newly introduced workspace (see cl/7533). This commit checks in the generated Cargo.nix file which can be regenerated with a parameterless invocation of `crate2nix generate` in `//tvix`. I tried generating this in IFD, but it turned out to be harder than what seemed worthwhile for now. In this setup, the various build targets for Rust projects end up being attributes of the imported `Cargo.nix` file at the `tvix.crates` attribute. These still lack configuration, however, which has been fixed in the various `default.nix` files of individual projects. Note that we (temporarily) lose the ability to build tvix-eval's benchmarks in CI. I haven't figured out what magic incantation summons them from the void again ... The `eval-okay-readDir` tests from both test suites have been disabled because they fail for unknown reasons when run in this new derivation. Somebody will have to debug it! Change-Id: I2014614ccb9c8951aedbd71df7966ca191a13695 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7538 Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2022-12-15 r/5419 refactor(tvix): share a Cargo.lock file between Rust projectsVincent Ambo2-525/+1
This relates to the (abandoned) cl/7256. Introduces a Cargo workspace at //tvix that is primarily intended to be used as a workaround for the annoying Nix+Rust tooling while having a consistent set of dependencies. This is driven in part by a desire to adopt crate2nix and get more granular Nix builds for Tvix's Rust projects, and in part by a need to split //tvix/eval into something providing the CLI (REPL etc.), and a library providing eval, without significantly altering the structure of build targets. To accomplish this the workspace has been designed to allow projects to remain independent build targets. I want to avoid lumping all the projects together - something like //tvix/eval should always be independent of other parts of tvix. A helper function in //tvix/default.nix lets downstream naersk projects construct a sparse root for the project which combines the workspace's `Cargo.lock` with the project's own `Cargo.toml`. Note that cargo commands in the workspace itself require the build dependencies of _all_ projects to be present, which is currently a bit annoying to accomplish. This introduces some breakage: 1. It breaks usage of rust-analyser without being in a shell with the dependencies of *all* Tvix projects, as it is not capable of respecting only the subset of dependencies for a part of the workspace. 2. It is no longer possible to run tests using `cargo test`, as the test generation crate we use does not work with workspaces: https://github.com/frehberg/test-generator/issues/6 This still works in the Nix build as we construct a Cargo project that looks like it's not in a workspace there. Until somebody fixes that crate / writes a new macro / does something else with the test suite, the way to run the tests is through the Nix build. Long-term we'll probably want to get rid of cargo completely, it's just a big wart and most tooling works without it if correctly configured, but we don't have time for that now. Change-Id: I846bff7a8429a25c077fd1e9ef4e3c34a299a4a1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7533 Reviewed-by: flokli <flokli@flokli.de> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-12-04 r/5386 feat(tvix/store): add size() and digest() functions for DirectoryFlorian Klink5-3/+307
Change-Id: I3a9e3c77da1d5d7bfb93da0cd7bce8d48dbc8cd4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7254 Reviewed-by: Adam Joseph <adam@westernsemico.com> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-12-04 r/5385 feat(tvix/store): build rust proto filesFlorian Klink4-0/+343
This uses prost-build to build the proto files. Change-Id: Ia99fcfa4a19c741683cf28a45202e1d9211f9131 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7286 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: Adam Joseph <adam@westernsemico.com>
2022-12-04 r/5384 feat(tvix/store): initFlorian Klink4-0/+35
Change-Id: Id846647fa456ff8b187ed4f16d035ce030f11d25 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7264 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org> Reviewed-by: tazjin <tazjin@tvl.su>
2022-12-04 r/5383 feat(tvix/store): add pathinfo.protoFlorian Klink1-0/+74
This adds the PathInfo message, which hosts information about a Nix Store path, mapping to either of a {Directory,File,Symlink}Node. Change-Id: I79d871b6fad450d6a4ae4101fb72c51f9a83471f Reviewed-on: https://cl.tvl.fyi/c/depot/+/7132 Reviewed-by: Adam Joseph <adam@westernsemico.com> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-12-04 r/5382 chore(tvix/store): move castore.protoFlorian Klink2-0/+80
This moves the castore.proto file into the //tvix/store/protos directory. Per- component protos might make more sense, than a "tvix-wide" proto directory. Change-Id: Ie728210174b041e9285a0e2ac605d715d0f0cbda Reviewed-on: https://cl.tvl.fyi/c/depot/+/7285 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI