depot/tvix/store/default.nix, branch refs/r/5866 monorepo for the virus lounge http://code.tvl.fyi/depot/atom?h=refs%2Fr%2F5866 2023-01-06T17:57:06+00:00 refactor(tvix/store): move protobuf build config one level up 2023-01-06T17:57:06+00:00 Vincent Ambo mail@tazj.in 2023-01-06T17:20:44+00:00 urn:sha1:3e03e59893b77b0b3915224967b377f6e940670d This embeds the build config directly at the point where `Cargo.nix` is imported, making it transparent to library consumers. Change-Id: I5586e12f02ed14587c32d9ef7d93f079366fb127 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7780 Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI chore(tvix/store): make importable 2023-01-03T13:03:07+00:00 Florian Klink flokli@flokli.de 2023-01-02T13:37:08+00:00 urn:sha1:ceb2c0ba895554c7cabb0ac20d3a80ea2aba1ab1 This allows other crates to import tvix_store. Rename the bin crate to tvix-store-bin, to avoid having multiple crates with the same name (https://github.com/rust-lang/cargo/issues/6313) Change-Id: I857768d6115640dbf102e79ed03e8474090df2fe Reviewed-on: https://cl.tvl.fyi/c/depot/+/7728 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> feat(tvix/store): implement reflection 2022-12-30T20:25:09+00:00 Florian Klink flokli@flokli.de 2022-11-26T01:14:02+00:00 urn:sha1:0bf2b0ef1164aae0ad692066e8cfc0b243a89e4d This implements grpc.reflection.v1alpha.ServerReflection, and will make tools like evans automatically discover available services, without having to specify the path to the .proto files client-side. It's behind a reflection feature flag, which is enabled by default. Change-Id: Icbcb5eb05ceede5b9952e38a2ba72eaa6fa8a437 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7435 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI feat(tvix/proto): add rpc_directory.proto 2022-12-22T14:35:32+00:00 Florian Klink flokli@flokli.de 2022-10-30T11:57:07+00:00 urn:sha1:2e04f68df8c69034151cce5b56ceedb0653687cd This provides an interface to retrieve and upload single Directory messages, or a DAG of them. Change-Id: Id9e7084bd697d843649a122da2c992a3e36d808c Reviewed-on: https://cl.tvl.fyi/c/depot/+/7137 Tested-by: BuildkiteCI Reviewed-by: Adam Joseph <adam@westernsemico.com> feat(tvix/proto): add rpc_pathinfo 2022-12-22T14:35:32+00:00 Florian Klink flokli@flokli.de 2022-10-29T11:28:04+00:00 urn:sha1:870cd533dd98e708f9870a8c85577681697db372 This defines a service that can be used to upload and retrieve metadata of nix paths. Change-Id: Id86eb531ce4ae316adb15934b0d1386a14ba2132 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7136 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: Adam Joseph <adam@westernsemico.com> feat(tvix/proto): add rpc_blobstore 2022-12-15T19:10:26+00:00 Florian Klink flokli@flokli.de 2022-10-29T11:27:00+00:00 urn:sha1:5f9796cf9411028451ffc39ffe2c7ab52fd4d337 This defines a service that can be used to get and put content-addressed chunks of data. Change-Id: I36cf2278ed1daf71848c04fdfd14450b2268c5de Reviewed-on: https://cl.tvl.fyi/c/depot/+/7135 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> refactor(tvix): build Rust projects using crate2nix 2022-12-15T17:26:45+00:00 Vincent Ambo mail@tazj.in 2022-12-07T11:05:02+00:00 urn:sha1:3aca3d3bba54eaedefc7c472af8bca16ce1f163d Introduces granular dependency builds using crate2nix, bootstrapped off the generated configuration from the newly introduced workspace (see cl/7533). This commit checks in the generated Cargo.nix file which can be regenerated with a parameterless invocation of `crate2nix generate` in `//tvix`. I tried generating this in IFD, but it turned out to be harder than what seemed worthwhile for now. In this setup, the various build targets for Rust projects end up being attributes of the imported `Cargo.nix` file at the `tvix.crates` attribute. These still lack configuration, however, which has been fixed in the various `default.nix` files of individual projects. Note that we (temporarily) lose the ability to build tvix-eval's benchmarks in CI. I haven't figured out what magic incantation summons them from the void again ... The `eval-okay-readDir` tests from both test suites have been disabled because they fail for unknown reasons when run in this new derivation. Somebody will have to debug it! Change-Id: I2014614ccb9c8951aedbd71df7966ca191a13695 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7538 Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI refactor(tvix): share a Cargo.lock file between Rust projects 2022-12-15T17:26:45+00:00 Vincent Ambo mail@tazj.in 2022-12-06T13:47:02+00:00 urn:sha1:d9d627cdf0507e44f606519384fbebd726eb0593 This relates to the (abandoned) cl/7256. Introduces a Cargo workspace at //tvix that is primarily intended to be used as a workaround for the annoying Nix+Rust tooling while having a consistent set of dependencies. This is driven in part by a desire to adopt crate2nix and get more granular Nix builds for Tvix's Rust projects, and in part by a need to split //tvix/eval into something providing the CLI (REPL etc.), and a library providing eval, without significantly altering the structure of build targets. To accomplish this the workspace has been designed to allow projects to remain independent build targets. I want to avoid lumping all the projects together - something like //tvix/eval should always be independent of other parts of tvix. A helper function in //tvix/default.nix lets downstream naersk projects construct a sparse root for the project which combines the workspace's `Cargo.lock` with the project's own `Cargo.toml`. Note that cargo commands in the workspace itself require the build dependencies of _all_ projects to be present, which is currently a bit annoying to accomplish. This introduces some breakage: 1. It breaks usage of rust-analyser without being in a shell with the dependencies of *all* Tvix projects, as it is not capable of respecting only the subset of dependencies for a part of the workspace. 2. It is no longer possible to run tests using `cargo test`, as the test generation crate we use does not work with workspaces: https://github.com/frehberg/test-generator/issues/6 This still works in the Nix build as we construct a Cargo project that looks like it's not in a workspace there. Until somebody fixes that crate / writes a new macro / does something else with the test suite, the way to run the tests is through the Nix build. Long-term we'll probably want to get rid of cargo completely, it's just a big wart and most tooling works without it if correctly configured, but we don't have time for that now. Change-Id: I846bff7a8429a25c077fd1e9ef4e3c34a299a4a1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7533 Reviewed-by: flokli <flokli@flokli.de> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI feat(tvix/store): init 2022-12-04T10:41:39+00:00 Florian Klink flokli@flokli.de 2022-11-11T22:48:24+00:00 urn:sha1:c432aae9b3a4924baa8b76cdc6f052f6fe70b730 Change-Id: Id846647fa456ff8b187ed4f16d035ce030f11d25 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7264 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org> Reviewed-by: tazjin <tazjin@tvl.su>
This XML file does not appear to have any style information associated with it. The document tree is shown below.
<feed xmlns="http://www.w3.org/2005/Atom">
<title>depot/tvix/store/default.nix, branch refs/r/5866</title>
<subtitle>monorepo for the virus lounge</subtitle>
<id>http://code.tvl.fyi/depot/atom?h=refs%2Fr%2F5866</id>
<link rel="self" href="http://code.tvl.fyi/depot/atom?h=refs%2Fr%2F5866"/>
<link rel="alternate" type="text/html" href="http://code.tvl.fyi/"/>
<updated>2023-01-06T17:57:06+00:00</updated>
<entry>
<title>refactor(tvix/store): move protobuf build config one level up</title>
<updated>2023-01-06T17:57:06+00:00</updated>
<author>
<name>Vincent Ambo</name>
<email>mail@tazj.in</email>
</author>
<published>2023-01-06T17:20:44+00:00</published>
<link rel="alternate" type="text/html" href="http://code.tvl.fyi/commit/?id=3e03e59893b77b0b3915224967b377f6e940670d"/>
<id>urn:sha1:3e03e59893b77b0b3915224967b377f6e940670d</id>
<content type="text"> This embeds the build config directly at the point where `Cargo.nix` is imported, making it transparent to library consumers. Change-Id: I5586e12f02ed14587c32d9ef7d93f079366fb127 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7780 Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI </content>
</entry>
<entry>
<title>chore(tvix/store): make importable</title>
<updated>2023-01-03T13:03:07+00:00</updated>
<author>
<name>Florian Klink</name>
<email>flokli@flokli.de</email>
</author>
<published>2023-01-02T13:37:08+00:00</published>
<link rel="alternate" type="text/html" href="http://code.tvl.fyi/commit/?id=ceb2c0ba895554c7cabb0ac20d3a80ea2aba1ab1"/>
<id>urn:sha1:ceb2c0ba895554c7cabb0ac20d3a80ea2aba1ab1</id>
<content type="text"> This allows other crates to import tvix_store. Rename the bin crate to tvix-store-bin, to avoid having multiple crates with the same name (https://github.com/rust-lang/cargo/issues/6313) Change-Id: I857768d6115640dbf102e79ed03e8474090df2fe Reviewed-on: https://cl.tvl.fyi/c/depot/+/7728 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> </content>
</entry>
<entry>
<title>feat(tvix/store): implement reflection</title>
<updated>2022-12-30T20:25:09+00:00</updated>
<author>
<name>Florian Klink</name>
<email>flokli@flokli.de</email>
</author>
<published>2022-11-26T01:14:02+00:00</published>
<link rel="alternate" type="text/html" href="http://code.tvl.fyi/commit/?id=0bf2b0ef1164aae0ad692066e8cfc0b243a89e4d"/>
<id>urn:sha1:0bf2b0ef1164aae0ad692066e8cfc0b243a89e4d</id>
<content type="text"> This implements grpc.reflection.v1alpha.ServerReflection, and will make tools like evans automatically discover available services, without having to specify the path to the .proto files client-side. It's behind a reflection feature flag, which is enabled by default. Change-Id: Icbcb5eb05ceede5b9952e38a2ba72eaa6fa8a437 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7435 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI </content>
</entry>
<entry>
<title>feat(tvix/proto): add rpc_directory.proto</title>
<updated>2022-12-22T14:35:32+00:00</updated>
<author>
<name>Florian Klink</name>
<email>flokli@flokli.de</email>
</author>
<published>2022-10-30T11:57:07+00:00</published>
<link rel="alternate" type="text/html" href="http://code.tvl.fyi/commit/?id=2e04f68df8c69034151cce5b56ceedb0653687cd"/>
<id>urn:sha1:2e04f68df8c69034151cce5b56ceedb0653687cd</id>
<content type="text"> This provides an interface to retrieve and upload single Directory messages, or a DAG of them. Change-Id: Id9e7084bd697d843649a122da2c992a3e36d808c Reviewed-on: https://cl.tvl.fyi/c/depot/+/7137 Tested-by: BuildkiteCI Reviewed-by: Adam Joseph <adam@westernsemico.com> </content>
</entry>
<entry>
<title>feat(tvix/proto): add rpc_pathinfo</title>
<updated>2022-12-22T14:35:32+00:00</updated>
<author>
<name>Florian Klink</name>
<email>flokli@flokli.de</email>
</author>
<published>2022-10-29T11:28:04+00:00</published>
<link rel="alternate" type="text/html" href="http://code.tvl.fyi/commit/?id=870cd533dd98e708f9870a8c85577681697db372"/>
<id>urn:sha1:870cd533dd98e708f9870a8c85577681697db372</id>
<content type="text"> This defines a service that can be used to upload and retrieve metadata of nix paths. Change-Id: Id86eb531ce4ae316adb15934b0d1386a14ba2132 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7136 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: Adam Joseph <adam@westernsemico.com> </content>
</entry>
<entry>
<title>feat(tvix/proto): add rpc_blobstore</title>
<updated>2022-12-15T19:10:26+00:00</updated>
<author>
<name>Florian Klink</name>
<email>flokli@flokli.de</email>
</author>
<published>2022-10-29T11:27:00+00:00</published>
<link rel="alternate" type="text/html" href="http://code.tvl.fyi/commit/?id=5f9796cf9411028451ffc39ffe2c7ab52fd4d337"/>
<id>urn:sha1:5f9796cf9411028451ffc39ffe2c7ab52fd4d337</id>
<content type="text"> This defines a service that can be used to get and put content-addressed chunks of data. Change-Id: I36cf2278ed1daf71848c04fdfd14450b2268c5de Reviewed-on: https://cl.tvl.fyi/c/depot/+/7135 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> </content>
</entry>
<entry>
<title>refactor(tvix): build Rust projects using crate2nix</title>
<updated>2022-12-15T17:26:45+00:00</updated>
<author>
<name>Vincent Ambo</name>
<email>mail@tazj.in</email>
</author>
<published>2022-12-07T11:05:02+00:00</published>
<link rel="alternate" type="text/html" href="http://code.tvl.fyi/commit/?id=3aca3d3bba54eaedefc7c472af8bca16ce1f163d"/>
<id>urn:sha1:3aca3d3bba54eaedefc7c472af8bca16ce1f163d</id>
<content type="text"> Introduces granular dependency builds using crate2nix, bootstrapped off the generated configuration from the newly introduced workspace (see cl/7533). This commit checks in the generated Cargo.nix file which can be regenerated with a parameterless invocation of `crate2nix generate` in `//tvix`. I tried generating this in IFD, but it turned out to be harder than what seemed worthwhile for now. In this setup, the various build targets for Rust projects end up being attributes of the imported `Cargo.nix` file at the `tvix.crates` attribute. These still lack configuration, however, which has been fixed in the various `default.nix` files of individual projects. Note that we (temporarily) lose the ability to build tvix-eval's benchmarks in CI. I haven't figured out what magic incantation summons them from the void again ... The `eval-okay-readDir` tests from both test suites have been disabled because they fail for unknown reasons when run in this new derivation. Somebody will have to debug it! Change-Id: I2014614ccb9c8951aedbd71df7966ca191a13695 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7538 Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI </content>
</entry>
<entry>
<title>refactor(tvix): share a Cargo.lock file between Rust projects</title>
<updated>2022-12-15T17:26:45+00:00</updated>
<author>
<name>Vincent Ambo</name>
<email>mail@tazj.in</email>
</author>
<published>2022-12-06T13:47:02+00:00</published>
<link rel="alternate" type="text/html" href="http://code.tvl.fyi/commit/?id=d9d627cdf0507e44f606519384fbebd726eb0593"/>
<id>urn:sha1:d9d627cdf0507e44f606519384fbebd726eb0593</id>
<content type="text"> This relates to the (abandoned) cl/7256. Introduces a Cargo workspace at //tvix that is primarily intended to be used as a workaround for the annoying Nix+Rust tooling while having a consistent set of dependencies. This is driven in part by a desire to adopt crate2nix and get more granular Nix builds for Tvix's Rust projects, and in part by a need to split //tvix/eval into something providing the CLI (REPL etc.), and a library providing eval, without significantly altering the structure of build targets. To accomplish this the workspace has been designed to allow projects to remain independent build targets. I want to avoid lumping all the projects together - something like //tvix/eval should always be independent of other parts of tvix. A helper function in //tvix/default.nix lets downstream naersk projects construct a sparse root for the project which combines the workspace's `Cargo.lock` with the project's own `Cargo.toml`. Note that cargo commands in the workspace itself require the build dependencies of _all_ projects to be present, which is currently a bit annoying to accomplish. This introduces some breakage: 1. It breaks usage of rust-analyser without being in a shell with the dependencies of *all* Tvix projects, as it is not capable of respecting only the subset of dependencies for a part of the workspace. 2. It is no longer possible to run tests using `cargo test`, as the test generation crate we use does not work with workspaces: https://github.com/frehberg/test-generator/issues/6 This still works in the Nix build as we construct a Cargo project that looks like it's not in a workspace there. Until somebody fixes that crate / writes a new macro / does something else with the test suite, the way to run the tests is through the Nix build. Long-term we'll probably want to get rid of cargo completely, it's just a big wart and most tooling works without it if correctly configured, but we don't have time for that now. Change-Id: I846bff7a8429a25c077fd1e9ef4e3c34a299a4a1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7533 Reviewed-by: flokli <flokli@flokli.de> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI </content>
</entry>
<entry>
<title>feat(tvix/store): init</title>
<updated>2022-12-04T10:41:39+00:00</updated>
<author>
<name>Florian Klink</name>
<email>flokli@flokli.de</email>
</author>
<published>2022-11-11T22:48:24+00:00</published>
<link rel="alternate" type="text/html" href="http://code.tvl.fyi/commit/?id=c432aae9b3a4924baa8b76cdc6f052f6fe70b730"/>
<id>urn:sha1:c432aae9b3a4924baa8b76cdc6f052f6fe70b730</id>
<content type="text"> Change-Id: Id846647fa456ff8b187ed4f16d035ce030f11d25 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7264 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org> Reviewed-by: tazjin <tazjin@tvl.su> </content>
</entry>
</feed>