about summary refs log tree commit diff
path: root/tvix/build
AgeCommit message (Collapse)AuthorFilesLines
2024-07-21 r/8385 chore(tvix): upgrade to tonic 0.12 / hyper 1.0Yureka1-6/+6
Change-Id: Idd8ce48869ddd869d51a10959b920f1290a8a9b3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11991 Autosubmit: yuka <yuka@yuka.dev> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-07-18 r/8368 refactor(tvix): use composition & registry for from_addrYureka1-9/+4
Change-Id: I3c94ecb5958294b5973c6fcdf5ee9c0d37fa54ad Reviewed-on: https://cl.tvl.fyi/c/depot/+/11976 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: yuka <yuka@yuka.dev>
2024-07-09 r/8360 fix(tvix/{store,castore,build}): Compile tonic w/o config in build.rsIlan Joselevich1-7/+3
Previously we had to make a mutable Config instance and set bytes and other values in it because they were not exposed to the builder pattern (https://github.com/hyperium/tonic/issues/908) but now they are, so we just set them through the builder. Change-Id: I8904c6b93f09173b56586024b1ced59d622bce66 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11966 Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-06-20 r/8297 feat(tvix/tracing): optional progressbarSimon Hauser1-1/+3
Disable the progressbar on default and provide a interface for optionally enabling the progressbar. Change-Id: I0e31b1957e80cf64a8dcf65c6ceb3713975b8220 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11861 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Simon Hauser <simon.hauser@helsinki-systems.de>
2024-06-14 r/8271 feat(tvix/tracing): correctly close otlp on exitSimon Hauser1-1/+1
Provide a new interface for forcing a flush of otlp traces and use this interface to shutdown otlp prior to exiting tvix-store, either if the tool was stopped with a SIGTERM or ended regularly. This also fixes an issue where traces were not even exported if for example we just imported 10 paths and never even emitted more than 256 traces. The implementation uses a mpsc channel so a flush can be done without having to wait for it to complete. If you want to wait for a flush to complete you can provide a oneshot channel which will receive a message once flushing is complete. Because of a otlp bug `force_flush` as well as `shutdown_tracer_provider` need to be executed using `spawn_blocking` otherwise the function will deadlock. See https://github.com/open-telemetry/opentelemetry-rust/issues/1395#issuecomment-1953280335 Change-Id: I0a828391adfb1f72dc8305f62ced8cba0515847c Reviewed-on: https://cl.tvl.fyi/c/depot/+/11803 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: Simon Hauser <simon.hauser@helsinki-systems.de>
2024-06-10 r/8242 feat(tvix/tracing): introduce common tvix-tracing crateSimon Hauser2-15/+9
Introduce a new common crate that contains tracing boilerplate which then can be used in the cli, tvix-store and tvix-build crates. It has otlp as an optional feature, which is currently only used by tvix-store. Change-Id: I41468ac4d9c65174515d721513b96fea463d6ed2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11758 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Simon Hauser <simon.hauser@helsinki-systems.de>
2024-06-03 r/8211 feat(tvix): add CI targets for checking crate features powersetIlan Joselevich1-3/+9
Closes: https://b.tvl.fyi/issues/401 With this change all crate features (and their combinations) will be built and tested in CI. From now on, when adding/removing a Cargo feature for a crate, you will want to add it to the features power set that gets tested in CI. For each crate there's a default.nix with a `mkFeaturePowerset` invocation, modify the list to include/remove the feature. Note that you don't want to add "collection" features, such as `fs` for tvix-[ca]store or `default`. Change-Id: I966dde1413d057770787da3296cce9c1924570e0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11717 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-05-26 r/8173 fix(tvix/build): tonic-reflection feature needs castore with it tooFlorian Klink1-1/+1
If building tvix-build with the tonic-reflection feature, it needs to import `tvix_castore::proto::FILE_DESCRIPTOR_SET`, which is only available if tvix-castore is built with the `tonic-reflection` feature. Change-Id: I355b4c5b4c1333d5cc56335de47ad5d2f1db6337 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11716 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de>
2024-05-16 r/8152 chore(tvix): drop json feature for tracing-subscriberFlorian Klink1-1/+1
Drop the (unused) feature flags here, and get rid of some crate dependencies. Change-Id: Id64852b498725467d56abb12b548301cfba6a760 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11678 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-05-16 r/8151 feat(tvix/build): drop json loggingFlorian Klink2-20/+6
This has already been dropped for tvix-store, drop it here as well. Change-Id: Ib3aa37bbe9fd4c498b5ba1213f7d922d0c64ffc7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11677 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-05-15 r/8149 chore(tvix): bump to tokio-listener 0.4.1Florian Klink1-1/+1
This gives multi-listener support, as well as listening on named socket FDs. Change-Id: I92b441f854e2faf80074463d0ca6bdc23cbd890a Reviewed-on: https://cl.tvl.fyi/c/depot/+/11663 Tested-by: BuildkiteCI Reviewed-by: yuka <yuka@yuka.dev> Autosubmit: flokli <flokli@flokli.de>
2024-05-13 r/8139 chore(tvix/clippy): configure clippy allow block_in_conditionsFlorian Klink1-0/+3
… in Cargo.toml. This gets an imperative `cargo clippy` run to pick up that config, so `-A clippy::blocks_in_conditions` doesn't need to be explicitly specified anymore. Change-Id: I32b6cc50c77c22cba0d816d0db508c2f94b2c383 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11659 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI
2024-04-19 r/7967 chore(tvix/build): migrate from test_case to rstestFlorian Klink3-36/+38
Change-Id: I75d8d61f836c76e8765e0e3b49022c056de84850 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11466 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-03-28 r/7795 refactor(tvix/castore/*): drop utils.rs and grpc directorysvc testsFlorian Klink1-8/+10
This drops pretty much all of castore/utils.rs. There were only two things left in there, both a bit messy and only used for tests: Some `gen_*_service()` helper functions. These can be expressed by `from_addr("memory://")`. The other thing was some plumbing code to test the gRPC layer, by exposing a in-memory implementation via gRPC, and then connecting to that channel via a gRPC client again. Previous CLs moved the connection setup code to {directory,blob}service::tests::utils, close to where we exercise them, the new rstest-based tests. The tests interacting directly on the gRPC types are removed, all scenarios that were in there show now be covered through the rstest ones on the trait level. Change-Id: I450ccccf983b4c62145a25d81c36a40846664814 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11223 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-03-16 r/7705 chore(tvix): bump tonic to 0.11.0Florian Klink1-4/+4
This bumps tonic and surrounding crates to 0.11.x. We added support for tonic 0.11.x into tokio-listener (https://github.com/vi/tokio-listener/pull/4), so that's bumped as well. Change-Id: Icfade5894403228299836fefb21b2f9ae59dbebb Reviewed-on: https://cl.tvl.fyi/c/depot/+/11156 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-03-16 r/7701 fix(tvix): don't emit rerun-if-changedFlorian Klink1-2/+2
`build.rs` emits rerun-if-changed statements for all proto files, as well as all include paths we pass it. Unfortunately, due to protobufs include path rules, we need to specify the path to the depot root itself as an include path, at least when building impurely with `cargo`. This causes cargo to essentially always rebuild, as it also puts its own temporary files in there. Unfortunately, tonic-build does not chase down to individual .proto files that are included. Disable emitting these `rerun-if-changed` statements for now. This could cause cargo to not rebuild protos every time, causing stale data until the next local `cargo clean`, but considering the protos change not that frequently, and it'll immediately surface if trying to build via Nix (either locally or in CI), it's a good-enough compromise. Change-Id: Ifd279a2216222ef3fc0e70c5a2fe6f87997f562e Reviewed-on: https://cl.tvl.fyi/c/depot/+/11157 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2024-03-15 r/7700 docs(tvix): fix some docstringsFlorian Klink1-1/+1
Change-Id: Ife599387d0472cd746b992bd6755a2fb6a0e0dc4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11158 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-01-15 r/7387 feat(tvix/build): add CLI entrypointFlorian Klink2-0/+135
This starts a BuildService as a separate process, currently defaulting to the DummyBuildService. Change-Id: Ic206f00831641d3ffebaa44883b7dc053700b9ca Reviewed-on: https://cl.tvl.fyi/c/depot/+/10631 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-01-15 r/7386 feat(tvix/build): add from_addr methodFlorian Klink3-0/+89
This allows constructing a BuildService from a URI, similar to how it's done in tvix-[ca]store. Change-Id: Ib962b329535c6c7e378ab7ac7f4dd254366497b3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10630 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de>
2024-01-15 r/7385 feat(tvix/build): add gRPC clientFlorian Klink3-1/+31
Change-Id: I0d917a9a308227b13d096def22945db917830d18 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10629 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-01-15 r/7384 feat(tvix/build): add GRPCBuildServiceWrapperFlorian Klink2-0/+39
This produces a gRPC BuildService server for anything implementing our BuildService trait. Change-Id: I59c690a432b5e1f59209fd67e2718cb8c935adf2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10628 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-01-15 r/7383 feat(tvix/build): add BuildService traitFlorian Klink4-0/+33
Also provide a dummy implementation that just fails on any build that's requested. Change-Id: I0df743a730c5331ec9ce6e97a966abe18ce067f5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10627 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-01-06 r/7356 feat(tvix/build): add BuildRequest validationFlorian Klink2-0/+265
Change-Id: I8182e4c4a7e5694c1e6f1f56ce092751c22adf4c Reviewed-on: https://cl.tvl.fyi/c/depot/+/10538 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-01-05 r/7353 fix(tvix/build): fix inconsistencies around pathsFlorian Klink1-1/+1
Change-Id: I76ce4e042f6fbe4bc897767b55ce4eb1f7269e5c Reviewed-on: https://cl.tvl.fyi/c/depot/+/10541 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de>
2024-01-05 r/7349 docs(tvix/build): document requirements for paths and sortingFlorian Klink1-2/+14
Change-Id: I466fac025b9b5b2279a6188fa35cb1f7d8ab7bc5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10536 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-01-03 r/7312 feat(tvix/build/protos): add additional_files fieldFlorian Klink1-0/+10
Nix builds also have some other inputs, outside of inputs_dir, used in derivations setting `passAsFile` and `__structuredAttrs`. This extends our `BuildRequest` structure to accomodate these usecases. Change-Id: I8de6516eb467fa48a4961d88933ebd548f131049 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10519 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de>
2023-12-29 r/7283 feat(tvix/build): don't force outputs to be in store_dirFlorian Klink1-6/+7
There's no need to require builds to also put their outputs in store_dir. So rename store_dir to inputs_dir and make outputs consume paths, not basenames. Also document the possibility for builds to write to inputs_dir. Let's see if we want to make this a constraint later. Change-Id: Ib4df20afcdde2d771c269c422f04c7e95587cd0f Reviewed-on: https://cl.tvl.fyi/c/depot/+/10447 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de>
2023-12-26 r/7265 feat(tvix/build/protos): add some missing fieldsFlorian Klink1-15/+34
- directory in which the castore input nodes are mounted - working directory for the build command - scratch paths - network access y/n - whether a (static) /bin/sh should be provided Populate these fields appropriately, and extend the tests in tvix-glue with a FOD example. Change-Id: I4f9de1483d6696d74694a09784910c407acb0be0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10412 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2023-12-11 r/7155 refactor(tvix/*/protos): separate lint targetFlorian Klink1-15/+36
Break up the go-bindings derivation. Keep "protos" containing all proto files (well, and the buf config), and use it for a check phase running linter and formatter, as well as the existing "go-bindings" attribute Change-Id: I52cb9d08570bb76452acb831eb711c5b6c0eacfb Reviewed-on: https://cl.tvl.fyi/c/depot/+/10239 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-12-11 r/7154 feat(tvix/build): add derivation_to_build_requestFlorian Klink1-0/+1
This function converts from a nix_compat::derivation::Derivation to a BuildRequest. In addition to the Derivation itself, it needs two lookup functions to map input paths to their castore nodes. Change-Id: I0332982f0bc7933a5fda137fe39d5a850639d929 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10236 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-12-11 r/7151 feat(tvix/build): initFlorian Klink9-0/+256
This adds the tvix-build crate, currently only containing a `tvix_build::proto` module, exposing the data structures defined in tvix/build/protos. Change-Id: I75f5d9196969ed0877b1fe640cacfecba0fb2e03 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10235 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-12-11T11·35+0000