about summary refs log tree commit diff
path: root/tvix/glue
AgeCommit message (Collapse)AuthorFilesLines
2024-02-17 r/7542 feat(tvix/glue): tune instrumentations in TvixStoreIOFlorian Klink1-2/+2
Print store paths with their ToString implementation for brevity, and don't log the sucessful return value of read_to_string. Change-Id: I01b6838398acd66b8818095622f361fcca26fa77 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10854 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-02-17 r/7541 test(tvix/glue/known_paths): add testsFlorian Klink3-0/+99
Change-Id: I02dc891c49ed650c66c403107767b11b6d5e2734 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10949 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Peter Kolloch <info@eigenvalue.net> Autosubmit: flokli <flokli@flokli.de>
2024-02-17 r/7540 refactor(tvix/glue/known_paths): use StorePathFlorian Klink2-9/+5
Passing a StorePathRef is annoying if we only (already) have a StorePath. Change-Id: Ic3b36c0041707230515a6745a57f0d25b2bafd16 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10948 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Reviewed-by: Peter Kolloch <info@eigenvalue.net>
2024-02-17 r/7539 feat(tvix/glue/known_paths): add get_drv_by_output_pathFlorian Klink1-0/+27
This allows getting a Derivation struct producing the passed output path. Change-Id: I89858d91bffc2ef7f1d86314c16fa4f850f21426 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10791 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Reviewed-by: Peter Kolloch <info@eigenvalue.net>
2024-02-17 r/7534 refactor(tvix/glue/KnownPaths): track Derivation struct tooFlorian Klink2-22/+58
We need to not only store a map from drv path to hash derivation modulo, but also keep the original Derivation struct - because we'll use that later to trigger builds. Change-Id: I78e2e8431ab5ae853188866b797a79025200de98 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10790 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-02-13 r/7508 feat(tvix/eval): Box Value::CatchableAspen Smith1-8/+8
This is now the only enum variant for Value that is larger than 8 bytes (it's 16 bytes), so boxing it (especially since it's not perf-critical) allows us to get the Value size down to only 16 bytes! Change-Id: I98598e2b762944448bef982e8ff7da6d6683c4aa Reviewed-on: https://cl.tvl.fyi/c/depot/+/10798 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: aspen <root@gws.fyi>
2024-02-13 r/7506 fix(tvix/eval): Replace inner NixString repr with Box<Bstr>Aspen Smith2-16/+13
Storing a full BString here incurs the extra overhead of the capacity for the inner byte-vector, which we basically never use as Nix strings are immutable (and we don't do any mutation / sharing analysis). Switching to a Box<BStr> cuts us from 72 bytes to 64 bytes per string (and there are a lot of strings!) Change-Id: I11f34c14a08fa02759f260b1c78b2a2b981714e4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10794 Autosubmit: aspen <root@gws.fyi> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-02-02 r/7467 refactor(tvix/eval): Box Value::StringAspen Smith3-7/+7
NixString is *quite* large - like 80 bytes - because of the extra capacity value for BString and because of the context. We want to keep Value small since we're passing it around a lot, so let's box the NixString inside Value::String to save on some memory, and make cloning ostensibly a little cheaper Change-Id: I343c8b4e7f61dc3dcbbaba4382efb3b3e5bbabb2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10729 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2024-01-31 r/7460 fix(tvix): Represent strings as byte arraysAspen Smith3-40/+48
C++ nix uses C-style zero-terminated char pointers to represent strings internally - however, up to this point, tvix has used Rust `String` and `str` for string values. Since those are required to be valid utf-8, we haven't been able to properly represent all the string values that Nix supports. To fix that, this change converts the internal representation of the NixString struct from `Box<str>` to `BString`, from the `bstr` crate - this is a wrapper around a `Vec<u8>` with extra functions for treating that byte vector as a "morally string-like" value, which is basically exactly what we need. Since this changes a pretty fundamental assumption about a pretty core type, there are a *lot* of changes in a lot of places to make this work, but I've tried to keep the general philosophy and intent of most of the code in most places intact. Most notably, there's nothing that's been done to make the derivation stuff in //tvix/glue work with non-utf8 strings everywhere, instead opting to just convert to String/str when passing things into that - there *might* be something to be done there, but I don't know what the rules should be and I don't want to figure them out in this change. To deal with OS-native paths in a way that also works in WASM for tvixbolt, this also adds a dependency on the "os_str_bytes" crate. Fixes: b/189 Fixes: b/337 Change-Id: I5e6eb29c62f47dd91af954f5e12bfc3d186f5526 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10200 Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de> Reviewed-by: sterni <sternenseemann@systemli.org> Autosubmit: aspen <root@gws.fyi> Tested-by: BuildkiteCI
2024-01-22 r/7439 fix(tvix/glue/tests): resolve logic error in hasContextInAttrKeyssterni1-1/+2
Change-Id: I7aa09c10f803c2b79363fe3eb52c1127276a333b Reviewed-on: https://cl.tvl.fyi/c/depot/+/10676 Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-01-20 r/7433 feat(tvix/store): enable `name` customization in the storeRyan Lahfa1-0/+1
Sometimes, Nix lets someone customize the `name` in the store for a path, this is the case for `builtins.path` which takes a `name` argument, we leave it to the caller to choose the name, which can be the basename by default of the path. Change-Id: Icdbf71d1d8f2dca5716b99d20aac885aab905b80 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10653 Tested-by: BuildkiteCI Autosubmit: raitobezarius <tvl@lahfa.xyz> Reviewed-by: flokli <flokli@flokli.de>
2024-01-20 r/7432 refactor(tvix/store): `import_path` → `import_path_as_nar_ca`Ryan Lahfa1-1/+1
Add multiple additional helpers such as: - `path_to_name`: derive the basename of a given path - `derive_nar_ca_path_info`: derive the `PathInfo` for a content addressed NAR which isolates further the tree walking feature and the ingestion feature. Additionally, we don't `expect` anymore and propagate properly ingestion errors up. Change-Id: I60edb5b633911c58ade7e19f5002e6f75f90e262 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10574 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: raitobezarius <tvl@lahfa.xyz>
2024-01-20 r/7425 fix(tvix/glue/benches/eval): add impure_builtinsFlorian Klink1-1/+2
Impure builtins need to be explicitly added. This was missed from cl/10640. Change-Id: I1f8a84aa02f1cf8a26247960d6eb15895e09c610 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10667 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de>
2024-01-19 r/7422 chore(3p/sources): Bump channels & overlayssterni1-2/+0
- Adjust to ecl 23.9.9 release - Regenerate go protos after protoc-gen-go update - Drop dhall fork which hasn't kept up with 1.42.* - Address new clippy warnings: - Variant naming of Error::ValidationError - Simplify .try_into().unwrap() - Drop unnecessary identity function - Test module must be last in file - Drop unused `pub use` - Update agenix to 0.15.0. Current master has a installCheckPhase that doesn't work with C++ Nix 2.3.*: https://github.com/ryantm/agenix/commit/a23aa271bec82d3e962bafb994595c1c4a62b133#commitcomment-137185861 Change-Id: Ic29eef20d6fd1362ce1031364a5ca6b4edf195bd Reviewed-on: https://cl.tvl.fyi/c/depot/+/10615 Reviewed-by: aspen <root@gws.fyi> Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org>
2024-01-18 r/7411 refactor(tvix/glue): add BuildService to TvixStoreIOFlorian Klink4-6/+16
TvixStoreIO triggers builds whenever IO into a not-yet-built store path is requested, if it knows how to build that path. Change-Id: If30e9db6be2f2a30cbc9d0576f357f3ecfa0d35a Reviewed-on: https://cl.tvl.fyi/c/depot/+/10645 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-01-18 r/7410 feat(tvix/glue): use TvixStoreIO as derivation builtin stateRyan Lahfa6-86/+106
We propagate a `TvixStoreIO` as the `state` of our derivation-specific builtins in the glue crate. The evaluators `io_handle` itself is using a Rc<dyn EvalIO>. An earlier version of TvixStoreIO was also introducing generics over the different internal services themselves, but we opted for instead hardcoding this to Arc<dyn …> for the sake of less macro voodoo. Change-Id: I535c476f06b840858fa3070c4a237ece47f7a15b Reviewed-on: https://cl.tvl.fyi/c/depot/+/10636 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-01-18 r/7407 refactor(tvix/eval): generalize EvalIO containerFlorian Klink2-3/+6
Don't restrict to a Box<dyn EvalIO>. There's still one or two places where we do restrict, this will be solved by b/262. Change-Id: Ic8d927d6ea81fa12d90b1e4352f35ffaafbd1adf Reviewed-on: https://cl.tvl.fyi/c/depot/+/10639 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-17 r/7404 feat(tvix/glue): introduce test suite for context stringsRyan Lahfa11-0/+363
This is an additional test suite on the top of the Nix ones for context strings matters. It already smoked out multiple mistakes and potential bugs and non-deterministic result from the evaluator. It uses a similar technology as the one in the tvix-eval albeit we instantiate a fully fledged evaluator with in-memory store. We copy the files instead of symlinking them because crates are built in isolation, so symlinks cannot work. Change-Id: I63ae225ce4f83c6e2c8ccd60d779c2f8eb9d08fb Reviewed-on: https://cl.tvl.fyi/c/depot/+/10619 Autosubmit: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-01-16 r/7389 refactor(nix-compat): use StorePathRef for hash derivation moduloFlorian Klink2-18/+17
Rather than passing strings around, use a StorePathRef. This makes things a bit more typesafe, and more aligned with what we want to do in b/264. Change-Id: Ib7080addf27e7f1a9c8da1d8aaa66744468e3b5a Reviewed-on: https://cl.tvl.fyi/c/depot/+/10633 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-15 r/7382 refactor(tvix/glue): have derivation_to_build_request consume inputsFlorian Klink1-58/+14
Determining the inputs might trigger additional builds/substitutions, so answering these lookups via a lambda in a lazy fashion gets complicated. You end up assembling the list of input nodes upfront, and the lambda will just be a dumb lookup into that preassembled list. Rather than doing that, simply have derivation_to_build_request leave the work of determining the inputs to the caller. Change-Id: I75880132916c76b930807c989090da298b6891bd Reviewed-on: https://cl.tvl.fyi/c/depot/+/10626 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-01-15 r/7381 refactor(tvix/glue/known_paths): drop some unused stuffFlorian Klink1-51/+3
This are leftovers from the "reference scanning" approach (which we didn't end up using). We still want a concept of known paths, so we can trace IO into storepaths back to the build recipe that'll produce it, so let's keep the rest of this struct around. Change-Id: I73d38e21e5b97950b8fc2a42176cae5f80d371c8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10632 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-01-14 r/7379 fix(tvix/eval): catchable-aware builtinsRyan Lahfa1-4/+23
A bunch of operations in Tvix are not aware of catchable values and does not propagate them. In the meantime, as we wait for a better solution, we just offer this commit for moving the needle. Change-Id: Ic3f0e1550126b0847b597dfc1402c35e0eeef469 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10473 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2024-01-12 r/7377 feat(tvix/glue): input derivation context testsRyan Lahfa1-0/+32
`args` was not propagating context, here's a regression test for it. Change-Id: I8b6a3148508d40df0077128f0bafe68c098a03bd Reviewed-on: https://cl.tvl.fyi/c/depot/+/10610 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-01-12 r/7376 feat(tvix/glue/derivationStrict): support __structuredAttrsFlorian Klink4-171/+220
This adds support to handle the __structuredAttrs argument, which can be passed to builtins.derivationStrict. If __structuredAttrs is passed, and set to true, most of the arguments passed to builtins.derivationStrict are not simply coerced to a string and passed down to "environments", but instead kept in a more structured fashion. Inside ATerm, which is what's relevant as far as path calculation is concerned, a virtual `__json` environment variable is present, containing these structured values. Inside Builds, these structured values are not made available as an environment variable, but a JSON file (and source-able bash script). This will need to be respected once we start emitting BuildRequests, and for that we can probably just parse the `__json` key in Derivation.environment again - or keep this additionally in non-serialized form around during Evaluation. No matter what, this is left for a followup CL. The existing handle_derivation_parameters and populate_outputs helper function were removed, as __structuredAttrs causes quite a change in behaviour, and so handling both in the same place makes it more readable. There's some open questions w.r.t. string contexts for structured attrs itself. A TODO is left for this, but at least path calculation for individual structured attrs derivations are correct now. Part of b/366. Change-Id: Ic293822266ced6f8c4826d8ef0d2e098a4adccaa Reviewed-on: https://cl.tvl.fyi/c/depot/+/10604 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-12 r/7374 docs(tvix/glue): misc reflowing of commentsRyan Lahfa1-7/+5
My OCD could not be stopped. Change-Id: I2bf504fe0865a5084ad02aee18e6180a8a3e19d7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10609 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-01-12 r/7373 fix(tvix/glue/tvix_build): fn_input_drvs_to_output_nodesFlorian Klink1-47/+36
The Derivation input_derivations field contains a list of input derivations and (a subset of their) output names. This means, multiple nodes can be returned, so return a Vec. Also, update the name to better reflect the nodes are the nodes of the selected outputs, not a node representing the .drv file itself. Additionally, use a proto::node::Node (the naked enum), rather than proto::Node, which wraps this in an optional struct field until realizing the BuildRequest. Change-Id: Iec5620b5d7ac0462f2c76acac4abcaeea2de0aad Reviewed-on: https://cl.tvl.fyi/c/depot/+/10608 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de>
2024-01-12 r/7372 refactor(tvix/glue/tvix_store_io): async store_path_to_nodeFlorian Klink1-17/+23
Provide a store_path_to_node_sync function which uses the runtime handle to block on the async function internally, but make store_path_to_node itself async, so it can call async functions internally. We'll use that later when triggering builds and waiting on their results. Change-Id: Idae9da7aa5b0878e0d3a2eba34ea2623e1ba84b2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10607 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-09 r/7361 refactor(tvix/castore/directorysvc): AsRef traverse_toFlorian Klink1-2/+1
Change-Id: I641bd4ab3de591a013f03137f1e16295946315f3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10579 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-01-09 r/7359 refactor(tvix): use AsRef<dyn …> instead of Deref<Target= …>Florian Klink1-16/+14
Removes some more needs for Arcs. Change-Id: I9a9f4b81641c271de260e9ffa98313a32944d760 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10578 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-08 r/7357 feat(tvix/glue): add test for output path calculationFlorian Klink1-0/+36
This test serves as a minimal reproducer for output path calculation. Derivations with the same name and output hash, but different build recipe should end up with the same outPath. However derivations with different name should end up with a different outPath. Change-Id: I555be59dd87ea675a0816188ed973f96c311e4e1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10416 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2024-01-06 r/7356 feat(tvix/build): add BuildRequest validationFlorian Klink1-2/+10
Change-Id: I8182e4c4a7e5694c1e6f1f56ce092751c22adf4c Reviewed-on: https://cl.tvl.fyi/c/depot/+/10538 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-01-05 r/7353 fix(tvix/build): fix inconsistencies around pathsFlorian Klink1-7/+7
Change-Id: I76ce4e042f6fbe4bc897767b55ce4eb1f7269e5c Reviewed-on: https://cl.tvl.fyi/c/depot/+/10541 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de>
2024-01-05 r/7352 refactor(tvix/glue): simplify TvixStoreIO furtherFlorian Klink1-45/+43
We don't need to spawn in all these places, we can just block_on directly, this is all IO bound. This also means, we don't need to clone any of the service handles (except preserving clone-ability of the BlobService). Change-Id: I7d90f4d6a263a98491caa071ada538a5197a5472 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10540 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-01-05 r/7351 refactor(tvix/glue): simplify store_path_to_[root]_nodeFlorian Klink1-41/+36
This was wrongly named, it returns a specific node at a subpath. Also, this code can be simplified a lot - we don't need to spawn additional tasks, and can get rid of some clones too. This is also where we need a certain build - so add some TODO to block / fetch here. Change-Id: Id26d7bd80f7a2095121e642b3f7716de78d6b6a6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10539 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-01-05 r/7350 chore(tvix): bump test-case dep to 3.3.1Florian Klink1-1/+1
Change-Id: I643548d95a5fab84563c7cbe51ca2ce640c186a9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10537 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-01-03 r/7339 feat(tvix/eval): impl `unsafeDiscardStringContext`Ryan Lahfa1-0/+45
Change-Id: I7f0cc42cbebfe5cd27bf6d4f58a4af927b83646a Reviewed-on: https://cl.tvl.fyi/c/depot/+/10423 Autosubmit: raitobezarius <tvl@lahfa.xyz> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2024-01-03 r/7338 feat(tvix/glue): contextful derivationRyan Lahfa4-214/+80
We calculate the input context by performing the union of context over all input of the derivation. Then, we just pass the rest of it to the remaining machinery. Finally, we re-emit an `outPath` and a `drvPath` containing the expected contexts. Change-Id: I74905fb258b5bee8b08d1208c9eb87f51b92a890 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10436 Autosubmit: raitobezarius <tvl@lahfa.xyz> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2024-01-03 r/7336 feat(tvix/glue): context-aware `toFile`Ryan Lahfa1-21/+14
This removes the reference tracking and uses instead the context for references and returns some. Change-Id: Ic359ca6b903b63f1a9c679c566004c617b792442 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10435 Autosubmit: raitobezarius <tvl@lahfa.xyz> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2024-01-03 r/7316 feat(tvix/glue): handle passAsFileFlorian Klink2-11/+168
This extends derivation_to_build_request to handle passAsFile the same way Nix does, and adds a unit test for it. I opted to making this function fallible (if passAsFile contains a non-existent env var), rather than pushing all of this into the Derivation validate function. Change-Id: I75b635f1f6c0c78d72b9a8fc7824f77e97b69951 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10522 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-03 r/7314 refactor(tvix/glue): create environment_vars Vec at the endFlorian Klink1-8/+7
Prevent code in this function from accessing the Vec, realize it only when returning the BuildRequest struct. Change-Id: I385c1adca552c9cb261cd70c5ca3ec45f0d7a9c3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10521 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2024-01-03 r/7313 test(tvix/glue): add test with passAsFileFlorian Klink1-0/+1
This already succeeds, as the output path calculation path does not need to be aware of any builder-specific custom handling for passAsFile. Change-Id: I9cc9e4e4351cdeaa3ec33ba58ee1569e7a368150 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10520 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-01-03 r/7312 feat(tvix/build/protos): add additional_files fieldFlorian Klink1-0/+4
Nix builds also have some other inputs, outside of inputs_dir, used in derivations setting `passAsFile` and `__structuredAttrs`. This extends our `BuildRequest` structure to accomodate these usecases. Change-Id: I8de6516eb467fa48a4961d88933ebd548f131049 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10519 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de>
2024-01-03 r/7311 feat(tvix/glue): emit a warning in case of bad SRI hashesFlorian Klink3-3/+42
And include a test to ensure we show the warning. Change-Id: Ib6a436dbba2592b398b54e44f15a48d1aa345099 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10470 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-01 r/7302 refactor(tvix/glue/tvix_store_io): use utils::import_pathFlorian Klink1-98/+25
This is doing the exact same thing. Change-Id: Iadc5e13dd192efc91cc3d36b2bdf4b8b99a312b2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10512 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-01 r/7299 feat(nix-compat/store_path): build_ca_path may failFlorian Klink1-1/+6
Change-Id: Ia74ee870f38b7966501458bace541092256c3213 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10509 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-12-31 r/7289 refactor(tvix/eval): remove code and location from structFlorian Klink3-6/+6
Instead, it's passed in the evaluate/compile_only functions, which feels more naturally. It lets us set up the Evaluation struct long before we actually feed it with data to evaluate. Now that Evaluation::new() would be accepting an empty list of arguments, we can simply implement Default, making things a bit more idiomatic. Change-Id: I4369658634909a0c504fdffa18242a130daa0239 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10475 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de>
2023-12-29 r/7283 feat(tvix/build): don't force outputs to be in store_dirFlorian Klink1-23/+16
There's no need to require builds to also put their outputs in store_dir. So rename store_dir to inputs_dir and make outputs consume paths, not basenames. Also document the possibility for builds to write to inputs_dir. Let's see if we want to make this a constraint later. Change-Id: Ib4df20afcdde2d771c269c422f04c7e95587cd0f Reviewed-on: https://cl.tvl.fyi/c/depot/+/10447 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de>
2023-12-26 r/7265 feat(tvix/build/protos): add some missing fieldsFlorian Klink2-12/+96
- directory in which the castore input nodes are mounted - working directory for the build command - scratch paths - network access y/n - whether a (static) /bin/sh should be provided Populate these fields appropriately, and extend the tests in tvix-glue with a FOD example. Change-Id: I4f9de1483d6696d74694a09784910c407acb0be0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10412 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2023-12-21 r/7244 docs(tvix/glue): fix docstringFlorian Klink1-1/+1
This is not an unclosed <html> tag. Change-Id: I2bd2426fc600de2d96dbab47743f1c7bd5fed35e Reviewed-on: https://cl.tvl.fyi/c/depot/+/10398 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: edef <edef@edef.eu>
2023-12-14 r/7219 test(tvix/glue): add nonexisting_path_without_import testFlorian Klink1-0/+18
This moves the failing example from cl/10285 into its separate test case. There were multiple complications: tvix-[ca]store was panicking in some places, rather than returning an error. This is now fixed. It needs to live in tvix-glue, so we actually have a "proper" EvalIO interface doing something. > toString ({ line = 42; col = 42; file = /deep/thought; }.file) Should not cause an error, because it shouldn't trigger an import, but leave the path as-is, and not care about it not being present. Change-Id: I76f70b3cb1f73a0fb05870375710fd9f67d5603c Reviewed-on: https://cl.tvl.fyi/c/depot/+/10342 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Reviewed-by: sterni <sternenseemann@systemli.org>
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-11-22T23·34+0000