about summary refs log tree commit diff
path: root/tvix/cli
AgeCommit message (Collapse)AuthorFilesLines
2023-05-25 r/6201 refactor(tvix/cli): wrap NixCompatIO import_cache in RwLockFlorian Klink1-4/+13
This allows dropping all &mut self from the EvalIO trait signature. Change-Id: Ie127b0a459d2996636385d159fcc5f7147e74e2e Reviewed-on: https://cl.tvl.fyi/c/depot/+/8630 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-05-24 r/6193 feat(tvix/cli): use TvixStoreIO instead of NixCompatIOFlorian Klink2-1/+19
This switches tvix-cli over from using `NixCompatIO` to `TvixStoreIO`. For now, we simply instantiate in-memory services instead of getting fancy with constructors inside tvix-store, but long-term, we might want to support some URI syntax, to make this configurable at runtime. nixpkgs eval tests might be fine (and fast!) with a purely in-memory backend, but other usages might involve talking to a local tvix-store over gRPC (using the gRPC client, either unix domain socket or even further away remote), or running tvix-store in "embedded" mode (using another client than the gRPC client). Change-Id: I509afd3dc5ce3f2d52b0fb7067748fab820e26ab Reviewed-on: https://cl.tvl.fyi/c/depot/+/8572 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-05-24 r/6190 feat(tvix/cli): check nixpkgs eval for pkgs.hello.outPathFlorian Klink1-0/+1
We already evaluate this the same as Nix does, so worth adding it into CI. Change-Id: I529faccac6e5e16e0bc985ab4c3e0cd07bb23293 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8624 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de>
2023-05-24 r/6189 feat(tvix/cli): introduce mkNixpkgsEvalCheck, add outPath checkFlorian Klink1-16/+23
This introduces a function that can be instantiated with an attribute path to instantiate, as well as the expected path (normally interpolated with the nix evaluator). Check both pkgs.stdenv.drvPath and pkgs.stdenv.outPath to match. Change-Id: Id667ed35fa159ff83fedb3017ef8d3271aa42695 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8606 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-05-22 r/6173 refactor(tvix/eval): use &Path instead of PathBufFlorian Klink2-6/+6
This allows getting rid of some clones in eval/src/vm/generators.rs. Change-Id: I330390307d3bcfeef19c98954c753ee55b1ccee3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8604 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-22 r/6171 refactor(tvix/eval/io): use io::Error instead of tvix_eval errorsFlorian Klink2-14/+12
We didn't return anything useful other than ErrorKind::IO anyways. We can use io::ErrorKind::Unsupported for DummyIO. Fixes b/271. Change-Id: Icb231e9b38168e8b6fa473bfa405d160357b317f Reviewed-on: https://cl.tvl.fyi/c/depot/+/8602 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-18 r/6160 chore(tvix/cli): remove unnecessary RefCellVincent Ambo1-7/+5
This remained around from some previous time where this type was being cloned around directly, but it's not anymore. Change-Id: I2c61cf9cecb8e5d7d08644ed20642ee61357bc21 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8580 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-05-18 r/6159 refactor(tvix/cli): split CLI-specific IO logic into TvixIO typeVincent Ambo3-11/+84
This adds a wrapper type TvixIO<T: EvalIO>, which can wrap around an arbitrary EvalIO implementation and perform actions needed for the Tvix CLI (marking imported paths as known, and handling __corepkgs__). Change-Id: I5fc1ca199b9f94b21a89103b84575e0f8f58dff9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8579 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-05-14 r/6140 feat(tvix/eval/io): allow &mut self in EvalIOFlorian Klink1-4/+4
It's okay if these calls mutate some internal state inside an implementation. Change-Id: I12bb11bde0310778c3da1275696bf7de058863a3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8571 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-11 r/6134 chore(tvix/*): bump to smol_str 0.2.0Florian Klink1-1/+1
Change-Id: Ic9ac1b6fecb564eafb41b265bf317cd385fdc170 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8560 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-04-09 r/6088 refactor(nix-compat): Properly encapsulate store path constructionJohn Ericson1-10/+8
Before there was code scattered about (e.g. text hashing module and derivation output computation) constructing store paths from low level building blocks --- there was some duplication and it was easy to make nonsense store paths. Now, we have roughly the same "safe-ish" ways of constructing them as C++ Nix, and only those are exposed: - Make text hashed content-addressed store paths - Make other content-addressed store paths - Make input-addressed fixed output hashes Change-Id: I122a3ee0802b4f45ae386306b95b698991be89c8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8411 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-03-30 r/6060 refactor(tvix/nix-compat): move build_store_path out of derivationFlorian Klink1-6/+13
This doesn't have anything to do with ATerms, we just happen to be using the aterm representation of a Derivation as contents. Moving this into store_path/utils.rs makes these things much cleaner - Have a build_store_path_from_references function, and a build_store_path_from_fingerprint helper function that makes use of it. build_store_path_from_references is invoked from the derivation module which can be used to calculate the derivation path. In the derivation module, we also invoke build_store_path_from_fingerprint during the output path calculation. Change-Id: Ia8d61a5e8e5d3f396f93593676ed3f5d1a3f1d66 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8367 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-03-22 r/6038 feat(tvix/cli): add `--strict` flag for evaluationVincent Ambo1-0/+9
This toggles whether tvix will evaluate the top-level value and deep-force it, or return it potentially still containing thunks. Change-Id: Ie910941e3b6a0f16c5c0cb896d73947626335f4b Reviewed-on: https://cl.tvl.fyi/c/depot/+/8326 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2023-03-15 r/6010 fix(tvix/cli): populate output configuration after walking argsVincent Ambo1-15/+14
Walking the arguments might encounter an `outputs` output, which might explicitly (for whatever reason) specify the `out` output. To prevent dropping FOD settings in this case, we have to populate that part of the configuration after walking the other attributes. Change-Id: Iee6a7f0a71e9c9699e79d35e6cb19e1ddb49395d Reviewed-on: https://cl.tvl.fyi/c/depot/+/8312 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-03-15 r/6009 refactor(tvix/nix-compat): -derivation::Hash, +NixHashFlorian Klink1-11/+10
This stops using our own custom Hash structure, which was mostly only used because we had to parse the JSON representation somehow. Since cl/8217, there's a `NixHash` struct, which is better suited to hold this data. Converting the format requires a bit of serde labor though, but that only really matters when interacting with JSON representations (which we mostly don't). Change-Id: Idc5ee511e36e6726c71f66face8300a441b0bf4c Reviewed-on: https://cl.tvl.fyi/c/depot/+/8304 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-14 r/5996 chore(tvix/cli): don't set COREPKGS anymoreFlorian Klink1-4/+0
Since cl/8213, tvix bundles corepkgs. Change-Id: I17fa4452a6a1d554ae67b4aed9d9b1e77cb495f5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8305 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-03-14 r/5995 refactor(tvix/nix-compat): replace calculate_drv_replacement_strFlorian Klink2-20/+30
Call this function derivation_or_fod_hash, and return a NixHash. This is more in line with how cppnix calls this, and allows using to_nix_hash_string() in some places. Change-Id: Iebf5355f08ed5c9a044844739350f829f874f0ce Reviewed-on: https://cl.tvl.fyi/c/depot/+/8293 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-03-13 r/5986 chore(tvix/cli): silence useless warningVincent Ambo1-0/+1
Change-Id: I18f567d5dcbbeb934428bf9111eb5fae5fef0342 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8261 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-03-13 r/5981 feat(tvix/cli): bundle corepkgs/fetchurl.nix with tvix-cliVincent Ambo3-1/+76
This file which ships with C++ Nix is required for evaluating nixpkgs. Like C++ Nix, we now inject a pseudo path in EvalIO from which this will resolve as <nix/fetchurl.nix> Change-Id: Ic948c476a2cfc6381d5655d308bc2d5fa25b7123 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8213 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-03-13 r/5964 refactor(tvix/eval): flatten call stack of VM using generatorsVincent Ambo2-291/+301
Warning: This is probably the biggest refactor in tvix-eval history, so far. This replaces all instances of trampolines and recursion during evaluation of the VM loop with generators. A generator is an asynchronous function that can be suspended to yield a message (in our case, vm::generators::GeneratorRequest) and receive a response (vm::generators::GeneratorResponsee). The `genawaiter` crate provides an interpreter for generators that can drive their execution and lets us move control flow between the VM and suspended generators. To do this, massive changes have occured basically everywhere in the code. On a high-level: 1. The VM is now organised around a frame stack. A frame is either a call frame (execution of Tvix bytecode) or a generator frame (a running or suspended generator). The VM has an outer loop that pops a frame off the frame stack, and then enters an inner loop either driving the execution of the bytecode or the execution of a generator. Both types of frames have several branches that can result in the frame re-enqueuing itself, and enqueuing some other work (in the form of a different frame) on top of itself. The VM will eventually resume the frame when everything "above" it has been suspended. In this way, the VM's new frame stack takes over much of the work that was previously achieved by recursion. 2. All methods previously taking a VM have been refactored into async functions that instead emit/receive generator messages for communication with the VM. Notably, this includes *all* builtins. This has had some other effects: - Some test have been removed or commented out, either because they tested code that was mostly already dead (nix_eq) or because they now require generator scaffolding which we do not have in place for tests (yet). - Because generator functions are technically async (though no async IO is involved), we lose the ability to use much of the Rust standard library e.g. in builtins. This has led to many algorithms being unrolled into iterative versions instead of iterator combinations, and things like sorting had to be implemented from scratch. - Many call sites that previously saw a `Result<..., ErrorKind>` bubble up now only see the result value, as the error handling is encapsulated within the generator loop. This reduces number of places inside of builtin implementations where error context can be attached to calls that can fail. Currently what we gain in this tradeoff is significantly more detailed span information (which we still need to bubble up, this commit does not change the error display). We'll need to do some analysis later of how useful the errors turn out to be and potentially introduce some methods for attaching context to a generator frame again. This change is very difficult to do in stages, as it is very much an "all or nothing" change that affects huge parts of the codebase. I've tried to isolate changes that can be isolated into the parent CLs of this one, but this change is still quite difficult to wrap one's mind and I'm available to discuss it and explain things to any reviewer. Fixes: b/238, b/237, b/251 and potentially others. Change-Id: I39244163ff5bbecd169fe7b274df19262b515699 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8104 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Reviewed-by: Adam Joseph <adam@westernsemico.com> Tested-by: BuildkiteCI
2023-02-16 r/5858 chore(tvix/cli): iter instead of into_iter for referencesAaqa Ishtyaq1-1/+1
This CL removes calling into_iter on a reference, as it will not move out it's content into resulting iterator. Change-Id: Ifcc10b7cf33b98453570cbcec3eb82ffaba2ffcb Signed-off-by: Aaqa Ishtyaq <aaqaishtyaq@gmail.com> Reviewed-on: https://cl.tvl.fyi/c/depot/+/8126 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-02-04 r/5836 fix(tvix/cli): use tvlfyi/wu-manber fork for refscannerVincent Ambo1-1/+1
Our fork fixes a small bug (https://github.com/jneem/wu-manber/pull/1) but it's not clear whether upstream will accept patches, so for now lets point this directly at our fork. Change-Id: Iccdcedae3e9a8b783241431787c952561d032694 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8031 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-02-03 r/5829 test(tvix/cli): eval nixpkgs.stdenv in CI and assert that it matchesVincent Ambo1-2/+26
Change-Id: If80194b5fdbf69512217bd4780e416e678045323 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8023 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-02-02 r/5827 fix(tvix/cli): keep tracking full paths in known_pathsVincent Ambo3-42/+84
We need to distinguish explicitly between the paths used for the scanner, and the paths that populate the derivation inputs. The full paths must be accessible from the result of the refscanner to populate drv fields correctly. This was previously hidden by debug changes that masked actual IO operations with no-ops. Change-Id: I037af6e6bbe2b573034d695f8779bee1b56bc125 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8022 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-02-02 r/5826 feat(tvix/cli): cache imported paths in NixCompatIOVincent Ambo1-2/+20
Creates a cache of imported literal files (e.g. `./default-builder.sh`) which avoids shelling out to Nix for each instance of the same file. Note that a better way to tackle this is to create memoizable thunks for these expressions in the compiler, but we are lacking a little bit of infrastructure for that at the moment. Change-Id: Ibc062b20d81e97dd3986e734d225a744e1779fe7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8015 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-02-02 r/5825 refactor(tvix/cli): use Wu-Manber string scanning for drv referencesVincent Ambo4-59/+37
Switch out the string-scanning algorithm used in the reference scanner. The construction of aho-corasick automata made up the vast majority of runtime when evaluating nixpkgs previously. While the actual scanning with a constructed automaton is relatively fast, we almost never scan for the same set of strings twice and the cost is not worth it. An algorithm that better matches our needs is the Wu-Manber multiple string match algorithm, which works efficiently on *long* and *random* strings of the *same length*, which describes store paths (up to their hash component). This switches the refscanner crate to a Rust implementation[0][1] of this algorithm. This has several implications: 1. This crate does not provide a way to scan streams. I'm not sure if this is an inherent problem with the algorithm (probably not, but it would need buffering). Either way, related functions and tests (which were actually unused) have been removed. 2. All strings need to be of the same length. For this reason, we truncate the known paths after their hash part (they are still unique, of course). 3. Passing an empty set of matches, or a match that is shorter than the length of a store path, causes the crate to panic. We safeguard against this by completely skipping the refscanning if there are no known paths (i.e. when evaluating the first derivation of an eval), and by bailing out of scanning a string that is shorter than a store path. On the upside, this reduces overall runtime to less 1/5 of what it was before when evaluating `pkgs.stdenv.drvPath`. [0]: Frankly, it's a random, research-grade MIT-licensed crate that I found on Github: https://github.com/jneem/wu-manber [1]: We probably want to rewrite or at least fork the above crate, and add things like a three-byte wide scanner. Evaluating large portions of nixpkgs can easily lead to more than 65k derivations being scanned for. Change-Id: I08926778e1e5d5a87fc9ac26e0437aed8bbd9eb0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8017 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-02-01 r/5807 refactor(tvix/cli): use nixhash module for output hash calculationFlorian Klink2-73/+20
This covers all the weird corner cases. Change-Id: I85637e82e8929828064ab562dc8a1c8bf161fffa Reviewed-on: https://cl.tvl.fyi/c/depot/+/7991 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-02-01 r/5799 feat(tvix/cli): implement builtins.placeholderVincent Ambo1-0/+26
This doesn't require any other corresponding handling *yet*, as the actual replacements happen in the builder logic (which we delegate to cppnix at the moment). Change-Id: I034147c933f05ae427c7a8794647132d108d0ede Reviewed-on: https://cl.tvl.fyi/c/depot/+/7972 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-01-31 r/5791 refactor(tvix/nix-compat): absorb //tvix/derivationFlorian Klink3-6/+7
Put this in its src/derivation. Change-Id: Ic047ab1c2da555a833ee454e10ef60c77537b617 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7967 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-01-31 r/5788 refactor(tvix): introduce nix-compat crateFlorian Klink1-1/+0
Move nixbase32 and store_path into this. This allows //tvix/cli to not pull in //tvix/store for now. Change-Id: Id3a32867205d95794bc0d33b21d4cb3d9bafd02a Reviewed-on: https://cl.tvl.fyi/c/depot/+/7964 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-31 r/5787 refactor(tvix/cli/derivation): use `if let` to destructureFlorian Klink1-5/+5
We only do logic here if algo and hash_mode are Some(_) (and there's an `out` output). The fact we don't do anything in all in other cases is a bit hidden at the bottom. Use if let for the destructuring, and drop the other case. Change-Id: Icc0e38e62947d52d48ef610f754749737977fca9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7966 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-01-31 r/5786 refactor(tvix/cli): remove unneeded cloneFlorian Klink1-2/+2
Change-Id: I6f4cb24bdd636af8918a2ade44075af92161c97d Reviewed-on: https://cl.tvl.fyi/c/depot/+/7965 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-01-31 r/5783 refactor(tvix/cli): absorb construct_output_hashFlorian Klink2-93/+116
This helper function only was created because populate_output_configuration was hard to test before cl/7939. With that out of the way, we can pull it in. Change-Id: I64b36c0eb34343290a8d84a03b0d29392a821fc7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7961 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-31 r/5782 refactor(tvix/cli): force outside of output configuration helperVincent Ambo1-54/+36
Change-Id: I28357fe131cefedcef9761b08a72f675f4a10789 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7939 Reviewed-by: flokli <flokli@flokli.de> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-01-29 r/5776 refactor(tvix/cli): add helper method for strong string coercionVincent Ambo1-29/+29
This is repetitive and error prone (e.g. switching around to_string/as_str has drastic consequences) due to the ToString overloads. Change-Id: I9b16a2e0e05e4c21e83f43e9f603746eb42e53f7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7947 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su>
2023-01-27 r/5773 refactor(tvix/cli): describe errors with thiserrorFlorian Klink2-64/+16
This is much less code, and makes it much easier to read. Change-Id: I9028f226105f905c2cc2cabd33907ff493e26225 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7938 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-01-27 r/5772 fix(tvix/cli): handle SRI hashes in outputHashFlorian Klink3-18/+146
Instead of being called with `md5`, `sha1`, `sha256` or `sha512`, `fetchurl.nix` (from corepkgs / `<nix`) can also be called with a `hash` attribute, being an SRI hash. In that case, `builtin.derivation` is called with `outputHashAlgo` being an empty string, and `outputHash` being an SRI hash string. In other cases, an SRI hash is passed as outputHash, but outputHashAlgo is set too. Nix does modify these values in (single, fixed) output specification it serializes to ATerm, but keeps it unharmed in `env`. Move this into a construct_output_hash helper function, that can be tested better in isolation. Change-Id: Id9d716a119664c44ea7747540399966752e20187 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7933 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-01-27 r/5770 feat(tvix/cli): faux implementation of builtins.toFileVincent Ambo1-0/+31
This adds an implementation of this builtin which correctly calculates paths, but does not actually write anything to the store or verify references. Change-Id: Ie9764cbc1d13a73d8dc9350910304e2b7cad3fe8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7910 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-01-27 r/5769 feat(tvix/cli): implement `builtins.derivation`Vincent Ambo3-0/+41
This uses the actual upstream Nix code for `builtins.derivation` (which is not a primop in C++ Nix) to implement `builtins.derivation` as a wrapper around `builtins.derivationStrict`. We're doing it this way to ensure that our thunking logic is correct. An initial Rust-native rewrite (see e.g. cl/7363) is pretty difficult to debug while there are still other issues to root out, but eventually we might want to turn this into native code. Change-Id: I5845e18073e103b8670e40648bd7fd9b511058e0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7902 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-01-27 r/5768 feat(tvix/cli): implement builtins.derivationStrictVincent Ambo2-4/+154
Implements the logic for converting an evaluator value supplied as arguments to builtins.derivationStrict into an actual, fully-functional derivation struct. This skips the implementation of structuredAttrs, which are left for a subsequent commit. Note: We will need to port some eval tests over to CLI to test this correct, which will be done in a separate commit later on. Change-Id: I0db69dcf12716180de0eb0b126e3da4683712966 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7756 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-01-27 r/5767 feat(tvix/cli): add helper for handling special drv parametersVincent Ambo1-0/+106
Adds a helper function which handles special parameters to `builtins.derivation` that are not just blindly passed through to the builder environment, but populate other specific fields of the derivation (outside of the ones handled by other, more complex helpers from previous commits). Change-Id: I82d1edf9af714fc4591e9071c0b83ece83be7eee Reviewed-on: https://cl.tvl.fyi/c/depot/+/7901 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-01-27 r/5766 feat(tvix/cli): add helper for populating drv output configurationVincent Ambo2-2/+125
This threads through the fields that control whether a derivation is a fixed-output derivation or not. Change-Id: I49739de178fed9f258291174ca1a2c15a7cf5c2a Reviewed-on: https://cl.tvl.fyi/c/depot/+/7900 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-01-27 r/5765 feat(tvix/cli): add helper for populating derivation inputsVincent Ambo1-0/+89
This adds a helper function which takes the output of the reference scanner used on derivation inputs and populates the `input_sources` and `input_derivations` field of the derivation accordingly. Note that we have a divergence from C++ Nix here, as we do not populate the entire FS closure of a literally referred derivation (and our standing theory is that this is unnecessary for nixpkgs). Change-Id: Id0f605dd8c0a82973c56605c2b8f478fc17777d6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7899 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-01-27 r/5764 feat(tvix/cli): add helper for populating derivation outputsVincent Ambo2-0/+89
Adds a small helper function which uses a Nix value supplied to `builtins.derivation{Strict}` to populate the `outputs` field of the `Derivation` struct. Change-Id: Iccc7a4f293b3d913140aed576a573a8992241e46 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7898 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-01-26 r/5760 fix(tvix/cli): correctly trim cppnix output in NixCompatIOVincent Ambo1-3/+4
We only stripped one of the two uses of this string, leading to extraneous newlines in the refscanner. Change-Id: I25d9119be082c487352f0cf66b97ecdcc3e1de06 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7932 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-01-20 r/5709 feat(tvix/cli): add `errors` module with drv construction errorsVincent Ambo3-0/+51
These will be threaded through to eval through the new `TvixError` variant. Change-Id: Ia0d3f8710dcf26bb95015cd2a6a2b2911f06343f Reviewed-on: https://cl.tvl.fyi/c/depot/+/7842 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-01-17 r/5677 refactor(tvix/cli): consistently assert type unity in known_pathsVincent Ambo1-46/+41
No situation should be allowed in which a path is inserted into known_paths with different types twice, which we previously enforced only for some path types. Change-Id: I8cb47d4b29c0aab3c58694f8b590e131deba7043 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7843 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-01-17 r/5673 feat(tvix/cli): add replacement strings tracking to KnownPathsVincent Ambo1-0/+29
Replacement strings are some weird internal feature of Nix that is required for calculating derivation hashes. We need to track these like other paths, as they need to be re-used on builds with dependencies on values from previous builds. Change-Id: Ie955b3fb5ae3685cfadfbe4d06ea6b5e219590c7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7828 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-01-17 r/5672 feat(tvix/cli): track known plain paths in NixCompatIOVincent Ambo3-27/+45
When adding things to a C++ Nix store, ensure that the path is tracked in the tracker. Since the mechanism for propagating the tracker instance isn't finalised yet, I've opted to take an Rc<RefCell> parameter for it. How exactly that ends up there is going to become clear in the next commits, but for now it's just instantiated in main with Default::default. Change-Id: I90f0b44f2d4f292dedc98ff1aa39041d279b61fd Reviewed-on: https://cl.tvl.fyi/c/depot/+/7833 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-01-17 r/5671 refactor(tvix/cli): reference scanner owns all the stringsVincent Ambo2-50/+29
This gets very complex very quickly otherwise, as all the construction paths for a reference scanner and all the access patterns for the KnownPaths structure are not yet fully understood. Change-Id: Ibadf1f18b476695f3c286fc6896ae557760edf63 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7827 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI