about summary refs log tree commit diff
path: root/tvix/eval/src/builtins/mod.rs
AgeCommit message (Collapse)AuthorFilesLines
2024-11-04 r/8890 refactor(tvix/eval): remove Value::Json and related functionalityBob van der Linden1-6/+9
Currently Value::Json is used in combination with VMRequest::ToJson to recursively convert tvix Value to serde_json::Value. This functionality is used in builtins.toJSON as well as derivation __structuredAttrs. Both Value::Json and VMRequest::ToJson were removed in this commit. Related functionality in vm.rs is also removed: vm.rs does not know about JSON anymore. Recursively converting to serde_json now happens without going through the VM. Thrown errors that are part of the value of toJSON are now directly propagated as ErrorKind, were-as previously there was a split between CatchableErrorKind and ErrorKind, where eventually CatchableErrorKind would be converted to ErrorKind::Catchable. Change-Id: I066f064926c491e4c087a984f07af43d19124cfe Reviewed-on: https://cl.tvl.fyi/c/depot/+/12732 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-08-19 r/8521 refactor(tvix/eval): remove use of imbl::OrdMapVincent Ambo1-3/+2
Removes imbl::OrdMap in favour of an Rc over the standard library's BTreeMap, which allows us to drop the imbl dependency completely. In my local tests this is actually slightly faster for `hello` and `firefox`. Change-Id: Ic9597ead4e98bf9530f290c6a94a3c5c3efd0acc Reviewed-on: https://cl.tvl.fyi/c/depot/+/12201 Reviewed-by: aspen <root@gws.fyi> Tested-by: BuildkiteCI
2024-08-19 r/8520 refactor(tvix/eval): remove use of imbl::VectorVincent Ambo1-28/+32
This vector type has served us well for now, but it contains internal refcounts which are incompatible with upcoming changes related to garbage collection. The performance impact of this change within all benchmarks I ran was within the margin of error: [nix-shell:/tmp/perf]$ hyperfine "./before -E '(import <nixpkgs> {}).firefox.outPath' --log-level ERROR --no-warnings" Benchmark 1: ./u64 -E '(import <nixpkgs> {}).firefox.outPath' --log-level ERROR --no-warnings Time (mean ± σ): 7.528 s ± 0.272 s [User: 6.578 s, System: 0.631 s] Range (min … max): 7.160 s … 8.012 s 10 runs nix-shell:/tmp/perf]$ hyperfine "./std-vec -E '(import <nixpkgs> {}).firefox.outPath' --log-level ERROR --no-warnings" Benchmark 1: ./std-vec -E '(import <nixpkgs> {}).firefox.outPath' --log-level ERROR --no-warnings Time (mean ± σ): 7.515 s ± 0.178 s [User: 6.508 s, System: 0.652 s] Range (min … max): 7.276 s … 7.861 s 10 runs Change-Id: Ib95f871956e336a1e5771f6293583854b1efb276 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12197 Reviewed-by: aspen <root@gws.fyi> Tested-by: BuildkiteCI
2024-08-07 r/8453 feat(tvix/eval): Forbid Hash{Map,Set}, use Fx insteadAspen Smith1-3/+4
Per https://nnethercote.github.io/perf-book/hashing.html, we have basically no reason to use the default hasher over a faster, non-DoS-resistant hasher. This gives a nice perf boost basically for free: hello outpath time: [704.76 ms 714.91 ms 725.63 ms] change: [-7.2391% -6.1018% -4.9189%] (p = 0.00 < 0.05) Performance has improved. Change-Id: If5587f444ed3af69f8af4eead6af3ea303b4ae68 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12046 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com> Autosubmit: aspen <root@gws.fyi>
2024-07-05 r/8343 refactor(tvix/eval): Drop LightSpan entirelyAspen Smith1-8/+4
This was made unnecessary in c92d06271 (feat(tvix/eval): drop LightSpan::Delayed, 2023-12-08) because it didn't improve benchmarks as much as expected and has been vestigial since; this continues the cleanup by just removing it altogether Change-Id: I21ec7ae9b52a5cccd2092696a5a87f658194d672 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11949 Autosubmit: aspen <root@gws.fyi> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-06-22 r/8303 refactor(tvix/eval): drop placeholder impls of builtins in glueIlan Joselevich1-16/+0
The builtins `placeholder` and `filterSource` are now implemented in tvix-glue and so we no longer need their placeholder "implementations" in tvix-eval. Change-Id: Ibf161ccf1ad40103e9fbb688e9f6be58e7772af1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11867 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
2024-06-11 r/8247 fix(tvix/eval): handle builtins.split matching the empty stringbinarycat1-0/+3
This prevents the following statements from looping endlessly: ``` builtins.split "(.*)" "" builtins.split "([abc]*)" "abc" builtins.split "(.*)" "abc" builtins.split ".*" "" ``` Cover these (and some more examples) in the test suite. Co-Authored-By: Florian Klink <flokli@flokli.de> Change-Id: Ibd339f971e0f4e3e5c229816e2be5a8e3836fec9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11743 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2024-05-23 r/8164 feat(tvix/eval): rm NixContext::join, add take_context & IntoIteratorFlorian Klink1-34/+24
In places where we want to extend context with that from another NixString, use take_context() to split it off, then call .extend(), making use of IntoIterator to avoid a bunch of clones. Change-Id: I2460141a3ed776c64c36132b2203b6a1d710b922 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11705 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: edef <edef@edef.eu>
2024-05-23 r/8163 feat(tvix/eval): teach builtins.toXML contextFlorian Klink1-2/+13
XmlEmitter gains a NixContext field, and `write_typed_value` extends it with all context elements present in the passed value. Once all serialization is done, a into_context() function returns the collected context, so we can construct a NixString with context. Tests for this live in tvix-glue, as we use builtins.derivation, which is not present in the tvix-eval crate. Fixes b/398. Change-Id: I85feaaa17b753885f8a017a54e419ec4e602af21 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11704 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Alyssa Ross <hi@alyssa.is>
2024-05-23 r/8161 fix(tvix/eval): NixString are bytesFlorian Klink1-1/+1
This is not necessarily valid UTF-8. Change-Id: I72f3157240772eb9c558e5699b4785e44d256fd4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11702 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2024-05-20 r/8156 fix(tvix/eval): use fake values for __curPos, rather than an errorFlorian Klink1-2/+9
Have this return the same values as builtins.unsafeGetAttrsPos, rather than returning a CatchableErrorKind, which crashes the CLI if it bubbles up. The environment we're in doesn't allow emitting a warning, as we don't have `co` in scope, but that's probably OK as a stopgap solution. Alternative to cl/11665. Change-Id: I5b2c2530842547c93b6533ed9601ee9b2923b1bf Reviewed-on: https://cl.tvl.fyi/c/depot/+/11685 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2024-04-13 r/7909 fix(tvix/eval): fix the `unsafeGetAttrPos` stub return valueRyan Lahfa1-1/+1
It was returning `{ line, col, file }` instead of `{ line, column, file }`. Note this is not a regression in Nix, Nix always had `column` since 2013. Change-Id: I4871a201b32324dff0432526d0449806a535828f Signed-off-by: Ryan Lahfa <tvl@lahfa.xyz> Reviewed-on: https://cl.tvl.fyi/c/depot/+/11409 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-04-13 r/7899 feat(tvix/eval): contextful JSON operationsRyan Lahfa1-3/+3
`toJSON` transform a Nix structure into a JSON string. For each context in that Nix structure, the JSON string must possess it. Thus, it is necessary to take the union of all contexts and attach it to the final structure. Unfortunately, the return type of `into_json` is a serde's JSON object, not a string. Therefore, it is not possible to reuse `NixString` machinery. Context tests are reinforced as Nix does not test those behaviors. Fixes b/393. Change-Id: I5afdbc4e18dd70469192c1aa657d1049ba330149 Signed-off-by: Ryan Lahfa <tvl@lahfa.xyz> Reviewed-on: https://cl.tvl.fyi/c/depot/+/11266 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-04-09 r/7882 fix(tvix): Avoid buffering file into memory in builtins.hashFileConnor Brewster1-3/+2
Right now `builtins.hashFile` always reads the entire file into memory before hashing, which is not ideal for large files. This replaces `read_to_string` with `open_file` which allows calculating the hash of the file without buffering it entirely into memory. Other callers can continue to buffer into memory if they choose, but they still use the `open_file` VM request and then call `read_to_string` or `read_to_end` on the `std::io::Reader`. Fixes b/380 Change-Id: Ifa1c8324bcee8f751604b0b449feab875c632fda Reviewed-on: https://cl.tvl.fyi/c/depot/+/11236 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-04-04 r/7851 feat(tvix/eval): implement `appendContext`Ryan Lahfa1-0/+80
`appendContext s ctx` will just append a user-crafted context attrs to `s`. The most important part of this builtin is to perform all the relevant invariant validations to avoid letting the user craft invalid contexts which can never be built, e.g. invalid store paths, inexistent derivations, etc. This version is incomplete and full of TODOs, but passes all the Nix's context strings tests, so we turn them on. Change-Id: I625dc5e7c4f5b784f078b390f04b0ee5a8d65a7c Signed-off-by: Ryan Lahfa <tvl@lahfa.xyz> Reviewed-on: https://cl.tvl.fyi/c/depot/+/11263 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-03-31 r/7806 feat(tvix/eval): implement `unsafeDiscardOutputDependency`Ryan Lahfa1-6/+36
This builtin only transforms any `NixContextElement::Derivation` into the trivial `NixContextElement::Plain`. This is a forgetful functor on derivation-deep context strings. The test coverage of this change is done in cl/11264. Change-Id: Icd00778c97766be6db8a6bdabaa59e9724353ec5 Signed-off-by: Ryan Lahfa <tvl@lahfa.xyz> Reviewed-on: https://cl.tvl.fyi/c/depot/+/11262 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-03-28 r/7801 feat(tvix/eval): non-implement `unsafeDiscardOutputDependency`Ryan Lahfa1-0/+13
Nixpkgs make use of it and this is a 2.3 builtin that was missed. Change-Id: If36d748eed4a5f777344556c0802392c76c8d53c Signed-off-by: Ryan Lahfa <tvl@lahfa.xyz> Reviewed-on: https://cl.tvl.fyi/c/depot/+/11260 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-03-19 r/7736 fix(tvix/eval): propagate errs from key comparison in genericClosuresterni1-7/+15
The accompanying test case shows that we need to bubble up the catchable error from the equality check if one is created. Change-Id: Ic9929a57aa7653c8aa5a72d1711cf3264798c731 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11159 Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: tazjin <tazjin@tvl.su>
2024-03-19 r/7735 fix(tvix/eval): don't force lambda arg in map, mapAttrs & genListsterni1-3/+7
It is pretty pointless to force the function argument if we are going to use a suspended call later since forcing the function may fail in ways that are not covered by Catchables (non-recoverable errors, infinite recursions). From this, it kind of seems as if using #[catch] is never correct and should be replaced by #[lazy]. Also we should probably try to come up with more test cases for stuff where laziness gets us out of the jam as an equivalent to the catchable tests for nonrecoverable errors. Fixes b/386. Change-Id: Ia926df4ac1b440ec430403ab7b40924a0c97221b Reviewed-on: https://cl.tvl.fyi/c/depot/+/11153 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org>
2024-03-16 r/7704 fix(tvix/eval): builtins.unsafeGetAttrPos returns path as a stringsterni1-1/+1
Change-Id: Ia1058a5ea687f413fbb62365da199032bdbd3fd2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11155 Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org>
2024-02-29 r/7625 feat(tvix/eval): implement `builtins.hashFile`Padraic-O-Mhuiris1-22/+3
The primary addition is the `builtin_hashFile` function which attempts value-path coercion and on success reads the file to a nix_string. Similar to the `builtins_hashString` implementation, the hash is then derived using the `hash_nix_string` function in the introduced hash module which has extracted hashing functionality initially specified in cl/11005. Change-Id: I5e3ef0317223af99488ebb79efc2fb49b4cbc1b6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11007 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-02-23 r/7597 feat(tvix/eval): implement `builtins.hashString`Padraic-O-Mhuiris1-9/+23
Implements md5, sha1, sha256 and sha512 using the related crates from the RustCrypto hashes project (https://github.com/RustCrypto/hashes) Change-Id: I00730dea44ec9ef85309edc27addab0ae88814b8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11005 Tested-by: BuildkiteCI Reviewed-by: aspen <root@gws.fyi>
2024-02-20 r/7580 fix(tvix/eval): Propagate catchables in builtins.groupByAspen Smith1-4/+6
One last place where we needed to wrap the forcing of the element of a list in `try_value!`. This fixes a previously `notyetpassing` test Change-Id: I8827a3e39630e6959013b70bdfa9cbcb93f4e91c Reviewed-on: https://cl.tvl.fyi/c/depot/+/10789 Autosubmit: aspen <root@gws.fyi> Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2024-02-13 r/7508 feat(tvix/eval): Box Value::CatchableAspen Smith1-9/+9
This is now the only enum variant for Value that is larger than 8 bytes (it's 16 bytes), so boxing it (especially since it's not perf-critical) allows us to get the Value size down to only 16 bytes! Change-Id: I98598e2b762944448bef982e8ff7da6d6683c4aa Reviewed-on: https://cl.tvl.fyi/c/depot/+/10798 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: aspen <root@gws.fyi>
2024-02-13 r/7507 revert(tvix/eval): Don't double-box Path valuesAspen Smith1-4/+4
This reverts commit d3d41552cf1f6485f8ebc597a2128a0d15b030a5. This was well-intentioned, but now the boxed Path values are actually the *largest* Value enum variants, at 16 bytes (because they're fat-pointers, with a len) instead of 8 bytes like all the other values. Having the double reference is a reasonable price to pay (it seems; more benchmarks may end up disagreeing) for a smaller Value repr. Change-Id: I0d3e84f646c8f5ffd0b7259c4e456637eea360f7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10797 Tested-by: BuildkiteCI Autosubmit: aspen <root@gws.fyi> Reviewed-by: sterni <sternenseemann@systemli.org>
2024-02-13 r/7506 fix(tvix/eval): Replace inner NixString repr with Box<Bstr>Aspen Smith1-25/+21
Storing a full BString here incurs the extra overhead of the capacity for the inner byte-vector, which we basically never use as Nix strings are immutable (and we don't do any mutation / sharing analysis). Switching to a Box<BStr> cuts us from 72 bytes to 64 bytes per string (and there are a lot of strings!) Change-Id: I11f34c14a08fa02759f260b1c78b2a2b981714e4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10794 Autosubmit: aspen <root@gws.fyi> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-02-10 r/7497 refactor(tvix/eval): Box the strings in CatchableErrorKindAspen Smith1-6/+4
These strings are allocated once and never changed, so they don't need the additional overhead of a capacity given by String - instead, we can use Box<str> and save on 16 bytes for each of these, *and* for each Value since this is currently the largest Value variant. Change-Id: I3e5cb070fe6c5bf82114c92d04f6bae775663a7e Reviewed-on: https://cl.tvl.fyi/c/depot/+/10796 Autosubmit: aspen <root@gws.fyi> Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2024-02-08 r/7488 fix(tvix/eval): Inline List.sort_by, and propagate errorsAspen Smith1-3/+45
In order to correctly propagate errors in the comparator passed to builtins.sort, we need to do all the sorting in a context where we can short-circuit return `Value`s (because catchables are Values on the `Ok` side of the Result , not `Err`s). Unfortunately this means we have to *inline* the List `sort_by` implementation into the builtin_sort function - fortunately this is the only place that was called so this is relatively low cost. This does that, and adds the requisite `try_value!` invocation to allow us to propagate comparator errors here. As before, this doesn't include tests, primarily since those are coming in the next commit. Change-Id: I8453c3aa2cd82299eae89828e2a2bb118da4cd48 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10754 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-02-08 r/7487 fix(tvix): Catch errors for generator in some builtinsAspen Smith1-2/+3
Nix doesn't propagate errors for the function argument to some builtins, like genList and map: ❯ nix repl Welcome to Nix version 2.3.17. Type :? for help. nix-repl> (builtins.tryEval (builtins.genList (builtins.throw "a") 10)).success true nix-repl> (builtins.tryEval (builtins.map (builtins.throw "a") [ "" ])).success true Note that this is untested as of this particular commit, only because a big test suite covering all sorts of catchable error propagation issues is coming next Change-Id: I48c8eb390a541204b1a6d438c753fa1ca9b3877e Reviewed-on: https://cl.tvl.fyi/c/depot/+/10753 Autosubmit: aspen <root@gws.fyi> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-02-08 r/7483 fix(tvix/eval): Propagate catchables in a few more placesAspen Smith1-9/+10
Propagate catchables that we get from forcing thunks in builtins in a few more places using the new try_value! macro Change-Id: I95fd41a231f877ff153f4adbabd944372d4cc7eb Reviewed-on: https://cl.tvl.fyi/c/depot/+/10738 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-02-08 r/7482 refactor(tvix/eval): Generalize propagation of catchable valuesAspen Smith1-177/+13
Rather than explicitly checking for Value::Catchable in all builtins, make the #[builtin] proc macro insert this for all strict arguments by default, with support for a #[catch] attribute on the argument to disable this behavior. That attribute hasn't actually been *used* anywhere here, primarily because the tests pass without it, even for those builtins which weren't previously checking for Value::Catchable - if some time passes without this being used I might get rid of support for it entirely. There's also a `try_value` macro in builtins directly for the places where builtins were eg forcing something, then explicitly propagating a catchable value. Change-Id: Ie22037b9d3e305e3bdb682d105fe467bd90d53e9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10732 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-02-02 r/7467 refactor(tvix/eval): Box Value::StringAspen Smith1-13/+12
NixString is *quite* large - like 80 bytes - because of the extra capacity value for BString and because of the context. We want to keep Value small since we're passing it around a lot, so let's box the NixString inside Value::String to save on some memory, and make cloning ostensibly a little cheaper Change-Id: I343c8b4e7f61dc3dcbbaba4382efb3b3e5bbabb2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10729 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2024-02-01 r/7461 refactor(tvix/eval): Don't double-box Path valuesAspen Smith1-4/+4
PathBuf internally contains a heap pointer (an OsString), so we were in effect double-boxing here. Removing the extra layer by making Tvix::Value represented by a Box<Path> rather than a Box<PathBuf> saves us an indirection, while still avoiding the extra memory overhead of the capacity which was the reason we were boxing PathBuf in the first place. Change-Id: I8c185b9d4646161d1921917f83e87421496a3e24 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10725 Reviewed-by: sterni <sternenseemann@systemli.org> Autosubmit: aspen <root@gws.fyi> Tested-by: BuildkiteCI
2024-01-31 r/7460 fix(tvix): Represent strings as byte arraysAspen Smith1-47/+62
C++ nix uses C-style zero-terminated char pointers to represent strings internally - however, up to this point, tvix has used Rust `String` and `str` for string values. Since those are required to be valid utf-8, we haven't been able to properly represent all the string values that Nix supports. To fix that, this change converts the internal representation of the NixString struct from `Box<str>` to `BString`, from the `bstr` crate - this is a wrapper around a `Vec<u8>` with extra functions for treating that byte vector as a "morally string-like" value, which is basically exactly what we need. Since this changes a pretty fundamental assumption about a pretty core type, there are a *lot* of changes in a lot of places to make this work, but I've tried to keep the general philosophy and intent of most of the code in most places intact. Most notably, there's nothing that's been done to make the derivation stuff in //tvix/glue work with non-utf8 strings everywhere, instead opting to just convert to String/str when passing things into that - there *might* be something to be done there, but I don't know what the rules should be and I don't want to figure them out in this change. To deal with OS-native paths in a way that also works in WASM for tvixbolt, this also adds a dependency on the "os_str_bytes" crate. Fixes: b/189 Fixes: b/337 Change-Id: I5e6eb29c62f47dd91af954f5e12bfc3d186f5526 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10200 Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de> Reviewed-by: sterni <sternenseemann@systemli.org> Autosubmit: aspen <root@gws.fyi> Tested-by: BuildkiteCI
2024-01-24 r/7446 feat(tvix/eval): expose value_to_xml for test casesFlorian Klink1-0/+3
It's debateable on whether the serialization code should be exposed a bit more prominently or not. Change-Id: Iff7a28f884b1490b12b145dfdadbedacb84fd387 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10684 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org> Reviewed-by: tazjin <tazjin@tvl.su>
2024-01-17 r/7406 fix(tvix/eval): catchable-aware `throw`Ryan Lahfa1-0/+4
`throw (throw "a")` should work and propagate the internal throw. Before this commit, it didn't work. Change-Id: Id5d46f74e484dba99e912ad9fa211f3bf1617bac Reviewed-on: https://cl.tvl.fyi/c/depot/+/10600 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-01-17 r/7405 fix(tvix/eval): catchable-aware `elem`Ryan Lahfa1-0/+4
`elem` did not catch the list being a catchable. This surfaced during Nixpkgs evaluation. Change-Id: Icf19b94e914e35a435c4412d769ee63ba59ab7b0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10599 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-01-17 r/7403 fix(tvix/eval): `getContext` merges underlying valuesRyan Lahfa1-17/+65
Previously, we were assembling very naively an attribute set composed of context we saw. But it was forgetting that `"${drv}${drv.drvPath}"` would contain 2 contexts with the same key, but with different values, one with `outputs = [ "out" ];` and `allOutputs = true;`. Following this reasoning and comparing with what Nix does, we ought to merge underlying values systematically. Hence, I bring `itertools` to perform a group by on the key and merge everything on the fly, it's not beautiful but it's the best I could find, notice that I don't use `group_by` but I talk about group by, that is, because `group_by` is a `group_by_consecutive`, see https://github.com/rust-itertools/itertools/issues/374. Initially, I tried to do it without a `into_grouping_map_by`, it was akin to assemble the final `NixAttrs` directly, it was less readable and harder to pull out because we don't have a lot of in-place mutable functions on our data structures. Change-Id: I9933c9bd88ffe04de50dda14f21879b60d8b8cd4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10620 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2024-01-17 r/7400 fix(tvix/eval): context-aware `dirOf`Ryan Lahfa1-1/+1
`dirOf` forgot to accepts contextful strings, e.g. derivations and propagates this context further. Change-Id: I6c05944a3ce5073e243e7676c9be56c48407d657 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10618 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-01-17 r/7399 fix(tvix/eval): context-aware… `hasContext`Ryan Lahfa1-1/+1
Yes, `hasContext e` should work where `e` is a contextful strings, otherwise, it is really useless. Change-Id: I5eb071fc257217d6e8a63fe519132ebd98186696 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10617 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-01-14 r/7379 fix(tvix/eval): catchable-aware builtinsRyan Lahfa1-0/+235
A bunch of operations in Tvix are not aware of catchable values and does not propagate them. In the meantime, as we wait for a better solution, we just offer this commit for moving the needle. Change-Id: Ic3f0e1550126b0847b597dfc1402c35e0eeef469 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10473 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2024-01-03 r/7340 feat(tvix/eval): context-aware `split`Ryan Lahfa1-5/+14
Nix does something like: ```cpp NixStringContext context; const auto str = state.forceString(*args[1], context, pos, "while evaluating the second argument passed to builtins.split"); ``` And then do nothing with that context, therefore, we follow them and make `split` aware of the context but still do nothing with it. Change-Id: I4fee1936600ce86d99d00893ca3f64013213935b Reviewed-on: https://cl.tvl.fyi/c/depot/+/10428 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: raitobezarius <tvl@lahfa.xyz>
2024-01-03 r/7339 feat(tvix/eval): impl `unsafeDiscardStringContext`Ryan Lahfa1-4/+19
Change-Id: I7f0cc42cbebfe5cd27bf6d4f58a4af927b83646a Reviewed-on: https://cl.tvl.fyi/c/depot/+/10423 Autosubmit: raitobezarius <tvl@lahfa.xyz> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2024-01-03 r/7332 feat(tvix/eval): `match` DO NOT propagate contextRyan Lahfa1-6/+13
`match` silently ignore the input context and do not propagate it and successful matches. The why is unclear but nixpkgs does rely implicitly on this behavior because dynamic attribute selection cannot be done with contextful strings. Change-Id: I5167fa9b2c2db8ecab0c2fb3e9895c9cfce6eeb2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10441 Autosubmit: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2024-01-03 r/7331 feat(tvix/eval): implement `getContext` primopRyan Lahfa1-0/+43
Change-Id: I2c5068a28f9883a01b0ff80a5e5ab32ba18bfc1a Reviewed-on: https://cl.tvl.fyi/c/depot/+/10437 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-01-03 r/7330 feat(tvix/eval): context-aware `replaceStrings`Ryan Lahfa1-6/+23
And it also preserve the original context if it exists. Change-Id: I904f7c13b7f003a267aace6301723780fccaafb7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10434 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: raitobezarius <tvl@lahfa.xyz>
2024-01-03 r/7329 chore(tvix/eval): note on context-aware `hashString`Ryan Lahfa1-0/+1
It must propagate context too. Change-Id: If57c22c9723ea02aa013f69d3dcf96054476d8de Reviewed-on: https://cl.tvl.fyi/c/depot/+/10433 Tested-by: BuildkiteCI Autosubmit: raitobezarius <tvl@lahfa.xyz> Reviewed-by: tazjin <tazjin@tvl.su>
2024-01-03 r/7327 feat(tvix/eval): context-aware `concatStringsSep`Ryan Lahfa1-3/+19
Change-Id: Id0e169084d26dc598091d157563c4d959b66279b Reviewed-on: https://cl.tvl.fyi/c/depot/+/10431 Autosubmit: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2024-01-03 r/7326 chore(tvix/eval): note on context-aware `toString`Ryan Lahfa1-0/+2
Change-Id: Ie26ebd16e95e6a7b6f81051d8269169842978058 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10430 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: raitobezarius <tvl@lahfa.xyz>
2024-01-03 r/7325 feat(tvix/eval): context-aware `throw`Ryan Lahfa1-1/+2
Change-Id: Ie552dabe4cf93cc396c883268a3bee67796dbbd8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10429 Autosubmit: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>