about summary refs log tree commit diff
path: root/tvix/eval/src/value
AgeCommit message (Collapse)AuthorFilesLines
2024-04-22 r/7991 fix(tvix/eval): don't impl From<NixString> for StringFlorian Klink1-5/+1
This caught me by accident in an earlier revision of cl/11500 - I had a `NixString`, wanted to return it as a `String`, so I was naively calling `s.into()`. That unfortunately gave me the `Display` implementation of `NixString`, which quotes strings, causing an annoying error further up the stack. NixStrings are bytes, we can keep the impl From<NixString> for BString, but having a `.into()` suddenly do quoting is more than unexpected. Change-Id: I5434ba94bfe6c493d0a57e68225ecc22daa4b948 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11505 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-04-13 r/7899 feat(tvix/eval): contextful JSON operationsRyan Lahfa2-19/+56
`toJSON` transform a Nix structure into a JSON string. For each context in that Nix structure, the JSON string must possess it. Thus, it is necessary to take the union of all contexts and attach it to the final structure. Unfortunately, the return type of `into_json` is a serde's JSON object, not a string. Therefore, it is not possible to reuse `NixString` machinery. Context tests are reinforced as Nix does not test those behaviors. Fixes b/393. Change-Id: I5afdbc4e18dd70469192c1aa657d1049ba330149 Signed-off-by: Ryan Lahfa <tvl@lahfa.xyz> Reviewed-on: https://cl.tvl.fyi/c/depot/+/11266 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-03-31 r/7806 feat(tvix/eval): implement `unsafeDiscardOutputDependency`Ryan Lahfa1-0/+6
This builtin only transforms any `NixContextElement::Derivation` into the trivial `NixContextElement::Plain`. This is a forgetful functor on derivation-deep context strings. The test coverage of this change is done in cl/11264. Change-Id: Icd00778c97766be6db8a6bdabaa59e9724353ec5 Signed-off-by: Ryan Lahfa <tvl@lahfa.xyz> Reviewed-on: https://cl.tvl.fyi/c/depot/+/11262 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-02-23 r/7597 feat(tvix/eval): implement `builtins.hashString`Padraic-O-Mhuiris1-0/+6
Implements md5, sha1, sha256 and sha512 using the related crates from the RustCrypto hashes project (https://github.com/RustCrypto/hashes) Change-Id: I00730dea44ec9ef85309edc27addab0ae88814b8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11005 Tested-by: BuildkiteCI Reviewed-by: aspen <root@gws.fyi>
2024-02-21 r/7591 feat(tvix/eval): Store string context alongside dataAspen Smith4-55/+359
Previously, Nix strings were represented as a Box (within Value) pointing to a tuple of an optional context, and another Box pointing to the actual string allocation itself. This is pretty inefficient, both in terms of memory usage (we use 48 whole bytes for a None context!) and in terms of the extra indirection required to get at the actual data. It was necessary, however, because with native Rust DSTs if we had something like `struct NixString(Option<NixContext>, BStr)` we could only pass around *fat* pointers to that value (with the length in the pointer) and that'd make Value need to be bigger (which is a waste of both memory and cache space, since that memory would be unused for all other Values). Instead, this commit implements *manual* allocation of a packed string representation, with the length *in the allocation* as a field past the context. This requires a big old pile of unsafe Rust, but the payoff is clear: hello outpath time: [882.18 ms 897.16 ms 911.23 ms] change: [-15.143% -13.819% -12.500%] (p = 0.00 < 0.05) Performance has improved. Fortunately this change can be localized entirely within value/string.rs, since we were abstracting things out nicely. Change-Id: Ibf56dd16c9c503884f64facbb7f0ac596463efb6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10852 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: aspen <root@gws.fyi>
2024-02-20 r/7577 fix(tvix/eval): propagate catchable errors at the top of an evalVincent Ambo1-0/+1
(Re-)Adds an error variant that wraps a catchable error kind, which is used for returning the result of an evaluation. Previously this would return the internal catchable value, which would lead to panics if users tried to use these. Somehow this was missed; I think we need error output tests. Change-Id: Id6e24aa2ce4ea4358a29b2e1cf4a6749986baf8c Reviewed-on: https://cl.tvl.fyi/c/depot/+/10991 Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de>
2024-02-13 r/7509 test(tvix/eval): Add test asserting size of ValueAspen Smith1-0/+8
Now that I've done a ton of things to make sure Value is small on the stack (16 bytes, which is a perfectly reasonable size for a programming language Value enum), add a test asserting it stays that way. These size improvements have a measurable impact, too - here's the `hello outpath` benchmark compared between canon (as of r/7495) and this commit: hello outpath time: [990.56 ms 995.83 ms 1.0070 s] change: [-7.1397% -6.1302% -5.1651%] (p = 0.00 < 0.05) Performance has improved. Change-Id: If99a0976eab28eb5e516fcd2f4a0e068145af23e Reviewed-on: https://cl.tvl.fyi/c/depot/+/10799 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Autosubmit: aspen <root@gws.fyi> Reviewed-by: sterni <sternenseemann@systemli.org>
2024-02-13 r/7508 feat(tvix/eval): Box Value::CatchableAspen Smith3-9/+14
This is now the only enum variant for Value that is larger than 8 bytes (it's 16 bytes), so boxing it (especially since it's not perf-critical) allows us to get the Value size down to only 16 bytes! Change-Id: I98598e2b762944448bef982e8ff7da6d6683c4aa Reviewed-on: https://cl.tvl.fyi/c/depot/+/10798 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: aspen <root@gws.fyi>
2024-02-13 r/7507 revert(tvix/eval): Don't double-box Path valuesAspen Smith3-10/+10
This reverts commit d3d41552cf1f6485f8ebc597a2128a0d15b030a5. This was well-intentioned, but now the boxed Path values are actually the *largest* Value enum variants, at 16 bytes (because they're fat-pointers, with a len) instead of 8 bytes like all the other values. Having the double reference is a reasonable price to pay (it seems; more benchmarks may end up disagreeing) for a smaller Value repr. Change-Id: I0d3e84f646c8f5ffd0b7259c4e456637eea360f7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10797 Tested-by: BuildkiteCI Autosubmit: aspen <root@gws.fyi> Reviewed-by: sterni <sternenseemann@systemli.org>
2024-02-13 r/7506 fix(tvix/eval): Replace inner NixString repr with Box<Bstr>Aspen Smith1-23/+57
Storing a full BString here incurs the extra overhead of the capacity for the inner byte-vector, which we basically never use as Nix strings are immutable (and we don't do any mutation / sharing analysis). Switching to a Box<BStr> cuts us from 72 bytes to 64 bytes per string (and there are a lot of strings!) Change-Id: I11f34c14a08fa02759f260b1c78b2a2b981714e4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10794 Autosubmit: aspen <root@gws.fyi> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-02-10 r/7496 refactor(tvix/eval): Box the inside of Value::JsonAspen Smith2-2/+2
serde_json::Value is pretty large, and is contributing (albeit not exclusively) to the large size of the Value repr. Putting it in a box is *especially* cheap (since it's rarely used) and allows us to (eventually) cut down on the size of Value. Change-Id: I005a802d8527b639beb4e938e3320b11ffa1ef23 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10795 Reviewed-by: sterni <sternenseemann@systemli.org> Autosubmit: aspen <root@gws.fyi> Tested-by: BuildkiteCI
2024-02-09 r/7492 fix(tvix/eval): Propagate catchables in NixAttrs::constructAspen Smith2-11/+25
Correctly propagate the case where the *key* of an attrset is a Value::Catchable (eg { "${builtins.throw "c"}" = "b"; }) in `NixAttrs::construct`, by converting the return type to `Result<Result<Self, CatchableErrorKind>, ErrorKind>` (ugh!!) and correctly handling that everywhere (including an `expect` in the Deserialize impl for NixAttrs, since afaict this is impossible to hit when deserializing from stuff like JSON). Change-Id: Ic4bc611fbfdab27c0bd8a40759689a87c4004a17 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10786 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-02-08 r/7488 fix(tvix/eval): Inline List.sort_by, and propagate errorsAspen Smith1-50/+0
In order to correctly propagate errors in the comparator passed to builtins.sort, we need to do all the sorting in a context where we can short-circuit return `Value`s (because catchables are Values on the `Ok` side of the Result , not `Err`s). Unfortunately this means we have to *inline* the List `sort_by` implementation into the builtin_sort function - fortunately this is the only place that was called so this is relatively low cost. This does that, and adds the requisite `try_value!` invocation to allow us to propagate comparator errors here. As before, this doesn't include tests, primarily since those are coming in the next commit. Change-Id: I8453c3aa2cd82299eae89828e2a2bb118da4cd48 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10754 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-02-02 r/7467 refactor(tvix/eval): Box Value::StringAspen Smith4-26/+26
NixString is *quite* large - like 80 bytes - because of the extra capacity value for BString and because of the context. We want to keep Value small since we're passing it around a lot, so let's box the NixString inside Value::String to save on some memory, and make cloning ostensibly a little cheaper Change-Id: I343c8b4e7f61dc3dcbbaba4382efb3b3e5bbabb2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10729 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2024-02-01 r/7463 feat(tvix/eval): Don't emit OpForce for non-thunk constantsAspen Smith1-0/+7
In the compiler, skip emitting an OpForce if the last op was an OpConstant for a non-thunk constant. This gives a small (~1% on my machine) perf boost, eg when evaluating hello.outPath: ❯ hyperfine \ "./before --no-warnings -E '(import <nixpkgs> {}).hello.outPath'" \ "./after --no-warnings -E '(import <nixpkgs> {}).hello.outPath'" Benchmark 1: ./before --no-warnings -E '(import <nixpkgs> {}).hello.outPath' Time (mean ± σ): 1.151 s ± 0.022 s [User: 1.003 s, System: 0.151 s] Range (min … max): 1.123 s … 1.184 s 10 runs Benchmark 2: ./after --no-warnings -E '(import <nixpkgs> {}).hello.outPath' Time (mean ± σ): 1.140 s ± 0.022 s [User: 0.989 s, System: 0.152 s] Range (min … max): 1.115 s … 1.175 s 10 runs Summary ./after --no-warnings -E '(import <nixpkgs> {}).hello.outPath' ran 1.01 ± 0.03 times faster than ./before --no-warnings -E '(import <nixpkgs> {}).hello.outPath' Change-Id: I2105fd431d4bad699087907e16c789418e9a4062 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10714 Reviewed-by: sterni <sternenseemann@systemli.org> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2024-02-01 r/7461 refactor(tvix/eval): Don't double-box Path valuesAspen Smith3-10/+10
PathBuf internally contains a heap pointer (an OsString), so we were in effect double-boxing here. Removing the extra layer by making Tvix::Value represented by a Box<Path> rather than a Box<PathBuf> saves us an indirection, while still avoiding the extra memory overhead of the capacity which was the reason we were boxing PathBuf in the first place. Change-Id: I8c185b9d4646161d1921917f83e87421496a3e24 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10725 Reviewed-by: sterni <sternenseemann@systemli.org> Autosubmit: aspen <root@gws.fyi> Tested-by: BuildkiteCI
2024-01-31 r/7460 fix(tvix): Represent strings as byte arraysAspen Smith5-87/+142
C++ nix uses C-style zero-terminated char pointers to represent strings internally - however, up to this point, tvix has used Rust `String` and `str` for string values. Since those are required to be valid utf-8, we haven't been able to properly represent all the string values that Nix supports. To fix that, this change converts the internal representation of the NixString struct from `Box<str>` to `BString`, from the `bstr` crate - this is a wrapper around a `Vec<u8>` with extra functions for treating that byte vector as a "morally string-like" value, which is basically exactly what we need. Since this changes a pretty fundamental assumption about a pretty core type, there are a *lot* of changes in a lot of places to make this work, but I've tried to keep the general philosophy and intent of most of the code in most places intact. Most notably, there's nothing that's been done to make the derivation stuff in //tvix/glue work with non-utf8 strings everywhere, instead opting to just convert to String/str when passing things into that - there *might* be something to be done there, but I don't know what the rules should be and I don't want to figure them out in this change. To deal with OS-native paths in a way that also works in WASM for tvixbolt, this also adds a dependency on the "os_str_bytes" crate. Fixes: b/189 Fixes: b/337 Change-Id: I5e6eb29c62f47dd91af954f5e12bfc3d186f5526 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10200 Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de> Reviewed-by: sterni <sternenseemann@systemli.org> Autosubmit: aspen <root@gws.fyi> Tested-by: BuildkiteCI
2024-01-25 r/7448 feat(tvix/eval): track pattern binding namesFlorian Klink1-0/+4
These need to be preserved at least for builtins.toXML. Also, we incorrectly only wrote an <attrspat> in case ellipsis was true, but that's not the case. Change-Id: I6bff9c47c2922f878d5c43e48280cda9c9ddb692 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10686 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: aspen <root@gws.fyi>
2024-01-24 r/7447 fix(tvix/eval/value/function): use BTreeMap for function arg namesFlorian Klink1-3/+3
At least toXML wants to get these out in a sorted fashion. Change-Id: I6373d7488fff7c40dc2ddeeecd03ba537c92c4af Reviewed-on: https://cl.tvl.fyi/c/depot/+/10685 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2024-01-19 r/7422 chore(3p/sources): Bump channels & overlayssterni1-5/+1
- Adjust to ecl 23.9.9 release - Regenerate go protos after protoc-gen-go update - Drop dhall fork which hasn't kept up with 1.42.* - Address new clippy warnings: - Variant naming of Error::ValidationError - Simplify .try_into().unwrap() - Drop unnecessary identity function - Test module must be last in file - Drop unused `pub use` - Update agenix to 0.15.0. Current master has a installCheckPhase that doesn't work with C++ Nix 2.3.*: https://github.com/ryantm/agenix/commit/a23aa271bec82d3e962bafb994595c1c4a62b133#commitcomment-137185861 Change-Id: Ic29eef20d6fd1362ce1031364a5ca6b4edf195bd Reviewed-on: https://cl.tvl.fyi/c/depot/+/10615 Reviewed-by: aspen <root@gws.fyi> Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org>
2024-01-14 r/7379 fix(tvix/eval): catchable-aware builtinsRyan Lahfa2-13/+22
A bunch of operations in Tvix are not aware of catchable values and does not propagate them. In the meantime, as we wait for a better solution, we just offer this commit for moving the needle. Change-Id: Ic3f0e1550126b0847b597dfc1402c35e0eeef469 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10473 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2024-01-12 r/7375 feat(tvix/eval): make into_json publicFlorian Klink1-1/+1
Allow other crates (like tvix-glue) to look at a Value in JSON, which is used by the structured attrs feature. Change-Id: Iba02ace6e11a74c3f9b19dcbef4b008b76dec046 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10602 Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-01-03 r/7335 feat(tvix/eval): contextful coercion of filesRyan Lahfa1-0/+5
In the past reference tracking system, `tvix-io` glue was appending plain paths in the known path state. Now, we make up for this by just making contextful coercion of file imports. Change-Id: Ieb9b04dd83302c77909252d5f7733857ac3cf8fd Reviewed-on: https://cl.tvl.fyi/c/depot/+/10443 Tested-by: BuildkiteCI Autosubmit: raitobezarius <tvl@lahfa.xyz> Reviewed-by: tazjin <tazjin@tvl.su>
2024-01-03 r/7333 feat(tvix/eval): contextful == of derivationsRyan Lahfa1-2/+6
Otherwise, you just fail because they are not... contextless strings! Change-Id: I0b8f63a18cd89c3841b613d41c12ec4ee336f953 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10442 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-01-03 r/7331 feat(tvix/eval): implement `getContext` primopRyan Lahfa1-0/+4
Change-Id: I2c5068a28f9883a01b0ff80a5e5ab32ba18bfc1a Reviewed-on: https://cl.tvl.fyi/c/depot/+/10437 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-01-03 r/7317 feat(tvix/eval): context-aware `coerce_to_string`Ryan Lahfa2-2/+18
I am still undecided whether we need a CoercionKind to control the coerced context, here's a simple attempt. Change-Id: Ibe59d09ef26c519a6acfdfe392014446646dd6d8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10426 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: raitobezarius <tvl@lahfa.xyz>
2023-12-29 r/7282 feat(tvix/eval): context-aware casting to stringsRyan Lahfa1-2/+23
By default, we don't want contextful strings and we almost always want contextless strings. To this end, we make taking a contextful string a very explicit operation under `to_contextful_str` and we implement manually the `to_str` cast which requires a `if !s.has_context()` guard that the macro cannot cover. Change-Id: I7aae8e57a7d73e547e62b1edb0b1cc7e8c0c69b6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10425 Autosubmit: raitobezarius <tvl@lahfa.xyz> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-12-29 r/7280 feat(tvix/eval): introduce `NixContext`Ryan Lahfa2-7/+192
This prepares the data structures to implement string contexts in Nix. Change-Id: Idd913c9c881daeb8d446907f4b940e462e730978 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10420 Autosubmit: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-12-29 r/7274 fix(tvix/eval): catchable in type field of nix_eq()Adam Joseph1-2/+10
Change-Id: I165ff77764e272cc94d18cb03ad6cbc9a8ebefde Reviewed-on: https://cl.tvl.fyi/c/depot/+/10348 Autosubmit: Adam Joseph <adam@westernsemico.com> Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2023-12-29 r/7272 refactor(tvix/eval): let OpCoerceToString select the CoercionKindAdam Joseph1-1/+1
Change-Id: I92d58ef216d7e0766af70f019b3dcd445284a95d Reviewed-on: https://cl.tvl.fyi/c/depot/+/10344 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-12-25 r/7264 fix(tvix/eval/value): correctly emit spaces when coercing listssterni1-2/+17
r/7176 introduced an incorrect assumption was the benefit of the nonrecursive coercion algorithm, namely that a coercion operation always returns a non empty string. This allows to detect whether we are coercing a list or not by checking if the intermediate result is empty or not. Unfortunately, coercing null and false yields an empty string, so we need to explicitly track whether we are coercing a list. Updated the test case to hopefully catch similar bugs in the future. I'm not a hundred percent certain I have not introduced a new edge case with this, so it may be interesting to add a prop test case for this to nix_oracle down the line. At least lists are the only nested data structures that can be serialized as nested data structures, so the problem is kind of limited. Change-Id: Ia41e904356f1c41a9d35e4e65ec02f2fe5a4100e Reviewed-on: https://cl.tvl.fyi/c/depot/+/10418 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2023-12-14 r/7218 fix(tvix/eval): remove incorrect imports when coercingsterni2-18/+43
The default behavior of string coercion in C++ Nix is to weakly coerce and import to store if necessary. There is a flag to make it strongly coerce (coerceMore) and a flag that controls whether path values have the corresponding file/directory imported into the store before returning the (store) path as a string (copyToStore). We need to implement our equivalent to the copyToStore (import_paths) flag for the benefit of weak coercions that don't import into the store (dirOf, baseNameOf, readFile, ...) and strong coercions that don't import into the store (toString). This makes coerce_to_string as well as CoercionKind weirder and more versatile, but prevents us from reimplementing parts of the coercion logic constantly as can be seen in the case of baseNameOf. Note that it is not possible to test this properly in //tvix/eval tests due to the lack of an appropriate EvalIO implementation being available. Tests should be added to //tvix/glue down the line. Change-Id: I8fb8ab99c7fe08e311d2ba1c36960746bf22f566 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10361 Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI Reviewed-by: Adam Joseph <adam@westernsemico.com>
2023-12-12 r/7178 fix(tvix/eval): fix branching on catchable defaults (b/343)Adam Joseph1-0/+1
This commit adds Opcode::OpJumpIfCatchable, which can be inserted ahead of most VM operations which expect a boolean on the stack, in order to handle catchables in branching position properly. Other than remembering to patch the jump, no other changes should be required. This commit also fixes b/343 by emitting this new opcode when compiling if-then-else. There are probably other places where we need to do the same thing. Change-Id: I48de3010014c0bbeba15d34fc0d4800e0bb5a1ef Reviewed-on: https://cl.tvl.fyi/c/depot/+/10288 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: Adam Joseph <adam@westernsemico.com>
2023-12-12 r/7176 feat(tvix/eval): nonrecursive coerce_to_string()Adam Joseph3-111/+113
After this commit, the only non-builtins uses of generators are: - coerce_to_string() uses generators::request_enter_lambda() - Thunk::force() uses generators::request_enter_lambda() That's it! Once those two are taken care of, GenCo can become an implementation detail of `builtins::BuiltinGen`. No more crazy nonlocal flow control within the interpreter: if you've got a GenCo floating around in your code it's because you're writing a builtin, which isn't part of the core interpreter. The interpreter won't need GenCos to talk to itself anymore. Technically generators::request_path_import() is also used by coerce_to_string(), but that's just because the io_handle happens to be part of the VM. There's no recursion-depth issue there, so the call doesn't need to go through the generator mechanism (request_path_import() doesn't call back to the interpreter!) Change-Id: I83ce5774d49b88fdafdd61160975b4937a435bb0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10256 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: Adam Joseph <adam@westernsemico.com>
2023-12-12 r/7175 feat(tvix/eval): nonrecursive deep_force()Adam Joseph2-67/+69
This commit implements deep_force() nonrecursively, by maintaining an explicit stack rather than using the call stack for recursion. As an added bonus, we don't need to pass around the SharedThunkSet anymore, and can in fact completely eliminate SharedThunkSet. Change-Id: I7c4f59f37834d451a28bf6be317eb0a90eac4ee6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10252 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: Adam Joseph <adam@westernsemico.com>
2023-12-12 r/7174 docs(tvix/eval): clarify difference between ThunkSet and BlackholeAdam Joseph1-2/+5
The comment explaining ThunkSet makes it seem like it does the same think as ThunkRepr::Blackhole. In fact neither one is a substitute for the other. Let's explain the difference. Change-Id: I89ceaaa9d3c499edbc7d48f70ca5d11f97666c43 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10250 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: Adam Joseph <adam@westernsemico.com>
2023-12-12 r/7169 fix(tvix/eval): preserve catchables in nix_cmp_ordering(), fix b/338Adam Joseph1-4/+6
This commit fixes b/338 by properly propagating catchables through comparison operations. Change-Id: I6b0283a40f228ecf9a6398d24c060bdacb1077cf Reviewed-on: https://cl.tvl.fyi/c/depot/+/10221 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: Adam Joseph <adam@westernsemico.com>
2023-12-12 r/7167 feat(tvix/eval): nonrecursive nix_cmp_ordering(), fixes b/339Adam Joseph1-40/+69
This commit rewrites Value::nix_cmp_ordering() into an equivalent nonrecursive form. Except for calls to Thunk::force(), the new form no longer uses generators, and is async only because of the fact that it calls Thunk::force(). I originally believed that this commit would make evaluation faster. In fact it is slightly slower. I believe this is due to the added vec![] allocation. I am investigating. Prev-Nixpkgs-Benchmark: {"attrpath":"pkgsCross.aarch64-multiplatform.hello.outPath","peak-kbytes":"460048","system-seconds":"0.68","user-seconds":"5.73"} This-Nixpkgs-Benchmark: {"attrpath":"pkgsCross.aarch64-multiplatform.hello.outPath","peak-kbytes":"460224","system-seconds":"0.67","user-seconds":"5.84"} Change-Id: Ic627bc220d9c5aa3c5e68b9b8bf199837cd55af5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10212 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: Adam Joseph <adam@westernsemico.com>
2023-12-12 r/7166 fix(tvix/eval): never use partial_cmp() (partial fix b/338)Adam Joseph1-18/+11
This is part of a fix for b/338. We should never use PartialOrd::partial_cmp(). All Nix types except floats are obviously totally-ordered. In addition, it turns out that because Nix treats division by zero rather than producing a NaN, and because it does not support "negative zero", even floats are in fact totally ordered in Nix. Therefore, every call to PartialOrd::partial_cmp() in tvix is an error. We have to *implement* this function, but we should never call it on built-in types. Moreover, nix_cmp_ordering() currently returns an Option<Ordering>. I'm not sure what was going on there, since it's impossible for it to return None. This commit fixes it to return simply Ordering rather than Option<Ordering>. Change-Id: If5c084164cf19cfb38c5a15554c0422faa5f895d Reviewed-on: https://cl.tvl.fyi/c/depot/+/10218 Autosubmit: Adam Joseph <adam@westernsemico.com> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-12-12 r/7164 feat(tvix/eval): nonrecursive nix_eq()Adam Joseph2-129/+151
This commit rewrites Value::nix_eq() into an equivalent. Except for calls to Thunk::force(), the new form no longer uses generators, and is async only because of the fact that it calls Thunk::force(). I believed that the nonrecursive form would be faster. It is, in fact, slightly slower. I believe this is due to the vec![] allocation; I am investigating. Prev-Nixpkgs-Benchmark: {"attrpath":"pkgsCross.aarch64-multiplatform.hello.outPath","peak-kbytes":"459068","system-seconds":"0.71","user-seconds":"5.39"} This-Nixpkgs-Benchmark: {"attrpath":"pkgsCross.aarch64-multiplatform.hello.outPath","peak-kbytes":"460048","system-seconds":"0.68","user-seconds":"5.73"} Change-Id: I10f4868891e4b7475df13f0cbc41ec78dd985dd8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10118 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: Adam Joseph <adam@westernsemico.com>
2023-12-09 r/7134 refactor(tvix/eval): address clippy lintsFlorian Klink1-1/+1
Change-Id: Ic2bd4e8291b30ceac9fa0e88a4f56e61ae99b603 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10227 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-12-09 r/7130 feat(tvix/eval): impl DoubleEndedIter for OwnedAttrsIteratorAdam Joseph1-0/+10
Change-Id: I4bd85dbe9c27047f4abbdeff4e2b796e9bcab3a1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10211 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: Adam Joseph <adam@westernsemico.com> Tested-by: BuildkiteCI
2023-12-06 r/7120 feat(tvix/eval): rewrite Thunk::force() in nonrecursive formAdam Joseph2-63/+96
This commit rewrites Thunk::force() so that it is not (directly) self-recursive. It maintains a Vec of all the previously-encountered thunks which point to the one it is currently forcing, rather than recursively calling itself. Benefits: - Short term: This commit saves the cost of a round-trip through the generator machinery for the generators::request_force() which is removed by this commit. - Medium term: Once a similar transformation has been applied to nix_cmp(), nix_add(), nix_eq(), and coerce_to_string(), those four functions, along with Thunk::force(), will make non-tail calls only to each other. They can then be merged into a single tail-recursive function which does not use the generator machinery at all: enum Task { Cmp, Add, Eq, CoerceToString, Force}; fn Value::walk(task:Task, v1:Value, v2:Value) { // ... - Long term: The long-term goal here is to use generators **only for builtins** and [Marionette]-style remote control of the VM. In other words: use `async` for things that actually involve concurrency. Calls from the VM to builtins can then be blocking calls, because even cppnix will overflow the stack if you make a MAX_STACK_DEPTH-deep recursive call which passes through a builtin at every stack frame (e.g. `{ func = builtins.sort (a: b: ... func ...) ...}`). This way the inner "tight loop" of the interpreter doesn't pay the costs of `async` and generators. These costs manifest in terms of: performance, complex nonlocal control flow, and language impediments (async Rust is a restricted subset of real Rust, and is missing things like traits). [Marionette]: https://firefox-source-docs.mozilla.org/testing/marionette/Intro.html Change-Id: I6179b8abb2ea0492180fcb347f37595a14665777 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10039 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-12-05 r/7119 fix(tvix/eval): Return error rather than panicking on bad substringAspen Smith1-0/+9
If builtins.substring is invoked with (byte!!) offsets that aren't at codepoint boundaries, return an error rather than panicking. This is still incorrect (see b/337) but pushes the incorrectness forward a step. Change-Id: I5a4261f2ff250874cd36489ef598dcf886669d04 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10199 Tested-by: BuildkiteCI Autosubmit: grfn <grfn@gws.fyi> Reviewed-by: sterni <sternenseemann@systemli.org>
2023-11-25 r/7064 fix(tvix): ensure PartialOrd/Ord agree for StorePath & NixStringVincent Ambo1-1/+1
This fixes a *future* clippy lint: https://rust-lang.github.io/rust-clippy/master/index.html#/incorrect_partial_ord_impl_on_ord_type In essence, because the implementation of *both* Ord and PartialOrd implies that ordering is not partial, all results of PartialOrd should simply be those of Ord. This is to avoid subtle bugs in future refactorings. Change-Id: I8fc6694010208752dd47746a2aaaeca0c788d574 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10109 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-11-25 r/7055 refactor(tvix/eval): add ThunkRepr::is_forced()Adam Joseph1-5/+9
Change-Id: I4eab5c81fb82337da06327248845cd2f3a4490d3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10038 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: Adam Joseph <adam@westernsemico.com>
2023-11-25 r/7054 feat(tvix/eval): add Thunk::unwrap_or_clone()Adam Joseph1-1/+34
This commit adds Thunk::unwrap_or_clone(), which uses Rc::try_unwrap() to avoid cloning the Value out of a an Rc which has only one strong reference. Change-Id: Icacefe0c823dcddf046d90c0c5cd5ed59fe976d4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10037 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: Adam Joseph <adam@westernsemico.com> Tested-by: BuildkiteCI
2023-11-05 r/6958 chore(tvix): add missing clippy attributes & configVincent Ambo1-0/+1
For cases where clippy lints don't apply to us, or something is misfiring, add appropriate configuration. Relates to b/321. Change-Id: I0af453910b4a4112bf685b2a8e9a73de10ec87ea Reviewed-on: https://cl.tvl.fyi/c/depot/+/9965 Reviewed-by: flokli <flokli@flokli.de> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-11-05 r/6957 refactor(tvix/eval): use IntoIterator trait for owned NixAttrs iterVincent Ambo1-15/+22
Uses the standard library IntoIterator trait for the construction of our iterators. Clippy complains about duplicating this. While doing this, I opted to rename the `IntoIter` type into something that is more useful to users, in case somebody ends up working with these manually. This fixes a clippy lint, and is related to b/321. Change-Id: I851fde0d7b8b38d182343a0fd6d9f8dd2a33ee11 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9963 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: tazjin <tazjin@tvl.su>
2023-11-05 r/6955 chore(tvix): fix trivial clippy lintsVincent Ambo5-18/+17
Relates to b/321. Change-Id: I37284f89b186e469eb432e2bbedb37aa125a6ad4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9961 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: tazjin <tazjin@tvl.su>
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-11-01T01·42+0000