about summary refs log tree commit diff
path: root/tvix/eval/src/value/mod.rs
AgeCommit message (Collapse)AuthorFilesLines
2023-03-13 r/5969 refactor(tvix/eval): box PathBufVincent Ambo1-4/+4
This shaves another 8 bytes off Value. How did that type get so big?! Change-Id: I65e9b59a1636bd57e3cc4aec5fea16887070b832 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8153 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-03-13 r/5964 refactor(tvix/eval): flatten call stack of VM using generatorsVincent Ambo1-247/+154
Warning: This is probably the biggest refactor in tvix-eval history, so far. This replaces all instances of trampolines and recursion during evaluation of the VM loop with generators. A generator is an asynchronous function that can be suspended to yield a message (in our case, vm::generators::GeneratorRequest) and receive a response (vm::generators::GeneratorResponsee). The `genawaiter` crate provides an interpreter for generators that can drive their execution and lets us move control flow between the VM and suspended generators. To do this, massive changes have occured basically everywhere in the code. On a high-level: 1. The VM is now organised around a frame stack. A frame is either a call frame (execution of Tvix bytecode) or a generator frame (a running or suspended generator). The VM has an outer loop that pops a frame off the frame stack, and then enters an inner loop either driving the execution of the bytecode or the execution of a generator. Both types of frames have several branches that can result in the frame re-enqueuing itself, and enqueuing some other work (in the form of a different frame) on top of itself. The VM will eventually resume the frame when everything "above" it has been suspended. In this way, the VM's new frame stack takes over much of the work that was previously achieved by recursion. 2. All methods previously taking a VM have been refactored into async functions that instead emit/receive generator messages for communication with the VM. Notably, this includes *all* builtins. This has had some other effects: - Some test have been removed or commented out, either because they tested code that was mostly already dead (nix_eq) or because they now require generator scaffolding which we do not have in place for tests (yet). - Because generator functions are technically async (though no async IO is involved), we lose the ability to use much of the Rust standard library e.g. in builtins. This has led to many algorithms being unrolled into iterative versions instead of iterator combinations, and things like sorting had to be implemented from scratch. - Many call sites that previously saw a `Result<..., ErrorKind>` bubble up now only see the result value, as the error handling is encapsulated within the generator loop. This reduces number of places inside of builtin implementations where error context can be attached to calls that can fail. Currently what we gain in this tradeoff is significantly more detailed span information (which we still need to bubble up, this commit does not change the error display). We'll need to do some analysis later of how useful the errors turn out to be and potentially introduce some methods for attaching context to a generator frame again. This change is very difficult to do in stages, as it is very much an "all or nothing" change that affects huge parts of the codebase. I've tried to isolate changes that can be isolated into the parent CLs of this one, but this change is still quite difficult to wrap one's mind and I'm available to discuss it and explain things to any reviewer. Fixes: b/238, b/237, b/251 and potentially others. Change-Id: I39244163ff5bbecd169fe7b274df19262b515699 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8104 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Reviewed-by: Adam Joseph <adam@westernsemico.com> Tested-by: BuildkiteCI
2023-03-13 r/5963 feat(tvix/eval): implement generator-based Nix equality logicVincent Ambo1-1/+181
Adds a `Value::neo_nix_eq` method (the `neo_` prefix will be dropped when we flip over to the generator implementation of the VM) which implements Nix equality semantics using async, generator-based comparisons. Instead of tracking the "kind" of equality that is being compared (see the pointer-equality doc) through a pair of booleans, I've introduced an enum that explicitly lists the possible comparisons. Change-Id: I3354cc1470eeccb3000a5ae24f2418db1a7a2edc Reviewed-on: https://cl.tvl.fyi/c/depot/+/8241 Tested-by: BuildkiteCI Reviewed-by: Adam Joseph <adam@westernsemico.com>
2023-03-07 r/5899 feat(tvix/eval): introduce generators moduleVincent Ambo1-1/+1
This module contains the request/response types for generators requesting actions from the VM. For most of these, an async helper function is added that will be used inside of generator functions to make use of these requests/responses instead of constructing them directly. Change-Id: I1e085f88adaf784a34867957a0e82532d3a83d7c Reviewed-on: https://cl.tvl.fyi/c/depot/+/8148 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-03-03 r/5870 chore(tvix/eval): fix clippy warningsVincent Ambo1-4/+4
Change-Id: I4c02f0104c455ac00a3f299c1fbf75cbb08e8972 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8142 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su>
2023-02-13 r/5849 chore(tvix/eval): clippy warn is length zeroAaqa Ishtyaq1-1/+1
This CL address clippy warning about finding the zero length of something using `is_empty()` instead of `len() == 0`. Change-Id: I2b36c7c7b65b733609fc0dcd33be06f9d772bc9b Signed-off-by: Aaqa Ishtyaq <aaqaishtyaq@gmail.com> Reviewed-on: https://cl.tvl.fyi/c/depot/+/8029 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-02-02 r/5821 chore(tvix/eval): elaborate on internal types in Value::type_ofVincent Ambo1-6/+8
This aids in debugging quite substantially. Change-Id: Ic43232aa6165ae1c3db7ac2701938e1dfeeb418c Reviewed-on: https://cl.tvl.fyi/c/depot/+/8013 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: tazjin <tazjin@tvl.su>
2023-01-25 r/5757 test(tvix/eval): add test for total_fmt_floatFlorian Klink1-0/+28
Change-Id: If6c478ee3d2e4ecf5ef92289614f86535ad05cb7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7927 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de>
2023-01-25 r/5756 refactor(tvix/eval): extract float formatting into a helperVincent Ambo1-71/+75
This keeps the actual TotalDisplay implementation readable, as this float formatting code suddenly made up the majority of its implementation. Change-Id: I2c0d00e4a691e0b8ffbc72680f680e16feef4bee Reviewed-on: https://cl.tvl.fyi/c/depot/+/7925 Reviewed-by: flokli <flokli@flokli.de> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-01-25 r/5753 feat(tvix/eval): use lexical-core to format floatFlorian Klink1-2/+86
Apparently our naive implementation of float formatting, which simply used {:.5}, and trimmed trailing "0" strings not sufficient. It wrongly trimmed numbers with zeroes but no decimal point, like `10000` got trimmed to `1`. Nix uses `std::to_string` on the double, which according to https://en.cppreference.com/w/cpp/string/basic_string/to_string is equivalent to `std::sprintf(buf, "%f", value)`. https://en.cppreference.com/w/cpp/io/c/fprintf mentions this is treated like this: > Precision specifies the exact number of digits to appear after > the decimal point character. The default precision is 6. In the > alternative implementation decimal point character is written even if > no digits follow it. For infinity and not-a-number conversion style > see notes. This doesn't seem to be the case though, and Nix uses scientific notation in some cases. There's a whole bunch of strategies to determine which is a more compact notation, and which notation should be used for a given number. https://github.com/rust-lang/rust/issues/24556 provides some pointers into various rabbit holes for those interested. This gist seems to be that currently a different formatting is not exposed in rust directly, at least not for public consumption. There is the [lexical-core](https://github.com/Alexhuszagh/rust-lexical) crate though, which provides a way to format floats with various strategies and formats. Change our implementation of `TotalDisplay` for the `Value::Float` case to use that. We still need to do some post-processing, because Nix always adds the sign in scientific notation (and there's no way to configure lexical-core to do that), and lexical-core in some cases keeps the trailing zeros. Even with all that in place, there as a difference in `eval-okay- fromjson.nix` (from tvix-tests), which I couldn't get to work. I updated the fixture to a less problematic number. With this, the testsuite passes again, and does for the upcoming CL introducing builtins.fromTOML, and enabling the nix testsuite bits for it, too. Change-Id: Ie6fba5619e1d9fd7ce669a51594658b029057acc Reviewed-on: https://cl.tvl.fyi/c/depot/+/7922 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-20 r/5715 refactor(tvix/eval): keep globals alive through VM structVincent Ambo1-4/+24
This forces users to pass the fully constructed set of globals to the VM, making it harder to accidentally "lose" the set while weak references to it still exist. This doesn't modify any functionality, but is laying the foundation for simplifying some of the builtins behaviour that has grown more complex again. Change-Id: I5120f97861c65dc46d90b8a4e2c92ad32cc53e03 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7877 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-01-17 r/5670 refactor(tvix/value): use proptest strategies from imbl crateVincent Ambo1-6/+0
Instead of going through Vec/BTreeMap for generating our internal types, use the proptest strategies from imbl. The one thing I couldn't figure out in the previous implementation is where the ranges/sizes of generated collections came from. The strategies in proptest use different types (Range, with an unknown default value, and SizeRange with 0..100). I've opted to specify 0..100 directly, but we can probably make it configurable. Change-Id: I749bc4c703fe424099240cab822b1642e5216361 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7791 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-01-16 r/5663 chore(tvix/eval): add other required items to public APIVincent Ambo1-2/+2
External implementors of builtins must be able to force values, which necessitates publishing a bunch more items from the crate. Change-Id: I8f6b8ae88156aae417dbe630a698d123d0c1c8d4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7830 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-01-12 r/5652 feat(tvix/eval): implement builtins.toJSONVincent Ambo1-3/+4
Implements `Serialize` for `tvix_eval::Value`. Special care is taken with serialisation of attribute sets, and forcing of thunks. The tests should cover both cases well. Change-Id: I9bb135bacf6f87bc6bd0bd88cef0a42308e6c335 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7803 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su>
2023-01-10 r/5640 feat(tvix/eval): implement serde::Deserialize for ValueRyan Lahfa1-42/+17
Co-Authored-By: Vincent Ambo <tazjin@tvl.su> Change-Id: Ib6f7d1f4f4faac36b44f5f75cccc57bf912cf606 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7626 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-01-08 r/5629 fix(tvix/eval): fix last uses of Vec<Value> -> NixList in builtinsVincent Ambo1-11/+6
Change-Id: I0d71b82eb7ddc1e457b0996b0668006f55f56751 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7790 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-12-29 r/5542 refactor(tvix/eval): remove extra Rc<..> around Value::AttrsVincent Ambo1-3/+3
The `im::OrdMap` is already small and cheap to copy while sharing memory, so this is not required anymore. Only the `KV` variant may have slightly larger content, but in practice this doesn't seem to make a difference when comparing the two variants and this one is less complicated. Change-Id: I64a563b209a2444125653777551373cb2989ca7d Reviewed-on: https://cl.tvl.fyi/c/depot/+/7677 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-12-29 r/5541 refactor(tvix/eval): persistent, memory-sharing OrdMap for NixAttrsVincent Ambo1-1/+1
This uses the `im::OrdMap` for `NixAttrs` to enable sharing of memory between different iterations of a map. This slightly speeds up eval, but not significantly. Future work might include benchmarking whether using a `HashMap` and only ordering in cases where order is actually required would help. This switches to a fork of `im` that fixes some bugs with its OrdMap implementation. Change-Id: I2f6a5ff471b6d508c1e8a98b13f889f49c0d9537 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7676 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-12-29 r/5540 refactor(tvix/eval): use im::Vector directly where possibleVincent Ambo1-3/+4
The conversion from im::Vector -> Vec is cheaper for NixList construction (of course), so where possible we should make use of that. This updates most builtins dealing with lists to use Vector directly, and marks the function constructing NixList from Vec as deprecated so that we get appropriate warnings in places where it's still in use. These places are currently inside of JSON serialisation logic which is in flux right now, so lets leave them as-is until it's stabilised. Change-Id: I037f12a2800f2576db4d9526bd935efd079163f0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7671 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-12-25 r/5486 fix(tvix/eval): fix current clippy warningsVincent Ambo1-2/+2
It's been a while since the last time, so quite a lot of stuff has accumulated here. Change-Id: I0762827c197b30a917ff470fd8ae8f220f6ba247 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7597 Reviewed-by: grfn <grfn@gws.fyi> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-12-25 r/5484 feat(tvix/eval): implement From<f64> for ValueRyan Lahfa1-0/+6
Change-Id: I287282a195d6f752260242739332b2357791974a Reviewed-on: https://cl.tvl.fyi/c/depot/+/7625 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-12-22 r/5477 feat(tvix/eval): display function names in documentationVincent Ambo1-1/+8
... if they are known. We currently do not propagate names correctly for curried functions. Change-Id: I19d57fb30a5c0000ccdf690b91076f6b2191de23 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7596 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-12-22 r/5475 feat(tvix/eval): add Value::explain methodVincent Ambo1-0/+33
This value creates a human-readable explanation of a value. This can be used to implement documentation related functionality. For some values, the amount of information displayed can be expanded quite a bit. Change-Id: Ie8c400feae909e7680af163596f99060262e4241 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7592 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-21 r/5467 feat(tvix/eval): use `EvalIO::import_path` when coercing pathsVincent Ambo1-1/+2
This "ties the knot" of importing files into a store when referring to them through path literals, e.g. inside of strings. I'm not yet sure if this interface is sufficient for builtins.path (which we haven't implemented at all yet), but it's enough to wire up eval & store initially. In the default implementations nothing interesting happens in this function at all. Change-Id: Ie01ff4161617d1e743a68dbd1a5e54c1b40c0990 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7582 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-21 r/5459 feat(tvix/eval): add EvalIO to public crate APIVincent Ambo1-4/+4
This lets users set the `io_handle` field on an `Evaluation`, which is then propagated to the VM. Change-Id: I616d7140724fb2b4db47c2ebf95451d5303a487a Reviewed-on: https://cl.tvl.fyi/c/depot/+/7566 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-21 r/5452 feat(tvix/eval): wrap Closure in Rc<> to match cppnix semanticsAdam Joseph1-2/+8
Change-Id: I595087eff943d38a9fc78a83d37e207bb2ab79bc Reviewed-on: https://cl.tvl.fyi/c/depot/+/7443 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-03 r/5379 feat(tvix/eval): Continue removing leakage of BTreeMap.Lyle Mantooth1-2/+3
Fixes b/212. Based on feedback in https://cl.tvl.fyi/c/depot/+/7492, all uses of `NixAttrs::from_map` have been removed. Only `from_iter` and `from_kv` remain. Change-Id: I52e25f73018c2aa1843197427516b7a852503e2c Reviewed-on: https://cl.tvl.fyi/c/depot/+/7500 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: IslandUsurper <lyle@menteeth.us>
2022-11-28 r/5349 feat(tvix/eval): add CoercionKind::ThunksOnlyAdam Joseph1-4/+11
Signed-off-by: Adam Joseph <adam@westernsemico.com> Change-Id: I92acb7e6099a4796d953b2d4d02cca4076ed0fb1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7426 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-11-27 r/5346 fix(tvix/eval): implement function/thunk ptr-equality for list ordAdam Joseph1-1/+1
With this change, the test introduced by cl/7370 passes. Change-Id: Ie7d2f02a59d61151f14ebd328e6cfa5892cacfb0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7375 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org> Autosubmit: Adam Joseph <adam@westernsemico.com>
2022-11-27 r/5345 feat(tvix/eval): non-recursive implementation of nix_eq()Adam Joseph1-26/+4
This passes all the function/thunk-pointer-equality tests in cl/7369. Change-Id: Ib47535ba2fc77a4f1c2cc2fd23d3a879e21d8b4c Signed-off-by: Adam Joseph <adam@westernsemico.com> Reviewed-on: https://cl.tvl.fyi/c/depot/+/7358 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-11-26 r/5326 feat(tvix/eval): declare function-pointer-equality dealt withAdam Joseph1-1/+0
Change-Id: If81ff414dba10a0448b905eec373730a68795376 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7376 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: Adam Joseph <adam@westernsemico.com>
2022-11-21 r/5299 fix(tvix/eval): aggressively fix a borrow error in nix_eqVincent Ambo1-3/+9
When comparing Nix values for equality, an issue can occur where recursive values contain thunks to themselves which causes borrow errors when forcing them for comparison later down the line. To work around this we clone the values for now. There might be some optimisations possible like checking for thunk equality directly and short-circuiting on that (we have to check what Nix does). Change-Id: I7e75c992ea68f100058f52b4b46168da7d671994 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7314 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-11-08 r/5268 feat(tvix/eval): Give names to builtin argumentsGriffin Smith1-1/+1
Refactor the arguments of a Builtin to be a vec of a new BuiltinArgument struct, which contains the old strictness boolean and also a static `name` str - this is automatically determined via the ident for the corresponding function argument in the proc-macro case, and passed in in the cases where we're still manually calling Builtin::new. Currently this name is unused, but in the future this can be used as part of a documentation system for builtins. Change-Id: Ib9dadb15b69bf8c9ea1983a4f4f197294a2394a6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7204 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-11-04 r/5236 fix(tvix/eval): remove impl PartialEq for ValueAdam Joseph1-1/+1
It isn't possible to implement PartialEq properly for Value, because any sensible implementation needs to force() thunks, which cannot be done without a `&mut VM`. The existing derive(PartialEq) has false negatives, which caused the bug which cl/7142 fixed. Fortunately that bug was easy to find, but a silent false negative deep within the bowels of nixpkgs could be a real nightmare to hunt down. Let's just remove the PartialEq impl for Value, and the other derive(PartialEq)'s that depend on it. Signed-off-by: Adam Joseph <adam@westernsemico.com> Change-Id: Iacd3726fefc7fc1edadcd7e9b586e04cf8466775 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7144 Reviewed-by: kanepyork <rikingcoding@gmail.com> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-10-31 r/5226 fix(tvix/eval): nix_eq() must recurseAdam Joseph1-3/+3
The current implementation of nix_eq will force one level of thunks and then switch to the (non-forcing) rust Eq::eq() method. This gives incorrect results for lists-of-thunks. This commit changes nix_eq() to be recursive. A regression test (which fails prior to this commit) is included. This fix also causes nix_tests/eval-okay-fromjson.nix to pass, so it is moved out of notyetpassing. Change-Id: I655fd7a5294208a7b39df8e2c3c12a8b9768292f Signed-off-by: Adam Joseph <adam@westernsemico.com> Reviewed-on: https://cl.tvl.fyi/c/depot/+/7142 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-29 r/5221 feat(tvix/eval): Implement comparison for listsGriffin Smith1-1/+14
Lists are compared lexicographically in C++ nix as of [0], and our updated nix test suites depend on this. This implements comparison of list values in `Value::nix_cmp` using a very similar algorithm to what C++ does - similarly to there, this requires passing in the VM so we can force thunks in the list elements as we go. [0]: https://github.com/NixOS/nix/commit/09471d2680292af48b2788108de56a8da755d661# Change-Id: I5d8bb07f90647a1fec83f775243e21af856afbb1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7070 Autosubmit: grfn <grfn@gws.fyi> Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-10-25 r/5198 feat(tvix/eval): add builtins.{floor,ceil}James Landrein1-0/+1
Change-Id: I4e6c4f96f6f5097a5c637eb3dbbd7bb8b34b7d52 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7032 Autosubmit: j4m3s <james.landrein@gmail.com> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: grfn <grfn@gws.fyi>
2022-10-24 r/5193 refactor(tvix/eval): Implement value comparison with a methodGriffin Smith1-0/+21
Rather than implementing all of the interesting semantics of value comparison with a macro bound to the VM, implement the bulk of the logic with a method on Value itself that returns an Ordering, and then use the macro to implement the comparison against that Ordering. This has no functional change, but paves the way to implementing lexicographic comparison of list values, which is supported in the latest version of upstream nix. Change-Id: I8af1a020b41577021af5939f5edc160c407d4a9e Reviewed-on: https://cl.tvl.fyi/c/depot/+/7069 Autosubmit: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-24 r/5189 feat(nix/eval): Implement builtins.groupByGriffin Smith1-0/+15
Change-Id: I3e0aa017a7100cbeb86d2e5747471b36affcc102 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7038 Autosubmit: grfn <grfn@gws.fyi> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-10-23 r/5178 fix(tvix/eval): detect cycles when printing infinite valuesVincent Ambo1-8/+18
Using the same method as in Thunk::deep_force, detect cycles when printing values by maintaining a set of already seen thunks. With this, display of infinite values matches that of Nix: > nix-instantiate --eval --strict -E 'let as = { x = 123; y = as; }; in as' { x = 123; y = { x = 123; y = <CYCLE>; }; } > tvix-eval -E 'let as = { x = 123; y = as; }; in as' => { x = 123; y = { x = 123; y = <CYCLE>; }; } :: set Change-Id: I007b918d5131d82c28884e46e46ff365ef691aa8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7056 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-10-22 r/5175 feat(tvix/eval): Implement builtins.deepSeqGriffin Smith1-0/+45
This is done via a new `deepForce` function on Value. Since values can be cyclical (for example, see the test-case), we need to do some extra work to avoid RefCell borrow errors if we ever hit a graph cycle: While deep-forcing values, we keep a set of thunks that we have already seen and avoid doing any work on the same thunk twice. The set is encapsulated in a separate type to stop potentially invalid pointers from leaking out. Finally, since deep_force is conceptually similar to `VM::force_for_output` (but more suited to usage in eval since it doesn't clone the values) this removes the latter, replacing it with the former. Co-Authored-By: Vincent Ambo <tazjin@tvl.su> Change-Id: Iefddefcf09fae3b6a4d161a5873febcff54b9157 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7000 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi> Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-17 r/5153 feat(tvix/eval): Record formals on lambdaGriffin Smith1-0/+1
In preparation for both implementing the `functionArgs` builtin and adding support for validating closed formals, record information about the formal arguments to a function *on the Lambda itself*. This may seem a little odd for the purposes of just closed formal checking, but is something we have to have anyway for builtins.functionArgs so I figured I'd do it this way to kill both birds with one stone. Change-Id: Ie3770a607bf352a1eb395c79ca29bb25d5978cd8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7001 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-10-17 r/5151 docs(tvix/eval) comments for various fieldsAdam Joseph1-0/+5
Change-Id: I8dcddf2b419761e475e71215c199eef2f7dc61dc Reviewed-on: https://cl.tvl.fyi/c/depot/+/7028 Autosubmit: Adam Joseph <adam@westernsemico.com> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-16 r/5146 refactor(tvix/eval): make OpFindFile use internal UnresolvedPathsterni1-2/+6
To assert that OpFindFile is only emitted for specially compiled SPATH expressions, as well as make sure it doesn't accidentally operate on “ordinary values”, introduce an UnresolvedPath internal value. If OpFindFile sees a non-UnresolvedPath value, it'll crash. Note that this change is not done purely for OpFindFile: We may want to compile SPATH expressions as function calls to __findFile (like C++ Nix does) in the future, so the UnresolvedPath value would definitely need to be an ordinary string again then. Rather, this change is done in preparation for resolving home dir relative paths at runtime (since they depend on the environment) for which we'll need a similar mechanism to OpFindFile. Change-Id: I6acf287f35197cd9e13377079f972b9d36e5b22e Reviewed-on: https://cl.tvl.fyi/c/depot/+/7023 Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: Adam Joseph <adam@westernsemico.com> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-16 r/5140 refactor(tvix/eval) remove Value::DynamicUpvalueMissingAdam Joseph1-6/+0
I believe this variant is left over from a previous implementation. If not, please let me know. Signed-off-by: Adam Joseph <adam@westernsemico.com> Change-Id: I02a3bf2f63794d09e96a5a92a034c0ad3d1ff221 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7027 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-15 r/5135 feat(tvix/eval): Implement builtins.fromJSONGriffin Smith1-0/+48
Using `serde_json` for parsing JSON here, plus an `impl FromJSON for Value`. The latter is primarily to stay "dependency light" for now - likely going with an actual serde `Deserialize` impl in the future is going to be way better as it allows saving significantly on intermediary allocations. Change-Id: I152a0448ff7c87cf7ebaac927c38912b99de1c18 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6920 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-15 r/5132 feat(tvix/eval): implement builtins.dirOfAdam Joseph1-0/+1
This commit causes the test eval-okay-builtins.nix to pass. It also adds tests/tvix_tests/eval-okay-dirof.nix which has better coverage than the nix tests for this builtin. Signed-off-by: Adam Joseph <adam@westernsemico.com> Change-Id: I71d96b48680696fd6e4fea3a9861742b35cfaa66 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6987 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-10-13 r/5119 refactor(tvix/eval): factor out all calls to canon_pathAdam Joseph1-0/+2
Right now we're pretending that the Rust library path_clean does the same thing that cppnix's canonPath() does. This is not true. It's close enough for the test suite, but may come back to bite us. Let's create our own canon_path() function and call that in all the places where we intend to match the behavior of cppnix's canonPath(). That way when we fix this we can fix it once, in one place. Signed-off-by: Adam Joseph <adam@westernsemico.com> Change-Id: Ia6f9577f62f49ef352ff9cfa5efdf37c32d31b11 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6993 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-10-12 r/5111 fix(tvix/eval): remove overlapping instancesAdam Joseph1-12/+0
I broke the build. I didn't understand that before hitting "submit" you need to re-test your changes on latest HEAD (and that CI doesn't do that for you); I failed to re-test cl/6912 following the merge of cl/6907. This commit fixes the build by removing the overlapping instances. Change-Id: I2a720d2c60cc7103b350f78102a8998f93bac828 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6965 Tested-by: BuildkiteCI Autosubmit: Adam Joseph <adam@westernsemico.com> Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-12 r/5110 feat(tvix/eval): From<Into<NixString>> for ValueAdam Joseph1-0/+9
Signed-off-by: Adam Joseph <adam@westernsemico.com> Change-Id: I27615baf4104ab96600708b12a780599a3714bae Reviewed-on: https://cl.tvl.fyi/c/depot/+/6912 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>