about summary refs log tree commit diff
path: root/tvix/eval (follow)
AgeCommit message (Collapse)AuthorFilesLines
2023-03-04 r/5888 refactor(tvix/eval): remove VM argument from suspended native thunksVincent Ambo3-13/+7
Because they do not use it, and it can not be passed with the coming generator refactoring. Change-Id: I0d96f2357a7ee79cd8a0f401583d4286230d4a6b Reviewed-on: https://cl.tvl.fyi/c/depot/+/8146 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-03-04 r/5887 refactor(tvix/eval): insert storeDir "builtin" in eval startupVincent Ambo2-10/+5
Instead of using a suspended native thunk, calculate and optionally insert the storeDir builtin when the VM is constructed. We already have the IO handle available at this point and can just check whether a storeDir is present, and insert its absolute value as a builtin. Change-Id: If966eee6ff26dc888b6e888e7c46170c0c346b05 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8145 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-03-04 r/5886 feat(tvix/eval): add SharedThunkSetVincent Ambo1-0/+11
This is a ThunkSet wrapped to be shareable, which will be required once ThunkSets are embedded in futures. Change-Id: I5a067b7972ac86e4d354c75ef05c86b2284c1137 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8144 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-03-04 r/5885 fix(tvix/eval): ThunkSet does not need mutable pointersVincent Ambo1-2/+2
Change-Id: Iea248870a0ea5d38cb02ff059c968fbd563570b6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8143 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-03-04 r/5884 chore(tvix/eval): implement From<OrdMap<..>> for NixAttrsVincent Ambo1-0/+6
Again simplifying some code down the line, where bits of code that construct attribute sets already have the final structure available. Change-Id: I0bb7a1daa63298122b51be73d35d695a4f73f8b0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8140 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-03-04 r/5883 refactor(tvix/eval): implement From<Span> for LightSpanVincent Ambo1-0/+6
This simplifies some code down the line. Change-Id: I58dd71e796e11479f44516cf24932f8061843d23 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8139 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-03-03 r/5870 chore(tvix/eval): fix clippy warningsVincent Ambo4-21/+21
Change-Id: I4c02f0104c455ac00a3f299c1fbf75cbb08e8972 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8142 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su>
2023-03-03 r/5869 refactor(tvix/eval): remove useless map callVincent Ambo1-5/+1
Change-Id: Ifb59ef148ea4fab613f2e4efb133c04baafa3a98 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8141 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-03-03 r/5868 refactor(tvix/eval): enhance debug output for bytecode dumpsVincent Ambo2-1/+25
This adds addresses of thunk and closure chunks to the debug output displayed when dumping bytecode. This makes it possible to see in the dump which thunks are referenced by constants in other thunks. Change-Id: I2c98de5227e7cb415666cd3134c947a56979dc80 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8137 Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-02-16 r/5857 refactor(tvix/eval): remove redundant cloneAaqa Ishtyaq3-6/+6
This CL removes redundant clone from value which is going to be dropped without further use. Change-Id: Ibd2a724853c5cfbf8ca40bf0b3adf0fab89b9be5 Signed-off-by: Aaqa Ishtyaq <aaqaishtyaq@gmail.com> Reviewed-on: https://cl.tvl.fyi/c/depot/+/8125 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-02-16 r/5856 docs(tvix/eval): add proposal for VM loop restructuringVincent Ambo1-0/+76
Change-Id: Ib991d68724a73886a8343d7f785b5b3aedd637ed Reviewed-on: https://cl.tvl.fyi/c/depot/+/8103 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2023-02-14 r/5851 fix(tvix/eval): correctly print lambda address in observerVincent Ambo1-1/+1
We want the address that the Rc is pointing to, not the address of the Rc. Change-Id: I8eba21677f242bbe4166c74d4aa4269c316076e3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8045 Reviewed-by: flokli <flokli@flokli.de> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-02-13 r/5850 chore(tvix/eval): use writeln for newline stringAaqa Ishtyaq1-2/+2
This CL address clippy warning which expects to use `writeln` instead of `write` for strings with new line. Change-Id: Ia72a07502c60cfd489ecf1e3833b9d42d44a8b17 Signed-off-by: Aaqa Ishtyaq <aaqaishtyaq@gmail.com> Reviewed-on: https://cl.tvl.fyi/c/depot/+/8030 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-02-13 r/5849 chore(tvix/eval): clippy warn is length zeroAaqa Ishtyaq1-1/+1
This CL address clippy warning about finding the zero length of something using `is_empty()` instead of `len() == 0`. Change-Id: I2b36c7c7b65b733609fc0dcd33be06f9d772bc9b Signed-off-by: Aaqa Ishtyaq <aaqaishtyaq@gmail.com> Reviewed-on: https://cl.tvl.fyi/c/depot/+/8029 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-02-13 r/5848 fix(tvix/eval): skip runtime completely on compiler errorsVincent Ambo2-0/+18
This branch was missing, and an assumption elsewhere just executed the returned (broken) bytecode. This fixes b/253. Change-Id: I015023ba921bc08ea03882167f1f560feca25e50 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8090 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: tazjin <tazjin@tvl.su>
2023-02-13 r/5847 fix(tvix/eval): make fields of eval's Error type publicVincent Ambo1-3/+3
These should be inspectable by callers. Change-Id: Ia9ef871aa63958d06066aaea61b2aecbd217369b Reviewed-on: https://cl.tvl.fyi/c/depot/+/8089 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-02-04 r/5837 fix(tvix/eval): fix the default case for path parsingVincent Ambo1-10/+4
Plain paths like `foo/bar.nix` are also allowed, so we can not determine this based on the prefix. The upstream PR that is referenced in a comment here has a significantly different interface than we expected, so I'm not touching that comment yet in this CL before I've had more time to digest it. Change-Id: Iea33bbb35de9c00a7d7fedf64d02253c75c1cc9e Reviewed-on: https://cl.tvl.fyi/c/depot/+/8032 Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: Alyssa Ross <hi@alyssa.is> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-02-03 r/5833 chore(tvix/eval): only use Rc with impure featureFlorian Klink1-2/+3
We only use Rc in `impl EvalIO for StdIO`, which is only included when building with the "impure" feature. Change-Id: Id29d647c899cbfcdda11abfb9fabd5aa7e24299f Reviewed-on: https://cl.tvl.fyi/c/depot/+/8025 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-02-03 r/5832 refactor(tvix/eval): wrap `Builtin` type in a BoxVincent Ambo1-20/+30
This reduces the size of `Builtin` from 88 (!) bytes to 8, and as the largest variant of `Value`, the size of that type from 96 to 64. The next largest type is NixList, clocking in at 64 bytes. This has noticeable performance impact. In an implementation without disk I/O, evaluating nixpkgs.stdenv looks like this: Benchmark 1: tvix -E '(import <nixpkgs> {}).stdenv.drvPath' Time (mean ± σ): 1.151 s ± 0.003 s [User: 1.041 s, System: 0.109 s] Range (min … max): 1.147 s … 1.155 s 10 runs After this change, it looks like this: Benchmark 1: tvix -E '(import <nixpkgs> {}).stdenv.drvPath' Time (mean ± σ): 1.046 s ± 0.004 s [User: 0.954 s, System: 0.092 s] Range (min … max): 1.041 s … 1.053 s 10 runs Change-Id: I5ab7cc02a9a450c0227daf1f1f72966358311ebb Reviewed-on: https://cl.tvl.fyi/c/depot/+/8027 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-02-03 r/5831 refactor(tvix/eval): statically resolve select from constant attrsVincent Ambo3-50/+94
When resolving a select expression (`attrs.name` or `attrs.name or default`), if the set compiles to a constant attribute set (as is most notably the case with `builtins`) we can backtrack and replace that attribute set directly with the compiled value. For something like `builtins.length`, this will directly emit an `OpConstant` that leaves the `length` builtin on the stack. Change-Id: I639654e065a06e8cfcbcacb528c6da7ec9e513ee Reviewed-on: https://cl.tvl.fyi/c/depot/+/7957 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-02-03 r/5828 fix(tvix/eval): ensure all evaluated thunks are correctly memoizedVincent Ambo5-51/+196
This fixes a very complicated bug (b/246). Evaluation progresses *much* further after this, leading to several less complicated bugs likely being uncovered by this What was the problem? ===================== Previously, when evaluating a thunk, we had a code path that looked like this: match *thunk { ThunkRepr::Evaluated(Value::Thunk(ref inner_thunk)) => { let inner_repr = inner_thunk.0.borrow().clone(); drop(thunk); self.0.replace(inner_repr); } /* ... */ } This code path created a copy of the inner `ThunkRepr` of a nested thunk, and moved that copy into the `ThunkRepr` of the parent. The effect of this was that the original `ThunkRepr` (unforced!) lived on in the original thunk, without the memoization of the subsequent forcing applying to it. This had the result that Tvix would repeatedly evaluate these thunks without ever memoizing them, if they occured repeatedly as shared inner thunks. Most notably, this would *always* occur when builtins.import was used. What's the solution? ==================== I have completely rewritten `Thunk::force_trampoline_self` to make all flows that can occur in it explicit. I have also removed the outer loop inside of that function, and resorted to more use of trampolining instead. The function is now well-commented and it should be possible to read it from top-to-bottom and get a general sense of what is going on, though the trampolining itself (which is implemented in the VM) needs to be at least partially understood for this. What's the new problem(s)? ========================== One new (known) problem is that we have to construct `Error` instances in all error types here, but we do not have spans available in some thunk-related situations. Due to b/238 we cannot ask the VM for an arbitrary span from the callsite leading to the force. This means that there are now code paths where, under certain conditions, causing an evaluation error during thunk forcing will panic. To fix this we will need to investigate and fix b/238, and/or add a span tracking mechanism to thunks themselves. What other impacts does this have? ================================== With this commit, eval of nixpkgs mostly succeeds (things like stdenv evaluate to the same hashes for us and C++ Nix, meaning we now construct identical derivations without eval breaking). Due to this we progress much further into nixpkgs, which lets us uncover more additional bugs. For example, after this commit we can quickly see that cl/7949 introduces some kind of behavioural issue and should not be merged as-is (this was not apparent before). Additionally, tvix-eval is now seemingly very fast. When doing performance analysis of a nixpkgs eval, we now mostly see the code path for shelling out to C++ Nix to add things to the store in there. We still need those code paths, so we can not (yet) do a performance analysis beyond that. Change-Id: I738525bad8bc5ede5d8c737f023b14b8f4160612 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8012 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-02-02 r/5823 fix(tvix/eval): unsafeDiscardStringContext is a no-opVincent Ambo1-4/+2
... not just a TODO. Most use-cases of unsafeDiscardStringContext are for cases where a string is processed in some ways and no longer contains a "physical" reference, but still has its context attached in C++ Nix. We don't need to do this. This does diverge in behaviour in use-cases related to build scheduling, but that whole behaviour will be different in Tvix. Change-Id: I4056d4c09f62d44d6bd52b791db03fe5556672b5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8016 Reviewed-by: flokli <flokli@flokli.de> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-02-02 r/5822 refactor(tvix/eval): import_cache can be a HashMapVincent Ambo1-2/+2
... instead of a BTreeMap, as we do not need ordering guarantees here. HashMaps are noticeably faster here (especially as we've been sorting essentially random data!). Change-Id: Ie92d74286df9f763c04c9b226ef1066ee8484c13 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8014 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su>
2023-02-02 r/5821 chore(tvix/eval): elaborate on internal types in Value::type_ofVincent Ambo1-6/+8
This aids in debugging quite substantially. Change-Id: Ic43232aa6165ae1c3db7ac2701938e1dfeeb418c Reviewed-on: https://cl.tvl.fyi/c/depot/+/8013 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: tazjin <tazjin@tvl.su>
2023-01-31 r/5795 fix(tvix/eval): allow builtins.toXML to serialise any functionVincent Ambo1-1/+13
This adds a fake argument name to builtins.toXML which allows toXML to serialise any value instead of panicking on functions. We do still have to fix the value itself, eventually, though. Change-Id: I2e330ecddcd80442b4fac5eced64431ac86123ba Reviewed-on: https://cl.tvl.fyi/c/depot/+/7962 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-01-31 r/5784 test(tvix/eval): add tests for internal formals dependenciesVincent Ambo4-0/+8
Formals can depend on each other when using another formal as a default value. This test ensures that the compiler's declaration and initialisation order of formals is consistent with what actually happens in the VM. Change-Id: Ibdabe262554e8066d67fac1ebc3b5a48ef626e18 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7948 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-01-26 r/5763 chore(tvix/eval): remove dead commentVincent Ambo1-1/+1
Change-Id: Icedb7f272e5067569b8dbf1c2d8b0fdd352b8e12 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7936 Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de>
2023-01-25 r/5758 docs(tvix/eval): builtins.add is not equivalent to +sterni1-3/+3
While it is in the given example, i.e. for integer addition, to claim that they are equivalent is a bit misleading: builtins.add is less overloaded than +, i.e. builtins.add "foo" "bar" will fail whereas "foo" + "bar" performs string concatenation. Change-Id: Ib52d530d1ab289b367565b286f06a76dd518d4fb Reviewed-on: https://cl.tvl.fyi/c/depot/+/7929 Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-01-25 r/5757 test(tvix/eval): add test for total_fmt_floatFlorian Klink1-0/+28
Change-Id: If6c478ee3d2e4ecf5ef92289614f86535ad05cb7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7927 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de>
2023-01-25 r/5756 refactor(tvix/eval): extract float formatting into a helperVincent Ambo1-71/+75
This keeps the actual TotalDisplay implementation readable, as this float formatting code suddenly made up the majority of its implementation. Change-Id: I2c0d00e4a691e0b8ffbc72680f680e16feef4bee Reviewed-on: https://cl.tvl.fyi/c/depot/+/7925 Reviewed-by: flokli <flokli@flokli.de> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-01-25 r/5754 feat(tvix/eval): implement builtins.fromTOMLFlorian Klink5-3/+26
This allows parsing TOML from Tvix. We can enable the eval-okay-fromTOML testcase from nix_tests. It uses the `toml` crate, and the serde integration it brings with it. Change-Id: Ic6f95aacf2aeb890116629b409752deac49dd655 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7920 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-01-25 r/5753 feat(tvix/eval): use lexical-core to format floatFlorian Klink4-4/+90
Apparently our naive implementation of float formatting, which simply used {:.5}, and trimmed trailing "0" strings not sufficient. It wrongly trimmed numbers with zeroes but no decimal point, like `10000` got trimmed to `1`. Nix uses `std::to_string` on the double, which according to https://en.cppreference.com/w/cpp/string/basic_string/to_string is equivalent to `std::sprintf(buf, "%f", value)`. https://en.cppreference.com/w/cpp/io/c/fprintf mentions this is treated like this: > Precision specifies the exact number of digits to appear after > the decimal point character. The default precision is 6. In the > alternative implementation decimal point character is written even if > no digits follow it. For infinity and not-a-number conversion style > see notes. This doesn't seem to be the case though, and Nix uses scientific notation in some cases. There's a whole bunch of strategies to determine which is a more compact notation, and which notation should be used for a given number. https://github.com/rust-lang/rust/issues/24556 provides some pointers into various rabbit holes for those interested. This gist seems to be that currently a different formatting is not exposed in rust directly, at least not for public consumption. There is the [lexical-core](https://github.com/Alexhuszagh/rust-lexical) crate though, which provides a way to format floats with various strategies and formats. Change our implementation of `TotalDisplay` for the `Value::Float` case to use that. We still need to do some post-processing, because Nix always adds the sign in scientific notation (and there's no way to configure lexical-core to do that), and lexical-core in some cases keeps the trailing zeros. Even with all that in place, there as a difference in `eval-okay- fromjson.nix` (from tvix-tests), which I couldn't get to work. I updated the fixture to a less problematic number. With this, the testsuite passes again, and does for the upcoming CL introducing builtins.fromTOML, and enabling the nix testsuite bits for it, too. Change-Id: Ie6fba5619e1d9fd7ce669a51594658b029057acc Reviewed-on: https://cl.tvl.fyi/c/depot/+/7922 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-23 r/5747 chore(tvix/eval): delete "useless parenthesis" warning/optimisationVincent Ambo2-30/+0
Two main reasons: 1. Traversing the structure to do this optimisation is actually *slower* than not optimising it. 2. There are literally hundreds of thousands of incidences of this in nixpkgs, and with some of the weird code there some of these (functionally) useless parens are actually required for readability reasons. Change-Id: I1044b1c5f9fe20df4b6085851fc3b191277c65dc Reviewed-on: https://cl.tvl.fyi/c/depot/+/7917 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-01-23 r/5746 fix(tvix/eval): force functors before applying themVincent Ambo3-0/+12
call_value in the VM expects the callable to be forced when calling it, which was not the case for functors. Change-Id: Id55a2fe32a9573be42aef8669e268df519a989cd Reviewed-on: https://cl.tvl.fyi/c/depot/+/7909 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2023-01-22 r/5735 feat(tvix/eval): support builtins implemented in Nix itselfVincent Ambo4-17/+108
This makes it possible to inject builtins into the builtin set that are written in Nix code, and which at runtime are represented by a thunk that will compile them the first time they are used. Change-Id: Ia632367328f66fb2f26cb64ae464f8f3dc9c6d30 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7891 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-01-22 r/5734 docs(tvix/eval): update some outdated commentsVincent Ambo1-8/+3
These don't apply anymore since the "antidote-CL". Change-Id: I40ee73ef43d44bbfc650a8fe6c2b33263dd06959 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7890 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-01-21 r/5725 docs(tvix): move most of //tvix/eval README up to //tvixFlorian Klink1-11/+4
While moving the CLI out of the evaluator, we forgot to update the README in //tvix/eval. Move this up to //tvix, so people know where to start. Keep the instructions on how to build only `//tvix/eval` in `//tvix/ eval/README.md`. Change-Id: Ie2755e8b5a0056225dbf3a0ee040f70f7f6a1f27 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7887 Reviewed-by: sterni <sternenseemann@systemli.org> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-21 r/5721 refactor(tvix/eval): administer antidote for poisonAdam Joseph8-135/+49
The codebase contains a lot of complexity and odd roundabout handling for shadowing globals. I'm pretty sure none of this is necessary, and all of it disappears if you simply make the globals part of the ordinary identifier resolution chain, with their own scope up above the root scope. Then the ordinary shadowing routines do the right thing, and no special cases or new terminology are required. This commit does that. Note by tazjin: This commit was originally abandoned when Adam decided not to take away reviewer bandwidth for this at the time (eval was still in a much earlier stage). As we've recently done some significant refactoring of globals initialisation this came up again, and it seems we can easily cover the use-cases of the poison tracking in other ways now, so I've rebased, updated and resurrected the CL. Co-Authored-By: Vincent Ambo <tazjin@tvl.su> Signed-off-by: Adam Joseph <adam@westernsemico.com> Change-Id: Ib3309a47a7b31fa5bf10466bade0d876b76ae462 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7089 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-01-20 r/5715 refactor(tvix/eval): keep globals alive through VM structVincent Ambo4-10/+59
This forces users to pass the fully constructed set of globals to the VM, making it harder to accidentally "lose" the set while weak references to it still exist. This doesn't modify any functionality, but is laying the foundation for simplifying some of the builtins behaviour that has grown more complex again. Change-Id: I5120f97861c65dc46d90b8a4e2c92ad32cc53e03 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7877 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-01-20 r/5714 docs(tvix/eval): add doc comments to VM fieldsVincent Ambo1-0/+8
Change-Id: Ia4857c217de15aec8b61e1abd39e22c50e2d816a Reviewed-on: https://cl.tvl.fyi/c/depot/+/7876 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su>
2023-01-20 r/5708 feat(tvix/eval): let builtin macro capture external stateVincent Ambo1-26/+98
This adds a feature to the `#[builtins]` macro which lets users specify an additional state type to (optionally) thread through to builtins when constructing them. This makes it possible for builtins-macro users to pass external state handles (specifically, in our case, known path tracking) into a set of builtins. Change-Id: I3ade20d333fc3ba90a80822cdfa5f87a9cfada75 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7840 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-01-20 r/5707 refactor(tvix/eval): directly return builtin tuples from macroVincent Ambo5-34/+23
All invocations of the builtin macro had to previously filter through the `builtin_tuple` function, but it's more sensible to directly return these from the macro. Change-Id: I45600ba84d56c9528d3e92570461c319eea595ce Reviewed-on: https://cl.tvl.fyi/c/depot/+/7825 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-01-20 r/5706 feat(tvix/eval): add error contexts to annotate error kindsVincent Ambo5-34/+99
This makes it possible for users to add additional context to an error, which will then be rendered as an additional secondary span in the formatted error output. We should strive to do this basically anywhere errors are raised that can occur multiple times, *especially* during type casts. This was triggered by me debugging a type cast error attached to a fairly large-ish span (a builtin invocation). Change-Id: I51be41fabee00cf04de973935daf34fe6424e76f Reviewed-on: https://cl.tvl.fyi/c/depot/+/7849 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-01-17 r/5676 refactor(tvix/eval): non-hacky suspended native thunksVincent Ambo1-55/+35
Instead of having a representation of suspended native thunks that involves constructing a fake code chunk, make these thunks a first-class part of the internal thunk representation. The previous code was not that simple to understand, and actually contained a critical bug which could lead to Tvix crashes. This version fixes the particular instance of that bug, but instead uncovers another (b/238) which can still lead to Tvix crashes. Fixes: b/237. Change-Id: I771d03864084d63953bdbb518fec94487481f839 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7750 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-01-17 r/5675 refactor(tvix/eval): remove `Box` in new_suspended_nativeVincent Ambo3-7/+5
This is unnecessary, Rc already provides all the boxing we need. Change-Id: I08cf0939c48da43f04c847526c7e5dae5336d528 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7749 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi> Reviewed-by: sterni <sternenseemann@systemli.org>
2023-01-17 r/5674 feat(tvix/eval): add builtins to builtinsVincent Ambo3-2/+34
This is a somewhat terrifying hack that enables us to support `builtins.builtins`, by running a "fake compilation" inside of a suspended native thunk that can resolve the weak pointer to the globals. With this implementation, the thunk at `builtins.builtins` actually resolves to the "real" `builtins` (verified with a new test). This is kind of ugly, and it's something users shouldn't use, but bubbling a warning out of this is difficult at the moment due to a little bit of trickery with how the spans in suspended native thunks work (they don't) (see b/237, b/238) Change-Id: I67d0e93246dd5b279c960aeda00402031aa12af3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7748 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-01-17 r/5670 refactor(tvix/value): use proptest strategies from imbl crateVincent Ambo5-71/+40
Instead of going through Vec/BTreeMap for generating our internal types, use the proptest strategies from imbl. The one thing I couldn't figure out in the previous implementation is where the ranges/sizes of generated collections came from. The strategies in proptest use different types (Range, with an unknown default value, and SizeRange with 0..100). I've opted to specify 0..100 directly, but we can probably make it configurable. Change-Id: I749bc4c703fe424099240cab822b1642e5216361 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7791 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-01-16 r/5665 feat(tvix/eval): add error variant for threading through errorsVincent Ambo1-2/+17
This variant is required for external builtins (which in our case includes `derivation`) to thread through reasonable error messages. This has some potential for improvement, but it's an improvement over the status quo of panicking in the external builtins when no appropriate error is available. Change-Id: I7e4bdb0a156c7717092dde30aa4785192182dc66 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7841 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-01-16 r/5664 feat(tvix/eval): implement builtins.toXMLVincent Ambo8-0/+158
Change-Id: I009efc53a8e98f0650ae660c4decd8216e8a06e7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7835 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-01-16 r/5663 chore(tvix/eval): add other required items to public APIVincent Ambo2-3/+7
External implementors of builtins must be able to force values, which necessitates publishing a bunch more items from the crate. Change-Id: I8f6b8ae88156aae417dbe630a698d123d0c1c8d4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7830 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
generated by cgit-pink 1.4.1 (git 2.44.1) at 2024-06-30T23·04+0000