about summary refs log tree commit diff
path: root/tvix/eval/src/value/thunk.rs
AgeCommit message (Collapse)AuthorFilesLines
2023-03-13 r/5964 refactor(tvix/eval): flatten call stack of VM using generatorsVincent Ambo1-229/+35
Warning: This is probably the biggest refactor in tvix-eval history, so far. This replaces all instances of trampolines and recursion during evaluation of the VM loop with generators. A generator is an asynchronous function that can be suspended to yield a message (in our case, vm::generators::GeneratorRequest) and receive a response (vm::generators::GeneratorResponsee). The `genawaiter` crate provides an interpreter for generators that can drive their execution and lets us move control flow between the VM and suspended generators. To do this, massive changes have occured basically everywhere in the code. On a high-level: 1. The VM is now organised around a frame stack. A frame is either a call frame (execution of Tvix bytecode) or a generator frame (a running or suspended generator). The VM has an outer loop that pops a frame off the frame stack, and then enters an inner loop either driving the execution of the bytecode or the execution of a generator. Both types of frames have several branches that can result in the frame re-enqueuing itself, and enqueuing some other work (in the form of a different frame) on top of itself. The VM will eventually resume the frame when everything "above" it has been suspended. In this way, the VM's new frame stack takes over much of the work that was previously achieved by recursion. 2. All methods previously taking a VM have been refactored into async functions that instead emit/receive generator messages for communication with the VM. Notably, this includes *all* builtins. This has had some other effects: - Some test have been removed or commented out, either because they tested code that was mostly already dead (nix_eq) or because they now require generator scaffolding which we do not have in place for tests (yet). - Because generator functions are technically async (though no async IO is involved), we lose the ability to use much of the Rust standard library e.g. in builtins. This has led to many algorithms being unrolled into iterative versions instead of iterator combinations, and things like sorting had to be implemented from scratch. - Many call sites that previously saw a `Result<..., ErrorKind>` bubble up now only see the result value, as the error handling is encapsulated within the generator loop. This reduces number of places inside of builtin implementations where error context can be attached to calls that can fail. Currently what we gain in this tradeoff is significantly more detailed span information (which we still need to bubble up, this commit does not change the error display). We'll need to do some analysis later of how useful the errors turn out to be and potentially introduce some methods for attaching context to a generator frame again. This change is very difficult to do in stages, as it is very much an "all or nothing" change that affects huge parts of the codebase. I've tried to isolate changes that can be isolated into the parent CLs of this one, but this change is still quite difficult to wrap one's mind and I'm available to discuss it and explain things to any reviewer. Fixes: b/238, b/237, b/251 and potentially others. Change-Id: I39244163ff5bbecd169fe7b274df19262b515699 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8104 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Reviewed-by: Adam Joseph <adam@westernsemico.com> Tested-by: BuildkiteCI
2023-03-04 r/5888 refactor(tvix/eval): remove VM argument from suspended native thunksVincent Ambo1-4/+4
Because they do not use it, and it can not be passed with the coming generator refactoring. Change-Id: I0d96f2357a7ee79cd8a0f401583d4286230d4a6b Reviewed-on: https://cl.tvl.fyi/c/depot/+/8146 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-03-04 r/5886 feat(tvix/eval): add SharedThunkSetVincent Ambo1-0/+11
This is a ThunkSet wrapped to be shareable, which will be required once ThunkSets are embedded in futures. Change-Id: I5a067b7972ac86e4d354c75ef05c86b2284c1137 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8144 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-03-04 r/5885 fix(tvix/eval): ThunkSet does not need mutable pointersVincent Ambo1-2/+2
Change-Id: Iea248870a0ea5d38cb02ff059c968fbd563570b6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8143 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-03-03 r/5870 chore(tvix/eval): fix clippy warningsVincent Ambo1-13/+13
Change-Id: I4c02f0104c455ac00a3f299c1fbf75cbb08e8972 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8142 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su>
2023-03-03 r/5868 refactor(tvix/eval): enhance debug output for bytecode dumpsVincent Ambo1-0/+16
This adds addresses of thunk and closure chunks to the debug output displayed when dumping bytecode. This makes it possible to see in the dump which thunks are referenced by constants in other thunks. Change-Id: I2c98de5227e7cb415666cd3134c947a56979dc80 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8137 Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-02-16 r/5857 refactor(tvix/eval): remove redundant cloneAaqa Ishtyaq1-4/+4
This CL removes redundant clone from value which is going to be dropped without further use. Change-Id: Ibd2a724853c5cfbf8ca40bf0b3adf0fab89b9be5 Signed-off-by: Aaqa Ishtyaq <aaqaishtyaq@gmail.com> Reviewed-on: https://cl.tvl.fyi/c/depot/+/8125 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-02-03 r/5828 fix(tvix/eval): ensure all evaluated thunks are correctly memoizedVincent Ambo1-45/+185
This fixes a very complicated bug (b/246). Evaluation progresses *much* further after this, leading to several less complicated bugs likely being uncovered by this What was the problem? ===================== Previously, when evaluating a thunk, we had a code path that looked like this: match *thunk { ThunkRepr::Evaluated(Value::Thunk(ref inner_thunk)) => { let inner_repr = inner_thunk.0.borrow().clone(); drop(thunk); self.0.replace(inner_repr); } /* ... */ } This code path created a copy of the inner `ThunkRepr` of a nested thunk, and moved that copy into the `ThunkRepr` of the parent. The effect of this was that the original `ThunkRepr` (unforced!) lived on in the original thunk, without the memoization of the subsequent forcing applying to it. This had the result that Tvix would repeatedly evaluate these thunks without ever memoizing them, if they occured repeatedly as shared inner thunks. Most notably, this would *always* occur when builtins.import was used. What's the solution? ==================== I have completely rewritten `Thunk::force_trampoline_self` to make all flows that can occur in it explicit. I have also removed the outer loop inside of that function, and resorted to more use of trampolining instead. The function is now well-commented and it should be possible to read it from top-to-bottom and get a general sense of what is going on, though the trampolining itself (which is implemented in the VM) needs to be at least partially understood for this. What's the new problem(s)? ========================== One new (known) problem is that we have to construct `Error` instances in all error types here, but we do not have spans available in some thunk-related situations. Due to b/238 we cannot ask the VM for an arbitrary span from the callsite leading to the force. This means that there are now code paths where, under certain conditions, causing an evaluation error during thunk forcing will panic. To fix this we will need to investigate and fix b/238, and/or add a span tracking mechanism to thunks themselves. What other impacts does this have? ================================== With this commit, eval of nixpkgs mostly succeeds (things like stdenv evaluate to the same hashes for us and C++ Nix, meaning we now construct identical derivations without eval breaking). Due to this we progress much further into nixpkgs, which lets us uncover more additional bugs. For example, after this commit we can quickly see that cl/7949 introduces some kind of behavioural issue and should not be merged as-is (this was not apparent before). Additionally, tvix-eval is now seemingly very fast. When doing performance analysis of a nixpkgs eval, we now mostly see the code path for shelling out to C++ Nix to add things to the store in there. We still need those code paths, so we can not (yet) do a performance analysis beyond that. Change-Id: I738525bad8bc5ede5d8c737f023b14b8f4160612 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8012 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-01-20 r/5706 feat(tvix/eval): add error contexts to annotate error kindsVincent Ambo1-4/+2
This makes it possible for users to add additional context to an error, which will then be rendered as an additional secondary span in the formatted error output. We should strive to do this basically anywhere errors are raised that can occur multiple times, *especially* during type casts. This was triggered by me debugging a type cast error attached to a fairly large-ish span (a builtin invocation). Change-Id: I51be41fabee00cf04de973935daf34fe6424e76f Reviewed-on: https://cl.tvl.fyi/c/depot/+/7849 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-01-17 r/5676 refactor(tvix/eval): non-hacky suspended native thunksVincent Ambo1-55/+35
Instead of having a representation of suspended native thunks that involves constructing a fake code chunk, make these thunks a first-class part of the internal thunk representation. The previous code was not that simple to understand, and actually contained a critical bug which could lead to Tvix crashes. This version fixes the particular instance of that bug, but instead uncovers another (b/238) which can still lead to Tvix crashes. Fixes: b/237. Change-Id: I771d03864084d63953bdbb518fec94487481f839 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7750 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-01-17 r/5675 refactor(tvix/eval): remove `Box` in new_suspended_nativeVincent Ambo1-3/+1
This is unnecessary, Rc already provides all the boxing we need. Change-Id: I08cf0939c48da43f04c847526c7e5dae5336d528 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7749 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi> Reviewed-by: sterni <sternenseemann@systemli.org>
2023-01-12 r/5652 feat(tvix/eval): implement builtins.toJSONVincent Ambo1-0/+11
Implements `Serialize` for `tvix_eval::Value`. Special care is taken with serialisation of attribute sets, and forcing of thunks. The tests should cover both cases well. Change-Id: I9bb135bacf6f87bc6bd0bd88cef0a42308e6c335 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7803 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su>
2022-12-25 r/5486 fix(tvix/eval): fix current clippy warningsVincent Ambo1-5/+5
It's been a while since the last time, so quite a lot of stuff has accumulated here. Change-Id: I0762827c197b30a917ff470fd8ae8f220f6ba247 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7597 Reviewed-by: grfn <grfn@gws.fyi> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-12-25 r/5485 refactor(tvix/eval): non-recursive thunk forcingAdam Joseph1-29/+115
Introduces continuation-passing-based trampolining of thunk forcing to avoid recursing when forcing deeply nested expressions. This is required for evaluating large expressions. This change was extracted out of cl/7362. Co-authored-by: Vincent Ambo <tazjin@tvl.su> Co-authored-by: Griffin Smith <grfn@gws.fyi> Change-Id: Ifc1747e712663684b2fff53095de62b8459a47f3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7551 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-12-21 r/5457 refactor(tvix/eval): add a LightSpan type for lighter span trackingVincent Ambo1-9/+12
This type carries the information required for calculating a span (i.e. the chunk and offset), instead of the span itself. The span is then only calculated in cases where it is required (when throwing errors). This reduces the eval time for `builtins.length (builtins.attrNames (import <nixpkgs> {}))` by *one third*! The data structure in chunks that carries span information reduces in-memory size by trading off the speed of retrieving span information. This is because the span information is only actually required when throwing errors (or emitting warnings). However, somewhere along the way we grew a dependency on carrying span information in thunks (for correctly reporting error chains). Hitting the code paths for span retrieval was expensive, and carrying the spans in a different way would still be less cache-efficient. This change is the best tradeoff I could come up with. Refs: b/229. Change-Id: I27d4c4b5c5f9be90ac47f2db61941e123a78a77b Reviewed-on: https://cl.tvl.fyi/c/depot/+/7558 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-21 r/5455 feat(tvix/eval): add thunks with suspended native Rust codeAdam Joseph1-1/+45
Having thunks which, when forced, execute native Rust code rather than interpreted opcodes lets us avoid having to bundle `src/libexpr/primops/derivation.nix` like cppnix does by implementing it in Rust instead. Change-Id: If91d77a6736234321eee87ba4b4777eed5a3fe1c Reviewed-on: https://cl.tvl.fyi/c/depot/+/7450 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-21 r/5453 feat(tvix/eval): remove `derive(Copy)` from UpvaluesAdam Joseph1-4/+4
Change-Id: I0fa069fbeff6718a765ece948c2c1bce285496f7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7449 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-21 r/5452 feat(tvix/eval): wrap Closure in Rc<> to match cppnix semanticsAdam Joseph1-39/+20
Change-Id: I595087eff943d38a9fc78a83d37e207bb2ab79bc Reviewed-on: https://cl.tvl.fyi/c/depot/+/7443 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-11-27 r/5345 feat(tvix/eval): non-recursive implementation of nix_eq()Adam Joseph1-0/+6
This passes all the function/thunk-pointer-equality tests in cl/7369. Change-Id: Ib47535ba2fc77a4f1c2cc2fd23d3a879e21d8b4c Signed-off-by: Adam Joseph <adam@westernsemico.com> Reviewed-on: https://cl.tvl.fyi/c/depot/+/7358 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-11-26 r/5324 feat(tvix/eval): wrap Closure::upvalues in RcAdam Joseph1-2/+3
See cl/7372; Nix equality semantics require the ability to track pointer equality of upvalue-sets. Signed-off-by: Adam Joseph <adam@westernsemico.com> Change-Id: I82ba517499cf370189a80355e4e46a5caaab7153 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7373 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-11-23 r/5303 feat(tvix/eval): improve panic!() messages in Thunk::value()Adam Joseph1-4/+4
Change-Id: I3b1284e28c350bfed84d643ae7f922f3487e1f2a Reviewed-on: https://cl.tvl.fyi/c/depot/+/7355 Autosubmit: Adam Joseph <adam@westernsemico.com> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-11-21 r/5297 fix(tvix/eval): ensure callable is forced when using call_withVincent Ambo1-0/+4
When passing multiple arguments, every intermediate callable needs to be forced as this is expected by the VM's call_value function. Also adds a debug assertion for this which makes it easier to spot exactly what went wrong. Change-Id: I3aa519cb6cdaab713bd18282bef901c4cd77c535 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7312 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-11-04 r/5236 fix(tvix/eval): remove impl PartialEq for ValueAdam Joseph1-2/+2
It isn't possible to implement PartialEq properly for Value, because any sensible implementation needs to force() thunks, which cannot be done without a `&mut VM`. The existing derive(PartialEq) has false negatives, which caused the bug which cl/7142 fixed. Fortunately that bug was easy to find, but a silent false negative deep within the bowels of nixpkgs could be a real nightmare to hunt down. Let's just remove the PartialEq impl for Value, and the other derive(PartialEq)'s that depend on it. Signed-off-by: Adam Joseph <adam@westernsemico.com> Change-Id: Iacd3726fefc7fc1edadcd7e9b586e04cf8466775 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7144 Reviewed-by: kanepyork <rikingcoding@gmail.com> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-10-23 r/5178 fix(tvix/eval): detect cycles when printing infinite valuesVincent Ambo1-5/+8
Using the same method as in Thunk::deep_force, detect cycles when printing values by maintaining a set of already seen thunks. With this, display of infinite values matches that of Nix: > nix-instantiate --eval --strict -E 'let as = { x = 123; y = as; }; in as' { x = 123; y = { x = 123; y = <CYCLE>; }; } > tvix-eval -E 'let as = { x = 123; y = as; }; in as' => { x = 123; y = { x = 123; y = <CYCLE>; }; } :: set Change-Id: I007b918d5131d82c28884e46e46ff365ef691aa8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7056 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-10-22 r/5175 feat(tvix/eval): Implement builtins.deepSeqGriffin Smith1-6/+23
This is done via a new `deepForce` function on Value. Since values can be cyclical (for example, see the test-case), we need to do some extra work to avoid RefCell borrow errors if we ever hit a graph cycle: While deep-forcing values, we keep a set of thunks that we have already seen and avoid doing any work on the same thunk twice. The set is encapsulated in a separate type to stop potentially invalid pointers from leaking out. Finally, since deep_force is conceptually similar to `VM::force_for_output` (but more suited to usage in eval since it doesn't clone the values) this removes the latter, replacing it with the former. Co-Authored-By: Vincent Ambo <tazjin@tvl.su> Change-Id: Iefddefcf09fae3b6a4d161a5873febcff54b9157 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7000 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi> Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-19 r/5159 feat(tvix/eval): deduplicate overlap between Closure and ThunkAdam Joseph1-38/+76
This commit deduplicates the Thunk-like functionality from Closure and unifies it with Thunk. Specifically, we now have one and only one way of breaking reference cycles in the Value-graph: Thunk. No other variant contains a RefCell. This should make it easier to reason about the behavior of the VM. InnerClosure and UpvaluesCarrier are no longer necessary. This refactoring allowed an improvement in code generation: `Rc<RefCell<>>`s are now created only for closures which do not have self-references or deferred upvalues, instead of for all closures. OpClosure has been split into two separate opcodes: - OpClosure creates non-recursive closures with no deferred upvalues. The VM will not create an `Rc<RefCell<>>` when executing this instruction. - OpThunkClosure is used for closures with self-references or deferred upvalues. The VM will create a Thunk when executing this opcode, but the Thunk will start out already in the `ThunkRepr::Evaluated` state, rather than in the `ThunkRepr::Suspeneded` state. To avoid confusion, OpThunk has been renamed OpThunkSuspended. Thanks to @sterni for suggesting that all this could be done without adding an additional variant to ThunkRepr. This does however mean that there will be mutating accesses to `ThunkRepr::Evaluated`, which was not previously the case. The field `is_finalised:bool` has been added to `Closure` to ensure that these mutating accesses are performed only on finalised Closures. Both the check and the field are present only if `#[cfg(debug_assertions)]`. Change-Id: I04131501029772f30e28da8281d864427685097f Signed-off-by: Adam Joseph <adam@westernsemico.com> Reviewed-on: https://cl.tvl.fyi/c/depot/+/7019 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-10 r/5097 fix(tvix/eval): Actually trace spans for thunksGriffin Smith1-18/+30
Currently, the span on *all* thunk force errors is the span at which the thunk is forced, which for recursive thunk forcing ends up just being the same span over and over again. This changes the span on thunk force errors to be the span at which point the thunk is *created*, which is a bit more helpful (though the printing atm is a little... crowded). To make this work, we have to thread through the span at which a thunk is created into a field on the thunk itself. Change-Id: I81474810a763046e2eb3a8f07acf7d8ec708824a Reviewed-on: https://cl.tvl.fyi/c/depot/+/6932 Autosubmit: grfn <grfn@gws.fyi> Reviewed-by: Adam Joseph <adam@westernsemico.com> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-10-10 r/5094 refactor(tvix/eval): after calling, the caller has to popVincent Ambo1-4/+3
Previously the various call functions either returned `EvalResult<()>` or `EvalResult<Value>`, which was confusing. Now only vm::call_with returns a Value directly, and other parts of the API just leave the stack top in the post-call state. This makes it easier to reason about what's going on in non-tail-call cases (which are making a comeback). Change-Id: I264ffc683a11aca72dd06e2220a5ff6e7c5fc2b0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6936 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-09-18 r/4908 refactor(tvix/eval): Don't (ab)use PartialEq for Nix equalityGriffin Smith1-2/+2
Using rust's PartialEq trait to implement Nix equality semantics is reasonably fraught with peril, both because the actual laws are different than what nix expects, and (more importantly) because certain things actually require extra context to compare for equality (for example, thunks need to be forced). This converts the manual PartialEq impl for Value (and all its descendants) to a *derived* PartialEq impl (which requires a lot of extra PartialEq derives on miscellanious other types within the codebase), and converts the previous nix-semantics equality comparison into a new `nix_eq` method. This returns an EvalResult, even though it can't currently return an error, to allow it to fail when eg forcing thunks (which it will do soon). Since the PartialEq impls for Value and NixAttrs are now quite boring, this converts the generated proptests for those into handwritten ones that cover `nix_eq` instead Change-Id: If3da7171f88c22eda5b7a60030d8b00c3b76f672 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6650 Autosubmit: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-11 r/4800 refactor(tvix/eval): introduce Upvalues struct in closures & thunksVincent Ambo1-6/+11
This struct will be responsible for tracking upvalues (and is a convenient place to introduce optimisations for reducing value clones) instead of a plain value vector. The main motivation for this is that the upvalues will have to capture the `with`-stack fully and I want to avoid duplicating the logic for this between the two capturing types. Change-Id: I6654f8739fc2e04ca046e6667d4a015f51724e99 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6485 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-08 r/4769 fix(tvix/eval): hold thunk borrow as shortly as possibleVincent Ambo1-7/+6
At the point where control flow exits Thunk::force (which may be due to recursing), it is vital that there is no longer a borrow to the inner thunk representation, otherwise this can cause accidental infinite recursion (which will be detected, but cause failures on valid code). Change-Id: I2846f3142830ae3110a4f5d2299e9d7928634504 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6436 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-08 r/4749 fix(tvix/eval): don't panic when printing a black holeVincent Ambo1-2/+6
This could occur when the disassembler is enabled and tracing the runtime while a thunk is being evaluated, as it would not be possible for the *tracer* to borrow the thunk at this exact moment. However, we know that if the borrowing fails here we are dealing with a not-fully evaluated thunk (blackhole), which should just print the internal representation. Change-Id: I4bdb4f17818d55795368e3d28842048f488f0a91 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6416 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-08 r/4748 refactor(tvix/eval): return call frame result from VM::callVincent Ambo1-2/+2
Previously, "calling" (setting up the VM run loop for executing a call frame) and "running" (running this loop to completion) were separate operations. This was basically an attempt to avoid nesting `VM::run` invocations. However, doing things this way introduced some tricky bugs for exiting out of the call frames of thunks vs. builtins & closures. For now, we unify the two operations and always return the value to the caller directly. For now this makes calls a little less effective, but it gives us a chance to nail down some other strange behaviours and then re-optimise this afterwards. To make sure we tackle this again further down I've added it to the list of known possible optimisations. Change-Id: I96828ab6a628136e0bac1bf03555faa4e6b74ece Reviewed-on: https://cl.tvl.fyi/c/depot/+/6415 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-08 r/4742 fix(tvix/eval): thread thunk forcing errors through correctlyVincent Ambo1-3/+3
With this, if an error occurs while forcing a thunk (which is very likely) it is threaded through to the top by wrapping it in the ErrorKind::ThunkForce variant. We could use this to generate "stacktrace-like" error output if we wanted, or simply jump through and discard everything except the innermost error. Change-Id: I3c1c8708c2f73ae062815adf490ce935b1979da8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6409 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-08 r/4741 feat(tvix/eval): ensure all errors always carry a spanVincent Ambo1-4/+6
Previously error spans were optional because the information about code spans was not available at runtime. Now that this information has been added, the error type will always carry a span. This change is very invasive all throughout the codebase. This is due to the fact that many functions that are called *by* the VM expected to return `EvalResult`, but this no longer works as the span information is not available to those functions - only to the VM itself. To work around this the majority of these functions have been changed to return `Result<T, ErrorKind>` instead and an accompanying macro in the VM constructs the "real" error. Note that this implementatino currently has a bug where errors occuring within thunks will yield the location at which the thunk was forced, not the location at which the error occured within the code. This will be fixed soon, but the commit is large enough as is. Change-Id: Ib1ecb81a4d09d464a95ea7ea9e589f3bd08d5202 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6408 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-07 r/4703 fix(tvix/eval): thread Display & PartialEq through to thunk valuesVincent Ambo1-0/+10
If a thunk is already evaluated, there are cases where due to the memoisation implementation something might observe a value wrapped in a thunk. In these cases, the implementation of `Display` and `PartialEq` must delegate to the underlying value. Note that there are a handful of other cases like these which we need to cover. It is a little tricky to write integration tests for these directly, especially as some of the open-upvalue optimisations coming down the pipe will reduce the number of observable thunks. One test that covers a part of this behaviour is currently disabled (needs some more machinery), but it's being brought back in the next commits. Change-Id: Iaa8cd338c12236af844bbc99d8cec2205f0d0095 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6370 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-07 r/4700 refactor(tvix/eval): encapsulate all thunk-forcing logic in moduleVincent Ambo1-20/+41
The VM previously took care of repeatedly forcing a thunk until it reached an evaluated state. This logic is now encapsulated inside of the `Thunk::force` implementation. In addition, force no longer returns a reference to the value by default, leaving it up to callers to decide whether they want to borrow the value or not (a helper is provided for this). Change-Id: I2aa7da922058ad1c57fbf8bfc7785aab7971c02b Reviewed-on: https://cl.tvl.fyi/c/depot/+/6365 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 r/4688 feat(tvix/eval): implement OpForce in VMVincent Ambo1-1/+31
This operation forces the evaluation of a thunk. There is some potential here for making an implementation that avoids some copies, but the thunk machinery is tricky to get right so the first priority is to make sure it is correct by keeping the implementation simple. Change-Id: Ib381455b02f42ded717faff63f55afed4c8fb7e3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6352 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 r/4682 fix(tvix/eval): allocate Thunk::upvalues with known capacityVincent Ambo1-1/+1
The capacity (i.e. number of builtins) is known from the lambda, so we can size it correctly right away. Change-Id: Iab0b5a3f47d450fa9866c091ebbbed935b934907 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6351 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 r/4677 refactor(tvix/eval): introduce UpvalueCarrier traitVincent Ambo1-2/+29
This trait abstracts over the commonalities of upvalue handling between closures and thunks. It allows the VM to simplify the code used for setting up upvalues, without duplicating between the two different types. Note that this does not yet refactor the VM code to optimally make use of this. Change-Id: If8de5181f26ae1fa00d554f1ae6ea473ee4b6070 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6347 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 r/4676 refactor(tvix/eval): simplify thunk representationsVincent Ambo1-7/+7
For now, do not distinguish between closing and non-closing thunks, it will make the initial implementation easier. See Knuth etc. Change-Id: I0bd51e0f89f2c77e90bac63b507e5027b649e3d8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6346 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 r/4673 feat(tvix/eval): introduce Value::Thunk variantVincent Ambo1-0/+55
Introduces the representation of runtime thunks, that is lazily evaluated values. Their representation is very similar to closures. Change-Id: I24d1ab7947c070ae72ca6260a7bbe6198bc8c7c5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6343 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>