about summary refs log tree commit diff
path: root/tvix/eval/src/compiler/mod.rs
AgeCommit message (Collapse)AuthorFilesLines
2023-01-06 r/5599 feat(tvix/eval): add Evaluation::compile_only methodVincent Ambo1-1/+3
This would make it possible to implement something like a linter based on the tvix-eval compiler warnings. Change-Id: I1feb4e7c4a44be7d1204b0a962ab522fd32b93c6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7763 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-01-04 r/5581 refactor(tvix/eval): streamline construction of globals/builtinsVincent Ambo1-21/+82
Previously the construction of globals (a compiler-only concept) and builtins (a (now) user-facing API) was intermingled between multiple different modules, and kind of difficult to understand. The complexity of this had grown in large part due to the implementation of `builtins.import`, which required the notorious "knot-tying" trick using Rc::new_cyclic (see cl/7097) for constructing the set of globals. As part of the new `Evaluation` API users should have the ability to bring their own builtins, and control explicitly whether or not impure builtins are available (regardless of whether they're compiled in or not). To streamline the construction and allow the new API features to work, this commit restructures things by making these changes: 1. The `tvix_eval::builtins` module is now only responsible for exporting sets of builtins. It no longer has any knowledge of whether or not certain sets (e.g. only pure, or pure+impure) are enabled, and it has no control over which builtins are globally available (this is now handled in the compiler). 2. The compiler module is now responsible for both constructing the final attribute set of builtins from the set of builtins supplied by a user, as well as for populating its globals (that is identifiers which are available at the top-level scope). 3. The `Evaluation` API now carries a `builtins` field which is populated with the pure builtins by default, and can be extended by users. 4. The `import` feature has been moved into the compiler, as a special case. In general, builtins no longer have the ability to reference the "fix point" of the globals set. This should not change any functionality, and in fact preserves minor differences between Tvix/Nix that we already had (such as `builtins.builtins` not existing). Change-Id: Icdf5dd50eb81eb9260d89269d6e08b1e67811a2c Reviewed-on: https://cl.tvl.fyi/c/depot/+/7738 Reviewed-by: sterni <sternenseemann@systemli.org> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2022-12-25 r/5486 fix(tvix/eval): fix current clippy warningsVincent Ambo1-3/+3
It's been a while since the last time, so quite a lot of stuff has accumulated here. Change-Id: I0762827c197b30a917ff470fd8ae8f220f6ba247 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7597 Reviewed-by: grfn <grfn@gws.fyi> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-12-21 r/5457 refactor(tvix/eval): add a LightSpan type for lighter span trackingVincent Ambo1-1/+2
This type carries the information required for calculating a span (i.e. the chunk and offset), instead of the span itself. The span is then only calculated in cases where it is required (when throwing errors). This reduces the eval time for `builtins.length (builtins.attrNames (import <nixpkgs> {}))` by *one third*! The data structure in chunks that carries span information reduces in-memory size by trading off the speed of retrieving span information. This is because the span information is only actually required when throwing errors (or emitting warnings). However, somewhere along the way we grew a dependency on carrying span information in thunks (for correctly reporting error chains). Hitting the code paths for span retrieval was expensive, and carrying the spans in a different way would still be less cache-efficient. This change is the best tradeoff I could come up with. Refs: b/229. Change-Id: I27d4c4b5c5f9be90ac47f2db61941e123a78a77b Reviewed-on: https://cl.tvl.fyi/c/depot/+/7558 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-21 r/5454 refactor(tvix/eval): add name-based index over compiler's localsVincent Ambo1-6/+7
Instead of finding locals by doing 2x O(n) walks over the compiler's locals list, use a secondary name-based index for resolving locals by name. Previously, almost 60% (!!) of eval time on some expressions over nixpkgs was spent in `Local::has_name`. This function doesn't even exist anymore now, and eval speed about doubles as a result. Note that this doesn't exactly make the locals code easier to read, but I'm also not sure what we can simplify in there in general. This fixes b/227. Change-Id: I29ce5eb9452b02d3b358c673e1f5cf8082e2fef9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7560 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-21 r/5452 feat(tvix/eval): wrap Closure in Rc<> to match cppnix semanticsAdam Joseph1-1/+1
Change-Id: I595087eff943d38a9fc78a83d37e207bb2ab79bc Reviewed-on: https://cl.tvl.fyi/c/depot/+/7443 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-11-05 r/5250 refactor(tvix/eval): rename Opcode::DataLocalIdx to DataStackIdxAdam Joseph1-1/+1
It is very confusing that this opcode is called DataLocalIdx, but it carries a StackIdx rather than a LocalIdx. It seems like this really ought to be called DataStackIdx, but maybe I've misunderstood; if so please explain it to me. Change-Id: I91f6ffa759412beef0b91d3c19ec0d873fe51b99 Signed-off-by: Adam Joseph <adam@westernsemico.com> Reviewed-on: https://cl.tvl.fyi/c/depot/+/7088 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-10-27 r/5214 refactor(tvix/eval): search-and-replace changesAdam Joseph1-5/+5
This commit contains two search-and-replace renames which are broken out from I04131501029772f30e28da8281d864427685097f in order to reduce the noise in that CL: - `is_thunk -> is_suspended_thunk`, since there are now OpThunkClosure and OpThunkSuspended - `compile_lambda_or_thunk` -> `compile_lambda_or_suspension` Change-Id: I7cc5bbb75ef6605e3428c7be27e812f41a10c127 Signed-off-by: Adam Joseph <adam@westernsemico.com> Reviewed-on: https://cl.tvl.fyi/c/depot/+/7037 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-10-27 r/5213 feat(tvix/eval): builtins.import without RefCellAdam Joseph1-45/+51
CL/6867 added support for builtins.import, which required a cyclic reference import->globals->builtins->import. This was implemented using a RefCell, which makes it possible to mutate the builtins during evaluation. The commit message for CL/6867 expressed a desire to eliminate this possibility: This opens up a potentially dangerous footgun in which we could mutate the builtins at runtime leading to different compiler invocations seeing different builtins, so it'd be nice to have some kind of "finalised" status for them or some such, but I'm not sure how to represent that atm. This CL replaces the RefCell with Rc::new_cyclic(), making the globals/builtins immutable once again. At VM runtime (once opcodes start executing) everything is the same as before this CL, except that the Rc<RefCell<>> introduced by CL/6867 is turned into an rc::Weak<>. The function passed to Rc::new_cyclic works very similarly to overlays in nixpkgs: a function takes its own result as an argument. However instead of laziness "breaking the cycle", Rust's Rc::new_cyclic() instead uses an rc::Weak. This is done to prevent memory leaks rather than divergence. This CL also resolves the following TODO from CL/6867: // TODO: encapsulate this import weirdness in builtins The main disadvantage of this CL is the fact that the VM now must ensure that it holds a strong reference to the globals while a program is executing; failure to do so will cause a panic when the weak reference in the builtins is upgrade()d. In theory it should be possible to create strong reference cycles the same way Rc::new_cyclic() creates weak cycles, but these cycles would cause a permanent memory leak -- without either an rc::Weak or RefCell there is no way to break the cycle. At some point we will have to implement some form of cycle collection; whatever library we choose for that purpose is likely to provide an "immutable strong reference cycle" primitive similar to Rc::new_cyclic(), and we should be able to simply drop it in. Signed-off-by: Adam Joseph <adam@westernsemico.com> Change-Id: I34bb5821628eb97e426bdb880b02e2097402adb7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7097 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-23 r/5186 feat(tvix/eval): initial attempt at setting lambda namesVincent Ambo1-2/+8
When compiling a lambda, take the name of the outer slot (if available) and store it as the name on the lambda. These names are then shown in the observer, and nowhere else (so far). It is of course common for these things to thread through many different context levels (e.g. `f = a: b: c: ...`), in this setup only the outermost closure or thunk gains the name, but it's better than nothing. Change-Id: I681ba74e624f2b9e7a147144a27acf364fe6ccc7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7065 Reviewed-by: grfn <grfn@gws.fyi> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-10-23 r/5184 fix(tvix/eval): thunk let-expressionVincent Ambo1-1/+5
There are some rare scope cases with deferred access where this doesn't behave correctly otherwise. Change-Id: I6c774f5e62c1cb50b598026c54727017a52cd22d Reviewed-on: https://cl.tvl.fyi/c/depot/+/7064 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-10-23 r/5182 refactor(tvix/eval): simplify check for deferring upvalue resolutionVincent Ambo1-9/+4
This check is now actually simply equivalent to checking whether the target has been initialised or not. Change-Id: I30660d11073ba313358f3a64234a90ed81abf74c Reviewed-on: https://cl.tvl.fyi/c/depot/+/7062 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-10-23 r/5180 refactor(tvix/eval): simplify self-reference checkVincent Ambo1-1/+1
Checking the computed depth and stack slot against the computed depth and stack slot is equivalent to just checking the indices into the locals vector against each other (i.e. "is the slot we're compiling into the slot we're accessing?") Change-Id: Ie85a68df073e3b2e3d9aba7fe8634c48eada81fc Reviewed-on: https://cl.tvl.fyi/c/depot/+/7059 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-10-21 r/5172 fix(tvix): distinguish search- and relative path resolution errorssterni1-2/+2
Failures to resolve a nix search path lookup in angle brackets can be caught using tryEval (if it reaches the runtime). Resolving relative paths (either to the current directory or the current user's home) can never be caught, even if they happen inside a thunk at runtime (which is currently the case for home-relative paths). Change-Id: I7f73221df66d82a381dd4063358906257826995a Reviewed-on: https://cl.tvl.fyi/c/depot/+/7025 Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: Adam Joseph <adam@westernsemico.com> Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-10-19 r/5159 feat(tvix/eval): deduplicate overlap between Closure and ThunkAdam Joseph1-4/+25
This commit deduplicates the Thunk-like functionality from Closure and unifies it with Thunk. Specifically, we now have one and only one way of breaking reference cycles in the Value-graph: Thunk. No other variant contains a RefCell. This should make it easier to reason about the behavior of the VM. InnerClosure and UpvaluesCarrier are no longer necessary. This refactoring allowed an improvement in code generation: `Rc<RefCell<>>`s are now created only for closures which do not have self-references or deferred upvalues, instead of for all closures. OpClosure has been split into two separate opcodes: - OpClosure creates non-recursive closures with no deferred upvalues. The VM will not create an `Rc<RefCell<>>` when executing this instruction. - OpThunkClosure is used for closures with self-references or deferred upvalues. The VM will create a Thunk when executing this opcode, but the Thunk will start out already in the `ThunkRepr::Evaluated` state, rather than in the `ThunkRepr::Suspeneded` state. To avoid confusion, OpThunk has been renamed OpThunkSuspended. Thanks to @sterni for suggesting that all this could be done without adding an additional variant to ThunkRepr. This does however mean that there will be mutating accesses to `ThunkRepr::Evaluated`, which was not previously the case. The field `is_finalised:bool` has been added to `Closure` to ensure that these mutating accesses are performed only on finalised Closures. Both the check and the field are present only if `#[cfg(debug_assertions)]`. Change-Id: I04131501029772f30e28da8281d864427685097f Signed-off-by: Adam Joseph <adam@westernsemico.com> Reviewed-on: https://cl.tvl.fyi/c/depot/+/7019 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-17 r/5154 feat(tvix/eval): Validate closed formalsGriffin Smith1-7/+6
Validate "closed formals" (formal parameters without an ellipsis) via a new ValidateClosedFormals op, which checks the arguments (in an attr set at the top of the stack) against the formal parameters on the Lambda in the current frame, and returns a new UnexpectedArgument error (including the span of the formals themselves!!) if any arguments aren't allowed Change-Id: Idcc47a59167a83be1832a6229f137d84e426c56c Reviewed-on: https://cl.tvl.fyi/c/depot/+/7002 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-10-17 r/5153 feat(tvix/eval): Record formals on lambdaGriffin Smith1-11/+20
In preparation for both implementing the `functionArgs` builtin and adding support for validating closed formals, record information about the formal arguments to a function *on the Lambda itself*. This may seem a little odd for the purposes of just closed formal checking, but is something we have to have anyway for builtins.functionArgs so I figured I'd do it this way to kill both birds with one stone. Change-Id: Ie3770a607bf352a1eb395c79ca29bb25d5978cd8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7001 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-10-16 r/5150 refactor(tvix/eval): unify compile_lambda() with thunk()Adam Joseph1-61/+59
This resolves a TODO. Signed-off-by: Adam Joseph <adam@westernsemico.com> Change-Id: If4d2124648ac88094e547e1ad7f1b446feb26182 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7010 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-16 r/5147 fix(tvix/eval): resolve home relative paths at runtimesterni1-12/+8
Home relative paths depend on the environment to be resolved. We have elected to do everything that depends on the environment, e.g. resolving SPATH expressions using NIX_PATH, at runtime, so tvix evaluation would continue to behave correctly even if we separated the compilation and execution phases more, e.g. via serializing bytecode. Then the value of HOME, NIX_PATH etc. could reasonably change in the time until execution, yielding wrong results if the resolution results were cached in the bytecode. We also take the opportunity to fix the broken path concatenation previously found in the compiler, fixing b/205. Another thing we could consider is emitting a warning for home relative path literals, as they are by nature relatively fragile. One sideeffect of this change is that home path resolution errors become catchable which is not the case in C++ Nix. This will need to be fixed up in a subsequent change. Change-Id: I30bd69b575667c49170a9fdea23a020565d0f9ec Reviewed-on: https://cl.tvl.fyi/c/depot/+/7024 Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: Adam Joseph <adam@westernsemico.com> Tested-by: BuildkiteCI
2022-10-16 r/5146 refactor(tvix/eval): make OpFindFile use internal UnresolvedPathsterni1-1/+1
To assert that OpFindFile is only emitted for specially compiled SPATH expressions, as well as make sure it doesn't accidentally operate on “ordinary values”, introduce an UnresolvedPath internal value. If OpFindFile sees a non-UnresolvedPath value, it'll crash. Note that this change is not done purely for OpFindFile: We may want to compile SPATH expressions as function calls to __findFile (like C++ Nix does) in the future, so the UnresolvedPath value would definitely need to be an ordinary string again then. Rather, this change is done in preparation for resolving home dir relative paths at runtime (since they depend on the environment) for which we'll need a similar mechanism to OpFindFile. Change-Id: I6acf287f35197cd9e13377079f972b9d36e5b22e Reviewed-on: https://cl.tvl.fyi/c/depot/+/7023 Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: Adam Joseph <adam@westernsemico.com> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-13 r/5121 fix(tvix/eval): fix Compiler::new on wasmVincent Ambo1-1/+3
This path normalisation business causes runtime panics on WebAssembly because those operations are unsupported. Maybe this shouldn't be happening in the compiler anyways, not sure, but for now this commit adds a workaround based on the target to disable the normalisation if we're compiling for wasm. Change-Id: I908a84fbdffc3401f8d443e2c73ec673e9f397ff Reviewed-on: https://cl.tvl.fyi/c/depot/+/7004 Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-10-13 r/5120 fix(tvix/eval): src/compiler: ensure root_dir is absoluteAdam Joseph1-8/+16
Cppnix immediately absolutizes pathnames at parse time; if you write `./foo`, it is immediately converted to `$(pwd)/foo` and manipulated as an absolute path at all times. To avoid having to introduce filesystem access operations in the implementation of otherwise-pure builtins, let's guarantee that the `root_dir` of the VM is always an absolute path. Signed-off-by: Adam Joseph <adam@westernsemico.com> Change-Id: I7cbbae2cba4b2716ff3f5ff7c9ce0ad529358c8a Reviewed-on: https://cl.tvl.fyi/c/depot/+/6995 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-10-13 r/5119 refactor(tvix/eval): factor out all calls to canon_pathAdam Joseph1-2/+1
Right now we're pretending that the Rust library path_clean does the same thing that cppnix's canonPath() does. This is not true. It's close enough for the test suite, but may come back to bite us. Let's create our own canon_path() function and call that in all the places where we intend to match the behavior of cppnix's canonPath(). That way when we fix this we can fix it once, in one place. Signed-off-by: Adam Joseph <adam@westernsemico.com> Change-Id: Ia6f9577f62f49ef352ff9cfa5efdf37c32d31b11 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6993 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-10-10 r/5097 fix(tvix/eval): Actually trace spans for thunksGriffin Smith1-1/+1
Currently, the span on *all* thunk force errors is the span at which the thunk is forced, which for recursive thunk forcing ends up just being the same span over and over again. This changes the span on thunk force errors to be the span at which point the thunk is *created*, which is a bit more helpful (though the printing atm is a little... crowded). To make this work, we have to thread through the span at which a thunk is created into a field on the thunk itself. Change-Id: I81474810a763046e2eb3a8f07acf7d8ec708824a Reviewed-on: https://cl.tvl.fyi/c/depot/+/6932 Autosubmit: grfn <grfn@gws.fyi> Reviewed-by: Adam Joseph <adam@westernsemico.com> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-10-10 r/5090 fix(tvix/eval): Thunk `if` exprGriffin Smith1-1/+3
Since the body of an `if` expr can refer to deferred upvalues, it needs to be thunked so when we actually compile those deferred upvalues we have something for the finalize op to point at. Without this all sorts of weird things can happen due to the finalize op being run in the wrong lambda context, up to and including a panic. Change-Id: I040d5e1a7232fd841cfa4953539898fa49cbbb83 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6929 Reviewed-by: sterni <sternenseemann@systemli.org> Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-10-10 r/5087 feat(tvix/eval): Initial resolution of `<...>` pathsGriffin Smith1-6/+17
This commit implements (lazy) resolution of `<...>` paths via either the NIX_PATH environment variable, or the -I command-line flag - both handled via EvalOptions. As a result, EvalOptions can no longer derive Copy, meaning we have to clone it at each line of the repl - this is probably not a huge deal as repl performance is not exactly an inner loop and we're not cloning very much. Internally, this works by creating a thunk which pushes a constant containing the string inside the brackets to the stack, then a new opcode to resolve that path via the `NixPath`. To get that opcode to work, we now have to pass in the NixPath when constructing the VM. This (intentionally) leaves out proper implementation of path resolution via `findFile` (cppnix just calls whatever identifier called findFile is in scope!!!) as that's widely considered a bit of a misfeature, but if we do decide to implement that down the road it likely wouldn't be more than a few extra ops within the thunk introduced here. Change-Id: Ibc979b7e425b65cbe88599940520239a4a10cee2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6918 Autosubmit: grfn <grfn@gws.fyi> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-10-10 r/5082 refactor(tvix/eval): Compile OpAssert using conditional jumpsGriffin Smith1-4/+22
In order to behave nicely with tryEval, asserts need to leave the instruction pointer in a reasonable place even if they fail - whereas with the previous implementation catching a failed assert would still end up running the op for the *body* of the assert. With this change, we compile asserts much more like an `if` expression with conditional jumps rather than having an OpAssert op. Change-Id: I1b266c3be90185c84000da6b1995ac3e6fd5471b Reviewed-on: https://cl.tvl.fyi/c/depot/+/6925 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-10-08 r/5057 refactor(tvix/eval): move `spans` module to crate rootVincent Ambo1-2/+7
This is also useful for error-handling related logic, outside of just the compiler module. Change-Id: I5c386e2b4c31cda0a0209b31136ca07f00e39e45 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6869 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-10-07 r/5048 feat(tvix/eval): insert `import` into the builtins itselfVincent Ambo1-5/+9
Adding `import` to builtins causes causes a bootstrap cycle because the `import` builtin needs to be initialised with the set of globals before being inserted into the globals, which also must contain itself. To break out of the cycle this hack wraps the builtins passed to the compiler in an `Rc` (probably sensible anyways, as they will end up getting cloned a bunch), containing a RefCell which gives us mutable access to the builtins. This opens up a potentially dangerous footgun in which we could mutate the builtins at runtime leading to different compiler invocations seeing different builtins, so it'd be nice to have some kind of "finalised" status for them or some such, but I'm not sure how to represent that atm. Change-Id: I25f8d4d2a7e8472d401c8ba2f4bbf9d86ab2abcb Reviewed-on: https://cl.tvl.fyi/c/depot/+/6867 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-10-04 r/5034 refactor(tvix/eval): remove unnecessary clones in compilerVincent Ambo1-128/+124
There's basically nothing that needs *ownership* of an AST node (which is just a little box full of references to other things anyways), so we can thread this through as references all the way. Change-Id: I35a1348a50c0e8e07d51dfc18847829379166fbf Reviewed-on: https://cl.tvl.fyi/c/depot/+/6853 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-10-04 r/5033 refactor(tvix/eval): split observer traits in twoVincent Ambo1-4/+4
There are actually two different types of observers, the ones that observe the compiler (and emitted chunks from different kinds of expressions), and the ones that trace runtime execution. Use of the NoOpObserver is unchanged, it simply implements both traits. Change-Id: I4277b82674c259ec55238a0de3bb1cdf5e21a258 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6852 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-09-29 r/4994 refactor(tvix/eval): emit OpAttrs inside of compile_bindingsVincent Ambo1-1/+1
This needs to move here so that we can reuse compile_bindings for the nested attribute sets we're about to start constructing. Change-Id: Ie83f52f7e1d128886e96a1da47792211fa826f21 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6796 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-29 r/4991 chore(tvix/eval): fix all current clippy lintsVincent Ambo1-2/+2
Change-Id: I28d6af8cb408f8427a75d30b9120aaa809a1ea40 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6784 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-29 r/4989 refactor(tvix/eval): merge all bindings creation logicVincent Ambo1-1/+0
As of this commit, all three types of bindings scopes are compiled the same way (i.e. compilation of non-recursive attribute sets has been switched over to the new code paths). This sets us up for doing the final implementation of nested attribute sets. HOWEVER, this breaks the existing implementation of nested attributes in non-recursive attribute sets. That implementation is flawed and unworkable in practice, so we need to do this dance to be able to implement it correctly. Change-Id: Iba2545c0d1d6b51f5e1a31a5d005b8d01da546d3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6782 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-28 r/4972 refactor(tvix/eval): bye compiler::attrs, hello compiler::bindingsVincent Ambo1-438/+121
Changes the module structure of the compiler to have a module dedicated to the logic of setting up bindings. This logic is in the process of being merged between attribute sets and `let`-expressions, and the structure of the modules makes more sense when ecapsulating that specifically. (Other bits of code related to e.g. attribute sets are pretty straightforward and can just live in the main compiler module). Change-Id: I9469b73a7034e5b5f3bb211694d97260c4c9ef54 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6766 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-22 r/4962 fix(tvix/eval): manually count entries in recursive scopesVincent Ambo1-2/+10
The previous version had a bug where we assumed that the number of entries in an attribute set AST node would be equivalent to the number of entries in the runtime attribute set, but due to inherit nodes containing a variable number of entries, this did not work out. Fixes b/199 Change-Id: I6f7f7729f3512b297cf29a2e046302ca28477854 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6749 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-22 r/4958 fix(tvix/eval): support string identifiers in inheritsVincent Ambo1-61/+89
This updates rnix-parser to a version where inherits provide an iterator over `ast::Attr` instead of `ast::Ident`, which mirrors the behaviour of Nix (inherits can have (statically known) strings as their identifiers). This actually required some fairly significant code reshuffling in the compiler, as there was an implicit assumption in many places that we would have an `ast::Ident` node available when dealing with variable access (which is then explicitly only not true in this case). Change-Id: I12f1e786c0030c85107b1aa409bd49adb5465546 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6747 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-20 r/4939 refactor(tvix/eval): Define a Compiler::new functionGriffin Smith1-27/+39
Change-Id: I6b9283d16447c83dd3978371d9a6ac1beb985926 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6657 Autosubmit: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-20 r/4937 fix(tvix/eval): force condition of an assertsterni1-0/+1
Change-Id: I3ad2234e8a8e4280e498c6d7af8ea0733ed4c7ea Reviewed-on: https://cl.tvl.fyi/c/depot/+/6699 Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-09-18 r/4907 refactor(tvix/eval): clone the Arc<codemap::File> for the compilerVincent Ambo1-5/+6
This disconnects ownership of the `File` reference in a compiler from the calling scope, which is required for when we implement `import`. `import` will need to carry an `Rc<RefCell<CodeMap>>` (or maybe, in the future, Arc) to give us the ability to add new detected code files at runtime. Note that the choice of `Arc` over `Rc` here is not ours - it's the codemap crate's. Change-Id: I3aeca4ffc167acbd1701846a332d93550b56ba7d Reviewed-on: https://cl.tvl.fyi/c/depot/+/6630 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-09-18 r/4898 fix(tvix/eval): Emit errors for invalid integersGriffin Smith1-1/+4
Invalid integers (eg integers that're too long) end up as error returns on the `.value()` returned from the literal in the AST - previously we'd unwrap this error, causing it to panic the compiler, but now we've got a nice error variant for it (which just unwraps the underlying std::num::ParseIntError). Change-Id: I50c3c5ba89407d86659e20d8991b9658415f39a0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6635 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-17 r/4889 refactor(tvix/eval): clean up implementation of `compile_literal`Vincent Ambo1-10/+7
Suggested by sterni in cl/6231 Change-Id: I58bbc8a922d360ea79a4dacb76cf8aa1fad93757 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6622 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-17 r/4884 refactor(tvix/eval): use new ToSpan trait wherever possibleVincent Ambo1-24/+16
... it would be nice if we could thread it through to the `Scope` stuff (declaring phantoms & locals). Change-Id: Id3b84e79032b8fbb12138b719e657565355fbc79 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6616 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-17 r/4883 feat(tvix/eval): introduce `ToSpan` trait in compiler moduleVincent Ambo1-18/+12
This trait can be used to convert most structures from rnix-parser into a codemap::Span. It uses a macro to implement the trait for the various expression types in the rnix AST, as Rust's silly semantic versioning restriction stops us from doing a blanket implementation. This will be used in the next commit to clean up the span handling in the compiler a bit. Change-Id: I0a437034e5fa203b5a49c6f25c45932a9f3b2bca Reviewed-on: https://cl.tvl.fyi/c/depot/+/6615 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-16 r/4877 feat(tvix/eval): implement legacy let syntaxVincent Ambo1-4/+12
... and emit a warning if anyone decides to use. Change-Id: Iaa6fe9fa932340e6d0fa9f357155e78823702576 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6611 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-16 r/4876 feat(tvix/eval): implement recursive attribute setsVincent Ambo1-22/+67
Yep. This is kind of ugly right now. The idea is that the recursive_scope compilation function is used for recursive sets as well by emitting the keys. The rest of the logic is pretty much identical. There is quite a lot of code here that can be cleaned up (duplication between attrs and let, duplication inside of the recursive scope compilation function etc.), but I'd like to get it working first and then make it nice. Note that nested keys are *not* supported yet. Change-Id: I45c7cdd5f0e1d35fd94797093904740af3a97134 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6610 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-16 r/4875 refactor(tvix/eval): introduce a type for tracking bindingsVincent Ambo1-24/+64
This type is used in the list temporarily populated by the *second* pass over all identifiers in a recursive scope. This first pass only serves to make all bindings known to the compiler, without populating their values yet. Having a type here is going to be useful once we implement `rec`, which needs to thread through slightly more information. Change-Id: Ie33e0f096c5fcb6c864c991255466748b6f0d1eb Reviewed-on: https://cl.tvl.fyi/c/depot/+/6609 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-16 r/4874 refactor(tvix/eval): extract recursive scope logic into a helperVincent Ambo1-7/+14
This needs to be reused between let & `rec` attrs. Change-Id: I4a3bb90af4be32771b0f9e405c19370e105c0fef Reviewed-on: https://cl.tvl.fyi/c/depot/+/6608 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-16 r/4873 refactor(tvix/eval): move compile_inherit_attrs to compiler::attrsVincent Ambo1-62/+0
Plain move, no other changes. Change-Id: Ic4f89709f5c2cbc03182a848af080c820e39a0fd Reviewed-on: https://cl.tvl.fyi/c/depot/+/6607 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-16 r/4872 refactor(tvix/eval): explicitly construct attrs in phasesVincent Ambo1-0/+1
This makes the phases of attribute set construction that Nix has very explicit (inherits, static keys, dynamic keys). This change focuses on the split between dynamic/static keys by collecting all dynamic ones while compiling the static ones, and then phasing them in afterwards. It's possible we also need to do some additional splitting inside of the inherits. Change-Id: Icae782e2a5c106e3ce0831dda47ed81c923c0a42 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6530 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-12-21T03·45+0000