about summary refs log tree commit diff
path: root/tvix/eval/src/compiler/mod.rs
AgeCommit message (Collapse)AuthorFilesLines
2022-09-08 r/4756 fix(tvix/eval): compare *stack* slots when deciding whether to deferVincent Ambo1-1/+2
When deciding whether an upvalue needs to have a deferred resolution step, the *stack* indexes should be compared - not the locals indexes. The results are almost always the same, but there are tricky situations where this can cause errors. It's difficult to reproduce these errors in isolation, as they depend on other scope behaviour, so this is one in a series of commits to address the combination of issues which will gain some tests at the end. Change-Id: Iaa400b8d9500af58f493ab10e4f95022f3b5dd21 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6423 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-08 r/4754 refactor(tvix/eval): refactor locals to use an enum for phantomsVincent Ambo1-8/+12
Instead of using sentinel values and an additional bool, this tracks the identifier of a local as an enum that is either a statically known name, or a phantom. To make this work correctly some more locals related logic has been encapsulated in the `scope` module, which is a good thing (that's the goal). Phantom values are now not initialised by default, but the only current call site of phantoms (`with` expression compilation) performs the initialisation right away. This commit changes no actual functionality right now, but paves the way for fixing an issue related to `let` bodies. Change-Id: I679f93a59a4daeacfe40f4012263cfb7bc05034e Reviewed-on: https://cl.tvl.fyi/c/depot/+/6421 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-08 r/4753 refactor(tvix/eval): always pass slot to compiler methodsVincent Ambo1-33/+28
The slot is now always known (at the root of the file it is simply stack slot 0 once the scope drops back down to 0), so it does not need to be wrapped in an `Option` and accessed in cumbersome ways anymore. Change-Id: I46bf67a4cf5cb96e4874dffd0e3fb07c551d44f0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6420 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-08 r/4747 feat(tvix/eval): thread codemap through to disassemblerVincent Ambo1-2/+25
If the disassembler feature is enabled, make sure that an Rc of the codemap is available through the chunk. Change-Id: I700f27ab665a704f73457b19bd2d7efc93828a16 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6414 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-08 r/4743 fix(tvix/eval): inherit scope poisoning data in nested contextsVincent Ambo1-4/+22
Scope poisoning must be inherited across lambda context boundaries, e.g. if an outer scope has a poisoned `null`, any lambdas defined on the same level must reference that poisoned identifier correctly. Change-Id: I1aac64e1c048a6f3bacadb6d78ed295fa439e8b4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6410 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-08 r/4741 feat(tvix/eval): ensure all errors always carry a spanVincent Ambo1-8/+9
Previously error spans were optional because the information about code spans was not available at runtime. Now that this information has been added, the error type will always carry a span. This change is very invasive all throughout the codebase. This is due to the fact that many functions that are called *by* the VM expected to return `EvalResult`, but this no longer works as the span information is not available to those functions - only to the VM itself. To work around this the majority of these functions have been changed to return `Result<T, ErrorKind>` instead and an accompanying macro in the VM constructs the "real" error. Note that this implementatino currently has a bug where errors occuring within thunks will yield the location at which the thunk was forced, not the location at which the error occured within the code. This will be fixed soon, but the commit is large enough as is. Change-Id: Ib1ecb81a4d09d464a95ea7ea9e589f3bd08d5202 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6408 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-07 r/4737 refactor(tvix/eval): store spans instead of nodes in Warning/ErrorVincent Ambo1-75/+76
Another step towards being able to report accurate errors. The codemap spans contain strictly more accessible information, as they now retain information about which input file something came from. This required some shuffling around in the compiler to thread all the right information to the right places. Change-Id: I18ccfb20f07b0c33e1c4f51ca00cd09f7b2d19c6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6404 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-07 r/4736 feat(tvix/eval): track source spans for builtin accessVincent Ambo1-20/+9
As of this commit, the source spans of all emitted bytecode are fully tracked. Change-Id: I4c83deee0fc3f5e6fd6acad5a39047aec693b388 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6403 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-07 r/4735 feat(tvix/eval): track source spans for `OpForce` instructionsVincent Ambo1-23/+23
These source spans will always point to the *value* that is being forced, not the instruction that caused the force to be emitted. This makes sense so that errors during forcing point at the value and not the surrounding expression. Change-Id: I4694414a3281a0de878f71634105b92148ec61f6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6402 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-07 r/4734 feat(tvix/eval): track source spans for scopesVincent Ambo1-6/+6
Change-Id: Iaedb0742940e4c2b9f4210579ed28e6deb32d5c8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6401 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-07 r/4733 feat(tvix/eval): track source spans for upvaluesVincent Ambo1-17/+42
With this change, the upvalue data instructions used by finalisers for thunks and closures track the source span of the first identifier that created the upvalue (if the same value is closed over multiple times the upvalue will be reused, hence only the first one). To do this the upvalue struct used by the compiler's scope now carries an identifier node, which had to be threaded through quite a few places. Change-Id: I15a5fcb4c8abbd48544a2325f297a5ad14ec06ae Reviewed-on: https://cl.tvl.fyi/c/depot/+/6400 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-07 r/4732 refactor(tvix/eval): split out Upvalue struct & UpvalueKind enumVincent Ambo1-13/+13
This separation makes it possible to annotate the upvalue itself with the span that created it, which (due to upvalue reuse) is only the first one for an instance of the given UpvalueKind. Change-Id: I9a991da6a3e8d71a92f981314bed900bcf434d44 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6399 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-07 r/4731 feat(tvix/eval): track source spans for thunksVincent Ambo1-8/+11
Change-Id: I73f13c914d84e052a43b4eddb09db1acd8a7076a Reviewed-on: https://cl.tvl.fyi/c/depot/+/6398 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-07 r/4730 feat(tvix/eval): track source spans for function callsVincent Ambo1-1/+1
Change-Id: If1bb463026414597acd561dc2032549fdb70f986 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6397 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-07 r/4729 feat(tvix/eval): track source spans for lambdasVincent Ambo1-2/+5
Change-Id: I2f420fc915b6bfc5b197e9d0c128b539c4bc7467 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6396 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-07 r/4728 feat(tvix/eval): track source spans for `with` expressionsVincent Ambo1-2/+2
Change-Id: I1d58ce548b5b47e967928e85bb64acf5ed69ecc8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6395 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-07 r/4727 feat(tvix/eval): track source spans for identifier accessVincent Ambo1-10/+8
Change-Id: I8e6ec0a84430d6e417fc7fd3e722a588913e4d18 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6394 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-07 r/4726 feat(tvix/eval): track source spans for `let` bindingsVincent Ambo1-2/+2
Change-Id: I9457917277a7fdd8bdbe227a567b28969312d06e Reviewed-on: https://cl.tvl.fyi/c/depot/+/6393 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-07 r/4725 feat(tvix/eval): track source spans for `if` expressionsVincent Ambo1-4/+7
These are again a bit tricky in terms of emitted errors. The main error is that the condition is not a boolean, which means that the jump inspecting the condition must derive from the condition itself to return an error at the correct position. For other parts of the expression, it is simply the node itself. Change-Id: I72411630e5d57dfc199f4c3c48afe443fe966322 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6392 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 r/4724 feat(tvix/eval): track source spans for `assert`Vincent Ambo1-1/+1
Change-Id: I5415f63ddde388847a261da4ce9a8d8235657535 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6391 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 r/4723 feat(tvix/eval): track source spans for `or` operatorVincent Ambo1-7/+4
This one is tricky, specifically the span used for the final jump. I decided that it makes sense to use the attrpath node, as the final jump is the one that jumps *over* the default value, so the effect of this is more closely related to the selector than the default. It might be more correct to pass through the `or` token itself and point to this for the jumps, but it depends a bit on what shape of errors we could end up producing from this. Change-Id: I29fbc97ba6b9e14e1a0e5f3a7759ddc299dd9c0c Reviewed-on: https://cl.tvl.fyi/c/depot/+/6390 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 r/4722 feat(tvix/eval): track source spans for attribute selectsVincent Ambo1-1/+1
Change-Id: Ifa8b0e7905f9d2746f83d6503ef0e8d42ce20f9c Reviewed-on: https://cl.tvl.fyi/c/depot/+/6389 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 r/4721 feat(tvix/eval): track source spans for attrsetsVincent Ambo1-3/+6
Change-Id: I0fcb07146b5a38c67bc46ed3f67533b056858390 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6388 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 r/4720 feat(tvix/eval): track source spans for listsVincent Ambo1-1/+1
Change-Id: Icfd057ee29180cac17264bb3299c67eae0051aee Reviewed-on: https://cl.tvl.fyi/c/depot/+/6387 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 r/4719 feat(tvix/eval): track source spans for literal identifiersVincent Ambo1-1/+4
Change-Id: I7ca08c6c0124f653e55fcc86413a847f0a4c50c5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6386 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 r/4718 feat(tvix/eval): track source spans for `?` operatorVincent Ambo1-2/+2
Change-Id: Idb842fb20fab14150455a81ea6c947e75b314d8a Reviewed-on: https://cl.tvl.fyi/c/depot/+/6385 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 r/4717 feat(tvix/eval): track source spans for binary operatorsVincent Ambo1-23/+23
Change-Id: I43e433ff3094232b244d38335d31e56a79ca70c1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6384 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 r/4716 feat(tvix/eval): track source spans for unary operatorsVincent Ambo1-1/+1
Change-Id: I79034a4aa04aea66ec598e33e6eab35e1e19c0d0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6383 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 r/4715 feat(tvix/eval): track source spans for stringsVincent Ambo1-2/+2
Change-Id: I29e6c7f9e25d1b2e6bafa602c463eb9ccee2131b Reviewed-on: https://cl.tvl.fyi/c/depot/+/6382 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 r/4714 feat(tvix/eval): track source spans for pathsVincent Ambo1-1/+1
Change-Id: I42fbd0bb6c2a8feb520e262a25f59ff27dcd035c Reviewed-on: https://cl.tvl.fyi/c/depot/+/6381 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 r/4713 feat(tvix/eval): track source spans for literalsVincent Ambo1-3/+3
Change-Id: Icfe77f85c4f65b6bf28b8752c2795419e8e396ce Reviewed-on: https://cl.tvl.fyi/c/depot/+/6380 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 r/4712 feat(tvix/eval): add methods for emitting code with tracked spansVincent Ambo1-79/+103
These are not actually used yet; this is in preparation for a multi-commit chain for emitting all the right spans in the right locations. Change-Id: Ie99d6add2696c1cc0acb9ab928917a10237159de Reviewed-on: https://cl.tvl.fyi/c/depot/+/6379 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 r/4711 chore(tvix/eval): thread a codemap::File reference to the compilerVincent Ambo1-4/+11
This instantiates a codemap outside of the compiler and passes a reference to the file currently under compilation to it. Note that the "file" might just be a REPL line. Change-Id: I131ae1ddb6d718e1374750da9ba0b99608c6058d Reviewed-on: https://cl.tvl.fyi/c/depot/+/6378 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 r/4710 refactor(tvix/eval): add and use Compiler::push_op methodVincent Ambo1-60/+65
This is currently just a wrapper around Chunk::push_op, but will gain the span resolution logic in a moment. Change-Id: I862bf9ecff0932f8da6708401ea044b9442c5d5b Reviewed-on: https://cl.tvl.fyi/c/depot/+/6377 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 r/4699 feat(tvix/eval): construct attribute sets lazilyVincent Ambo1-2/+10
This thunks the construction of attribute sets. Because Tvix does not currently have a "strict output" mode, a test had to be disabled that now displays a thunk representation. The test will be re-enabled once that is available. Change-Id: I360332be64cd5c154f9caea21828f6f1b37a265c Reviewed-on: https://cl.tvl.fyi/c/depot/+/6363 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 r/4696 fix(tvix/eval): correctly resolve identifiers in inheritVincent Ambo1-21/+4
This should not have grown a second implementation of the identifier resolution logic, but it somehow did. This implementation ended up being incorrect because it did not account for upvalues inside of thunks. Change-Id: Ieb1364d8fe43c96aaf4b125fd4b8a522aedff167 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6360 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 r/4694 feat(tvix/eval): insert strictness points for attribute set keysVincent Ambo1-2/+10
All attribute set *key* related operations strictly evaluate all key fragments, including during construction of an attribute set. Change-Id: I3519e5e9b0886c2cdc8615ea7dcb5f7be0c59b3f Reviewed-on: https://cl.tvl.fyi/c/depot/+/6358 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 r/4693 feat(tvix/eval): insert strictness points for unary/binary operatorsVincent Ambo1-1/+17
The arguments of all unary/binary operators that are built in to Nix are forced when encountered. This emits the necessary OpForce operations. Change-Id: I691fcdbebfe7586cfe217c68d44b10b1192f82d1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6357 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 r/4692 chore(tvix/eval): thread `slot` value through all compiler methodsVincent Ambo1-54/+58
With this change any compilation of an expression is aware of its own stack slot if it is leaving identifiers on the stack. Change-Id: I0c9f148ae06b078a46b25180c4961686d5f2e166 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6356 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 r/4691 chore(tvix/eval): thread `slot` value through Compiler:compile_attrVincent Ambo1-17/+23
This threads the current local slot through the `compile_attr` function and all of its callers. At the moment this does not improve any user-facing behaviour, just internally changes the way in which some correct expressions would fail to run. Eventually this slot will need to reach everywhere ... Change-Id: Iba73123dd1ced421093d8fc18ebeeffc16efacf8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6355 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 r/4689 feat(tvix/eval): always emit OpForce as the last instructionVincent Ambo1-0/+6
Change-Id: Id70c987f654dc5d9b47db74e395281309762b468 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6353 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 r/4676 refactor(tvix/eval): simplify thunk representationsVincent Ambo1-1/+1
For now, do not distinguish between closing and non-closing thunks, it will make the initial implementation easier. See Knuth etc. Change-Id: I0bd51e0f89f2c77e90bac63b507e5027b649e3d8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6346 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 r/4675 feat(tvix/eval): emit thunks for recursive local scope resolutionVincent Ambo1-6/+13
When resolving a value on the same level that is known but not yet defined, emit a thunk. Consider for example: let # v--- requires a thunk a = 1 * b; b = 10; in a Change-Id: I922cb50973ebe05e335a7bc7cb851960cf34733b Reviewed-on: https://cl.tvl.fyi/c/depot/+/6345 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 r/4674 feat(tvix/eval): Add Compiler::thunk method for emitting thunksVincent Ambo1-1/+36
The logic in this method is *very* similar to `compile_lambda`. It is intended to be called around any expression that should be thunked (such as function applications, attribute set values, etc.). Change-Id: Idfbb2daa9f4b735095378fb9c39a2fd07c8cff91 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6344 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 r/4672 refactor(tvix/eval): extract compiler's upvalue logic into helperVincent Ambo1-14/+20
This exact same logic is reused for thunk creation. Change-Id: I731db9cc659a1f2ca87db55d58d6ff632f417812 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6342 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 r/4670 fix(tvix/eval): distinguish statically between StackIdx and LocalIdxVincent Ambo1-73/+36
Previously the functions in the scope module returned usize values, which - sometimes from the same function - were either indexes into the runtime stack *or* indexes into the compiler's local stack. This is extremely confusing because it requires the caller to be aware of the difference, and it actually caused subtle bugs. To avoid this, there is now a new LocalIdx wrapper type which is used by the scope module to return indexes into the compiler's stack, as well as helpers for accounting for the differences between these indexes and the runtime indexes. Change-Id: I58f0b50ad94b28a304e3372fd9731b6590b3fdb8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6340 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 r/4669 feat(tvix/eval): emit OpFinalise when local scopes are completeVincent Ambo1-0/+8
With this change, the runtime can correctly capture deferred upvalues. Change-Id: I1e43b7b1ac2553b1812424adfc8bd08ef77bf1ea Reviewed-on: https://cl.tvl.fyi/c/depot/+/6339 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 r/4668 feat(tvix/eval): track whether locals needs to be finalisedVincent Ambo1-0/+8
When encountering a deferred local upvalue, the compiler will now mark the corresponding local as needing a finaliser which makes it possible to emit the OpFinalise instruction for this stack slot a little bit down the line. Change-Id: I3962066f10fc6c6e1472722b8bdb415a811e0740 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6338 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-06 r/4665 feat(tvix/eval): detect deferred upvalue capturingVincent Ambo1-3/+16
Uses the threaded through slot offset to determine whether initialisation of a captured local upvalue must be defered to a later point where all values of a scope are available. This adds a new data representation to the opcode for this situation, but the equivalent runtime handling is not yet implemented. This is in part because there is more compiler machinery needed to find the resolution point. Change-Id: Ifd0c393f76abfe6e2d91483faf0f58947ab1dedc Reviewed-on: https://cl.tvl.fyi/c/depot/+/6329 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-06 r/4664 chore(tvix/eval): thread current slot index through compilerVincent Ambo1-34/+34
While compiling local bindings, we now know the stack slot of the value currently being compiled. This will let us determine whether an upvalue can be captured directly or whether it needs to wait for a synchronisation point at which the upvalue can be instantiated. This machinery lets us avoid unnecessary work at runtime when instantiating closures that actually do not need complicated recursive resolution. This change itself introduces no new functionality, but since the threading is noisy it is split out as a separate change. Change-Id: I847c677ee8f6725fda1d2efd689b6a58bdccb779 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6328 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-11-12T03·56+0000