about summary refs log tree commit diff
path: root/tvix/eval/src/builtins/mod.rs (unfollow)
AgeCommit message (Collapse)AuthorFilesLines
2022-10-12 r/5114 feat(tvix/eval): builtins: implement parseDrvNameAdam Joseph1-0/+25
This commit passes nix_eval_okay_src_tests_nix_tests_eval_okay_versions_nix. See also: https://github.com/NixOS/nix/pull/7149 Signed-off-by: Adam Joseph <adam@westernsemico.com> Change-Id: I24605c2a0cd0da434f37f6c518f20693bfa1b799 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6913 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-12 r/5113 refactor(tvix/eval) s/NixPath/NixSearchPath/Adam Joseph5-46/+49
Since NixString is the Rust type for nix strings, people might mistake NixPath for the Rust type of nix paths, which it is not. Let's call it NixSearchPath instead. Signed-off-by: Adam Joseph <adam@westernsemico.com> Change-Id: Ib2ea155c4b27cb90d6180a04ea7b951d86607373 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6927 Reviewed-by: kanepyork <rikingcoding@gmail.com> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-12 r/5112 chore(tazjin/keys): update SSH key setVincent Ambo1-2/+1
frog is defunct for now, unihertz phone is replaced with palm (on which I don't run SSH), and for b/210 I'd like a non-sk key on zamalek Change-Id: Ie99e650dcf9deef91a0f63d4171a332a8e596de0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6950 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-10-12 r/5111 fix(tvix/eval): remove overlapping instancesAdam Joseph1-12/+0
I broke the build. I didn't understand that before hitting "submit" you need to re-test your changes on latest HEAD (and that CI doesn't do that for you); I failed to re-test cl/6912 following the merge of cl/6907. This commit fixes the build by removing the overlapping instances. Change-Id: I2a720d2c60cc7103b350f78102a8998f93bac828 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6965 Tested-by: BuildkiteCI Autosubmit: Adam Joseph <adam@westernsemico.com> Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-12 r/5110 feat(tvix/eval): From<Into<NixString>> for ValueAdam Joseph1-0/+9
Signed-off-by: Adam Joseph <adam@westernsemico.com> Change-Id: I27615baf4104ab96600708b12a780599a3714bae Reviewed-on: https://cl.tvl.fyi/c/depot/+/6912 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-12 r/5109 feat(tvix/eval): From<Utf8Error> for ErrorKindAdam Joseph1-0/+7
Change-Id: I074d9e862fbdd4e78e443cbaf0e77c7ffe825a10 Signed-off-by: Adam Joseph <adam@westernsemico.com> Reviewed-on: https://cl.tvl.fyi/c/depot/+/6911 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-11 r/5108 fix(nix/dependency-analyzer): don't read from non-drv store pathssterni1-1/+7
Emitting dependencies on non-drv store paths from drv directDrvDeps is fine and actually correct, even though the Nix 2.3 version can't do it at the moment (but this would change when the placeholder implementation is replaced using a drv parser). However, we can't necessarily determine the dependencies of non-drv store paths because such store paths may be binary files that can't be read in by readFile due to NUL bytes. Change-Id: Ifbd101adaee4f32f10c010fa79e19b9b1127fc6a Reviewed-on: https://cl.tvl.fyi/c/depot/+/6945 Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-10-11 r/5107 feat(sterni/nix/int): add sum functionsterni1-0/+3
Change-Id: I7c3109b27ba91c848fd8b64d979d5e3c5c4583aa Reviewed-on: https://cl.tvl.fyi/c/depot/+/6944 Reviewed-by: sterni <sternenseemann@systemli.org> Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-10-11 r/5106 fix(nix/buildGo): fix for Nix >= 2.6 readFile changessterni1-1/+4
The string context retained by readFile would leak into attribute keys in fromJSON which may not have string context in any Nix version. We don't need Nix >= 2.6 support, but buildGo may have external users and this change is simple enough. Change-Id: I593f1ef513502691119428d26d508a5f4d378543 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6946 Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-10-11 r/5105 fix(tvix/eval/builtins): force acc not list element in foldl'sterni4-1/+12
When investigating discrepancies between foldl' in tvix and C++ Nix, I discovered that C++ Nix's foldl' doesn't seem to be strict at all. Since this seemed wrong, I looked into Haskell's foldl' implementation which doesn't force the list elements (`val` in our code), but the accumulation value (`res` in our code). You can look at the code here: https://hackage.haskell.org/package/base-4.17.0.0/docs/src/GHC.List.html#foldl%27 This actually makes a lot of sense: If `res` is not forced after each application of `op`, we'll end up thunks nested as deeply as the list is long, potentially taking up a lot of space. This can be limited by forcing the `res` thunk before applying `op` again (and creating a new thunk). I've also PR-ed an equivalent change for C++ Nix at https://github.com/NixOS/nix/pull/7158. Since this is not merged nor backported to our Nix 2.3 fork, I've not copied the eval fail test yet, since it wouldn't when checking our tests against C++ Nix in depot. Change-Id: I34edf6fc3031fc1485c3e714f2280b4fba8f004b Reviewed-on: https://cl.tvl.fyi/c/depot/+/6947 Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-10-11 r/5104 fix(nix/buildkite): split extra steps based on all known phasesEvgeny Zemtsov1-1/+1
Fix a bug introduced by 2ca153141 (I merged the wrong patchset). Issue happens when pipeline is split by phases into independent evals (e.g. build/release/deploy). Splitting extra steps requires knowledge of all known phases, otherwise pipeline evaluation fails due to extra steps from inactive phases. Change-Id: Iab0f2dc3eadda281e483055e26f00a95442e15b0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6942 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-11 r/5103 feat(tvix/eval): Allow directly evaluating an expr via mainGriffin Smith1-0/+7
This *maybe* should do something to check that we don't pass both a file and an expr, but for now this is useful enough to cut corners (plus we're probably due for a CLI revamp eventually anyway). Change-Id: Id44357074150b336b6215ba596cc52d01d037dbd Reviewed-on: https://cl.tvl.fyi/c/depot/+/6941 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-10-11 r/5102 feat(tvix/eval): Implement builtins.removeAttrsGriffin Smith4-1/+31
Change-Id: I28910991a0108436a42ac7bf3458f9180a44154e Reviewed-on: https://cl.tvl.fyi/c/depot/+/6928 Reviewed-by: Adam Joseph <adam@westernsemico.com> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-10-11 r/5101 feat(tvix/eval): Implement builtins.traceGriffin Smith1-0/+12
This is currently implemented with a simple println inline, but in the future we could hook into this via something pluggable on the VM. Change-Id: Idd9cc3b34aa13d6ebc64c02aade81ecdf439656a Reviewed-on: https://cl.tvl.fyi/c/depot/+/6938 Autosubmit: grfn <grfn@gws.fyi> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-10-11 r/5100 fix(tvix/eval): Pop frames even if running op failsGriffin Smith3-269/+285
Previously, the VM assumed that if an error was returned from `run()`, the evaluation was "finished" and the state of the VM didn't matter. This used to be a reasonable assumption, but now that we've got `tryEval` around we need to actually make sure that we clean up after ourselves if we're about to return an error. Specifically, if the *last* instruction in an evaluation frame returns an error, we previously wouldn't pop that evaluation frame, which could cause all sorts of bizarre errors based on what happened to be in the stack at the time. This commit splits out a `run_op` method from `VM::run`, and uses that to check the evaluation frame return condition even if the op we're running is about to return an error, and pop the evaluation frame if we're at the last instruction. Change-Id: Ib40649d8915ee1571153cb71e3d76492542fc3d7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6940 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-11 r/5099 feat(tvix/eval): observe stack after exiting call frames/builtinsVincent Ambo2-8/+22
Change-Id: I1937d37551503a0b6bb0ac899d067302e4791e5f Reviewed-on: https://cl.tvl.fyi/c/depot/+/6939 Reviewed-by: grfn <grfn@gws.fyi> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-10-11 r/5098 fix(tvix/eval): implement functor calling for non-tail-callsVincent Ambo1-0/+15
Change-Id: I1113921c46010021b61c412d74d60193c156e0f1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6937 Reviewed-by: grfn <grfn@gws.fyi> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-10-10 r/5097 fix(tvix/eval): Actually trace spans for thunksGriffin Smith4-22/+34
Currently, the span on *all* thunk force errors is the span at which the thunk is forced, which for recursive thunk forcing ends up just being the same span over and over again. This changes the span on thunk force errors to be the span at which point the thunk is *created*, which is a bit more helpful (though the printing atm is a little... crowded). To make this work, we have to thread through the span at which a thunk is created into a field on the thunk itself. Change-Id: I81474810a763046e2eb3a8f07acf7d8ec708824a Reviewed-on: https://cl.tvl.fyi/c/depot/+/6932 Autosubmit: grfn <grfn@gws.fyi> Reviewed-by: Adam Joseph <adam@westernsemico.com> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-10-10 r/5096 feat(tvix/eval): Implement builtins.getEnvGriffin Smith1-1/+7
Change-Id: I09f512a60989a37184e73e521d4a3aa23f33a1a8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6922 Autosubmit: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI Reviewed-by: Adam Joseph <adam@westernsemico.com> Reviewed-by: kanepyork <rikingcoding@gmail.com>
2022-10-10 r/5095 feat(tvix/eval): Implement builtins.readFileGriffin Smith4-1/+16
Change-Id: If3fd0b087009a2bfbad8bb7aca0aa20de906eb12 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6921 Tested-by: BuildkiteCI Reviewed-by: Adam Joseph <adam@westernsemico.com> Reviewed-by: kanepyork <rikingcoding@gmail.com> Autosubmit: grfn <grfn@gws.fyi> Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-10 r/5094 refactor(tvix/eval): after calling, the caller has to popVincent Ambo3-21/+29
Previously the various call functions either returned `EvalResult<()>` or `EvalResult<Value>`, which was confusing. Now only vm::call_with returns a Value directly, and other parts of the API just leave the stack top in the post-call state. This makes it easier to reason about what's going on in non-tail-call cases (which are making a comeback). Change-Id: I264ffc683a11aca72dd06e2220a5ff6e7c5fc2b0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6936 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-10-10 r/5093 feat(wpcarro/scratch): Rewrite Python compiler in OCamlWilliam Carroll4-0/+154
Just to see how productive I could be in OCaml with little familiarity. Overall I really like it. Change-Id: I8affc65a5ee86a29d4f8c01426529ae9948660f9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6934 Reviewed-by: wpcarro <wpcarro@gmail.com> Autosubmit: wpcarro <wpcarro@gmail.com> Tested-by: BuildkiteCI
2022-10-10 r/5092 fix(tvix/eval): format nested compiler errors in fancy outputVincent Ambo1-10/+22
Instead of just printing the number of errors (useless!) actually emit separate diagnostics for each nested error. Change-Id: I97b53c3276c906af5def89077b5b6ba6ec108b37 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6933 Reviewed-by: grfn <grfn@gws.fyi> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-10-10 r/5091 fix(tvix/eval): path resolution errors are catchableGriffin Smith3-2/+3
Despite this not being documented, `tryEval` is empirically able to catch errors caused by a <...> path not resolving (and nixpkgs depends on this). Change-Id: Ia3b78a2d9d2d0c603aba829518b351102dc55396 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6926 Reviewed-by: sterni <sternenseemann@systemli.org> Autosubmit: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-10-10 r/5090 fix(tvix/eval): Thunk `if` exprGriffin Smith3-1/+10
Since the body of an `if` expr can refer to deferred upvalues, it needs to be thunked so when we actually compile those deferred upvalues we have something for the finalize op to point at. Without this all sorts of weird things can happen due to the finalize op being run in the wrong lambda context, up to and including a panic. Change-Id: I040d5e1a7232fd841cfa4953539898fa49cbbb83 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6929 Reviewed-by: sterni <sternenseemann@systemli.org> Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-10-10 r/5089 feat(tvix/eval): Implement builtins.tryEvalGriffin Smith4-0/+32
With asserts compiled using conditional jumps, this ends up being quite straightforward - the only real tricky bit is that we have to know whether an error can or can't be handled. Change-Id: I75617da73b7a9c5cdd888c0e26ae81d2c5c0d714 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6924 Reviewed-by: sterni <sternenseemann@systemli.org> Autosubmit: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-10-10 r/5088 feat(tvix/eval): Implement builtins.pathExistsGriffin Smith3-6/+10
Change-Id: Ife8a690e9036868964771893ab29a9ae3a2d2365 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6919 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-10-10 r/5087 feat(tvix/eval): Initial resolution of `<...>` pathsGriffin Smith9-21/+77
This commit implements (lazy) resolution of `<...>` paths via either the NIX_PATH environment variable, or the -I command-line flag - both handled via EvalOptions. As a result, EvalOptions can no longer derive Copy, meaning we have to clone it at each line of the repl - this is probably not a huge deal as repl performance is not exactly an inner loop and we're not cloning very much. Internally, this works by creating a thunk which pushes a constant containing the string inside the brackets to the stack, then a new opcode to resolve that path via the `NixPath`. To get that opcode to work, we now have to pass in the NixPath when constructing the VM. This (intentionally) leaves out proper implementation of path resolution via `findFile` (cppnix just calls whatever identifier called findFile is in scope!!!) as that's widely considered a bit of a misfeature, but if we do decide to implement that down the road it likely wouldn't be more than a few extra ops within the thunk introduced here. Change-Id: Ibc979b7e425b65cbe88599940520239a4a10cee2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6918 Autosubmit: grfn <grfn@gws.fyi> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-10-10 r/5086 feat(tvix/eval): Add a struct implementing NIX_PATHGriffin Smith2-0/+208
Add a simple struct implementing both the string parsing and path resolution rules of Nix's `NIX_PATH` environment variable, for use in resolving `<...>`-style paths Change-Id: Ife75f39aa5c12928278d81fe428fbadc98bac5cc Reviewed-on: https://cl.tvl.fyi/c/depot/+/6917 Autosubmit: grfn <grfn@gws.fyi> Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: Adam Joseph <adam@westernsemico.com> Tested-by: BuildkiteCI
2022-10-10 r/5085 feat(tvix/eval): Allow adding strings to pathsGriffin Smith5-6/+40
Implement adding paths and strings via OpAdd. Since the nix rules are quite obscure, I'm electing to test this one with an oracle test to avoid the danger of getting the actual asserted result wrong. Change-Id: Icdcca3690ca2e8459e386c1f29cc48eaaa39e9a3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6914 Autosubmit: grfn <grfn@gws.fyi> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-10-10 r/5084 feat(tvix/eval): Implement builtins.concatStringsSepGriffin Smith5-0/+46
Change-Id: I6e46bcdbf3b5258b60edb017709fee577eb8ec74 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6907 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-10-10 r/5083 feat(tvix/eval): Support builtins.readDirWilliam Carroll7