about summary refs log tree commit diff
path: root/tvix/eval
AgeCommit message (Collapse)AuthorFilesLines
2023-01-04 r/5582 fix(tvix/eval): ' is allowed in nonfirst position in Nix identifierssterni3-1/+32
With this is_valid_nix_identifier should line up with the upstream lexer definition: ID [a-zA-Z\_][a-zA-Z0-9\_\'\-]* While we're working on this, add a simple test checking the various formatting rules. Interestingly, it would not be suitable as an identity test, since you have to write { "assert" = null; } in order to avoid an evaluation error, but C++ Nix is happy to print this as { assert = null; } – maybe should be considered to be a bug. Change-Id: I0a4e1ccb5033a80f3767fb8d1c4bba08d303c5d8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7744 Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-04 r/5581 refactor(tvix/eval): streamline construction of globals/builtinsVincent Ambo6-218/+269
Previously the construction of globals (a compiler-only concept) and builtins (a (now) user-facing API) was intermingled between multiple different modules, and kind of difficult to understand. The complexity of this had grown in large part due to the implementation of `builtins.import`, which required the notorious "knot-tying" trick using Rc::new_cyclic (see cl/7097) for constructing the set of globals. As part of the new `Evaluation` API users should have the ability to bring their own builtins, and control explicitly whether or not impure builtins are available (regardless of whether they're compiled in or not). To streamline the construction and allow the new API features to work, this commit restructures things by making these changes: 1. The `tvix_eval::builtins` module is now only responsible for exporting sets of builtins. It no longer has any knowledge of whether or not certain sets (e.g. only pure, or pure+impure) are enabled, and it has no control over which builtins are globally available (this is now handled in the compiler). 2. The compiler module is now responsible for both constructing the final attribute set of builtins from the set of builtins supplied by a user, as well as for populating its globals (that is identifiers which are available at the top-level scope). 3. The `Evaluation` API now carries a `builtins` field which is populated with the pure builtins by default, and can be extended by users. 4. The `import` feature has been moved into the compiler, as a special case. In general, builtins no longer have the ability to reference the "fix point" of the globals set. This should not change any functionality, and in fact preserves minor differences between Tvix/Nix that we already had (such as `builtins.builtins` not existing). Change-Id: Icdf5dd50eb81eb9260d89269d6e08b1e67811a2c Reviewed-on: https://cl.tvl.fyi/c/depot/+/7738 Reviewed-by: sterni <sternenseemann@systemli.org> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-01-02 r/5565 chore(tvix/eval): implement std::error::Error for tvix_eval::ErrorVincent Ambo2-1/+20
This makes it easier to interface this error with other crates. Change-Id: I4947ea6097608f8c0427fb94a819ef748d94ea4b Reviewed-on: https://cl.tvl.fyi/c/depot/+/7711 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2022-12-29 r/5542 refactor(tvix/eval): remove extra Rc<..> around Value::AttrsVincent Ambo5-21/+24
The `im::OrdMap` is already small and cheap to copy while sharing memory, so this is not required anymore. Only the `KV` variant may have slightly larger content, but in practice this doesn't seem to make a difference when comparing the two variants and this one is less complicated. Change-Id: I64a563b209a2444125653777551373cb2989ca7d Reviewed-on: https://cl.tvl.fyi/c/depot/+/7677 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-12-29 r/5541 refactor(tvix/eval): persistent, memory-sharing OrdMap for NixAttrsVincent Ambo7-74/+73
This uses the `im::OrdMap` for `NixAttrs` to enable sharing of memory between different iterations of a map. This slightly speeds up eval, but not significantly. Future work might include benchmarking whether using a `HashMap` and only ordering in cases where order is actually required would help. This switches to a fork of `im` that fixes some bugs with its OrdMap implementation. Change-Id: I2f6a5ff471b6d508c1e8a98b13f889f49c0d9537 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7676 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-12-29 r/5540 refactor(tvix/eval): use im::Vector directly where possibleVincent Ambo3-28/+31
The conversion from im::Vector -> Vec is cheaper for NixList construction (of course), so where possible we should make use of that. This updates most builtins dealing with lists to use Vector directly, and marks the function constructing NixList from Vec as deprecated so that we get appropriate warnings in places where it's still in use. These places are currently inside of JSON serialisation logic which is in flux right now, so lets leave them as-is until it's stabilised. Change-Id: I037f12a2800f2576db4d9526bd935efd079163f0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7671 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-12-29 r/5534 refactor(tvix/eval): use im::Vector for NixList representationVincent Ambo4-44/+49
This is a persistent, structurally sharing data structure which is more efficient in some of our use-cases. I have verified the efficiency improvement using `hyperfine` repeatedly over expressions on nixpkgs. Lists are not the most performance-critical structure in Nix (that would be attribute sets), but we can already see a small (~5-10%) improvement. Note that there are a handful of cases where we still go via `Vec` that need to be fixed, most notable for `builtins.sort` which can not currently be implemented directly using `im::Vector` because of a restrictive type bound. Change-Id: I237cc50cbd7629a046e5a5e4601fbb40355e551d Reviewed-on: https://cl.tvl.fyi/c/depot/+/7670 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-12-29 r/5530 docs(tvix/eval): sketch in place list/attr set update ideasterni1-0/+24
Change-Id: Ic7debbd8cbd3acdf5f3947288f2aa2964bd163a0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7660 Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-12-25 r/5486 fix(tvix/eval): fix current clippy warningsVincent Ambo17-108/+69
It's been a while since the last time, so quite a lot of stuff has accumulated here. Change-Id: I0762827c197b30a917ff470fd8ae8f220f6ba247 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7597 Reviewed-by: grfn <grfn@gws.fyi> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-12-25 r/5485 refactor(tvix/eval): non-recursive thunk forcingAdam Joseph2-67/+243
Introduces continuation-passing-based trampolining of thunk forcing to avoid recursing when forcing deeply nested expressions. This is required for evaluating large expressions. This change was extracted out of cl/7362. Co-authored-by: Vincent Ambo <tazjin@tvl.su> Co-authored-by: Griffin Smith <grfn@gws.fyi> Change-Id: Ifc1747e712663684b2fff53095de62b8459a47f3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7551 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-12-25 r/5484 feat(tvix/eval): implement From<f64> for ValueRyan Lahfa1-0/+6
Change-Id: I287282a195d6f752260242739332b2357791974a Reviewed-on: https://cl.tvl.fyi/c/depot/+/7625 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-12-22 r/5477 feat(tvix/eval): display function names in documentationVincent Ambo1-1/+8
... if they are known. We currently do not propagate names correctly for curried functions. Change-Id: I19d57fb30a5c0000ccdf690b91076f6b2191de23 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7596 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-12-22 r/5475 feat(tvix/eval): add Value::explain methodVincent Ambo1-0/+33
This value creates a human-readable explanation of a value. This can be used to implement documentation related functionality. For some values, the amount of information displayed can be expanded quite a bit. Change-Id: Ie8c400feae909e7680af163596f99060262e4241 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7592 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-22 r/5474 feat(tvix/cli): implement `NixCompatIO` helper typeVincent Ambo1-1/+1
This type allows for temporarily compatibility with the C++ Nix store, specifically (for now) it gives us the store directory used by Nix and imports files the same way. Change-Id: I4767794ef2863eba49661315c63c4e17de946d60 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7587 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-21 r/5469 chore(tvix/eval): fix a broken commentVincent Ambo1-1/+0
Change-Id: I0a6edb51685f94d4712089ae805170da3fb7faae Reviewed-on: https://cl.tvl.fyi/c/depot/+/7609 Reviewed-by: grfn <grfn@gws.fyi> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-12-21 r/5468 fix(tvix/builtin-macros): parse multi-line docstrings correctlyVincent Ambo2-5/+28
Having a multi-line docstring yields multiple doc-attributes in order, however we were previously discarding all but the first one. This reduces them into a single string instead, which can then be displayed as multi-line documentation. Change-Id: I1f237956cdea2e4c746d3f13744e0373c1c645a6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7594 Reviewed-by: grfn <grfn@gws.fyi> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-12-21 r/5467 feat(tvix/eval): use `EvalIO::import_path` when coercing pathsVincent Ambo3-3/+28
This "ties the knot" of importing files into a store when referring to them through path literals, e.g. inside of strings. I'm not yet sure if this interface is sufficient for builtins.path (which we haven't implemented at all yet), but it's enough to wire up eval & store initially. In the default implementations nothing interesting happens in this function at all. Change-Id: Ie01ff4161617d1e743a68dbd1a5e54c1b40c0990 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7582 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-21 r/5466 refactor(tvix/eval): use light spans in builtins.importVincent Ambo2-12/+8
Change-Id: I05732073155b430575babb6f076bf465aef98857 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7581 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-12-21 r/5465 feat(tvix/eval): builtins.storeDirAdam Joseph2-1/+17
Returns the store directory through EvalIO::store_dir. Note that this is _optional_ in Tvix, as an evaluation can occur in a context where there simply is no store directory. In those contexts, `builtins.storeDir` returns `null` in Tvix. This would only happen in contexts like Tvixbolt (or completely unrelated use-cases) in practice. Co-Authored-By: Vincent Ambo <tazjin@tvl.su> Change-Id: I5a752c7e89b2f75bd7efb082dbfa5b25e3b1ff3b Reviewed-on: https://cl.tvl.fyi/c/depot/+/7452 Autosubmit: Adam Joseph <adam@westernsemico.com> Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-12-21 r/5464 refactor(tvix/eval): use `EvalIO::read_dir` for equivalent builtinVincent Ambo2-30/+77
Change-Id: I6d782c07166f51587d2f1d06607823268debb5d5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7574 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-21 r/5463 refactor(tvix/eval): use `EvalIO::path_exists` for the builtinVincent Ambo2-2/+18
Change-Id: I49822ce30137777865e7370ee86666636e277b35 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7573 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-12-21 r/5462 chore(tvix/eval): gate tvix_eval::StdIO behind the `impure` featureVincent Ambo2-1/+6
This shouldn't be available if we've built a "pure" crate. Change-Id: I7c85827ee212890252ff7e0b6242e2c52618cba5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7572 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-12-21 r/5460 refactor(tvix/eval): use EvalIO::read_to_string in impure builtinsVincent Ambo3-38/+21
With this change, the behaviour of reading a string from a file path is controlled by the provided `EvalIO` structure. This is a huge step towards abstracting away I/O behaviour correctly. Change-Id: Ifde8e46cd863b16e0301dca45a434ad27560399f Reviewed-on: https://cl.tvl.fyi/c/depot/+/7567 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-12-21 r/5459 feat(tvix/eval): add EvalIO to public crate APIVincent Ambo6-14/+42
This lets users set the `io_handle` field on an `Evaluation`, which is then propagated to the VM. Change-Id: I616d7140724fb2b4db47c2ebf95451d5303a487a Reviewed-on: https://cl.tvl.fyi/c/depot/+/7566 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-21 r/5458 feat(tvix/eval): introduce initial EvalIO traitVincent Ambo2-0/+53
This trait is going to be used to abstract filesystem interactions in Tvix. For now, it only contains a `read_to_string` method that closely mirrors `std::fs::read_to_string`. As a first step, to see how this works in practice, we will thread through only this function to the various relevant parts. Two implementations are provided in tvix-eval itself: A dummy implementation (which just returns ErrorKind::NotImplemented for all operations), and a std implementation which delegates to `std` functions. Change-Id: Ied3e3bf4bd0e874dd84e166190e3873a0f923ddb Reviewed-on: https://cl.tvl.fyi/c/depot/+/7565 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-21 r/5457 refactor(tvix/eval): add a LightSpan type for lighter span trackingVincent Ambo5-12/+61
This type carries the information required for calculating a span (i.e. the chunk and offset), instead of the span itself. The span is then only calculated in cases where it is required (when throwing errors). This reduces the eval time for `builtins.length (builtins.attrNames (import <nixpkgs> {}))` by *one third*! The data structure in chunks that carries span information reduces in-memory size by trading off the speed of retrieving span information. This is because the span information is only actually required when throwing errors (or emitting warnings). However, somewhere along the way we grew a dependency on carrying span information in thunks (for correctly reporting error chains). Hitting the code paths for span retrieval was expensive, and carrying the spans in a different way would still be less cache-efficient. This change is the best tradeoff I could come up with. Refs: b/229. Change-Id: I27d4c4b5c5f9be90ac47f2db61941e123a78a77b Reviewed-on: https://cl.tvl.fyi/c/depot/+/7558 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-21 r/5455 feat(tvix/eval): add thunks with suspended native Rust codeAdam Joseph1-1/+45
Having thunks which, when forced, execute native Rust code rather than interpreted opcodes lets us avoid having to bundle `src/libexpr/primops/derivation.nix` like cppnix does by implementing it in Rust instead. Change-Id: If91d77a6736234321eee87ba4b4777eed5a3fe1c Reviewed-on: https://cl.tvl.fyi/c/depot/+/7450 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-21 r/5454 refactor(tvix/eval): add name-based index over compiler's localsVincent Ambo2-34/+108
Instead of finding locals by doing 2x O(n) walks over the compiler's locals list, use a secondary name-based index for resolving locals by name. Previously, almost 60% (!!) of eval time on some expressions over nixpkgs was spent in `Local::has_name`. This function doesn't even exist anymore now, and eval speed about doubles as a result. Note that this doesn't exactly make the locals code easier to read, but I'm also not sure what we can simplify in there in general. This fixes b/227. Change-Id: I29ce5eb9452b02d3b358c673e1f5cf8082e2fef9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7560 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-21 r/5453 feat(tvix/eval): remove `derive(Copy)` from UpvaluesAdam Joseph3-12/+26
Change-Id: I0fa069fbeff6718a765ece948c2c1bce285496f7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7449 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-21 r/5452 feat(tvix/eval): wrap Closure in Rc<> to match cppnix semanticsAdam Joseph6-61/+34
Change-Id: I595087eff943d38a9fc78a83d37e207bb2ab79bc Reviewed-on: https://cl.tvl.fyi/c/depot/+/7443 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-21 r/5445 refactor(tvix/eval): consume `self` in Evaluation::evaluateVincent Ambo1-7/+6
This simplifies lifetime management for observers in callers of tvix_eval. Change-Id: I2f47c8d89f22b1c766526e5d1483c0d026b500ae Reviewed-on: https://cl.tvl.fyi/c/depot/+/7546 Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-21 r/5444 feat(tvix/eval): add observer configuration to public APIVincent Ambo1-11/+21
With this change, it should be possible to have both existing use-cases (CLI & Tvixbolt) use the same API. Change-Id: I2195264f08cc892177b559a28660dc5f98e48e41 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7545 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-12-21 r/5443 feat(tvix/eval): add configuration of Nix search path to public APIVincent Ambo2-2/+33
This is required for passing through NIX_PATH from the CLI. Change-Id: If129df79ef9c3ffab31408adb85679909276c4f0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7544 Reviewed-by: grfn <grfn@gws.fyi> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-12-21 r/5442 fix(tvix/eval): use test-generator fork that supports workspacesVincent Ambo1-1/+6
This should make no difference in Nix builds, but allows running tests locally again with `cargo test` for //tvix/eval. Change-Id: I97d61840143d5c14db61d5862781bf635f9a28e7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7590 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2022-12-21 r/5441 chore(tvix): upgrade to clap 4.0Vincent Ambo2-2/+5
In //tvix/eval: * criterion bumped to 4.0, which at least depends on clap 3.x instead of 2.x, which is less incompatible In //tvix/cli: * no changes required In //tvix/nix_cli: * some minor changes for compatibility with clap 4.0, no functionality changes Change-Id: If793f64b59fcaa2402d3d483ddbab4092f32df03 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7588 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-21 r/5440 refactor(tvix): split binary (REPL etc.) out from evaluator libraryVincent Ambo4-263/+1
The tvix-eval project is independent from any *uses* of the evaluator, such as the tvix-repl. This functionality has been split out into a separate "tvix-cli" crate. Note that this doesn't have to mean that this CLI crate is the "final" CLI crate for tvix, the point of this is not "getting the CLI structure right" but rather "getting the evaluator structure right". This reshuffling is part of restructuring the way that functionality like store communication is injected into language evaluation. Note that at this commit the new CLI crate is not at feature-parity. Change-Id: Id0af03dc8e07ef09a9f882a89612ad555eca8f93 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7541 Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-21 r/5439 refactor(tvix/eval): use new public API in test codeVincent Ambo2-41/+48
This removes internal uses of the previous crate::eval module, which is being removed. Change-Id: I5fb3c53460a9c5381853d0258f9ed074ab23c630 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7543 Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: grfn <grfn@gws.fyi>
2022-12-21 r/5438 feat(tvix/eval): begin introducing new public APIVincent Ambo1-2/+154
A step towards something more like how I imagine the future public API for tvix-eval. Please note that this is definitely not the final version yet, but it's better than the previous API that either exposed a side-effecting blackbox, or a very low-level "interface". The basic idea is that an evaluation of some Nix code is requested by a caller with various parameters, but not all callers are interested in all of these parameters. There are also some bits of information that are returned from an evaluation that are not necessarily relevant to all callers. To support this somewhat ergonomically, the API is built around an `Evaluation` struct that is configured by the caller with the various parameters and then "executed". Change-Id: I71826f3897126898adc2873d31c44d3eaf5c2be0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7542 Reviewed-by: grfn <grfn@gws.fyi> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-12-15 r/5420 refactor(tvix): build Rust projects using crate2nixVincent Ambo6-32/+7
Introduces granular dependency builds using crate2nix, bootstrapped off the generated configuration from the newly introduced workspace (see cl/7533). This commit checks in the generated Cargo.nix file which can be regenerated with a parameterless invocation of `crate2nix generate` in `//tvix`. I tried generating this in IFD, but it turned out to be harder than what seemed worthwhile for now. In this setup, the various build targets for Rust projects end up being attributes of the imported `Cargo.nix` file at the `tvix.crates` attribute. These still lack configuration, however, which has been fixed in the various `default.nix` files of individual projects. Note that we (temporarily) lose the ability to build tvix-eval's benchmarks in CI. I haven't figured out what magic incantation summons them from the void again ... The `eval-okay-readDir` tests from both test suites have been disabled because they fail for unknown reasons when run in this new derivation. Somebody will have to debug it! Change-Id: I2014614ccb9c8951aedbd71df7966ca191a13695 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7538 Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2022-12-15 r/5419 refactor(tvix): share a Cargo.lock file between Rust projectsVincent Ambo4-2484/+1
This relates to the (abandoned) cl/7256. Introduces a Cargo workspace at //tvix that is primarily intended to be used as a workaround for the annoying Nix+Rust tooling while having a consistent set of dependencies. This is driven in part by a desire to adopt crate2nix and get more granular Nix builds for Tvix's Rust projects, and in part by a need to split //tvix/eval into something providing the CLI (REPL etc.), and a library providing eval, without significantly altering the structure of build targets. To accomplish this the workspace has been designed to allow projects to remain independent build targets. I want to avoid lumping all the projects together - something like //tvix/eval should always be independent of other parts of tvix. A helper function in //tvix/default.nix lets downstream naersk projects construct a sparse root for the project which combines the workspace's `Cargo.lock` with the project's own `Cargo.toml`. Note that cargo commands in the workspace itself require the build dependencies of _all_ projects to be present, which is currently a bit annoying to accomplish. This introduces some breakage: 1. It breaks usage of rust-analyser without being in a shell with the dependencies of *all* Tvix projects, as it is not capable of respecting only the subset of dependencies for a part of the workspace. 2. It is no longer possible to run tests using `cargo test`, as the test generation crate we use does not work with workspaces: https://github.com/frehberg/test-generator/issues/6 This still works in the Nix build as we construct a Cargo project that looks like it's not in a workspace there. Until somebody fixes that crate / writes a new macro / does something else with the test suite, the way to run the tests is through the Nix build. Long-term we'll probably want to get rid of cargo completely, it's just a big wart and most tooling works without it if correctly configured, but we don't have time for that now. Change-Id: I846bff7a8429a25c077fd1e9ef4e3c34a299a4a1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7533 Reviewed-by: flokli <flokli@flokli.de> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-12-06 r/5396 fix(tvix/eval): use rnix-parser from crates.ioVincent Ambo2-8/+4
A few weeks ago, oberblastmeister did a release to crates.io so we can stop importing it via GitHub. Change-Id: I9d5fa5cd281685779c71b12fed45ed201a1db17e Reviewed-on: https://cl.tvl.fyi/c/depot/+/7532 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-12-03 r/5379 feat(tvix/eval): Continue removing leakage of BTreeMap.Lyle Mantooth4-52/+40
Fixes b/212. Based on feedback in https://cl.tvl.fyi/c/depot/+/7492, all uses of `NixAttrs::from_map` have been removed. Only `from_iter` and `from_kv` remain. Change-Id: I52e25f73018c2aa1843197427516b7a852503e2c Reviewed-on: https://cl.tvl.fyi/c/depot/+/7500 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: IslandUsurper <lyle@menteeth.us>
2022-12-02 r/5375 feat(tvix/eval): impl FromIterator for NixAttrsLyle Mantooth2-52/+64
Allows for the removal of some BTreeMap usage when constructing NixAttrs by allowing any iterator over 2-tuples to build a NixAttrs. Some instances of BTreeMap didn't have anything to do with making NixAttrs, and some were just the best tool for the job, so they are left using the old `from_map` interface. Change-Id: I668ea600b0d93eae700a6b1861ac84502c968d78 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7492 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-12-02 r/5374 test(tvix/eval): check inner forcing despite declaring pointersterni2-0/+12
Maybe counter-intuitively the inner elements of a list or the attribute values of an attribute set will be forced despite pointer equality (but only one layer deep). Change-Id: I485d96452fb56f5fb342d39039c9137725b33d3f Reviewed-on: https://cl.tvl.fyi/c/depot/+/7371 Reviewed-by: Adam Joseph <adam@westernsemico.com> Tested-by: BuildkiteCI
2022-12-02 r/5373 test(tvix/eval): add a test for repeated keys in listToAttrsVincent Ambo2-0/+14
This came up in the Nix Language channel today and I thought it warranted a test case. We did actually implement this correctly. Change-Id: I4b37c92d06eb6e3a7f59ea3d10af38f2b0a93d53 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7493 Reviewed-by: sterni <sternenseemann@systemli.org> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-12-02 r/5369 test(tvix/eval): verify pointer equality checkssterni2-0/+47
Change-Id: I9baf2810fbd5b6ee8bfe10fce5b64801a35c1d67 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7369 Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: Adam Joseph <adam@westernsemico.com> Tested-by: BuildkiteCI
2022-12-02 r/5368 test(tvix/eval): verify pointer equality in list comparisonssterni2-0/+7
Change-Id: I617d402c8ecc7aaf607c4bdcd58a06ebddb71fac Reviewed-on: https://cl.tvl.fyi/c/depot/+/7370 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: Adam Joseph <adam@westernsemico.com>
2022-12-02 r/5367 feat(tvix/eval): inline(always) tail_call_valueAdam Joseph1-0/+1
Rust doesn't do tail-call elimination (still!) so the best we can hope for here is to inline non-recursive invocations. Change-Id: I78949967e48b006fcbf31786d8f6281cd122f36f Reviewed-on: https://cl.tvl.fyi/c/depot/+/7360 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: Adam Joseph <adam@westernsemico.com>
2022-12-02 r/5366 feat(tvix/eval): crude caching builtins.importAdam Joseph2-7/+20
Before this, tvix was spending most of its time furiously re-parsing and re-compiling nixpkgs, each time hoping to get a different result... Change-Id: I1c0cfbf9af622c276275b1f2fb8d4e976f1b5533 Signed-off-by: Adam Joseph <adam@westernsemico.com> Reviewed-on: https://cl.tvl.fyi/c/depot/+/7361 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-12-01 r/5356 feat(tvix/eval): placeholder for builtins.placeholderAdam Joseph1-0/+7
Change-Id: I8d11f2db4489a7d82910256069d10f8bed3bdf9a Reviewed-on: https://cl.tvl.fyi/c/depot/+/7451 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: Adam Joseph <adam@westernsemico.com> Tested-by: BuildkiteCI