about summary refs log tree commit diff
path: root/tvix (follow)
AgeCommit message (Collapse)AuthorFilesLines
2022-12-26 r/5496 feat(txix/store/protos): init go.mod, generate .pb.go filesFlorian Klink9-0/+2390
This allows importing the generated .pb.go files into other go projects. I initially looked at buildGo.protos, but it doesn't work for multi-.proto files, and actually having LSP support for the generated structs is nice, too. Change-Id: Idbd448008010790a10a0ea42e4059dbb609eaf1a Reviewed-on: https://cl.tvl.fyi/c/depot/+/7322 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-12-26 r/5495 feat(tvix/store/protos): add go_package optionFlorian Klink5-0/+10
Change-Id: I0898b8a0a78e704219da38e5acaabef1e640d4e4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7321 Reviewed-by: Adam Joseph <adam@westernsemico.com> Tested-by: BuildkiteCI
2022-12-26 r/5494 docs(tvix/store): add README, document services and store modelFlorian Klink4-2/+391
These are intended to help digest the protocol definitions for tvix- store, and how they tie into the whole concept. Co-Authored-By: Vincent Ambo <mail@tazj.in> Change-Id: Ic1ba3ba41ef599209453f15d0ac2e07a6144bcca Reviewed-on: https://cl.tvl.fyi/c/depot/+/7439 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-12-26 r/5492 feat(tvix/store): use tonic-build for protosFlorian Klink5-20/+2334
While prost-build already exposes protobuf message types as structs, we actually need tonic-build too, to be able to get traits for all the RPC services defined in the proto files. Change-Id: I7f4c08454bf0d280d577975c7cdae13ccc2d933b Reviewed-on: https://cl.tvl.fyi/c/depot/+/7320 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-12-25 r/5486 fix(tvix/eval): fix current clippy warningsVincent Ambo17-108/+69
It's been a while since the last time, so quite a lot of stuff has accumulated here. Change-Id: I0762827c197b30a917ff470fd8ae8f220f6ba247 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7597 Reviewed-by: grfn <grfn@gws.fyi> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-12-25 r/5485 refactor(tvix/eval): non-recursive thunk forcingAdam Joseph2-67/+243
Introduces continuation-passing-based trampolining of thunk forcing to avoid recursing when forcing deeply nested expressions. This is required for evaluating large expressions. This change was extracted out of cl/7362. Co-authored-by: Vincent Ambo <tazjin@tvl.su> Co-authored-by: Griffin Smith <grfn@gws.fyi> Change-Id: Ifc1747e712663684b2fff53095de62b8459a47f3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7551 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-12-25 r/5484 feat(tvix/eval): implement From<f64> for ValueRyan Lahfa1-0/+6
Change-Id: I287282a195d6f752260242739332b2357791974a Reviewed-on: https://cl.tvl.fyi/c/depot/+/7625 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-12-22 r/5477 feat(tvix/eval): display function names in documentationVincent Ambo1-1/+8
... if they are known. We currently do not propagate names correctly for curried functions. Change-Id: I19d57fb30a5c0000ccdf690b91076f6b2191de23 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7596 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-12-22 r/5476 feat(tvix/cli): add `:d` REPL-prefix to print explanations of valuesVincent Ambo1-5/+14
Change-Id: I1dd8de8d996e07840f9b0aaebf932b812103a43a Reviewed-on: https://cl.tvl.fyi/c/depot/+/7593 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-22 r/5475 feat(tvix/eval): add Value::explain methodVincent Ambo1-0/+33
This value creates a human-readable explanation of a value. This can be used to implement documentation related functionality. For some values, the amount of information displayed can be expanded quite a bit. Change-Id: Ie8c400feae909e7680af163596f99060262e4241 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7592 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-22 r/5474 feat(tvix/cli): implement `NixCompatIO` helper typeVincent Ambo6-2/+92
This type allows for temporarily compatibility with the C++ Nix store, specifically (for now) it gives us the store directory used by Nix and imports files the same way. Change-Id: I4767794ef2863eba49661315c63c4e17de946d60 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7587 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-22 r/5472 feat(tvix): add crate2nixGenerate targetFlorian Klink2-9/+11
`mg run //tvix:crate2nixGenerate --` runs crate2nix generate and depotfmt's it afterwards. This removes a frequent point of friction, because the Cargo.nix emitted by crate2nix needs another formatter. Change-Id: I649495980718cd3847d4cff77c9d4bfcb599387c Reviewed-on: https://cl.tvl.fyi/c/depot/+/7612 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-12-22 r/5471 feat(tvix/proto): add rpc_directory.protoFlorian Klink3-0/+48
This provides an interface to retrieve and upload single Directory messages, or a DAG of them. Change-Id: Id9e7084bd697d843649a122da2c992a3e36d808c Reviewed-on: https://cl.tvl.fyi/c/depot/+/7137 Tested-by: BuildkiteCI Reviewed-by: Adam Joseph <adam@westernsemico.com>
2022-12-22 r/5470 feat(tvix/proto): add rpc_pathinfoFlorian Klink3-0/+44
This defines a service that can be used to upload and retrieve metadata of nix paths. Change-Id: Id86eb531ce4ae316adb15934b0d1386a14ba2132 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7136 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: Adam Joseph <adam@westernsemico.com>
2022-12-21 r/5469 chore(tvix/eval): fix a broken commentVincent Ambo1-1/+0
Change-Id: I0a6edb51685f94d4712089ae805170da3fb7faae Reviewed-on: https://cl.tvl.fyi/c/depot/+/7609 Reviewed-by: grfn <grfn@gws.fyi> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-12-21 r/5468 fix(tvix/builtin-macros): parse multi-line docstrings correctlyVincent Ambo2-5/+28
Having a multi-line docstring yields multiple doc-attributes in order, however we were previously discarding all but the first one. This reduces them into a single string instead, which can then be displayed as multi-line documentation. Change-Id: I1f237956cdea2e4c746d3f13744e0373c1c645a6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7594 Reviewed-by: grfn <grfn@gws.fyi> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-12-21 r/5467 feat(tvix/eval): use `EvalIO::import_path` when coercing pathsVincent Ambo3-3/+28
This "ties the knot" of importing files into a store when referring to them through path literals, e.g. inside of strings. I'm not yet sure if this interface is sufficient for builtins.path (which we haven't implemented at all yet), but it's enough to wire up eval & store initially. In the default implementations nothing interesting happens in this function at all. Change-Id: Ie01ff4161617d1e743a68dbd1a5e54c1b40c0990 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7582 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-21 r/5466 refactor(tvix/eval): use light spans in builtins.importVincent Ambo2-12/+8
Change-Id: I05732073155b430575babb6f076bf465aef98857 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7581 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-12-21 r/5465 feat(tvix/eval): builtins.storeDirAdam Joseph2-1/+17
Returns the store directory through EvalIO::store_dir. Note that this is _optional_ in Tvix, as an evaluation can occur in a context where there simply is no store directory. In those contexts, `builtins.storeDir` returns `null` in Tvix. This would only happen in contexts like Tvixbolt (or completely unrelated use-cases) in practice. Co-Authored-By: Vincent Ambo <tazjin@tvl.su> Change-Id: I5a752c7e89b2f75bd7efb082dbfa5b25e3b1ff3b Reviewed-on: https://cl.tvl.fyi/c/depot/+/7452 Autosubmit: Adam Joseph <adam@westernsemico.com> Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-12-21 r/5464 refactor(tvix/eval): use `EvalIO::read_dir` for equivalent builtinVincent Ambo2-30/+77
Change-Id: I6d782c07166f51587d2f1d06607823268debb5d5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7574 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-21 r/5463 refactor(tvix/eval): use `EvalIO::path_exists` for the builtinVincent Ambo2-2/+18
Change-Id: I49822ce30137777865e7370ee86666636e277b35 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7573 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-12-21 r/5462 chore(tvix/eval): gate tvix_eval::StdIO behind the `impure` featureVincent Ambo2-1/+6
This shouldn't be available if we've built a "pure" crate. Change-Id: I7c85827ee212890252ff7e0b6242e2c52618cba5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7572 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-12-21 r/5461 fix(tvix/cli): use tvix_eval::StdIO in CLIVincent Ambo1-0/+1
... until we have a store-I/O layer, or something that intercepts the store-related stuff appropriately. Change-Id: I22f63435b3f9e118e3faeb2924fda8373a23ea7f Reviewed-on: https://cl.tvl.fyi/c/depot/+/7568 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-21 r/5460 refactor(tvix/eval): use EvalIO::read_to_string in impure builtinsVincent Ambo3-38/+21
With this change, the behaviour of reading a string from a file path is controlled by the provided `EvalIO` structure. This is a huge step towards abstracting away I/O behaviour correctly. Change-Id: Ifde8e46cd863b16e0301dca45a434ad27560399f Reviewed-on: https://cl.tvl.fyi/c/depot/+/7567 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-12-21 r/5459 feat(tvix/eval): add EvalIO to public crate APIVincent Ambo6-14/+42
This lets users set the `io_handle` field on an `Evaluation`, which is then propagated to the VM. Change-Id: I616d7140724fb2b4db47c2ebf95451d5303a487a Reviewed-on: https://cl.tvl.fyi/c/depot/+/7566 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-21 r/5458 feat(tvix/eval): introduce initial EvalIO traitVincent Ambo2-0/+53
This trait is going to be used to abstract filesystem interactions in Tvix. For now, it only contains a `read_to_string` method that closely mirrors `std::fs::read_to_string`. As a first step, to see how this works in practice, we will thread through only this function to the various relevant parts. Two implementations are provided in tvix-eval itself: A dummy implementation (which just returns ErrorKind::NotImplemented for all operations), and a std implementation which delegates to `std` functions. Change-Id: Ied3e3bf4bd0e874dd84e166190e3873a0f923ddb Reviewed-on: https://cl.tvl.fyi/c/depot/+/7565 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-21 r/5457 refactor(tvix/eval): add a LightSpan type for lighter span trackingVincent Ambo5-12/+61
This type carries the information required for calculating a span (i.e. the chunk and offset), instead of the span itself. The span is then only calculated in cases where it is required (when throwing errors). This reduces the eval time for `builtins.length (builtins.attrNames (import <nixpkgs> {}))` by *one third*! The data structure in chunks that carries span information reduces in-memory size by trading off the speed of retrieving span information. This is because the span information is only actually required when throwing errors (or emitting warnings). However, somewhere along the way we grew a dependency on carrying span information in thunks (for correctly reporting error chains). Hitting the code paths for span retrieval was expensive, and carrying the spans in a different way would still be less cache-efficient. This change is the best tradeoff I could come up with. Refs: b/229. Change-Id: I27d4c4b5c5f9be90ac47f2db61941e123a78a77b Reviewed-on: https://cl.tvl.fyi/c/depot/+/7558 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-21 r/5456 chore(tvix/cli): rename binary to just 'tvix'Vincent Ambo2-1/+5
Change-Id: If735c81a04f60dcd77f1c40b595b52ed88bbbe95 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7580 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi> Reviewed-by: flokli <flokli@flokli.de>
2022-12-21 r/5455 feat(tvix/eval): add thunks with suspended native Rust codeAdam Joseph1-1/+45
Having thunks which, when forced, execute native Rust code rather than interpreted opcodes lets us avoid having to bundle `src/libexpr/primops/derivation.nix` like cppnix does by implementing it in Rust instead. Change-Id: If91d77a6736234321eee87ba4b4777eed5a3fe1c Reviewed-on: https://cl.tvl.fyi/c/depot/+/7450 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-21 r/5454 refactor(tvix/eval): add name-based index over compiler's localsVincent Ambo2-34/+108
Instead of finding locals by doing 2x O(n) walks over the compiler's locals list, use a secondary name-based index for resolving locals by name. Previously, almost 60% (!!) of eval time on some expressions over nixpkgs was spent in `Local::has_name`. This function doesn't even exist anymore now, and eval speed about doubles as a result. Note that this doesn't exactly make the locals code easier to read, but I'm also not sure what we can simplify in there in general. This fixes b/227. Change-Id: I29ce5eb9452b02d3b358c673e1f5cf8082e2fef9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7560 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-21 r/5453 feat(tvix/eval): remove `derive(Copy)` from UpvaluesAdam Joseph3-12/+26
Change-Id: I0fa069fbeff6718a765ece948c2c1bce285496f7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7449 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-21 r/5452 feat(tvix/eval): wrap Closure in Rc<> to match cppnix semanticsAdam Joseph6-61/+34
Change-Id: I595087eff943d38a9fc78a83d37e207bb2ab79bc Reviewed-on: https://cl.tvl.fyi/c/depot/+/7443 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-21 r/5451 docs(tvix/nar): document how to use NAR writerVincent Ambo1-0/+49
This is done in the form of some comments on the functions, as well as a functional doctest that writes a single file to a NAR. Change-Id: Ic97ebd439e91d6b076685807fe70de098ec02575 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7599 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2022-12-21 r/5450 chore(tvix/nar): add CI build targetVincent Ambo6-8/+33
Change-Id: Ic73a391da9a733cade26114ab1127907c8d62a57 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7598 Reviewed-by: flokli <flokli@flokli.de> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-12-21 r/5449 chore(tvix/cli): re-add observer flagsVincent Ambo1-2/+34
Users can again pass flags for dumping the AST, bytecode, and runtime trace. With this commit the CLI is at feature-parity with what we had before, but entirely through the new API. Change-Id: I30fe26f243224b25d1e4f828fec607325ef88306 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7550 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-12-21 r/5448 chore(tvix/cli): re-add NIX_PATH handlingVincent Ambo1-1/+7
Change-Id: I5595d6f5141ed4a533ca44a46264dc604fca6be1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7549 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-12-21 r/5447 chore(tvix/cli): re-add `--raw` argumentVincent Ambo1-14/+15
Change-Id: I82f469f7f186c754c40bd941587b30aa3fc48045 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7548 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-21 r/5445 refactor(tvix/eval): consume `self` in Evaluation::evaluateVincent Ambo2-9/+8
This simplifies lifetime management for observers in callers of tvix_eval. Change-Id: I2f47c8d89f22b1c766526e5d1483c0d026b500ae Reviewed-on: https://cl.tvl.fyi/c/depot/+/7546 Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-21 r/5444 feat(tvix/eval): add observer configuration to public APIVincent Ambo1-11/+21
With this change, it should be possible to have both existing use-cases (CLI & Tvixbolt) use the same API. Change-Id: I2195264f08cc892177b559a28660dc5f98e48e41 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7545 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-12-21 r/5443 feat(tvix/eval): add configuration of Nix search path to public APIVincent Ambo2-2/+33
This is required for passing through NIX_PATH from the CLI. Change-Id: If129df79ef9c3ffab31408adb85679909276c4f0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7544 Reviewed-by: grfn <grfn@gws.fyi> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-12-21 r/5442 fix(tvix/eval): use test-generator fork that supports workspacesVincent Ambo4-4/+16
This should make no difference in Nix builds, but allows running tests locally again with `cargo test` for //tvix/eval. Change-Id: I97d61840143d5c14db61d5862781bf635f9a28e7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7590 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2022-12-21 r/5441 chore(tvix): upgrade to clap 4.0Vincent Ambo7-445/+478
In //tvix/eval: * criterion bumped to 4.0, which at least depends on clap 3.x instead of 2.x, which is less incompatible In //tvix/cli: * no changes required In //tvix/nix_cli: * some minor changes for compatibility with clap 4.0, no functionality changes Change-Id: If793f64b59fcaa2402d3d483ddbab4092f32df03 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7588 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-21 r/5440 refactor(tvix): split binary (REPL etc.) out from evaluator libraryVincent Ambo9-198/+105
The tvix-eval project is independent from any *uses* of the evaluator, such as the tvix-repl. This functionality has been split out into a separate "tvix-cli" crate. Note that this doesn't have to mean that this CLI crate is the "final" CLI crate for tvix, the point of this is not "getting the CLI structure right" but rather "getting the evaluator structure right". This reshuffling is part of restructuring the way that functionality like store communication is injected into language evaluation. Note that at this commit the new CLI crate is not at feature-parity. Change-Id: Id0af03dc8e07ef09a9f882a89612ad555eca8f93 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7541 Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-12-21 r/5439 refactor(tvix/eval): use new public API in test codeVincent Ambo2-41/+48
This removes internal uses of the previous crate::eval module, which is being removed. Change-Id: I5fb3c53460a9c5381853d0258f9ed074ab23c630 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7543 Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: grfn <grfn@gws.fyi>
2022-12-21 r/5438 feat(tvix/eval): begin introducing new public APIVincent Ambo1-2/+154
A step towards something more like how I imagine the future public API for tvix-eval. Please note that this is definitely not the final version yet, but it's better than the previous API that either exposed a side-effecting blackbox, or a very low-level "interface". The basic idea is that an evaluation of some Nix code is requested by a caller with various parameters, but not all callers are interested in all of these parameters. There are also some bits of information that are returned from an evaluation that are not necessarily relevant to all callers. To support this somewhat ergonomically, the API is built around an `Evaluation` struct that is configured by the caller with the various parameters and then "executed". Change-Id: I71826f3897126898adc2873d31c44d3eaf5c2be0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7542 Reviewed-by: grfn <grfn@gws.fyi> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-12-20 r/5428 feat(tvix/nar): init writeredef4-0/+214
Change-Id: I101f35840cb14c58af708c91e59f997f6d167f75 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7505 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su>
2022-12-15 r/5422 feat(tvix/proto): add rpc_blobstoreFlorian Klink3-0/+34
This defines a service that can be used to get and put content-addressed chunks of data. Change-Id: I36cf2278ed1daf71848c04fdfd14450b2268c5de Reviewed-on: https://cl.tvl.fyi/c/depot/+/7135 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-12-15 r/5421 refactor(tvix): provide a combined //tvix:shell mkShell targetVincent Ambo2-17/+20
Manually maintained shell target based on the previous "loose" `shell.nix` file. We might want to have a function that combines the dependencies of all the targets automatically, but at a quick glance that was actually non-trivial so I'm leaving it as an exercise for someone else. Change-Id: I74754940088f1b58e3b6754fb782470c80ea4292 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7540 Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-12-15 r/5420 refactor(tvix): build Rust projects using crate2nixVincent Ambo11-56/+6345
Introduces granular dependency builds using crate2nix, bootstrapped off the generated configuration from the newly introduced workspace (see cl/7533). This commit checks in the generated Cargo.nix file which can be regenerated with a parameterless invocation of `crate2nix generate` in `//tvix`. I tried generating this in IFD, but it turned out to be harder than what seemed worthwhile for now. In this setup, the various build targets for Rust projects end up being attributes of the imported `Cargo.nix` file at the `tvix.crates` attribute. These still lack configuration, however, which has been fixed in the various `default.nix` files of individual projects. Note that we (temporarily) lose the ability to build tvix-eval's benchmarks in CI. I haven't figured out what magic incantation summons them from the void again ... The `eval-okay-readDir` tests from both test suites have been disabled because they fail for unknown reasons when run in this new derivation. Somebody will have to debug it! Change-Id: I2014614ccb9c8951aedbd71df7966ca191a13695 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7538 Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2022-12-15 r/5419 refactor(tvix): share a Cargo.lock file between Rust projectsVincent Ambo10-1707/+262
This relates to the (abandoned) cl/7256. Introduces a Cargo workspace at //tvix that is primarily intended to be used as a workaround for the annoying Nix+Rust tooling while having a consistent set of dependencies. This is driven in part by a desire to adopt crate2nix and get more granular Nix builds for Tvix's Rust projects, and in part by a need to split //tvix/eval into something providing the CLI (REPL etc.), and a library providing eval, without significantly altering the structure of build targets. To accomplish this the workspace has been designed to allow projects to remain independent build targets. I want to avoid lumping all the projects together - something like //tvix/eval should always be independent of other parts of tvix. A helper function in //tvix/default.nix lets downstream naersk projects construct a sparse root for the project which combines the workspace's `Cargo.lock` with the project's own `Cargo.toml`. Note that cargo commands in the workspace itself require the build dependencies of _all_ projects to be present, which is currently a bit annoying to accomplish. This introduces some breakage: 1. It breaks usage of rust-analyser without being in a shell with the dependencies of *all* Tvix projects, as it is not capable of respecting only the subset of dependencies for a part of the workspace. 2. It is no longer possible to run tests using `cargo test`, as the test generation crate we use does not work with workspaces: https://github.com/frehberg/test-generator/issues/6 This still works in the Nix build as we construct a Cargo project that looks like it's not in a workspace there. Until somebody fixes that crate / writes a new macro / does something else with the test suite, the way to run the tests is through the Nix build. Long-term we'll probably want to get rid of cargo completely, it's just a big wart and most tooling works without it if correctly configured, but we don't have time for that now. Change-Id: I846bff7a8429a25c077fd1e9ef4e3c34a299a4a1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7533 Reviewed-by: flokli <flokli@flokli.de> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-07-05T21·54+0000