about summary refs log tree commit diff
path: root/tvix
AgeCommit message (Collapse)AuthorFilesLines
2023-03-27 r/6046 feat(tvix/store/directorysvc): add DirectoryService::get_recursive()Florian Klink5-101/+320
This moves the recursive BFS traversal of Directory closures from the GRPCDirectoryServiceWrapper out into a a DirectoryTraverser struct implementing Iterator. It is then used from various implementors of DirectoryService in the `get_recursive()` method. This allows distinguishing between recursive requests and non-recursive requests in the gRPC client trait implementation. Change-Id: I50bfd4a0d9eb11832847329b78c587ec7c9dc7b1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8351 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-03-27 r/6045 refactor(tvix/eval): retain call frames when entering callsVincent Ambo1-6/+13
This grows the frame stack as the call stack grows, which yields *much* better user-facing error messages. I haven't measured the performance impact this has yet, for now I'm still just trying to add more information to errors and then cut down again where necessary. Change-Id: I89f058ef31979edacf4667775d460b60704ce4d7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8334 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su>
2023-03-27 r/6044 feat(tvix/store/directorysvc): add gRPC clientFlorian Klink5-4/+204
This provides a GRPCDirectoryService struct implementing DirectoryService, allowing a client to Directory objects from a (remote) tvix-store. Remote in this case is anything outside the current process, be it another process, or an endpoint on the network. To keep the sync interface in the `DirectoryService` trait, a handle to some tokio runtime needs to be passed into the constructor, and the two methods use `self.tokio_handle.spawn` to start an async function, and `self.tokio_handle.block_on` to wait for its completion. The client handle, called `grpc_client` itself is easy to clone, and treats concurrent requests internally. This means, even though we keep the `DirectoryService` trait sync, there's nothing preventing it from being used concurrently, let's say from multiple threads. There's still two limitations for now: 1) The trait doesn't make use of the `recursive` request, which currently leads to a N+1 query problem. This can be fixed by `GRPCDirectoryService` having a reference to another `DirectoryService` acting as the local side. I want to wait for general store composition code to pop up before manually coding this here. 2) It's currently only possible to put() leaf directory nodes, as the request normally requires uploading a whole closure. We might want to add another batch function to upload a whole closure, and/or do this batching in certain cases. This still needs some more thinking. Change-Id: I7ffec791610b72c0960cf5307cefbb12ec946dc9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8336 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-27 r/6043 feat(tvix/store/pathinfosvc): add gRPC clientFlorian Klink3-0/+80
Change-Id: Ie8e205c691bd11db99fcf097357c1e49161c6e19 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8349 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-03-27 r/6042 docs(tvix/store/blobreader): fix docstringFlorian Klink1-1/+1
Change-Id: Ia8ce1dd1663663903b9f60feada12e6a16fb4af2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8353 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-03-27 r/6041 docs(tvix/store): fix typo in commentFlorian Klink1-1/+1
We query the blob service for detailled blob info, not the chunk service. Change-Id: I85a6a57b1dae74a950f734be7d4455c5c35ae355 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8348 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-22 r/6038 feat(tvix/cli): add `--strict` flag for evaluationVincent Ambo1-0/+9
This toggles whether tvix will evaluate the top-level value and deep-force it, or return it potentially still containing thunks. Change-Id: Ie910941e3b6a0f16c5c0cb896d73947626335f4b Reviewed-on: https://cl.tvl.fyi/c/depot/+/8326 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2023-03-22 r/6037 feat(tvix/eval): add Evaluation::strict to toggle top-level deepseqVincent Ambo5-5/+26
This makes it possible for callers to control whether they can receive partially evaluated values from an evaluation or not. We're actually flipping the default behaviour to non-strict top-level evaluation, which means that callers have to set `strict = true` on the Evaluation to get the previous behaviour. Change-Id: Ic048e9ba09c88866d4c3177d5fa07db11c4eb20e Reviewed-on: https://cl.tvl.fyi/c/depot/+/8325 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2023-03-22 r/6036 fix(tvix/eval): print unevaluated thunks like Nix doesVincent Ambo1-0/+1
Change-Id: Ie4c563e933f571f45cb4f4efe650d1b65f119e8d Reviewed-on: https://cl.tvl.fyi/c/depot/+/8324 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org> Autosubmit: tazjin <tazjin@tvl.su>
2023-03-21 r/6029 chore(3p/sources): Bump channels & overlayssterni8-20/+46
* //users/wpcarro/emacs: use top level (ELPA) version of eglot, as it was removed from MELPA: https://github.com/melpa/melpa/commit/dc2ead17a8e5d5df59fc3729c8f0435cfcbf55ef * //3p/nixpkgs:tdlib: 1.8.11 -> 1.8.12 * //tvix/store: regenerate protos after buf update Change-Id: I782a8d91fda5ed461788055a3721104e8c032207 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8327 Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: wpcarro <wpcarro@gmail.com> Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: sterni <sternenseemann@systemli.org>
2023-03-19 r/6028 feat(tvix/store/bin): print store pathFlorian Klink1-0/+38
After ingestion of the contents into the store, this will use the NonCachingNARCalculationService to create a NAR stream or the contents of the path, and use our Derivation output path calculation machinery to determine the output path (using recursive hashing strategy). In a real-world scenario, we obviously want to cache these calculations, but this should be sufficient to tinker around with it. Change-Id: I9b2e69384414f0be1bdcb5a99a4bfd46e8db9932 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8317 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-03-17 r/6027 fix(tvix/eval): use span of `set` for OpForce in attribute accessVincent Ambo1-2/+2
Emits the span of the `set` that is being accessed in the `force` operation of an attribute access. Looking at traces, it's a lot more useful to get information about *what* is being forced, as in cases like `foo.bar` it can be misleading to have an error highlight `bar`, when the error occured while forcing `foo` to be able to access `bar` in the first place. Change-Id: Id46ff28f20c67cb4971727ac52cc4811795cea2d Reviewed-on: https://cl.tvl.fyi/c/depot/+/8272 Reviewed-by: flokli <flokli@flokli.de> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-03-17 r/6026 feat(tvix/eval): report all known spans on infinite recursionVincent Ambo3-16/+80
This reports the span 1. of the code within a thunk, 2. of the place where the thunk was instantiated, 3. of the place where the thunk was first forced, 4. of the place where the thunk was forced again, when yielding an infinite recursion error, which hopefully makes it easier to debug them. The spans are tracked in the ThunkRepr::Blackhole variant when putting a thunk under evaluation. Note that we currently have some loss of span precision in the VM loop when switching between frame types, so spans 3/4 are currently a bit wonky. Working on it. Change-Id: Icbd2a9df903d00e8c2545b3fc46dcd2a9e3e3e55 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8270 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su>
2023-03-17 r/6025 feat(tvix/eval): track span of first force in a thunk blackholeVincent Ambo5-17/+48
This is step 1 towards being able to use all 4 spans that we know when dealing with infinite recursion. It tracks the span at which the force of a thunk was first requested when constructing a blackhole, so that we can highlight the spans of the first and second forces. These are actually the least relevant spans, but the easiest to put in place, more coming soon. Change-Id: I4c7e82f6211b98756439d4148a4191457cc46807 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8269 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-03-17 r/6024 feat(tvix/eval): add generator "name" to NativeError kindVincent Ambo3-12/+29
This produces traces in which we can see what kind of native code was run. Note that these "names" are named after the generator message, so these aren't *really* intended for end-user consumption, but we can give them saner names later. Example: https://gist.github.com/tazjin/82b24e92ace8e821008954867ee05057 This already makes the traces a little easier to parse. Change-Id: Idcd601baf84f492211b732ea0f04b377112e10d0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8268 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su>
2023-03-17 r/6023 feat(tvix/eval): enrich errors with VM's frame stack informationVincent Ambo4-129/+242
When emitting an error at runtime, the VM will now use the new `NativeError` and `BytecodeError` error kinds (which just wrap inner errors) to create a set of diagnostics to emit. The primary diagnostic is emitted last, with `error` type (so it will be coloured red in terminals), the other ones will be emitted with `note` type, highlighting the causal chain. Example: https://gist.github.com/tazjin/25feba7d211702453c9ebd5f8fd378e4 This is currently quite verbose, and we can cut down on this further, but the purpose of this commit is to surface more information first of all before worrying about the exact display. Change-Id: I058104a178c37031c0db6b4b3e4f4170cf76087d Reviewed-on: https://cl.tvl.fyi/c/depot/+/8266 Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-03-17 r/6022 fix(tvix/eval): use coerce_to_string in builtins.substringVincent Ambo3-1/+7
This actually uses coercion under the hood in C++ Nix. See the test for an example. Change-Id: Id56b364acf269225b6829d0b600e0222f8b3608d Reviewed-on: https://cl.tvl.fyi/c/depot/+/8322 Reviewed-by: andi <andi@notmuch.email> Tested-by: BuildkiteCI
2023-03-17 r/6020 chore(tvix/eval): remove some dead codeVincent Ambo1-31/+0
This was commented out and forgotten during the generator refactor, oh well. Change-Id: I474b685159a955a846db462da0dd0067af177b04 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8321 Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-03-17 r/6019 refactor(tvix/store/nar): pass in &proto::node::NodeFlorian Klink5-21/+29
Passing in a &proto::node::Node into all this allows us consumers to keep ownership of the proto::node::Node. Change-Id: I44882a86c46826b06a8a8a0b24c18adfc7052662 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8316 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de>
2023-03-17 r/6018 chore(tvix/store/grpcblobsvc): clippy lintFlorian Klink1-1/+1
Change-Id: Idb78e0417a962599cdfdef5e7346f7fa41e3fa1b Reviewed-on: https://cl.tvl.fyi/c/depot/+/8320 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-03-17 r/6017 chore(tvix/store/chunkservice): clippy lintFlorian Klink3-5/+5
Change-Id: I2073408fe6e87ffadbeb35ef2f9aaa0316c54966 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8319 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de>
2023-03-17 r/6016 chore(tvix/store/blobreader): clippy lintFlorian Klink1-1/+4
Change-Id: I4c85425782878295277abe6dc91a1b570768f7fa Reviewed-on: https://cl.tvl.fyi/c/depot/+/8318 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-03-16 r/6015 refactor(tvix/store/chunksvc): use [u8; 32] instead of Vec<u8>Florian Klink6-54/+70
Change-Id: Ie2b94aa5d69ff2c61fb77e13ae844f81f6270273 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8314 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de>
2023-03-16 r/6014 refactor(tvix/store/directorysvc): use [u8; 32] instead of Vec<u8>Florian Klink12-130/+128
Also, simplify the trait interface, only allowing lookups of Directory objects by their digest. Change-Id: I6eec28a8cb0557bed9b69df8b8ff99a5e0f8fe35 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8313 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-16 r/6013 chore(tvix): run `cargo update`Vincent Ambo2-656/+549
Change-Id: I710f0920da118cb1b94cd8ff364adadbeef998bb Reviewed-on: https://cl.tvl.fyi/c/depot/+/8315 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-03-15 r/6012 refactor(tvix/nix-compat): remove get_fixed_output() funcFlorian Klink1-66/+52
The behaviour of this function is a bit unintuitive, and cl/8310 already inlined the other consumer of it. Rewrite the last consumer of the function, so we can drop it. Change-Id: I59c8486037ce3f777667d1d9e4f4a9316d5a0cb9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8311 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-15 r/6011 refactor(tvix/nix-compat): simplify fingerprint calculationFlorian Klink1-76/+54
Instead of having two very similar match branches for the FOD and non- FOD case, detect the FOD case while looping over all outputs. In the case of anything other than recursive sha256 FODs, the fingerprint and output path calculation is exactly the same. Change-Id: Ieb6995653d008766e595cf29d7cd4fb1334e33dd Reviewed-on: https://cl.tvl.fyi/c/depot/+/8310 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de>
2023-03-15 r/6010 fix(tvix/cli): populate output configuration after walking argsVincent Ambo1-15/+14
Walking the arguments might encounter an `outputs` output, which might explicitly (for whatever reason) specify the `out` output. To prevent dropping FOD settings in this case, we have to populate that part of the configuration after walking the other attributes. Change-Id: Iee6a7f0a71e9c9699e79d35e6cb19e1ddb49395d Reviewed-on: https://cl.tvl.fyi/c/depot/+/8312 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-03-15 r/6009 refactor(tvix/nix-compat): -derivation::Hash, +NixHashFlorian Klink8-87/+201
This stops using our own custom Hash structure, which was mostly only used because we had to parse the JSON representation somehow. Since cl/8217, there's a `NixHash` struct, which is better suited to hold this data. Converting the format requires a bit of serde labor though, but that only really matters when interacting with JSON representations (which we mostly don't). Change-Id: Idc5ee511e36e6726c71f66face8300a441b0bf4c Reviewed-on: https://cl.tvl.fyi/c/depot/+/8304 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-14 r/6008 style(tvix): add new logo to the Tvix READMEVincent Ambo1-2/+5
Change-Id: Ibb8061be75d6fd16e8f813bc94e3bacaae890454 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8309 Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de>
2023-03-14 r/6007 chore(tvix/store): drop separate bin packageFlorian Klink5-29/+8
Apparently, having multiple packages with the same path is a bad thing: ``` The bin target `tvix-store` in package `tvix-store-bin v0.1.0 (/home/flokli/tvl/tvix/store)` has the same output filename as the lib target `tvix_store` in package `tvix-store-bin v0.1.0 (/home/flokli/tvl/tvix/store)`. Colliding filename is: /home/flokli/tvl/tvix/target/doc/tvix_store/index.html The output filenames should be unique. This is a known bug where multiple crates with the same name use the same path; see <https://github.com/rust-lang/cargo/issues/6313>. ``` Change-Id: Ic785c0349070783baf5e8fd23f5fb60603a3c995 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8308 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-14 r/6006 fix(tvix): build all tvix crate docsFlorian Klink1-1/+3
There's still some shadowing going on, but that's left for a followup CL. Change-Id: I02992f1eb494faca99857a3a5ee4dcd47f1b9fd0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8306 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-03-14 r/6004 feat(tvix): build (some of) our Rust documentation in NixVincent Ambo1-1/+32
This will be published on docs.tvix.dev Change-Id: I348e057351d5295ad20953c9e9a32c257abab089 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8298 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-03-14 r/6002 feat(tvix): add simple initial websiteVincent Ambo3-0/+68
The website itself contains no useful information that we don't already have published anywhere, but it's a decent landing place. Mostly I want to have something on the root of tvix.dev, so that we can start using it for other purposes. Change-Id: Id2b4dabc7f6e4dd26b61484b86dbde2f39aa1719 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8296 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-03-14 r/6000 refactor(tvix/nix-compat): rename helper funcFlorian Klink1-3/+3
This only trims the output paths from a Derivation struct, not the output hashes. Change-Id: I9250fec4602ed05bb64540c4a89ddb6fb052be1f Reviewed-on: https://cl.tvl.fyi/c/depot/+/8303 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de>
2023-03-14 r/5999 refactor(tvix/nix-compat): remove manual mapFlorian Klink1-8/+6
Change-Id: I1652e24c9be28112d98683de2d2db51dc46001ed Reviewed-on: https://cl.tvl.fyi/c/depot/+/8302 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-03-14 r/5998 refactor(tvix/nix-compat): rename test nameFlorian Klink1-1/+1
Change-Id: Ia6af8d4b53fbd4d9ea26f0094cd98dde66a91f78 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8301 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-03-14 r/5997 fix(tvix/store/derivation): fix commentFlorian Klink1-1/+1
The digest is encoded in LOWERHEX here, not nixbase32. Change-Id: I8e14742f0135966424073bfdb93ada8cd7bd52e2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8300 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-14 r/5996 chore(tvix/cli): don't set COREPKGS anymoreFlorian Klink1-4/+0
Since cl/8213, tvix bundles corepkgs. Change-Id: I17fa4452a6a1d554ae67b4aed9d9b1e77cb495f5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8305 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-03-14 r/5995 refactor(tvix/nix-compat): replace calculate_drv_replacement_strFlorian Klink4-71/+98
Call this function derivation_or_fod_hash, and return a NixHash. This is more in line with how cppnix calls this, and allows using to_nix_hash_string() in some places. Change-Id: Iebf5355f08ed5c9a044844739350f829f874f0ce Reviewed-on: https://cl.tvl.fyi/c/depot/+/8293 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-03-14 r/5994 docs(tvix/eval): suggested layout adjustment to VM loop diagramAdam Joseph1-11/+11
Change-Id: I5467cd66801ad8fe6c4ec0ae337763f1762cea1c Reviewed-on: https://cl.tvl.fyi/c/depot/+/8252 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: Adam Joseph <adam@westernsemico.com>
2023-03-14 r/5993 docs(tvix/eval): document inner workings of the new VM loopVincent Ambo1-45/+284
Change-Id: I8224bf039f739c401900b5a2ddc839810c87cf6e Reviewed-on: https://cl.tvl.fyi/c/depot/+/8226 Tested-by: BuildkiteCI Reviewed-by: Adam Joseph <adam@westernsemico.com>
2023-03-14 r/5992 chore(tvix): Generator{Request|Response} -> VM{Request|Response}Vincent Ambo3-130/+121
We settled on this being the most reasonable name for this construct. Change-Id: Ic31c45461a842f22aa05f4446123fe3a61dfdbc0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8291 Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-03-13 r/5991 chore(tvix/eval): mark async functions which are called by the VMAdam Joseph3-0/+9
Given Rust's current lack of support for tail calls, we cannot avoid using `async` for builtins. This is the only way to avoid overflowing the cpu stack when we have arbitrarily deep builtin/interpreted/builtin/interpreted/... "sandwiches" There are only five `async fn` functions which are not builtins (some come in multiple "flavors"): - add_values - resolve_with - force, final_deep_force - nix_eq, nix_cmp_eq - coerce_to_string These can be written iteratively rather than recursively (and in fact nix_eq used to be written that way!). I volunteer to rewrite them. If written iteratively they would no longer need to be `async`. There are two motivations for limiting our reliance on `async` to only the situation (builtins) where we have no other choice: 1. Performance. We don't really have any good measurement of the performance hit that the Box<dyn Future>s impose on us. Right now all of our large (nixpkgs-eval) tests are swamped by the cost of other things (e.g. fork()ing `nix-store`) so we can't really measure it. Builtins tend to be expensive operations anyways (regexp-matching, sorting, etc) that are likely to already cost more than the `async` overhead. 2. Preserving the ability to switch to `musttail` calls. Clang/LLVM recently got `musttail` (mandatory-elimination tail calls). Rust has refused to add this mainly because WASM doesn't support, but WASM `tail_call` has been implemented and was recently moved to phase 4 (standardization). It is very likely that Rust will get tail calls sometime in the next year; if it does, we won't need async anymore. In the meantime, I'd like to avoid adding any further reliance on `async` in places where it wouldn't be straightforward to replace it with a tail call. https://reviews.llvm.org/D99517 https://github.com/WebAssembly/proposals/pull/157 https: //github.com/rust-lang/rfcs/issues/2691#issuecomment-1462152908 Change-Id: Id15945d5a92bf52c16d93456e3437f91d93bdc57 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8290 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: Adam Joseph <adam@westernsemico.com>
2023-03-13 r/5990 refactor(tvix/eval): reduce fetch{forced|captured}_with visibilityAdam Joseph1-22/+22
This commit moves fetch_forced_with and fetch_captured_with into the scope of their only caller (resolve_with). Change-Id: I9a8bc27228888729d591e8cb021c431b2b6468f5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8289 Autosubmit: Adam Joseph <adam@westernsemico.com> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-03-13 r/5989 feat(tvix/eval): rewrite nix_cmp_ordering to be nonrecursiveAdam Joseph1-45/+46
This rewrites nix_cmp_ordering as an iterative loop, which eliminates the extra pinned-boxing helper function. Change-Id: I33d0ecc913e02affd8fd4c7bc1c9ecfdf4c7deb9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8288 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: Adam Joseph <adam@westernsemico.com>
2023-03-13 r/5988 chore(tvix/eval): prune some dependencies & featuresVincent Ambo4-597/+28
* We no longer need backtrace-on-stack-overflow, as we no longer overflow the stack with the recent eval refactorings. This was weird voodoo anyways, introduced earlier to debug some cases where stack overflows occured. * default features of genawaiter crate are not needed, as we don't use their proc macros Change-Id: I346fc5a18d7f117ee805909a8be8f535b96be76c Reviewed-on: https://cl.tvl.fyi/c/depot/+/8263 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-03-13 r/5987 refactor(tvix/eval): reorder bytecode operations match by frequencyVincent Ambo1-265/+264
This reorders the operations in the VM's main `match` statement while evaluating bytecode according to the frequency with which these operations appear in some nixpkgs evaluations. I used raw data that looks like this: https://gist.github.com/tazjin/63d0788a78eb8575b04defaad4ef610d This has a small but noticeable impact on evaluation performance. No operations have changed in any way, this is purely moving code around. Change-Id: Iaa4ef4f0577e98144e8905fec88149c41e8c315c Reviewed-on: https://cl.tvl.fyi/c/depot/+/8262 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-03-13 r/5986 chore(tvix/cli): silence useless warningVincent Ambo1-0/+1
Change-Id: I18f567d5dcbbeb934428bf9111eb5fae5fef0342 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8261 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-03-13 r/5985 docs(tvix/eval): fix reference to `Empty` message in a commentVincent Ambo1-1/+1
Change-Id: I3dc30cca33fbbd8e8686655635ee471f5937d9f8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8257 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-11-10T09·05+0000