depot - monorepo for the virus lounge

Age	Commit message (Collapse)	Author	Files	Lines
2023-03-13	r/5982 fix(tvix/eval): more closely line up path resolution with cppnix	Vincent Ambo	1	-1/+4
	... except now the tests fail, but at least it works Change-Id: I05e86c173f40533ae65548585c1ddaa200ac5235 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8214 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-03-13	r/5980 refactor(tvix/eval): VM struct no longer needs to be public	Vincent Ambo	1	-1/+1
	Change-Id: I93b485ddd280cc15fcbaecf4aed5fcd22e28a8a8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8212 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-03-13	r/5979 fix(tvix/eval): implement cppnix JSON-serialisation semantics	Vincent Ambo	1	-0/+23
	This drops the usage of serde::Serialize, as the trait can not be used to implement the correct semantics (function colouring!). Instead, a manual JSON serialisation function is written which correctly handles toString, outPath and other similar weirdnesses. Unexpectedly, the eval-okay-tojson test from the C++ Nix test suite now passes, too. This fixes an issue where serialising data structures containing derivations to JSON would fail. Change-Id: I5c39e3d8356ee93a07eda481410f88610f6dd9f8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8209 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-03-13	r/5978 feat(tvix/eval): give generators human-readable names	Vincent Ambo	3	-35/+56
	This adds static strings to generator frames that describe the generator in a human-readable fashion, which are then logged in observers. This makes runtime traces very precise, explaining exactly what is being requested from where. Change-Id: I695659a6bd0b7b0bdee75bc8049651f62b150e0c Reviewed-on: https://cl.tvl.fyi/c/depot/+/8206 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-03-13	r/5974 refactor(tvix/eval): print only types when observing generators	Vincent Ambo	1	-7/+11
	Do not print the entire value (they're likely to be thunks anyways). This is useful because there can be cases where something like `nixpkgs` itself is sent through one of these messages, in which case the observer trying to print it will just blow up. Change-Id: I1fa37ea071d75efa0eb3428c6e2fe4351c62be6b Reviewed-on: https://cl.tvl.fyi/c/depot/+/8202 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-03-13	r/5969 refactor(tvix/eval): box PathBuf	Vincent Ambo	1	-2/+2
	This shaves another 8 bytes off Value. How did that type get so big?! Change-Id: I65e9b59a1636bd57e3cc4aec5fea16887070b832 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8153 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-03-13	r/5964 refactor(tvix/eval): flatten call stack of VM using generators	Vincent Ambo	3	-51/+1424
	Warning: This is probably the biggest refactor in tvix-eval history, so far. This replaces all instances of trampolines and recursion during evaluation of the VM loop with generators. A generator is an asynchronous function that can be suspended to yield a message (in our case, vm::generators::GeneratorRequest) and receive a response (vm::generators::GeneratorResponsee). The `genawaiter` crate provides an interpreter for generators that can drive their execution and lets us move control flow between the VM and suspended generators. To do this, massive changes have occured basically everywhere in the code. On a high-level: 1. The VM is now organised around a frame stack. A frame is either a call frame (execution of Tvix bytecode) or a generator frame (a running or suspended generator). The VM has an outer loop that pops a frame off the frame stack, and then enters an inner loop either driving the execution of the bytecode or the execution of a generator. Both types of frames have several branches that can result in the frame re-enqueuing itself, and enqueuing some other work (in the form of a different frame) on top of itself. The VM will eventually resume the frame when everything "above" it has been suspended. In this way, the VM's new frame stack takes over much of the work that was previously achieved by recursion. 2. All methods previously taking a VM have been refactored into async functions that instead emit/receive generator messages for communication with the VM. Notably, this includes all builtins. This has had some other effects: - Some test have been removed or commented out, either because they tested code that was mostly already dead (nix_eq) or because they now require generator scaffolding which we do not have in place for tests (yet). - Because generator functions are technically async (though no async IO is involved), we lose the ability to use much of the Rust standard library e.g. in builtins. This has led to many algorithms being unrolled into iterative versions instead of iterator combinations, and things like sorting had to be implemented from scratch. - Many call sites that previously saw a `Result<..., ErrorKind>` bubble up now only see the result value, as the error handling is encapsulated within the generator loop. This reduces number of places inside of builtin implementations where error context can be attached to calls that can fail. Currently what we gain in this tradeoff is significantly more detailed span information (which we still need to bubble up, this commit does not change the error display). We'll need to do some analysis later of how useful the errors turn out to be and potentially introduce some methods for attaching context to a generator frame again. This change is very difficult to do in stages, as it is very much an "all or nothing" change that affects huge parts of the codebase. I've tried to isolate changes that can be isolated into the parent CLs of this one, but this change is still quite difficult to wrap one's mind and I'm available to discuss it and explain things to any reviewer. Fixes: b/238, b/237, b/251 and potentially others. Change-Id: I39244163ff5bbecd169fe7b274df19262b515699 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8104 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Reviewed-by: Adam Joseph <adam@westernsemico.com> Tested-by: BuildkiteCI
2023-03-13	r/5963 feat(tvix/eval): implement generator-based Nix equality logic	Vincent Ambo	1	-5/+1
	Adds a `Value::neo_nix_eq` method (the `neo_` prefix will be dropped when we flip over to the generator implementation of the VM) which implements Nix equality semantics using async, generator-based comparisons. Instead of tracking the "kind" of equality that is being compared (see the pointer-equality doc) through a pair of booleans, I've introduced an enum that explicitly lists the possible comparisons. Change-Id: I3354cc1470eeccb3000a5ae24f2418db1a7a2edc Reviewed-on: https://cl.tvl.fyi/c/depot/+/8241 Tested-by: BuildkiteCI Reviewed-by: Adam Joseph <adam@westernsemico.com>
2023-03-07	r/5899 feat(tvix/eval): introduce generators module	Vincent Ambo	1	-0/+538
	This module contains the request/response types for generators requesting actions from the VM. For most of these, an async helper function is added that will be used inside of generator functions to make use of these requests/responses instead of constructing them directly. Change-Id: I1e085f88adaf784a34867957a0e82532d3a83d7c Reviewed-on: https://cl.tvl.fyi/c/depot/+/8148 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>