about summary refs log tree commit diff
path: root/tvix/eval/src/observer.rs
AgeCommit message (Collapse)AuthorFilesLines
2024-01-31 r/7459 feat(tvix/eval/observer): Allow capturing timing of eventsAspen Smith1-0/+28
Add a new --trace-runtime-timing flag (probably a better bikeshed for this) that enables capturing the time, relative to the last event, of each event recorded with the tracing observer. This probably isn't *super* useful yet, but I'd like to start here in adding new profiling tools to the VM, specifically based on the runtime observer Change-Id: Id7f12077291c39bf3eef42ab6744bfba53687a65 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10713 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2023-08-20 r/6506 refactor(tvix/eval/observer): cargo clippyFlorian Klink1-1/+1
Change-Id: I11d7d55f31ddd27952f201a0abbed7445013f434 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9102 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-08-20 r/6505 refactor(tvix/eval): cargo clippyFlorian Klink1-1/+4
This passes a unit value to the function. Change-Id: I4df3ad8fb0f35c0f110cee3349971ae28ce2878c Reviewed-on: https://cl.tvl.fyi/c/depot/+/9101 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-03-14 r/5992 chore(tvix): Generator{Request|Response} -> VM{Request|Response}Vincent Ambo1-3/+3
We settled on this being the most reasonable name for this construct. Change-Id: Ic31c45461a842f22aa05f4446123fe3a61dfdbc0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8291 Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-03-13 r/5978 feat(tvix/eval): give generators human-readable namesVincent Ambo1-14/+22
This adds static strings to generator frames that describe the generator in a human-readable fashion, which are then logged in observers. This makes runtime traces very precise, explaining exactly what is being requested from where. Change-Id: I695659a6bd0b7b0bdee75bc8049651f62b150e0c Reviewed-on: https://cl.tvl.fyi/c/depot/+/8206 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-03-13 r/5973 fix(tvix/eval): don't print full stack in observerVincent Ambo1-1/+8
Print only the top 6 values of the stack, not the entire stack. There's very few operations that deal with more values anyways, so the rest are not likely to be useful. This gets us one step closer to tracing VERY large executions without blowing up. Change-Id: I97472321b0321b25d534d9f53b3aadfacc2318fa Reviewed-on: https://cl.tvl.fyi/c/depot/+/8201 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-03-13 r/5972 fix(tvix/eval): don't print full values in observerVincent Ambo1-1/+14
This can actually blow up when tracing arbitrary execution, as some of the data structures just get too large to run through a tabwriter. Change-Id: I6ec4c30ee48655b8a62954ca219107404fb2c256 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8200 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-03-13 r/5971 refactor(tvix/eval): condense observer's stack writing logicVincent Ambo1-53/+24
Change-Id: I1282c3387ac1e0d1528b894814f2a495ca5a6a32 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8199 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-03-13 r/5964 refactor(tvix/eval): flatten call stack of VM using generatorsVincent Ambo1-1/+1
Warning: This is probably the biggest refactor in tvix-eval history, so far. This replaces all instances of trampolines and recursion during evaluation of the VM loop with generators. A generator is an asynchronous function that can be suspended to yield a message (in our case, vm::generators::GeneratorRequest) and receive a response (vm::generators::GeneratorResponsee). The `genawaiter` crate provides an interpreter for generators that can drive their execution and lets us move control flow between the VM and suspended generators. To do this, massive changes have occured basically everywhere in the code. On a high-level: 1. The VM is now organised around a frame stack. A frame is either a call frame (execution of Tvix bytecode) or a generator frame (a running or suspended generator). The VM has an outer loop that pops a frame off the frame stack, and then enters an inner loop either driving the execution of the bytecode or the execution of a generator. Both types of frames have several branches that can result in the frame re-enqueuing itself, and enqueuing some other work (in the form of a different frame) on top of itself. The VM will eventually resume the frame when everything "above" it has been suspended. In this way, the VM's new frame stack takes over much of the work that was previously achieved by recursion. 2. All methods previously taking a VM have been refactored into async functions that instead emit/receive generator messages for communication with the VM. Notably, this includes *all* builtins. This has had some other effects: - Some test have been removed or commented out, either because they tested code that was mostly already dead (nix_eq) or because they now require generator scaffolding which we do not have in place for tests (yet). - Because generator functions are technically async (though no async IO is involved), we lose the ability to use much of the Rust standard library e.g. in builtins. This has led to many algorithms being unrolled into iterative versions instead of iterator combinations, and things like sorting had to be implemented from scratch. - Many call sites that previously saw a `Result<..., ErrorKind>` bubble up now only see the result value, as the error handling is encapsulated within the generator loop. This reduces number of places inside of builtin implementations where error context can be attached to calls that can fail. Currently what we gain in this tradeoff is significantly more detailed span information (which we still need to bubble up, this commit does not change the error display). We'll need to do some analysis later of how useful the errors turn out to be and potentially introduce some methods for attaching context to a generator frame again. This change is very difficult to do in stages, as it is very much an "all or nothing" change that affects huge parts of the codebase. I've tried to isolate changes that can be isolated into the parent CLs of this one, but this change is still quite difficult to wrap one's mind and I'm available to discuss it and explain things to any reviewer. Fixes: b/238, b/237, b/251 and potentially others. Change-Id: I39244163ff5bbecd169fe7b274df19262b515699 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8104 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Reviewed-by: Adam Joseph <adam@westernsemico.com> Tested-by: BuildkiteCI
2023-03-13 r/5962 feat(tvix/eval): add generator-related functions to RuntimeObserverVincent Ambo1-5/+87
These functions will be used by the changes in the VM to observe the runtime execution of generator frames, and provide a more linear view of the execution of the Tvix VM. Change-Id: I10b1b1933dedc065e7c61d5d6062f0aaeee0097e Reviewed-on: https://cl.tvl.fyi/c/depot/+/8240 Tested-by: BuildkiteCI Reviewed-by: Adam Joseph <adam@westernsemico.com>
2023-02-14 r/5851 fix(tvix/eval): correctly print lambda address in observerVincent Ambo1-1/+1
We want the address that the Rc is pointing to, not the address of the Rc. Change-Id: I8eba21677f242bbe4166c74d4aa4269c316076e3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8045 Reviewed-by: flokli <flokli@flokli.de> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-10-23 r/5186 feat(tvix/eval): initial attempt at setting lambda namesVincent Ambo1-4/+14
When compiling a lambda, take the name of the outer slot (if available) and store it as the name on the lambda. These names are then shown in the observer, and nowhere else (so far). It is of course common for these things to thread through many different context levels (e.g. `f = a: b: c: ...`), in this setup only the outermost closure or thunk gains the name, but it's better than nothing. Change-Id: I681ba74e624f2b9e7a147144a27acf364fe6ccc7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7065 Reviewed-by: grfn <grfn@gws.fyi> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-10-11 r/5099 feat(tvix/eval): observe stack after exiting call frames/builtinsVincent Ambo1-6/+18
Change-Id: I1937d37551503a0b6bb0ac899d067302e4791e5f Reviewed-on: https://cl.tvl.fyi/c/depot/+/6939 Reviewed-by: grfn <grfn@gws.fyi> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-10-05 r/5035 refactor(tvix/eval): introduce source::SourceCode typeVincent Ambo1-5/+5
This type hides away the lower-level handling of most codemap data structures, especially to library consumers (see corresponding changes in tvixbolt). This will help with implement `import` by giving us central control over how the codemap works. Change-Id: Ifcea36776879725871b30c518aeb96ab5fda035a Reviewed-on: https://cl.tvl.fyi/c/depot/+/6855 Tested-by: BuildkiteCI Reviewed-by: wpcarro <wpcarro@gmail.com>
2022-10-04 r/5033 refactor(tvix/eval): split observer traits in twoVincent Ambo1-10/+14
There are actually two different types of observers, the ones that observe the compiler (and emitted chunks from different kinds of expressions), and the ones that trace runtime execution. Use of the NoOpObserver is unchanged, it simply implements both traits. Change-Id: I4277b82674c259ec55238a0de3bb1cdf5e21a258 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6852 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-09-13 r/4838 refactor(tvix/eval): use CodeIdx wrapper for instruction pointerVincent Ambo1-3/+3
As suggested by sterni in cl/6453. Change-Id: I3cf80d97c11fd7d085ab510f6be4b5f937c791ec Reviewed-on: https://cl.tvl.fyi/c/depot/+/6562 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-11 r/4799 fix(tvix/eval): use correct lambda address in observerVincent Ambo1-1/+1
Instead of the reference to the Rc, print the address of the Rc itself. Change-Id: I4560598924db7d2864d5c4ae9af847aee2ea7eff Reviewed-on: https://cl.tvl.fyi/c/depot/+/6471 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-10 r/4783 feat(tvix/eval): implement OpTailCallVincent Ambo1-0/+12
If the last operation within a chunk is a function call, the call can be executed in the same call frame without increasing the depth of the call stack. To enable this, a new OpTailCall instruction (similar to OpCall) is introduced, but not yet emitted by the compiler. Change-Id: I9ffbd7da6d2d6a8ec7a724646435dc6ee89712f2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6457 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-09 r/4779 feat(tvix/eval): implement TracingObserver for runtime tracingVincent Ambo1-0/+54
This produces similar output to the previous tracing feature, but can redirect the output somewhere else. Change-Id: I9493c260f480904f3932cb74809b622c24d7be96 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6453 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-09 r/4778 feat(tvix/eval): implement runtime tracing methods for ObserverVincent Ambo1-1/+18
These methods make it possible to trace the runtime execution of the VM through an observer. Change-Id: I90e26853ba2fe44748613e7f761ed5c1c5fc9ff7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6452 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-09 r/4777 refactor(tvix/eval): move `disassemble_op` to the Chunk structureVincent Ambo1-2/+1
Change-Id: Ic6710c609ed647bfa47d673aaf22c4da96c0f319 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6451 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-09 r/4775 feat(tvix/eval): implement DisassemblingObserver for compilerVincent Ambo1-2/+64
This type implements an observer that is called whenever the compiler emits a chunk (after the toplevel, thunks, or lambdas) and prints the output of the disassembler to its internal writer. This replaces half of the uses of the `disassembler` feature, which has been removed from the Cargo configuration. Note that at this commit runtime tracing is not yet implemented as an observer. Change-Id: I7894ca1ba445761aba4ad51d98e4a7b6445f1aea Reviewed-on: https://cl.tvl.fyi/c/depot/+/6449 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-09 r/4774 feat(tvix/eval): implement (compilation) observer traitVincent Ambo1-0/+36
This trait will enable library users of tvix-eval to observe internal happenings of the compilation and runtime processes. The initial methods of the observer will be called whenever the compiler emits a chunk. Change-Id: I668f6c2cfe3d6f4c1a1612c0f293831011768437 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6448 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-11-10T08·10+0000