From fd49dfa6ca9db0f3a026a581ae2a14f760ac251d Mon Sep 17 00:00:00 2001 From: Florian Klink Date: Fri, 10 May 2024 08:46:38 +0300 Subject: docs(tvix/todo): update test suite section nix_oracle is quite misleading, and fooled me into thinking this validates all .exp files in the repo to match Nix output (which it doesn't, that's done by verify-lang-tests). Also, the whole test suite structure thing can use a bit more explanation. Change-Id: I2fadcc871843143270ad2ed9ac98de8287280e6f Reviewed-on: https://cl.tvl.fyi/c/depot/+/11609 Tested-by: BuildkiteCI Autosubmit: flokli Reviewed-by: Connor Brewster --- tvix/docs/src/TODO.md | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) (limited to 'tvix/docs/src/TODO.md') diff --git a/tvix/docs/src/TODO.md b/tvix/docs/src/TODO.md index 6644bb6bac03..50b41e3e50bd 100644 --- a/tvix/docs/src/TODO.md +++ b/tvix/docs/src/TODO.md @@ -10,11 +10,20 @@ Feel free to add new ideas. Before picking something, ask in `#tvix-dev` to make sure noone is working on this, or has some specific design in mind already. ## Cleanups +### Nix language test suite + - Think about how to merge, but "categorize" `tvix_tests` in `glue` and `eval`. + We currently only have this split as they need a different feature set / + builtins. - move some of the rstest cases in `tvix-glue` to the `.nix`/`.exp` mechanism. - - Parts requiring test fixtures need some special convention. - Some of these also cannot be checked into the repo, like the import tests - adding special files to test filtering. - - add `nix_oracle` mechanism from `tvix-eval` to `tvix-glue`. + Some of them need test fixtures, which cannot be represented in git (special + file types in the import tests for example). Needs some support from the test + suite to create these fixtures on demand. + - extend `verify-lang-tests/default.nix` mechanism to validate `tvix-eval` and + `tvix-glue` test cases (or the common structure above). + - absorb `eval/tests/nix_oracle.rs` into `tvix_tests`, or figure out why it's + not possible (and document) it. It looks like it's only as nix is invoked + with a different level of `--strict`, but the toplevel doc-comment suggests + its generic? ## Fixes towards correctness - `builtins.toXML` is missing string context. See b/398. -- cgit 1.4.1