about summary refs log tree commit diff
path: root/tvix/verify-lang-tests
AgeCommit message (Collapse)AuthorFilesLines
2024-03-19 r/7735 fix(tvix/eval): don't force lambda arg in map, mapAttrs & genListsterni1-1/+4
It is pretty pointless to force the function argument if we are going to use a suspended call later since forcing the function may fail in ways that are not covered by Catchables (non-recoverable errors, infinite recursions). From this, it kind of seems as if using #[catch] is never correct and should be replaced by #[lazy]. Also we should probably try to come up with more test cases for stuff where laziness gets us out of the jam as an equivalent to the catchable tests for nonrecoverable errors. Fixes b/386. Change-Id: Ia926df4ac1b440ec430403ab7b40924a0c97221b Reviewed-on: https://cl.tvl.fyi/c/depot/+/11153 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org>
2024-02-10 r/7495 feat(tvix/eval): strengthen significantly catchable test suiteRyan Lahfa1-0/+7
Adds a bunch (notably certain overlapping) tests for catchable situations. This should cover many scenarios, argument is catchable, element in argument is catchable, function returns catchable in the middle of the processing, etc. Co-authored-by: Aspen Smith <root@gws.fyi> Change-Id: Icd722cf8dbc91a24f45cd540a328711e5826f76c Reviewed-on: https://cl.tvl.fyi/c/depot/+/10621 Reviewed-by: aspen <root@gws.fyi> Tested-by: BuildkiteCI
2023-12-12 r/7165 test(tvix/eval): add test case for b/339Adam Joseph1-0/+1
Not yet passing. Change-Id: I1de3f72d8b3f46567fdba010fc3ab4bace3f1699 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10219 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: Adam Joseph <adam@westernsemico.com> Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2023-09-15 r/6589 fix(tvix/eval): update identifier quoting to match cppnix 2.17Adam Joseph1-3/+3
In cppnix 2.17, commit b72bc4a972fe568744d98b89d63adcd504cb586c, the libexpr pretty-printing routine was fixed so that it would no longer pretty-print attrsets with keywords in their attrnames incorrectly. This commit implements the corresponding fix for tvix, fixes our tests to work with cppnix>=2.17 oracles, and expands our test cases to cover all the keywords. Change-Id: I4b51389cd3a9c44babc8ab2a84b383b7b0b116ca Reviewed-on: https://cl.tvl.fyi/c/depot/+/9283 Autosubmit: Adam Joseph <adam@westernsemico.com> Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2023-09-06 r/6560 chore(3p/sources): bump channels & overlays (2023-08-21)Vincent Ambo1-4/+5
Included changes: * tvix/eval: enable some lang tests on nix_latest Nix 2.16 contains some breaking language changes which Tvix does not yet implement, but the existing tests for them are now passed by Nix 2.16 (but not yet by Tvix). * tvix/eval: disable a lang test on nix_latest In Nix 2.17, the identifier formatting test fails because some behaviour changed. We have not investigated further yet. * 3p/overlays: use version of ihp-hsx that works with GHC 9.4 Originally from the separate cl/9185. * top-level: introduce a mechanism to exclude build targets from CI in the top level. This fixes b/296. * users/grfn: disable builds of xanthous (and dependents) until the CLs fixing its build are submitted * 3p/overlays: build nixos-option against Nix 2.15, the only version with which it builds * 3p/overlays: bump tdlib to 1.8.16 Change-Id: Ia377f39dbdb08ac45ff830a615e64babc091e5ee Reviewed-on: https://cl.tvl.fyi/c/depot/+/9125 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su>
2023-06-15 r/6314 test(tvix/eval): update nix_tests suite to C++ Nix mastersterni1-0/+12
Adds new tests for foldl', intersectAttrs as well as fills in missing .exp files. New test cases we don't pass: - fromTOML timestamp capabilities - path antiquotation - replaceStrings is lazier on C++ Nix master The C++ Nix revision used is 7066d21a0ddb421967980094222c4bc1f5a0f45a. Change-Id: Ic619c96e2d41e6c5ea6fa93f9402b12e564af3c5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8778 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2023-06-15 r/6309 test(tvix/eval): genericClosure (pointer) comparison supportsterni1-0/+1
genericClosure has very limited support for pointer equality: It relies on comparison (not equality!) in C++ Nix, so as soon as C++ Nix supports comparing lists (langVersion >= 6) we can rely on pointer equality for key. Since Tvix uses equality, not comparison for the insert, our behavior is currently different, as documented by the notyetpassing tests. Change-Id: Ifcd741ed4fc3ccc3825f7038875d56a9918b786a Reviewed-on: https://cl.tvl.fyi/c/depot/+/8720 Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-15 r/6308 fix(tvix/eval): make tvix display values like nix-instantiate(1)sterni1-0/+4
In order for the test suite we have currently to be comparable to C++ Nix, we need to display values in the same way. This was largely the case except in some weird cases. * <CODE> for thunks and <CYCLE> for repeated thunks (?) are already in use. <CODE> formatting is tested by the oracle test suite already. * Instead of lambda, we need to use <LAMBDA> * <<primop>> and <<primop-app>> (a formatting C++ Nix uses nowhere) now are <PRIMOP> and <PRIMOP-APP>. We'll probably want to have a fancier display of values (in a separate trait) down the line. This could be used for interactive usage, e.g. the REPL or a potential debugger. There is a peculiarity with C++ Nix 2.3 formatting primops: import is considered a <<PRIMOP-APP>>, since it is internally implemented by means of scopedImport. This implementation detail no longer leaks in C++ Nix 2.13 nor in Tvix. <CYCLE> display is untested at the moment, since we exhibit a discrepancy to C++ Nix 2.3. Our current detection is more similar to C++ Nix 2.13—luckily it is also the more consistent of the two. See also b/245. Change-Id: I1d534434b02e470bf5475b3758920ea81e3420dc Reviewed-on: https://cl.tvl.fyi/c/depot/+/8760 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2023-01-06 r/5594 test(tvix/eval): add test for builtins paritysterni1-1/+1
This will eventually force us to have a base builtins set in common with C++ Nix, i.e. all 2.3 builtins except the controversial builtins.valueSize. Change-Id: I2c767f07d6a14711911658e87da9f18ede57a143 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7747 Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-12-02 r/5368 test(tvix/eval): verify pointer equality in list comparisonssterni1-0/+1
Change-Id: I617d402c8ecc7aaf607c4bdcd58a06ebddb71fac Reviewed-on: https://cl.tvl.fyi/c/depot/+/7370 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: Adam Joseph <adam@westernsemico.com>
2022-11-04 r/5242 test(tvix/eval): add a test case for groupBy with thunksAdam Joseph1-0/+2
We have to be careful implementing `builtins.groupBy`, since the list may contain thunks, and tvix's to_xxx() functions do not work on thunks. Signed-off-by: Adam Joseph <adam@westernsemico.com> Change-Id: I182b6fc2d4296f864ed16744ef70b153e8e6978a Reviewed-on: https://cl.tvl.fyi/c/depot/+/7039 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-10-29 r/5221 feat(tvix/eval): Implement comparison for listsGriffin Smith1-0/+1
Lists are compared lexicographically in C++ nix as of [0], and our updated nix test suites depend on this. This implements comparison of list values in `Value::nix_cmp` using a very similar algorithm to what C++ does - similarly to there, this requires passing in the VM so we can force thunks in the list elements as we go. [0]: https://github.com/NixOS/nix/commit/09471d2680292af48b2788108de56a8da755d661# Change-Id: I5d8bb07f90647a1fec83f775243e21af856afbb1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7070 Autosubmit: grfn <grfn@gws.fyi> Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-10-25 r/5198 feat(tvix/eval): add builtins.{floor,ceil}James Landrein1-0/+2
Change-Id: I4e6c4f96f6f5097a5c637eb3dbbd7bb8b34b7d52 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7032 Autosubmit: j4m3s <james.landrein@gmail.com> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: grfn <grfn@gws.fyi>
2022-10-20 r/5170 test(tvix/verify-lang-tests): also check notyetpassing testssterni1-12/+12
Adding these to the C++ Nix CI ensures that it is possible for the tests in that directory to pass at all. Mainly this would catch situations like fixed in a previous CL when moving the tests around would break them so that they wouldn't even pass in C++ Nix. For this to work, we need to track skips by basename to be directory-independent (assuming that every skipped test name is unique is hopefully okay). Change-Id: If6cb63ebdef0fc19b082b6a04e79ada2e47c658e Reviewed-on: https://cl.tvl.fyi/c/depot/+/7048 Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-20 r/5164 test(tvix/eval): update C++ Nix test suite to current mastersterni1-1/+9
The language test suite actually doesn't require flakes and the new features are mostly sensible (added builtins) as well as some tests for regressions the C++ implementation experienced. The path interpolation test is not included in this update because there is no way to construct an location-independent .exp file for it (the C++ repo also doesn't have one). We may still want to implement that feature eventually (in case rnix adds support for it). The C++ Nix revision used is ac0fb38e8a5a25a84fa17704bd31b453211263eb. Change-Id: I75f1e780ddeeee6f6b1f28cf3c66c288dca2c20c Reviewed-on: https://cl.tvl.fyi/c/depot/+/7043 Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-20 r/5162 test(tvix): nix-planned test verification using C++ Nix 2.3 and 2.11sterni1-0/+183
Reimplement the test discovery of the lang tests script in Nix which allows for a more flexible skipping logic that can e.g. react to the C++ Nix version used. This allows us to run the test suite against both C++ Nix 2.3 and the latest C++ Nix version 2.11. The latter is mainly useful, so we can implement newer Nix features and still verify them against the C++ implementation. Change-Id: I30c802844133b86b5e49f5e4f4fefacdb6215e0e Reviewed-on: https://cl.tvl.fyi/c/depot/+/7042 Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-11-01T22·30+0000