about summary refs log tree commit diff
path: root/nix/runTestsuite (follow)
AgeCommit message (Collapse)AuthorFilesLines
2021-09-17 r/2881 refactor(nix/runTestsuite): clean up runTestsuitesterni1-16/+6
* goodAss wasn't used before. Simplify it to just return a boolean, so we can use it for partitionTests later. * goodIt also returns unnecessary extra meta information which is not used. Cleaning that up makes the condition extremely small, so we can inline it into (what was) goodIts. * goodIts is just called in one place, so we can inline it into res. Change-Id: I70cf4fa3f61ce1467a2ee5319f841cdd42db6a66 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3548 Tested-by: BuildkiteCI Reviewed-by: Profpatsch <mail@profpatsch.de>
2021-09-14 r/2857 feat(nix/runTestsuite): rebuild if test case list changessterni1-2/+5
If the result of the assertions changes for a successful test suite (this happens if tests are reworded, added or removed), this makes sure the no-op derivation is rebuilt. This makes sure that test suites show up in buildkite on ocassions other than channel bumps, since they are only added to the job list if their `outPath` is missing nowadays (see cl/3427). Change-Id: Ia1050cca5eeed8b7da84c40f6154b40760a3047f Reviewed-on: https://cl.tvl.fyi/c/depot/+/3536 Tested-by: BuildkiteCI Reviewed-by: Profpatsch <mail@profpatsch.de>
2021-03-26 r/2336 fix(nix/runTestsuite): use s6-portable-utilsProfpatsch1-2/+3
Same as 221698c603dcb318c609b4d21cb2a9fada44a14c We had a bunch of instances of https://github.com/NixOS/nix/issues/2176, where nix would exit with a “killed by signal 9” error. According to Eelco in that issue, this is perfectly normal behaviour of course, and appears if the last command in a loop closes `stdout` or `stdin`, then the builder will SIGKILL it immediately. This is of course also a perfectly fine error message for that case. It turns out that mainly GNU coreutils exhibit this behaviour … Let’s see if using a more sane tool suite fixes that. Change-Id: Iaf9e542952ca36c02208a3f067f575ba978272b4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2663 Reviewed-by: Profpatsch <mail@profpatsch.de> Tested-by: BuildkiteCI
2021-02-09 r/2198 feat(nix/runTestsuite): add assertDoesNotThrowsterni1-0/+7
assertDoesNotThrow is like assertThrows, but fails if the expression throws. In that case the new unexpected-throw branch of AssertErrorContext is returned. Change-Id: I7195eb5df8965456e9ab9b69e35ec96b33f00a35 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2476 Tested-by: BuildkiteCI Reviewed-by: Profpatsch <mail@profpatsch.de>
2021-02-09 r/2197 refactor(nix/runTestsuite): implement in terms of generic assertBoolsterni1-26/+46
Refactor assertEq and assertThrows to be implemented in terms of a more generic assertBool to reduce code duplication and ease adding new assert types. To preserve meaningful error messages AssertResult is changed slightly: nope-eq and nope-throw have been replaced by a single nope branch which contains an AssertErrorContext which contains error information. To implement an assert assertBoolContext (which is not exposed) can be used: It takes an AssertErrorContext which is returned in case of an error and a boolean determining whether the assert was successful. The currently possible AssertErrorContext are: * should-throw: error result of assertThrows, formerly nope-throw * not-equal: error result of assertEq, formerly nope-eq Change-Id: Ifd6b3aa4187c90c3add2df63fa7c906c8f03fd2d Reviewed-on: https://cl.tvl.fyi/c/depot/+/2473 Tested-by: BuildkiteCI Reviewed-by: Profpatsch <mail@profpatsch.de>
2021-01-30 r/2162 feat(nix/runTestsuite): add assertThrowsProfpatsch1-4/+22
Uses `builtins.tryEval` to check that the expression throws when `deepSeq`-ed. Change-Id: I0d57cc37f473bb733f57a1b1c0d889084152fd2f Reviewed-on: https://cl.tvl.fyi/c/depot/+/2463 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2021-01-30 r/2161 fix(nix/runTestsuite): wrap runTestsuite into derivationProfpatsch1-8/+26
Previously we would throw or return `{}`, which doesn’t integrate nicely into our CI; thus, let’s wrap it into a derivation which either fails the build or doesn’t. Change-Id: I65880d86b8393094661e57a0b32aafe748bf1dd5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2462 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2020-06-28 r/1104 feat(nix/runTestsuite): add runTestsuiteProfpatsch1-0/+121
This is a very simple test suite for nix expressions. It should help us set up a good suite of unit tests for our nix-based stuff. Since we allow import from derivation, these tests can also depend on derivations and e.g. use `builtins.readFile` to check outputs. This is a first PoC to get us going, we can always replace it by something different in the future if we don’t like it. Change-Id: I206c7b624db2b1dabd9c73ffce4f87e658919958 Reviewed-on: https://cl.tvl.fyi/c/depot/+/662 Reviewed-by: tazjin <mail@tazj.in> Tested-by: tazjin <mail@tazj.in>