about summary refs log tree commit diff
path: root/nix
AgeCommit message (Collapse)AuthorFilesLines
2021-09-09 r/2830 feat(nix/sparseTree): get a directory with only selected childrensterni2-0/+65
Given a path (which points to a directory and a list of paths which are below that path, build a “sparse” version of that directory, so that it only contains the listed paths (and their children): $ nix-build -E 'with import ./. {}; nix.sparseTree ./. [ ./default.nix ./nix/readTree ./nix/buildLisp ./third_party/nixpkgs ./third_party/overlays ]' /nix/store/0ynj0gc613fs6mfp9snqcvdj5gfxbdzg-sparse-depot $ lr -t 'type == d' result/ result/ result/nix result/nix/buildLisp result/nix/buildLisp/example result/nix/readTree result/nix/readTree/tests […] result/third_party result/third_party/nixpkgs result/third_party/overlays result/third_party/overlays/haskell result/third_party/overlays/haskell/patches result/third_party/overlays/patches This is useful if a derivation depends on depot.path (e. g. if it wants to import depot at runtime). Usually this means that on every depot commit (or even worse, every change of .git on a local machine), this derivation has to be rebuild. By using sparseTree you can instead depend on a stripped down version of depot which only contains the bits you actually depend on, avoiding unrelated rebuilds. Change-Id: I127b108c8b177c657fb46786d0a6256516fd2c52 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3503 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-09-09 r/2829 refactor(nix/readTree): use throw instead of assertMsgsterni1-4/+3
This gives a slightly prettier error message and won't leak the error message when builtins.tryEval is used. Currently an error message from the tests is always part of the pipeline evaluation log. Change-Id: I9b488a440368091ed42d707ba4124f592a64bd86 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3502 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-09-09 r/2825 feat(readTree): Support scoped import argumentsVincent Ambo2-9/+15
This makes it possible to override Nix builtins within a readTree structure. Why would you want to do that, you might ask? Well ... Change-Id: Icc9cb32e5db4a2eba370cf81769c642d237d4937 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3499 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2021-09-08 r/2824 refactor(readTree): Pass all readTree parameters as function argsVincent Ambo3-20/+47
Instead of having a mix of depot-passed args (for the filter) and args to the readTree function itself, make everything a single attribute set of arguments passed to the function. This also makes it a bit easier to extend this in the future. Change-Id: I633c1fc96026d137b451bb604ef92be32571a0f5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3498 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2021-08-29 r/2800 refactor(bufCheck): Assume bufCheck is run in depot checkoutVincent Ambo1-2/+2
Rather than copying the depot path into the store on each commit, assume bufCheck is run in the depot checkout (which it is, in Buildkite land). Change-Id: I4a4af2e5b45acad2d18218e503880ee63b20f078 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3462 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2021-08-26 r/2792 feat(readTree): Add support for path-dependent args filteringVincent Ambo1-2/+7
Adds another argument to readTree itself which can be passed when importing readTree (e.g. in our default.nix) to filter the arguments passed to a target based on that target's location in the tree. This is intentionally not yet mentioned in the docs, and also intentionally implemented in such a way that the API surface of readTree doesn't change. The reason for this is that I want to figure out whether these filter functions are actually useful, e.g. within depot by filtering user-folder passing, and then refactor the readTree API to find a public way of exposing this as part of the readTree function itself (and not its import). Relates to b/143. Change-Id: I2cdf09f67916527d2337f4bfb578749aeac51a6a Reviewed-on: https://cl.tvl.fyi/c/depot/+/3433 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi> Reviewed-by: sterni <sternenseemann@systemli.org>
2021-08-26 r/2779 docs(nix/buildLisp): document NIX_BUILDLISP_LISP_ARGSsterni1-0/+24
Change-Id: I54294da08ee08bcf6cba6c792a2a2235b988a778 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3422 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-08-26 r/2778 docs(nix/buildLisp): move example further upsterni1-29/+29
Make "Example" the second section again since it got a bit buried under a lot of detailed documentation you won't necessarily need right away. Change-Id: I481354d1761c590e5872dfce8c3cf9934e278673 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3421 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2021-08-26 r/2777 fix(nix/buildLisp): prevent image loader from parsing argumentssterni1-3/+22
CCL and SBCL create executables by dumping their image. As a consequence, some part of the respective compiler is embedded in the resulting executable which is executed and loads the image. For CCL and SBCL this piece of software seems to unconditionally parse arguments which can't be prevented since it happens before any lisp is loaded. Luckily in both cases the parsing stops at `--`, so we can just pass this via the wrapper — we just need to work around the problem that this will of course be left in argv and confuse any later code. This can be rectified by deleting everything prior to the first `--` in the global argument list on startup in both cases. In cases we do want to pass arguments to the image loader, we can use the special NIX_BUILDLISP_LISP_ARGS environment variable which is understood by the wrapper. Note: This fix doesn't interfere with ECL since it is not using the wrapper script at the moment. Fixes b/136. Change-Id: I3f95aa61e945e51428021ca18232ff15c923f870 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3357 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: grfn <grfn@gws.fyi>
2021-08-24 r/2774 docs(nix/buildLisp): document recent changes to buildLispsterni1-10/+119
Doing this in a separate CL to avoid having to track the intermediate changes no one will ever see in documentation as well which would be unnecessary effort. * Multi-implementation support introduced in cl/3292 and refined in cl/3368 in terms of the user interface. * Implementation specific srcs and deps introduced in cl/3321 * Implementation passthru attrs and rename from .sbcl -> .repl was done in cl/3359 * ECL added in cl/3297, CCL in cl/3350 Change-Id: Ia13f2aea4e7e091c00991fcbfc601de364413979 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3380 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-08-24 r/2773 feat(nix/buildLisp): pass implementation description instead of namesterni1-30/+32
Instead of using a string to refer to an internal set defined in buildLisp, we just expose the relevant sets (as nix.buildLisp.sbcl, nix.buildLisp.ecl, …) and receive them as the `implementation` argument directly. This has several advantages: * It becomes easier to extend buildLisp, even for downstream users: Since you can just pass your own set, there's nothing stopping you from adding support for another implementation in a downstream derivation without having to edit the buildLisp file in any way which is great if you're using e. g. builtins.fetchGit to import it. * Users can mess with the implementation set by changing out some parts of it for customization purposes. Note that currently the sets use a lot of self-references which aren't even bound by a fix-point, so to make this work smoothly, we'd need to add some overriding mechanism. * The buildLisp code becomes quite a bit clearer. Since we're now always dealing with the implementation set, the confusing distinction between `impl`, `impl.name` and `implementation` no longer exists. `impl` is now exclusively an abbreviation of `implementation` (we could make this more consistent in the future even). Change-Id: I36d68069dd1315610b2f7159941507b465469b7c Reviewed-on: https://cl.tvl.fyi/c/depot/+/3368 Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2021-08-24 r/2772 feat(nix/buildLisp): add cclsterni1-1/+103
This adds support for Clozure's CL implementation to buildLisp. This is quite trivial in comparison to ECL since SBCL and CCL have very similar in how they work (so much so that CCL also suffers from b/136). Also the similarities in the code actually added here are striking, so I'll try to make an effort to reduce the code duplication in the future. To fix builds with CCL the following changes were made: * //3p/lisp/nibbles: The double inclusion of the types.lisp file was fixed. CCL doesn't like double definitions and refuses to compile otherwise. * //3p/lisp/physical-quantities: Update to a new bug fix release which contains a compilation fix for CCL. * //3p/lisp/routes: apply a patch fixing the build which was previously failing due to a double definition. * //3p/lisp/usocket: only depend on sb-bsd-sockets for SBCL and ECL, the latter of which seems to have a SBCL compatible implementation of the package. * Conditionally include a few CCL-specific source files and add `badImplementation` entries for the remaining failures which are //fun/gemma (to be expected) and //web/panettone which fails with an incredibly vague message. Change-Id: I666efdc39a0f16ee1bb6e23225784c709b04e740 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3350 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-08-24 r/2771 feat(nix/buildLisp): add eclsterni1-4/+157
Adds ECL as a second supported implementation, specifically a statically linked ECL. This is interesting because we can create statically linked binaries, but has a few drawbacks which doesn't make it generally useful: * Loading things is very slow: The statically linked ECL only has byte compilation available, so when we do load things or use the REPL it is significantly worse than with e. g. SBCL. * We can't load shared objects via the FFI since ECL's dffi is not available when linked statically. This means that as it stands, we can't build a statically linked //web/panettone for example. Since ECL is quite slow anyways, I think these drawbacks are worth it since the biggest reason for using ECL would be to get a statically linked binary. If we change our minds, it shouldn't be too hard to provide ecl-static and ecl-dynamic as separate implementations. ECL is LGPL and some libraries it uses as part of its runtime are as well. I've outlined in the ecl-static overlay why this should be of no concern in the context of depot even though we are statically linking. Currently everything is building except projects that are using cffi to load shared libaries which have gotten an appropriate `badImplementations` entry. To get the rest building the following changes were made: * Anywhere a dependency on UIOP is expressed as `bundled "uiop"` we now use `bundled "asdf"` for all implementations except SBCL. From my testing, SBCL seems to be the only implementation to support using `(require 'uiop)` to only load the UIOP package. Where both a dependency on ASDF and UIOP exists, we just delete the UIOP one. `(require 'asdf)` always causes UIOP to be available. * Where appropriate only conditionally compile SBCL-specific code and if any build the corresponding files for ECL. * //lisp/klatre: Use the standard condition parse-error for all implementations except SBCL in try-parse-integer. * //3p/lisp/ironclad: disable SBCL assembly optimization hack for all other platforms as it may interfere with compilation. * //3p/lisp/trivial-mimes: prevent call to asdf function by substituting it out of the source since it always errors out in ECL and we hardcode the correct path elsewhere anyways. As it stands ECL still suffers from a very weird problem which happens when compiling postmodern and moptilities: https://gitlab.com/embeddable-common-lisp/ecl/-/issues/651 Change-Id: I0285924f92ac154126b4c42145073c3fb33702ed Reviewed-on: https://cl.tvl.fyi/c/depot/+/3297 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: eta <tvl@eta.st>
2021-08-24 r/2770 feat(nix/buildLisp): expose drvs built w/ the other implementationssterni1-10/+48
For every implementation we support an extra passthru attribute with the name of the implementation is created which points to a version of the derivation built with that implementation. E. g. if we support CCL, ECL and SBCL, third_party.lisp.alexandria would have: * third_party.lisp.alexandria.sbcl * third_party.lisp.alexandria.ecl * third_party.lisp.alexandria.ccl To make this possible, the REPL derivation which was called `sbcl` originally has been renamed to `repl`. Since some things won't build with all implementations, we introduce a brokenOn argument which influences the meta.targets list that is created, but won't prevent the passthru attrs from being created to ease debugging failures. Change-Id: Icd6af345143593fac30ded10deabf31172e5d48a Reviewed-on: https://cl.tvl.fyi/c/depot/+/3359 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-08-24 r/2769 feat(nix/buildLisp): implementation specific deps and srcssterni1-13/+67
Both the deps and srcs arguments may now have special “filter sets” in the lists they receive as arguments. When building, buildLisp checks if such sets either have a attribute named like the current implementation or a "default" attribute. If yes, the set is replaced by the respective attribute's value. If no, the set is removed from the list without replacement. This can be used to add elements for (a) specific implementation(s): { sbcl = buildLisp.bundled "sb-posix"; } { sbcl = ./sbcl/optional-sbcl.lisp; } or to switch between files for different implementations: # If a implementation case is missing and no default set present, # no file will be added. Compilation will likely fail as a result. { ecl = ./tf-ecl.lisp; ccl = ./tf-ccl.lisp; sbcl = ./tf-sbcl.lisp; } or to account for special behavior for a certain implementation: { ccl = ./ccl-quirk-impl.lisp default = ./ansi-impl.lisp; } Change-Id: I082c3701d1f5063b92100bf336a83425471c269d Reviewed-on: https://cl.tvl.fyi/c/depot/+/3321 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-08-24 r/2768 feat(nix/buildLisp): allow implementation-specifc bundled functionssterni1-6/+32
By implementing a bundled function for an implementation, we can use a custom one for a specific implementation. This is useful for implementations like ECL where a require will be compiled as an instruction rather than importing all new symbols into a dump, so using the underlying static or shared object directly would be beneficial. overrideLisp for bundled libraries now only allows overriding the name and implementation arguments. Change-Id: I9036b29157e8daa4d86ff87d603b044373711dbf Reviewed-on: https://cl.tvl.fyi/c/depot/+/3301 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-08-24 r/2767 refactor(nix/buildLisp): prepare multi implementation supportsterni1-105/+188
Concept is roughly: * receive extra argument `implementation` that refers to the name of an implementation or rather an attribute in an internal attribute set telling buildLisp how to do certain build steps. * We assume an implementation can execute lisp files as scripts and that we can implement the following main tasks in lisp: - Building a library (`genCompileLisp`) - Building an executable (`genDumpLisp`) - Loading a library dynamically (`genLoadLisp`) Based on that we can implement: - Running a test suite (`genTestLisp`) - A REPL preloaded with a libraries and their dependencies (`lispWith`) Additional attributes for implementing these parts genericly are added as needed (`faslExt` and `runScript`). * `genCompileLisp` no longer prints a shell script which concatenates the individual FASLs. Instead it does the step previously done by the shell script itself. In essence `genCompileLisp` now writes a lisp script which compiles and installs the library to build. This will allow us extra freedom for different implementations, e. g. for ECL we'll want to build a object file archive additionally to fasl files in order to be able to link proper executables. * `genLoadLisp` and `genTestLisp` are almost generic (the former just sometimes would need to use different file extensions), but we integrate them into the implementation “API” to facilitate minor tweaks we need to do like the `fasc` extension for ECL's native FASL files. Change-Id: I1b8ccc0063159638ec7af534e9a6b5384e750193 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3292 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-08-24 r/2753 refactor(users/grfn/gws.fyi): implement isDirectory in pure nixsterni1-0/+65
Another day, another import from derivation avoided by builtins.unsafeDiscardStringContext! Change-Id: I67274b1ba13ba980bb3346b22f2955c702aa3151 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3372 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi> Reviewed-by: tazjin <mail@tazj.in>
2021-08-12 r/2722 docs(nix/buildLisp): mention drawback of non-parallel compilationsterni1-0/+4
To be fair this hardly matters since SBCL is quite fast, but compiling ironclad with ECL is quite the experience… Change-Id: Ib89cc50e5d557acec51fdb085bcbdfc99736221e Reviewed-on: https://cl.tvl.fyi/c/depot/+/3342 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-08-10 r/2716 fix(nix/buildLisp): remove misplaced `makeOverridable`ssterni1-2/+1
`makeOverriddable` doesn't work for bundled sbclWith as is because it uses the `//` operator internally which doesn't work with the types `bundled` and `sbclWith` accept as arguments (string and list respectively). What's more, `bundled` already uses `makeOverridable` and allows to override the internal call to `library` via `overrideLisp`. For `sbclWith` no such mechanism exists, but this seems to be no concern for now: Using `overrideLisp` for this hasn't worked so far (and failed with a _hideous_ evaluation error), so there doesn't seem to be any real demand for this feature. Maybe a feature for another CL. Change-Id: I0b2f34c00a2143cd66dd43a6b1b2880af997ee50 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3296 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-08-01 r/2701 refactor(nix/buildLisp): use passthru for extra attributessterni1-15/+17
Using passthru and appending the attributes via `//` have the same effect with a subtle difference: In the latter case re-evaluating the derivation when using the underlying `mkDerivation`'s `overrideAttrs` will delete all appended attributes. Using passthru at least preserves the attributes although the self reference to the derivation in `passthru.sbcl` will become outdated (unless updated manually). Change-Id: I8b85009f386b9375b86a23fd50c4ec8c6a9dea7f Reviewed-on: https://cl.tvl.fyi/c/depot/+/3257 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: grfn <grfn@gws.fyi>
2021-05-17 r/2594 fix(nix/writers/rust): remove args overrideProfpatsch1-9/+5
We can be closed world, so let’s restrict the arguments to the subset we need for now. The existing override was wrong, in that `// args` would use the arguments we already added, again. So instead of deliberating about how to make this work right in all cases, we don’t need it, we trim it. Change-Id: I6443a0808b8bfd5e4db939b669c6afc741954db8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3057 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2021-04-30 r/2555 feat(users/Profpatsch/blog): implement blog engine lolProfpatsch1-0/+1
My horrible blog engine v0.0.0.0.1. Don’t judge. Change-Id: I427b4e84d67aa49793cb7625e4d8ca2824f00943 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2456 Tested-by: BuildkiteCI Reviewed-by: Profpatsch <mail@profpatsch.de>
2021-04-24 r/2547 chore(nix): move rustSimple from users.Profpatsch.writersProfpatsch2-0/+165
I think it’s solid enough to use in a wider context. Change-Id: If53e8bbb6b90fa88d73fb42730db470e822ea182 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3055 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org> Reviewed-by: lukegb <lukegb@tvl.fyi>
2021-04-24 r/2546 feat(nix): add basic netstring nix generation functionsProfpatsch2-0/+43
Moving to toplevel so I can use them with `runExecline`. They should be pretty atomic, and are proven to work (tests are still in my user dir, since they test the producers indirectly via the python parser and I don’t want to pull it out right now). Change-Id: Id0baa3adcb2ec646458a104c7868c2889b8c64f5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3054 Reviewed-by: Profpatsch <mail@profpatsch.de> Tested-by: BuildkiteCI
2021-04-23 r/2535 feat(nix/buildGo): add the import position to errmsgProfpatsch2-11/+22
When a foreign dep is missing a dependency, it is good to have a context. e.g. the `github.com/charmbracelet/bubblegum` package has a lot of dependencies that are only used in its `examples/` dir; this is not obvious, unless we also print where the imports come from. New error message: ``` error: missing foreign dependency 'github.com/containerd/console' in 'github.com/charmbracelet/bubbletea, imported at /nix/store/0cry4sg6bzxqwk5zl2nxhas6k5663svg-source/tea.go:22:2' ``` Change-Id: If34a3c62b9d77d4aea108b5e011e16fbd03e8554 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2852 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-04-20 r/2532 feat(nix/utils): add storePathName, a more generic baseNameOfsterni1-0/+42
This is a wrapper around baseNameOf which also can deal with derivations. Added to //nix/utils since I've found myself introducing an ad-hoc implementation of this for both //web/bubblegum and //nix/buildC. Change-Id: I2fcd97a150d6eda21ab323fa0d881ff7442a892e Reviewed-on: https://cl.tvl.fyi/c/depot/+/3049 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-04-12 r/2497 refactor: Replace 'depotPath' with 'depot.path'Vincent Ambo1-2/+2
Instead of having two ways of accessing the path to the depot (one of which was stuttering, depot.depotPath) we settle on only one: depot.path. This was mostly used for NixOS module imports. Co-Authored-By: Florian Klink <flokli@flokli.de> Change-Id: I2c0db23383fc34f6ca76baaad4cc4af2d9dfae15 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2962 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi> Reviewed-by: sterni <sternenseemann@systemli.org>
2021-04-12 r/2496 refactor(readTree): Initialise repo roots without recursingVincent Ambo2-4/+19
Plumbs an additional internal argument through readTree that indicates whether the top-level of a tree is being read, and avoids recursing into itself in that case. This changes the externally visible behaviour of readTree (it is now expected to be called a level higher than previously). This allows us to reduce the amount of boilerplate needed to bootstrap the TVL repository (by not having to specify the individual folders that need to be read). For reasons related to an infinite recursion we could not (be bothered to) debug, the top-level `config` key (which held the attribute set passed on by readTree) has been removed. This is not needed, as it is already passed on by readTree ... Co-Authored-By: Florian Klink <flokli@flokli.de> Change-Id: Id6e39b57b2f5b3473c4b695a72dd1d01fcfb7a66 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2961 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org> Reviewed-by: grfn <grfn@gws.fyi>
2021-04-12 r/2483 docs(readTree): Add SPDX license header on readTreeVincent Ambo1-0/+6
Change-Id: I9a6eafa7d4bb3d590dfa35d368adfd25aeed7f64 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2936 Reviewed-by: Profpatsch <mail@profpatsch.de> Tested-by: BuildkiteCI
2021-04-10 r/2460 refactor(sterni): use pkgs over third_party to import from nixpkgssterni1-3/+2
This should ease migrating to a distinction between depot.third_party and pkgs (as in nixpkgs) in the future. Ref cl/2910, b/108. Change-Id: I53a854071fddd7c0d0526cc4c5b16998202082c6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2913 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-03-31 r/2380 feat(nix/buildManPages): infra for generating man page dirs from nixsterni2-0/+80
Very simple builder which builds (optionally) gzipped man pages from a list of attrsets and links them into a common man directory with the correct layout, so it should be installable immediately. Additionally runs mandoc -T lint, but by default only for informational purposes as it is very strict and some things are almost never true (for example all Xrs being present in the respective directory). buildManPages.single exposes the internal builder for a single, optionally gzipped man page from a nix attrset. Change-Id: I43fce011716f4a7cc80521f222800ca99ba54060 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2654 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-03-26 r/2336 fix(nix/runTestsuite): use s6-portable-utilsProfpatsch1-2/+3
Same as 221698c603dcb318c609b4d21cb2a9fada44a14c We had a bunch of instances of https://github.com/NixOS/nix/issues/2176, where nix would exit with a “killed by signal 9” error. According to Eelco in that issue, this is perfectly normal behaviour of course, and appears if the last command in a loop closes `stdout` or `stdin`, then the builder will SIGKILL it immediately. This is of course also a perfectly fine error message for that case. It turns out that mainly GNU coreutils exhibit this behaviour … Let’s see if using a more sane tool suite fixes that. Change-Id: Iaf9e542952ca36c02208a3f067f575ba978272b4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2663 Reviewed-by: Profpatsch <mail@profpatsch.de> Tested-by: BuildkiteCI
2021-03-25 r/2329 refactor: Replace some uses of builtins.toFile with pkgs.writeTextVincent Ambo1-2/+2
I'm looking at removing some of these because they can cause unnecessary build steps during CI pipeline generation. Change-Id: I84742968918090c050d2eedab8a1b42692632a42 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2655 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2021-03-15 r/2282 feat(users/Profpatsch): build attrset members on CIsterni2-0/+27
Setting meta.targets to include all derivations in the different package sets in Profpatsch's user folder makes them checked by CI until they do the readTree refactor as promised. To reduce code duplication we handle this in a simple function which is exposed from nix.utils which may be a good place for depot specific bits and bops we accumulate over time. To get around the issue of too nested sets we perform the following renames: * users.Profpatsch.tests gets moved into its own directory * users.Profpatsch.arglib.netencode now lives in its own file instead of the default.nix * users.Profpatsch.netstring.tests gets moved into its own directory Change-Id: Icd039c29d7760a711c1c53554504d6b0cd19e120 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2603 Tested-by: BuildkiteCI Reviewed-by: Profpatsch <mail@profpatsch.de>
2021-02-24 r/2233 fix(buildGo): Fix stdlib list under Go 1.16.Luke Granger-Brown1-1/+1
Go 1.16 makes "go list all" not work. "go list std" is what we should be using instead anyway. Change-Id: I3f867fde477030d2358085b3d64b5856fb9c421b Reviewed-on: https://cl.tvl.fyi/c/depot/+/2551 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-02-19 r/2224 feat(nix/readTree): give better error message when not a functionProfpatsch4-1/+27
When a file is added to the depot tree that is picked up by read-tree, but it’s not a function like ({...}: {}), `readTree` will fail on the function application, leading to a bad error message. We can do slightly better, by checking the type and throwing a nicer trace message. `assertMsg` is copied from `nixpkgs/lib/assert.nix`, since at this point we don’t have a reference to the lib. There is another evaluation failure that can happen, which is when the function we try to call does not have dots; however, nix does not provide any inflection capabilies for checking whether a function attrset is open (`builtins.functionArgs` only tells us the attrs it mentions explicitly). Maybe the locality of the error could be improved somehow. Change-Id: Ibe38ce78bb56902075f7c31f2eeeb93485b34be3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2469 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-02-19 r/2222 fix(nix/yants): make (typedef …).checkType return a result setsterni1-3/+7
Previously, for types defined using typedef (like all primitive types) type.checkType would return a boolean. This is largely fine since in most places `type.checkToBool (type.checkType x)` or similar is used. However, some functions actually take type.checkType up on the promise that it returns a set of the form: { ok = <bool>; err = <option string>; } This is the case for restrict which has checkToBool = v: v.ok; and will generate a proper set except if `t.checkToBool (t.checkType v) == false` in which case it will return t.checkType v. If t was a primitive type or defined using typedef, previously `t.checkType v` would be a boolean which meant as soon as (restrict …).checkToBool was called on a restrict checkType result in cases where the wrapped type didn't match, an unrelated error would be thrown: nix-repl> with nix.yants; restrict "foo" (_: true) int "lol" error: value is a boolean while a set was expected, at /home/lukas/src/depot/nix/yants/default.nix:38:39 This is fixed by making typedef return a proper set from checkType and adjusting its checkToBool accordingly. Unfortunately I don't think we can easily add test cases for this except by using recursive nix or VM tests as there is no way to introspect error messages. Change-Id: I96a7be065630f04ca33358f21809284911ec14fe Reviewed-on: https://cl.tvl.fyi/c/depot/+/2536 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: Profpatsch <mail@profpatsch.de>
2021-02-18 r/2218 fix(nix/readTree): make nixFileName only match .nix extensionssterni1-1/+1
Nix unfortunately has terrible escaping syntax: If something is an escape sequence it does not know, like \0, it just swallows the backslash and returns the second character (byte) as is (yes, "\0" == "0" is true). This created the following bug in nixFileName which should have resulted in at least a parse error: "(.*)\.nix" is the same as "(.*).nix" which meant that nixFileName matched anything that is at least 4 characters long and ends in "nix". This lead to readTree creating double attributes when directories are involved or attributes for non-nix files. Change-Id: Ibf3be2bd189d48881c82ee795c50151bfb365627 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2535 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2021-02-14 r/2214 feat(nix/drvSeqL): make build of a drv depend on unrelated drvssterni1-0/+43
This adds the star of NixCon 2017 from vuizvui, slightly reformatted and now using yants. While it has some flaws, I realized that it is ideal to run the tests of rustSimple{Lib,Bin} where the normal and the -tests variant would have to be rebuilt if either the tests or the library / executable itself changes. Change-Id: Ie8f84f98c51c9fafc046eff916c8f0df7e8f224b Reviewed-on: https://cl.tvl.fyi/c/depot/+/2528 Tested-by: BuildkiteCI Reviewed-by: Profpatsch <mail@profpatsch.de>
2021-02-12 r/2202 feat(nix/yants/tests): test drv typesterni1-0/+10
By using an extremely trivial derivation we can ensure that it will not throw if evaluated using deepSeq. When using stdenv.mkDerivation or similar at some point something will most likely throw or generate some kind of error which is alright in the context of nixpkgs, but makes testing yants harder than you'd think it should be. Change-Id: I61ff7dc01a00a4815ef39066e4e223123356ddd5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2507 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-02-09 r/2199 feat(nix/yants/tests): port to runTestsuitesterni1-48/+92
Port existing tests to runTestsuite and add some obvious additional tests that wouldn't be possible before (using assertThrows and assertEq). Change-Id: Ibe950a7a0cda3e23ebb226bdff35f52cdfec5ddf Reviewed-on: https://cl.tvl.fyi/c/depot/+/2479 Tested-by: BuildkiteCI Reviewed-by: Profpatsch <mail@profpatsch.de> Reviewed-by: tazjin <mail@tazj.in>
2021-02-09 r/2198 feat(nix/runTestsuite): add assertDoesNotThrowsterni1-0/+7
assertDoesNotThrow is like assertThrows, but fails if the expression throws. In that case the new unexpected-throw branch of AssertErrorContext is returned. Change-Id: I7195eb5df8965456e9ab9b69e35ec96b33f00a35 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2476 Tested-by: BuildkiteCI Reviewed-by: Profpatsch <mail@profpatsch.de>
2021-02-09 r/2197 refactor(nix/runTestsuite): implement in terms of generic assertBoolsterni1-26/+46
Refactor assertEq and assertThrows to be implemented in terms of a more generic assertBool to reduce code duplication and ease adding new assert types. To preserve meaningful error messages AssertResult is changed slightly: nope-eq and nope-throw have been replaced by a single nope branch which contains an AssertErrorContext which contains error information. To implement an assert assertBoolContext (which is not exposed) can be used: It takes an AssertErrorContext which is returned in case of an error and a boolean determining whether the assert was successful. The currently possible AssertErrorContext are: * should-throw: error result of assertThrows, formerly nope-throw * not-equal: error result of assertEq, formerly nope-eq Change-Id: Ifd6b3aa4187c90c3add2df63fa7c906c8f03fd2d Reviewed-on: https://cl.tvl.fyi/c/depot/+/2473 Tested-by: BuildkiteCI Reviewed-by: Profpatsch <mail@profpatsch.de>
2021-01-30 r/2166 feat(nix): add nix.tag, a way of discriminating via tagged unionsProfpatsch2-0/+237
Introduces the concept of a “tag”, a single-keyed attrset which annotates a nix value with a name. This can be used to implement tagged unions (by implying the list of possible tags is well-known), which has some overlap with how `nix.yants` does it. However, the more fascinating use-case is in concert with a so-called discriminator, `match` and hylomorphisms. The discriminator can take a nix value, and add tags to it based on some predicate. With `match`, we can then use that information to convert the discriminated values again. With `hylo`, we can combine both the “constructive” discriminator step with the “destructive” match step to recursively walk over a nix data structure (based on a description of how to recurse, e.g. through attrset values or list values), and then apply a transformation in one go. Change-Id: Ia335ca8b0881447fbbcb6bcd80f49feb835f1715 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2434 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2021-01-30 r/2165 feat(nix/readTree): add tests of the traversal logic in READMEProfpatsch15-1/+95
This should closely match the documented behaviour. It might still be missing some edge cases of course. Change-Id: I5c75fa045d5f3be8cf5eab787a02644500c14522 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2466 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2021-01-30 r/2164 feat(nix/readTree): add tests of the README exampleProfpatsch7-0/+47
First step to slowly giving readTree some coverage, so we can do refactoring without breaking functionality. Change-Id: If25a8c0fa9c4ac7472c0473372f10a9326cccaf7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2465 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2021-01-30 r/2163 chore(nix/readTree): move function into __functorProfpatsch1-12/+13
We are going to export some tests under `nix.readTree.tests`, so in order to do that and still have `nix.readTree` be a function, let’s move it to `__functor`. This requires wiring the `args` and `initPath` arguments through explicitly. Change-Id: Ife7956b85d35e59c22174b42dcb7cca83ed868ea Reviewed-on: https://cl.tvl.fyi/c/depot/+/2464 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2021-01-30 r/2162 feat(nix/runTestsuite): add assertThrowsProfpatsch1-4/+22
Uses `builtins.tryEval` to check that the expression throws when `deepSeq`-ed. Change-Id: I0d57cc37f473bb733f57a1b1c0d889084152fd2f Reviewed-on: https://cl.tvl.fyi/c/depot/+/2463 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2021-01-30 r/2161 fix(nix/runTestsuite): wrap runTestsuite into derivationProfpatsch1-8/+26
Previously we would throw or return `{}`, which doesn’t integrate nicely into our CI; thus, let’s wrap it into a derivation which either fails the build or doesn’t. Change-Id: I65880d86b8393094661e57a0b32aafe748bf1dd5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2462 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI