about summary refs log tree commit diff
path: root/nix (follow)
AgeCommit message (Collapse)AuthorFilesLines
2021-12-20 r/3328 fix(nix/buildLisp): don't emit a target for impl used at top levelsterni1-1/+1
These targets would be the same derivation, but named differently which is noisy and causes a few, mostly subtle issues: * Buildkite struggles with large pipelines a bit, we can save quite a few steps by removing these. * Having two jobs for the same derivation sometimes causes the annoying situation that an agent would do nothing except waiting for a lock. * Non-nix CI we add in the future may not be able to recognize that these targets are the same and do extra work unnecessarily. Change-Id: I1103e719ade1d3859d222b713969ac34a8765cba Reviewed-on: https://cl.tvl.fyi/c/depot/+/4515 Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-12-17 r/3285 feat(nix/utils): handle toString-able sets produced by builtins.pathsterni2-3/+8
When preparing cl/4381 I noticed that we actually handle this case properly. depot.nix.utils.storePathName depot.path now works as expected. Change-Id: Ice9329c67b2e2210852012f5abe82fbbb13193de Reviewed-on: https://cl.tvl.fyi/c/depot/+/4382 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in> Autosubmit: sterni <sternenseemann@systemli.org>
2021-12-16 r/3271 docs(nix/readTree): Tweak grammar of docs for `gather`Griffin Smith1-1/+1
This seemed to be missing a word previously. Change-Id: Ifa860051d6b692a626dbaddbaee44b761f2274ff Reviewed-on: https://cl.tvl.fyi/c/depot/+/4386 Autosubmit: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-12-08 r/3158 fix(readTree): Apply .skip-subtree to Nix-file children as wellVincent Ambo1-1/+2
This behaviour was previously confusing, since readTree's data structure treats children from Nix files and directories as identical but only one of them would be affected by .skip-subtree The "subtree" to be skipped here refers to all children of the structure. Change-Id: Idf596c9823f09cc2acf49523916bde4b801b8519
2021-11-25 r/3093 fix(nix/runTestsuite): don't crash on big test suitessterni1-8/+9
Having `prettyRes` in the execline script causes it to fail because of the argv limit if your test suite is long enough. For the succeeding one we can work around this by hashing it (since we only care that something changes if the test suite changes), in the case of the failing one where we want to print the results, we use runExecline's stdin mechanism. Change-Id: I2489f76acfbe809351f51caefe2a477328a70ee3
2021-11-23 r/3088 refactor(readTree): Move 'drvTargets' into readTreeVincent Ambo3-18/+18
This function is also generally useful for readTree consumers that have the concept of subtargets. Change-Id: Ic7fc03380dec6953fb288763a28e50ab3624d233
2021-11-23 r/3087 refactor(readTree): Move 'gather' into readTree itselfVincent Ambo1-0/+34
Discovering CI targets is relevant to all readTree consumers and this logic is not TVL-specific. Change-Id: I81ed3d3f76a6c36119f04bee28ca995a013f0e35
2021-11-23 r/3084 refactor(readTree): Move copy of 'fix' into readTreeVincent Ambo1-0/+7
This is often used when bootstrapping a repository with readTree, before lib is available. Having this definition in readTree is more convenient than copy&pasting it around to callsites. Change-Id: I6d5d27ed142bea704843fe289ad2674be8c4d360
2021-11-23 r/3082 refactor(readTree): Move 'restrictFolder' function into readTreeVincent Ambo1-2/+33
This is generally useful for readTree users and should be part of readTree itself. This is a move towards exposing several readTree-related features from the library itself, in the future also including logic like 'gather'. Note that this has a small functional change: In error messages of the function, the notation for accessing Nix attributes is now used rather than the Perforce-style `//` notation common in TVL. For example, an error at `//web/tvl/logo` will produce `web.tvl.logo` in the error message (which corresponds to the readTree attribute itself). This makes more sense for non-TVL consumers of readTree, as the Perforce-style notation is custom to us specifically. Change-Id: I8e199e473843c40db40b404c20d2c71f48a0f658
2021-11-15 r/3070 refactor(readTree): Flip argument order of argFilterVincent Ambo1-2/+2
Since the filters return 'args', this makes nesting of filters more readable. Change-Id: I775252460e3e077cc6db2fab6f3948414a95ecbf Reviewed-on: https://cl.tvl.fyi/c/depot/+/3873 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2021-10-24 r/2991 refactor(nix/tag): use throw over lib.assertMsgsterni1-1/+1
Slightly prettier error message etc. Change-Id: I9af4ec6ce337c65b5ca0820dc15a4e084832c529 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3542 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-10-19 r/2975 chore(nix/renderMarkdown): Move out of //users/tazjinVincent Ambo1-0/+8
Change-Id: Ia5e873420b57436abead8e02cef3e5b7149de123 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3731 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2021-10-10 r/2957 feat(nix/sparseTree): allow specifying subtrees as relative pathssterni1-2/+10
Passed strings will be treated as a relative path below the given root, which is quite convenient when using depot.path by eliminating a lot of repetition. Change-Id: I3da6058094484f4a6ffbb84f89ad4472b502a00c Reviewed-on: https://cl.tvl.fyi/c/depot/+/3704 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-10-05 r/2950 test(nix/readTree): test marking behavior of readTreesterni5-0/+35
This ensures in a simple example that __readTree and __readTreeChildren are populated correctly. Change-Id: I69a46b2ddde0d1f9bf0dff1c4780f033ac8fc27a Reviewed-on: https://cl.tvl.fyi/c/depot/+/3655 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-10-02 r/2947 feat(nix/utils): expose pathType of symlink targetsterni2-13/+104
In order to make readTree import symlinked directories I've been looking into how to detect if a symlink points to a directory (since this would allow us to use symlinks for //nix/sparseTree). I've found a hack for this: symlinkPointsToDir = path: isSymlink path && builtins.pathExists (toString path + "/.") Unfortunately it doesn't seem to be possible to distinguish whether the symlink target does not exist or is a regular file. Since it's possible, I thought might as well add this to `pathType`. To make returning the extra information workable, I've elected to use the attribute set layout used by `//nix/tag`. This doesn't require us to depend anything (as opposed to yants), but gives us pattern matching (via `nix.tag.match`) and also quite idiomatic checking of pathTypes: pathType ./foo ? file (pathType ./foo).symlink or null == "symlink-directory" Nonexistent paths are encoded like this: pathType ./foo ? missing Of course we can't use this in readTree (since it must be zero dependency), but we can easily inline this hack at some point. Change-Id: I15b64a1ea69953c95dc3239ef5860623652b3089 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3535 Tested-by: BuildkiteCI Reviewed-by: Profpatsch <mail@profpatsch.de> Reviewed-by: tazjin <mail@tazj.in>
2021-09-30 r/2938 fix(nix/readTree): generate the correct marker for nix file childrensterni1-1/+1
This was a regression introduced in cl/3554. Change-Id: I0721693a6eb1b28976b28499875812b1c3d1c910 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3654 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-09-18 r/2882 feat(nix/nint): accept attribute set with stdout, stderr and exitsterni2-3/+57
This extends the calling convention for nint in a non-breaking way: If the called script returns an attribute set instead of a string the following is done: * If the attributes `stdout` and/or `stderr` exist, their content (which must be a string currently) is written to the respective output. * If the attribute `exit` exists, nint will exit with the given exit code. Must be a number that can be converted to an `i32`. If it's missing, nint will exit without indicating an error. Change-Id: I209cf178fee3d970fdea3b26e4049e944af47457 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3547 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-09-17 r/2881 refactor(nix/runTestsuite): clean up runTestsuitesterni1-16/+6
* goodAss wasn't used before. Simplify it to just return a boolean, so we can use it for partitionTests later. * goodIt also returns unnecessary extra meta information which is not used. Cleaning that up makes the condition extremely small, so we can inline it into (what was) goodIts. * goodIts is just called in one place, so we can inline it into res. Change-Id: I70cf4fa3f61ce1467a2ee5319f841cdd42db6a66 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3548 Tested-by: BuildkiteCI Reviewed-by: Profpatsch <mail@profpatsch.de>
2021-09-17 r/2880 test(nix/utils): unit test storePathNamesterni1-0/+16
Change-Id: I4208cf9e3c5e1d922ee5b5bffd034e4ac6d0e2c0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3543 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-09-15 r/2869 feat(nix/readTree): record list of children added by readTreesterni1-18/+33
This change adds a new attribute to readTree nodes, `__readTreeChildren` which is a list of attribute names added to this node by readTree. This is then used by `gather` for `ci.targets` to avoid evaluating attributes unnecessarily. Especially since Nix is not as lazy as we'd like when determining types (i. e. child ? __readTree needs to force `child` even when it's not an attribute set), evaluating attributes unnecessarily is sometimes problematic. Change-Id: I0a98691d41f987e23ee7e9ba21fbe465da5fe402
2021-09-15 r/2862 refactor(nix/nint): use stdout().write_all() over print!()sterni1-3/+3
Change-Id: Ic372442c5156f1e13fea28c1b3e57d64e89ba49b Reviewed-on: https://cl.tvl.fyi/c/depot/+/3546 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2021-09-15 r/2861 refactor(nix/readTree): remove unused assertMsg functionsterni1-5/+0
Change-Id: I4048a59ca3daeb541a7294ce72eebe32982d852c Reviewed-on: https://cl.tvl.fyi/c/depot/+/3553 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-09-14 r/2858 test(nix/utils): unit test path predicatessterni8-0/+69
Change-Id: I85f38ef8f284c4accd1ea744f61358622785358d Reviewed-on: https://cl.tvl.fyi/c/depot/+/3534 Tested-by: BuildkiteCI Reviewed-by: Profpatsch <mail@profpatsch.de>
2021-09-14 r/2857 feat(nix/runTestsuite): rebuild if test case list changessterni1-2/+5
If the result of the assertions changes for a successful test suite (this happens if tests are reworded, added or removed), this makes sure the no-op derivation is rebuilt. This makes sure that test suites show up in buildkite on ocassions other than channel bumps, since they are only added to the job list if their `outPath` is missing nowadays (see cl/3427). Change-Id: Ia1050cca5eeed8b7da84c40f6154b40760a3047f Reviewed-on: https://cl.tvl.fyi/c/depot/+/3536 Tested-by: BuildkiteCI Reviewed-by: Profpatsch <mail@profpatsch.de>
2021-09-14 r/2856 feat(nix/tag): add functions for querying a tag's name and valuesterni1-1/+14
Both are just trivial wrappers around assertIsTag to make these lookups more ergonomic. This also allows us to demote assertIsTag to an implemtation detail. Change-Id: Ib6ba2a858f4839354a57b660042b418976c4b1d9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3541 Tested-by: BuildkiteCI Reviewed-by: Profpatsch <mail@profpatsch.de>
2021-09-14 r/2855 docs(nix/tag): correct comment for assertIsTagsterni1-1/+1
Change-Id: Id54cc3a1d7c83c6a94b9446c76b00ca2ce1ab48e Reviewed-on: https://cl.tvl.fyi/c/depot/+/3540 Tested-by: BuildkiteCI Reviewed-by: Profpatsch <mail@profpatsch.de>
2021-09-12 r/2852 fix(nix/buildLisp): preserve argv[0] in buildLisp.programsterni1-7/+6
I *thought* I was being clever with the (cdr (member …)) call, but somehow completely forgot that *posix-argv* and *command-line-argument-list* are equivalent to argv, so they also contain the program name as the first element. Dropping that made argument parsing completely break down, so let's revert back to the older solution which works quite well. Change-Id: If7d3321cda0ca512bc8c23b6541ce390b81a3e24 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3538 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-09-10 r/2834 docs(nix/tag): fix example for discrDefsterni1-2/+2
Change-Id: I8b7accb399448f57ace502642bd91fcfae416f15 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3507 Tested-by: BuildkiteCI Reviewed-by: Profpatsch <mail@profpatsch.de>
2021-09-10 r/2833 chore(nint): move from //users/sterni to //nixsterni4-0/+212
Since //web/bubblegum depends on nint, we need to move it to a non user directory to conform with the policy established via cl/3434. Note that this likely doesn't mean greater stability (which isn't really implied in depot anyways), since I still would like to use a more elaborate calling convention to allow for additional useful features. Change-Id: I616f905d8df13e3363674aab69a797b0d39fdd79 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3506 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-09-09 r/2830 feat(nix/sparseTree): get a directory with only selected childrensterni2-0/+65
Given a path (which points to a directory and a list of paths which are below that path, build a “sparse” version of that directory, so that it only contains the listed paths (and their children): $ nix-build -E 'with import ./. {}; nix.sparseTree ./. [ ./default.nix ./nix/readTree ./nix/buildLisp ./third_party/nixpkgs ./third_party/overlays ]' /nix/store/0ynj0gc613fs6mfp9snqcvdj5gfxbdzg-sparse-depot $ lr -t 'type == d' result/ result/ result/nix result/nix/buildLisp result/nix/buildLisp/example result/nix/readTree result/nix/readTree/tests […] result/third_party result/third_party/nixpkgs result/third_party/overlays result/third_party/overlays/haskell result/third_party/overlays/haskell/patches result/third_party/overlays/patches This is useful if a derivation depends on depot.path (e. g. if it wants to import depot at runtime). Usually this means that on every depot commit (or even worse, every change of .git on a local machine), this derivation has to be rebuild. By using sparseTree you can instead depend on a stripped down version of depot which only contains the bits you actually depend on, avoiding unrelated rebuilds. Change-Id: I127b108c8b177c657fb46786d0a6256516fd2c52 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3503 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-09-09 r/2829 refactor(nix/readTree): use throw instead of assertMsgsterni1-4/+3
This gives a slightly prettier error message and won't leak the error message when builtins.tryEval is used. Currently an error message from the tests is always part of the pipeline evaluation log. Change-Id: I9b488a440368091ed42d707ba4124f592a64bd86 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3502 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-09-09 r/2825 feat(readTree): Support scoped import argumentsVincent Ambo2-9/+15
This makes it possible to override Nix builtins within a readTree structure. Why would you want to do that, you might ask? Well ... Change-Id: Icc9cb32e5db4a2eba370cf81769c642d237d4937 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3499 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2021-09-08 r/2824 refactor(readTree): Pass all readTree parameters as function argsVincent Ambo3-20/+47
Instead of having a mix of depot-passed args (for the filter) and args to the readTree function itself, make everything a single attribute set of arguments passed to the function. This also makes it a bit easier to extend this in the future. Change-Id: I633c1fc96026d137b451bb604ef92be32571a0f5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3498 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2021-08-29 r/2800 refactor(bufCheck): Assume bufCheck is run in depot checkoutVincent Ambo1-2/+2
Rather than copying the depot path into the store on each commit, assume bufCheck is run in the depot checkout (which it is, in Buildkite land). Change-Id: I4a4af2e5b45acad2d18218e503880ee63b20f078 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3462 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2021-08-26 r/2792 feat(readTree): Add support for path-dependent args filteringVincent Ambo1-2/+7
Adds another argument to readTree itself which can be passed when importing readTree (e.g. in our default.nix) to filter the arguments passed to a target based on that target's location in the tree. This is intentionally not yet mentioned in the docs, and also intentionally implemented in such a way that the API surface of readTree doesn't change. The reason for this is that I want to figure out whether these filter functions are actually useful, e.g. within depot by filtering user-folder passing, and then refactor the readTree API to find a public way of exposing this as part of the readTree function itself (and not its import). Relates to b/143. Change-Id: I2cdf09f67916527d2337f4bfb578749aeac51a6a Reviewed-on: https://cl.tvl.fyi/c/depot/+/3433 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi> Reviewed-by: sterni <sternenseemann@systemli.org>
2021-08-26 r/2779 docs(nix/buildLisp): document NIX_BUILDLISP_LISP_ARGSsterni1-0/+24
Change-Id: I54294da08ee08bcf6cba6c792a2a2235b988a778 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3422 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-08-26 r/2778 docs(nix/buildLisp): move example further upsterni1-29/+29
Make "Example" the second section again since it got a bit buried under a lot of detailed documentation you won't necessarily need right away. Change-Id: I481354d1761c590e5872dfce8c3cf9934e278673 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3421 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2021-08-26 r/2777 fix(nix/buildLisp): prevent image loader from parsing argumentssterni1-3/+22
CCL and SBCL create executables by dumping their image. As a consequence, some part of the respective compiler is embedded in the resulting executable which is executed and loads the image. For CCL and SBCL this piece of software seems to unconditionally parse arguments which can't be prevented since it happens before any lisp is loaded. Luckily in both cases the parsing stops at `--`, so we can just pass this via the wrapper — we just need to work around the problem that this will of course be left in argv and confuse any later code. This can be rectified by deleting everything prior to the first `--` in the global argument list on startup in both cases. In cases we do want to pass arguments to the image loader, we can use the special NIX_BUILDLISP_LISP_ARGS environment variable which is understood by the wrapper. Note: This fix doesn't interfere with ECL since it is not using the wrapper script at the moment. Fixes b/136. Change-Id: I3f95aa61e945e51428021ca18232ff15c923f870 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3357 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: grfn <grfn@gws.fyi>
2021-08-24 r/2774 docs(nix/buildLisp): document recent changes to buildLispsterni1-10/+119
Doing this in a separate CL to avoid having to track the intermediate changes no one will ever see in documentation as well which would be unnecessary effort. * Multi-implementation support introduced in cl/3292 and refined in cl/3368 in terms of the user interface. * Implementation specific srcs and deps introduced in cl/3321 * Implementation passthru attrs and rename from .sbcl -> .repl was done in cl/3359 * ECL added in cl/3297, CCL in cl/3350 Change-Id: Ia13f2aea4e7e091c00991fcbfc601de364413979 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3380 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-08-24 r/2773 feat(nix/buildLisp): pass implementation description instead of namesterni1-30/+32
Instead of using a string to refer to an internal set defined in buildLisp, we just expose the relevant sets (as nix.buildLisp.sbcl, nix.buildLisp.ecl, …) and receive them as the `implementation` argument directly. This has several advantages: * It becomes easier to extend buildLisp, even for downstream users: Since you can just pass your own set, there's nothing stopping you from adding support for another implementation in a downstream derivation without having to edit the buildLisp file in any way which is great if you're using e. g. builtins.fetchGit to import it. * Users can mess with the implementation set by changing out some parts of it for customization purposes. Note that currently the sets use a lot of self-references which aren't even bound by a fix-point, so to make this work smoothly, we'd need to add some overriding mechanism. * The buildLisp code becomes quite a bit clearer. Since we're now always dealing with the implementation set, the confusing distinction between `impl`, `impl.name` and `implementation` no longer exists. `impl` is now exclusively an abbreviation of `implementation` (we could make this more consistent in the future even). Change-Id: I36d68069dd1315610b2f7159941507b465469b7c Reviewed-on: https://cl.tvl.fyi/c/depot/+/3368 Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2021-08-24 r/2772 feat(nix/buildLisp): add cclsterni1-1/+103
This adds support for Clozure's CL implementation to buildLisp. This is quite trivial in comparison to ECL since SBCL and CCL have very similar in how they work (so much so that CCL also suffers from b/136). Also the similarities in the code actually added here are striking, so I'll try to make an effort to reduce the code duplication in the future. To fix builds with CCL the following changes were made: * //3p/lisp/nibbles: The double inclusion of the types.lisp file was fixed. CCL doesn't like double definitions and refuses to compile otherwise. * //3p/lisp/physical-quantities: Update to a new bug fix release which contains a compilation fix for CCL. * //3p/lisp/routes: apply a patch fixing the build which was previously failing due to a double definition. * //3p/lisp/usocket: only depend on sb-bsd-sockets for SBCL and ECL, the latter of which seems to have a SBCL compatible implementation of the package. * Conditionally include a few CCL-specific source files and add `badImplementation` entries for the remaining failures which are //fun/gemma (to be expected) and //web/panettone which fails with an incredibly vague message. Change-Id: I666efdc39a0f16ee1bb6e23225784c709b04e740 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3350 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-08-24 r/2771 feat(nix/buildLisp): add eclsterni1-4/+157
Adds ECL as a second supported implementation, specifically a statically linked ECL. This is interesting because we can create statically linked binaries, but has a few drawbacks which doesn't make it generally useful: * Loading things is very slow: The statically linked ECL only has byte compilation available, so when we do load things or use the REPL it is significantly worse than with e. g. SBCL. * We can't load shared objects via the FFI since ECL's dffi is not available when linked statically. This means that as it stands, we can't build a statically linked //web/panettone for example. Since ECL is quite slow anyways, I think these drawbacks are worth it since the biggest reason for using ECL would be to get a statically linked binary. If we change our minds, it shouldn't be too hard to provide ecl-static and ecl-dynamic as separate implementations. ECL is LGPL and some libraries it uses as part of its runtime are as well. I've outlined in the ecl-static overlay why this should be of no concern in the context of depot even though we are statically linking. Currently everything is building except projects that are using cffi to load shared libaries which have gotten an appropriate `badImplementations` entry. To get the rest building the following changes were made: * Anywhere a dependency on UIOP is expressed as `bundled "uiop"` we now use `bundled "asdf"` for all implementations except SBCL. From my testing, SBCL seems to be the only implementation to support using `(require 'uiop)` to only load the UIOP package. Where both a dependency on ASDF and UIOP exists, we just delete the UIOP one. `(require 'asdf)` always causes UIOP to be available. * Where appropriate only conditionally compile SBCL-specific code and if any build the corresponding files for ECL. * //lisp/klatre: Use the standard condition parse-error for all implementations except SBCL in try-parse-integer. * //3p/lisp/ironclad: disable SBCL assembly optimization hack for all other platforms as it may interfere with compilation. * //3p/lisp/trivial-mimes: prevent call to asdf function by substituting it out of the source since it always errors out in ECL and we hardcode the correct path elsewhere anyways. As it stands ECL still suffers from a very weird problem which happens when compiling postmodern and moptilities: https://gitlab.com/embeddable-common-lisp/ecl/-/issues/651 Change-Id: I0285924f92ac154126b4c42145073c3fb33702ed Reviewed-on: https://cl.tvl.fyi/c/depot/+/3297 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: eta <tvl@eta.st>
2021-08-24 r/2770 feat(nix/buildLisp): expose drvs built w/ the other implementationssterni1-10/+48
For every implementation we support an extra passthru attribute with the name of the implementation is created which points to a version of the derivation built with that implementation. E. g. if we support CCL, ECL and SBCL, third_party.lisp.alexandria would have: * third_party.lisp.alexandria.sbcl * third_party.lisp.alexandria.ecl * third_party.lisp.alexandria.ccl To make this possible, the REPL derivation which was called `sbcl` originally has been renamed to `repl`. Since some things won't build with all implementations, we introduce a brokenOn argument which influences the meta.targets list that is created, but won't prevent the passthru attrs from being created to ease debugging failures. Change-Id: Icd6af345143593fac30ded10deabf31172e5d48a Reviewed-on: https://cl.tvl.fyi/c/depot/+/3359 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-08-24 r/2769 feat(nix/buildLisp): implementation specific deps and srcssterni1-13/+67
Both the deps and srcs arguments may now have special “filter sets” in the lists they receive as arguments. When building, buildLisp checks if such sets either have a attribute named like the current implementation or a "default" attribute. If yes, the set is replaced by the respective attribute's value. If no, the set is removed from the list without replacement. This can be used to add elements for (a) specific implementation(s): { sbcl = buildLisp.bundled "sb-posix"; } { sbcl = ./sbcl/optional-sbcl.lisp; } or to switch between files for different implementations: # If a implementation case is missing and no default set present, # no file will be added. Compilation will likely fail as a result. { ecl = ./tf-ecl.lisp; ccl = ./tf-ccl.lisp; sbcl = ./tf-sbcl.lisp; } or to account for special behavior for a certain implementation: { ccl = ./ccl-quirk-impl.lisp default = ./ansi-impl.lisp; } Change-Id: I082c3701d1f5063b92100bf336a83425471c269d Reviewed-on: https://cl.tvl.fyi/c/depot/+/3321 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-08-24 r/2768 feat(nix/buildLisp): allow implementation-specifc bundled functionssterni1-6/+32
By implementing a bundled function for an implementation, we can use a custom one for a specific implementation. This is useful for implementations like ECL where a require will be compiled as an instruction rather than importing all new symbols into a dump, so using the underlying static or shared object directly would be beneficial. overrideLisp for bundled libraries now only allows overriding the name and implementation arguments. Change-Id: I9036b29157e8daa4d86ff87d603b044373711dbf Reviewed-on: https://cl.tvl.fyi/c/depot/+/3301 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-08-24 r/2767 refactor(nix/buildLisp): prepare multi implementation supportsterni1-105/+188
Concept is roughly: * receive extra argument `implementation` that refers to the name of an implementation or rather an attribute in an internal attribute set telling buildLisp how to do certain build steps. * We assume an implementation can execute lisp files as scripts and that we can implement the following main tasks in lisp: - Building a library (`genCompileLisp`) - Building an executable (`genDumpLisp`) - Loading a library dynamically (`genLoadLisp`) Based on that we can implement: - Running a test suite (`genTestLisp`) - A REPL preloaded with a libraries and their dependencies (`lispWith`) Additional attributes for implementing these parts genericly are added as needed (`faslExt` and `runScript`). * `genCompileLisp` no longer prints a shell script which concatenates the individual FASLs. Instead it does the step previously done by the shell script itself. In essence `genCompileLisp` now writes a lisp script which compiles and installs the library to build. This will allow us extra freedom for different implementations, e. g. for ECL we'll want to build a object file archive additionally to fasl files in order to be able to link proper executables. * `genLoadLisp` and `genTestLisp` are almost generic (the former just sometimes would need to use different file extensions), but we integrate them into the implementation “API” to facilitate minor tweaks we need to do like the `fasc` extension for ECL's native FASL files. Change-Id: I1b8ccc0063159638ec7af534e9a6b5384e750193 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3292 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-08-24 r/2753 refactor(users/grfn/gws.fyi): implement isDirectory in pure nixsterni1-0/+65
Another day, another import from derivation avoided by builtins.unsafeDiscardStringContext! Change-Id: I67274b1ba13ba980bb3346b22f2955c702aa3151 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3372 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi> Reviewed-by: tazjin <mail@tazj.in>
2021-08-12 r/2722 docs(nix/buildLisp): mention drawback of non-parallel compilationsterni1-0/+4
To be fair this hardly matters since SBCL is quite fast, but compiling ironclad with ECL is quite the experience… Change-Id: Ib89cc50e5d557acec51fdb085bcbdfc99736221e Reviewed-on: https://cl.tvl.fyi/c/depot/+/3342 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-08-10 r/2716 fix(nix/buildLisp): remove misplaced `makeOverridable`ssterni1-2/+1
`makeOverriddable` doesn't work for bundled sbclWith as is because it uses the `//` operator internally which doesn't work with the types `bundled` and `sbclWith` accept as arguments (string and list respectively). What's more, `bundled` already uses `makeOverridable` and allows to override the internal call to `library` via `overrideLisp`. For `sbclWith` no such mechanism exists, but this seems to be no concern for now: Using `overrideLisp` for this hasn't worked so far (and failed with a _hideous_ evaluation error), so there doesn't seem to be any real demand for this feature. Maybe a feature for another CL. Change-Id: I0b2f34c00a2143cd66dd43a6b1b2880af997ee50 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3296 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-08-01 r/2701 refactor(nix/buildLisp): use passthru for extra attributessterni1-15/+17
Using passthru and appending the attributes via `//` have the same effect with a subtle difference: In the latter case re-evaluating the derivation when using the underlying `mkDerivation`'s `overrideAttrs` will delete all appended attributes. Using passthru at least preserves the attributes although the self reference to the derivation in `passthru.sbcl` will become outdated (unless updated manually). Change-Id: I8b85009f386b9375b86a23fd50c4ec8c6a9dea7f Reviewed-on: https://cl.tvl.fyi/c/depot/+/3257 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: grfn <grfn@gws.fyi>