buildLisp.nix
This is a build system for Common Lisp, written in Nix.
It aims to offer an alternative to ASDF for users who live in a Nix-based ecosystem. This offers several advantages over ASDF:
- Simpler (almost logic-less) package definitions
- Easy linking of native dependencies (from Nix)
- Composability with Nix tooling for other languages
- Effective, per-system caching strategies
- Easy overriding of dependencies and whatnot
- Convenient support for multiple Common Lisp implementations
- ... and more!
The project is still in its early stages and some important restrictions should be highlighted:
- Extending
buildLisp
with support for a custom implementation currently requires some knowledge of internals and may not be considered stable yet. - Parallel compilation is not possible: Since buildLisp doesn't encode dependencies between components (i. e. source files) like ASDF, it must compile source files in sequence to avoid errors due to undefined symbols.
Usage
buildLisp
exposes four different functions:
-
buildLisp.library
: Builds a collection of Lisp files into a library.parameter type use required? name
string
Name of the library yes srcs
list<path>
List of paths to source files yes deps
list<drv>
List of dependencies no native
list<drv>
List of native dependencies no test
see "Tests" Specification for test suite no implementation
see "Implementations" Common Lisp implementation to use no The output of invoking this is a directory containing a FASL file that is the concatenated result of all compiled sources.
-
buildLisp.program
: Builds an executable program out of Lisp files.parameter type use required? name
string
Name of the program yes srcs
list<path>
List of paths to source files yes deps
list<drv>
List of dependencies no native
list<drv>
List of native dependencies no main
string
Entrypoint function no test
see "Tests" Specification for test suite no implementation
see "Implementations" Common Lisp implementation to use no The
main
parameter should be the name of a function and defaults to${name}:main
(i.e. the exportedmain
function of the package named after the program).The output of invoking this is a directory containing a
bin/${name}
. -
buildLisp.bundled
: Creates a virtual dependency on a built-in library.Certain libraries ship with Lisp implementations, for example UIOP/ASDF are commonly included but many implementations also ship internals (such as SBCLs various
sb-*
libraries).This function takes a single string argument that is the name of a built-in library and returns a "package" that simply requires this library.
Tests
Both buildLisp.library
and buildLisp.program
take an optional argument
tests
, which has the following supported fields:
parameter | type | use | required? |
---|---|---|---|
name |
string |
Name of the test suite | no |
expression |
string |
Lisp expression to run tests | yes |
srcs |
list<path> |
List of paths to source files | no |
native |
list<drv> |
List of native dependencies | no |
the expression
parameter should be a Lisp expression and will be evaluated
after loading all sources and dependencies (including library/program
dependencies). It must return a non-NIL
value if the test suite has passed.
Example
Using buildLisp could look like this:
{ buildLisp, lispPkgs }: let libExample = buildLisp.library { name = "lib-example"; srcs = [ ./lib.lisp ]; deps = with lispPkgs; [ (buildLisp.bundled "sb-posix") iterate cl-ppcre ]; }; in buildLisp.program { name = "example"; deps = [ libExample ]; srcs = [ ./main.lisp ]; tests = { deps = [ lispPkgs.fiveam ]; srcs = [ ./tests.lisp ]; expression = "(fiveam:run!)"; }; }
Development REPLs
buildLisp
builds loadable variants of both program
and library
derivations
(usually FASL files). Therefore it can provide a convenient way to obtain an
instance of any implementation preloaded with buildLisp
-derivations. This
is especially useful to use as a host for Sly or SLIME.
-
buildLisp.sbcl.lispWith
,buildLisp.ccl.lispWith
, ...: Creates a wrapper script preloading a Lisp implementation with various dependencies.This function takes a single argument which is a list of Lisp libraries programs or programs. The desired Lisp implementation will load all given derivations and all their dependencies on startup.
The shortcut
buildLisp.sbclWith
forbuildLisp.sbcl.lispWith
is also provided. -
repl
passthru attribute:derivation.repl
is provided as a shortcut forbuildLisp.${implementationName}.lispWith [ derivation ]
.derivation.ccl.repl
,derivation.sbcl.repl
etc. work as well, of course (see also "Implementations" section).
Implementations
Both buildLisp.library
and buildLisp.program
allow specifying a different
Common Lisp implementation than the default one (which is SBCL). When an
implementation is passed, buildLisp
makes sure all dependencies are built
with that implementation as well since build artifacts from different
implementation will be incompatible with each other.
The argument taken by implementation
is a special attribute set which
describes how to do certain tasks for a given implementation, like building
or loading a library. In case you want to use a custom implementation
description, the precise structure needed is documented in buildLisp
's
source code for now. buildLisp
also exposes the following already
working implementation sets:
-
buildLisp.sbcl
: SBCL, our default implementation -
buildLisp.ccl
: CCL, similar to SBCL, but with very good macOS support -
buildLisp.ecl
: ECL setup to produce statically linked binaries and libraries. Note that its runtime library is LGPL, so extra conditions must be fulfilled when distributing binaries produced this way. -
Support for ABCL is planned.
For every of these “known” implementations, buildLisp
will create a passthru
attribute named like the implementation which points to a variant of the derivation
built with said implementation. Say we have a derivation, myDrv
, built using SBCL:
While myDrv
and myDrv.sbcl
are built using SBCL, myDrv.ecl
, myDrv.ccl
etc.
build the derivation and all its dependencies using ECL and CCL respectively.
This is useful to test portability of your derivation, but is also used internally to speed up the “normalization” of the dependency graph. Thus it is important to make sure that your custom implementation's name doesn't clash with one of the “known” ones.
Handling Implementation Specifics
When targeting multiple Common Lisp implementation, it is often necessary to
handle differing interfaces for OS interaction or to make use of special
implementation features. For this reason, buildLisp
allows specifying
dependencies and source files for specific implementations only. This can
be utilized by having an attribute set in the list for the deps
or srcs
argument: buildLisp
will pick the value of the attribute named like the
used implementation or default
and ignore the set completely if both
are missing.
{ buildLisp, lispPkgs }: buildLisp.library { name = "mylib"; srcs = [ # These are included always of course ./package.lisp ./portable-lib.lisp # Choose right impl-* file { sbcl = ./impl-sbcl.lisp; ccl = ./impl-ccl.lisp; ecl = ./impl-ecl.lisp; } # We can also use this to inject extra files { ecl = ./extra-ecl-optimizations.lisp; } ]; deps = [ # Use SBCL's special bundled package, flexi-streams otherwise { sbcl = buildLisp.bundled "sb-rotate-byte"; default = lispPkgs.flexi-streams; } ]; }
Additionally a brokenOn
parameter is accepted which takes a list of
implementation names on which the derivation is not expected to work.
This only influences meta.targets
which is read by depot's CI to
check which variants (see "Implementations") of the derivation to
build, so it may not be useful outside of depot.
Influencing the Lisp Runtime
Lisp implementations which create an executable by dumping an image
usually parse a few implementation-specific command line options on
executable startup that influence runtime settings related to things
like GC. buildLisp
generates a wrapper which makes sure that this
never interferes with the argument parsing implemented in the actual
application, but sometimes it is useful to run an executable with
special settings. To allow this, the content of NIX_BUILDLISP_LISP_ARGS
is passed to the lisp implementation.
For example, you can make the underlying SBCL print its version for
any executable built with buildLisp
(and SBCL) like this:
$ env NIX_BUILDLISP_LISP_ARGS="--version" ./result/bin/🕰️ SBCL 2.1.2.nixos
In practice you'd probably want to specify options like
--dynamic-space-size
or --tls-limit
(try passing --help
for a
full list). Naturally, these options are completely different for
different implementations.