From 8901acc97664aa8ebf687ee904428aa57a5192be Mon Sep 17 00:00:00 2001 From: Mikey Ariel Date: Wed, 27 Aug 2014 18:41:09 +0200 Subject: Restructuring the Nix manual --- doc/manual/writing-nix-expressions.xml | 1901 -------------------------------- 1 file changed, 1901 deletions(-) delete mode 100644 doc/manual/writing-nix-expressions.xml (limited to 'doc/manual/writing-nix-expressions.xml') diff --git a/doc/manual/writing-nix-expressions.xml b/doc/manual/writing-nix-expressions.xml deleted file mode 100644 index 0470625ff6d3..000000000000 --- a/doc/manual/writing-nix-expressions.xml +++ /dev/null @@ -1,1901 +0,0 @@ - - -Writing Nix Expressions - - -This chapter shows you how to write Nix expressions, which are -the things that tell Nix how to build packages. It starts with a -simple example (a Nix expression for GNU Hello), and then moves -on to a more in-depth look at the Nix expression language. - -This chapter is mostly about the Nix expression language. -For more extensive information on adding packages to the Nix Packages -collection (such as functions in the standard environment and coding -conventions), please consult its -manual. - - -
A simple Nix expression - -This section shows how to add and test the GNU Hello -package to the Nix Packages collection. Hello is a program -that prints out the text Hello, world!. - -To add a package to the Nix Packages collection, you generally -need to do three things: - - - - Write a Nix expression for the package. This is a - file that describes all the inputs involved in building the package, - such as dependencies, sources, and so on. - - Write a builder. This is a - shell scriptIn fact, it can be written in any - language, but typically it's a bash shell - script. that actually builds the package from - the inputs. - - Add the package to the file - pkgs/top-level/all-packages.nix. The Nix - expression written in the first step is a - function; it requires other packages in order - to build it. In this step you put it all together, i.e., you call - the function with the right arguments to build the actual - package. - - - - - - -
The Nix expression - -Nix expression for GNU Hello -(<filename>default.nix</filename>) - -{ stdenv, fetchurl, perl }: - -stdenv.mkDerivation { - name = "hello-2.1.1"; - builder = ./builder.sh; - src = fetchurl { - url = ftp://ftp.nluug.nl/pub/gnu/hello/hello-2.1.1.tar.gz; - md5 = "70c9ccf9fac07f762c24f2df2290784d"; - }; - inherit perl; -} - - - shows a Nix expression for GNU -Hello. It's actually already in the Nix Packages collection in -pkgs/applications/misc/hello/ex-1/default.nix. -It is customary to place each package in a separate directory and call -the single Nix expression in that directory -default.nix. The file has the following elements -(referenced from the figure by number): - - - - - - This states that the expression is a - function that expects to be called with three - arguments: stdenv, fetchurl, - and perl. They are needed to build Hello, but - we don't know how to build them here; that's why they are function - arguments. stdenv is a package that is used - by almost all Nix Packages packages; it provides a - standard environment consisting of the things you - would expect in a basic Unix environment: a C/C++ compiler (GCC, - to be precise), the Bash shell, fundamental Unix tools such as - cp, grep, - tar, etc. fetchurl is a - function that downloads files. perl is the - Perl interpreter. - - Nix functions generally have the form { x, y, ..., - z }: e where x, y, - etc. are the names of the expected arguments, and where - e is the body of the function. So - here, the entire remainder of the file is the body of the - function; when given the required arguments, the body should - describe how to build an instance of the Hello package. - - - - - - So we have to build a package. Building something from - other stuff is called a derivation in Nix (as - opposed to sources, which are built by humans instead of - computers). We perform a derivation by calling - stdenv.mkDerivation. - mkDerivation is a function provided by - stdenv that builds a package from a set of - attributes. A set is just a list of - key/value pairs where each key is a string and each value is an - arbitrary Nix expression. They take the general form { - name1 = - expr1; ... - nameN = - exprN; }. - - - - - - The attribute name specifies the symbolic - name and version of the package. Nix doesn't really care about - these things, but they are used by for instance nix-env - -q to show a human-readable name for - packages. This attribute is required by - mkDerivation. - - - - - - The attribute builder specifies the - builder. This attribute can sometimes be omitted, in which case - mkDerivation will fill in a default builder - (which does a configure; make; make install, in - essence). Hello is sufficiently simple that the default builder - would suffice, but in this case, we will show an actual builder - for educational purposes. The value - ./builder.sh refers to the shell script shown - in , discussed below. - - - - - - The builder has to know what the sources of the package - are. Here, the attribute src is bound to the - result of a call to the fetchurl function. - Given a URL and an MD5 hash of the expected contents of the file - at that URL, this function builds a derivation that downloads the - file and checks its hash. So the sources are a dependency that - like all other dependencies is built before Hello itself is - built. - - Instead of src any other name could have - been used, and in fact there can be any number of sources (bound - to different attributes). However, src is - customary, and it's also expected by the default builder (which we - don't use in this example). - - - - - - Since the derivation requires Perl, we have to pass the - value of the perl function argument to the - builder. All attributes in the set are actually passed as - environment variables to the builder, so declaring an attribute - - -perl = perl; - - will do the trick: it binds an attribute perl - to the function argument which also happens to be called - perl. However, it looks a bit silly, so there - is a shorter syntax. The inherit keyword - causes the specified attributes to be bound to whatever variables - with the same name happen to be in scope. - - - - - - - -
- - -
The builder - -Build script for GNU Hello -(<filename>builder.sh</filename>) - -source $stdenv/setup - -PATH=$perl/bin:$PATH - -tar xvfz $src -cd hello-* -./configure --prefix=$out -make -make install - - - shows the builder referenced -from Hello's Nix expression (stored in -pkgs/applications/misc/hello/ex-1/builder.sh). -The builder can actually be made a lot shorter by using the -generic builder functions provided by -stdenv, but here we write out the build steps to -elucidate what a builder does. It performs the following -steps: - - - - - - When Nix runs a builder, it initially completely clears the - environment (except for the attributes declared in the - derivation). For instance, the PATH variable is - emptyActually, it's initialised to - /path-not-set to prevent Bash from setting it - to a default value.. This is done to prevent - undeclared inputs from being used in the build process. If for - example the PATH contained - /usr/bin, then you might accidentally use - /usr/bin/gcc. - - So the first step is to set up the environment. This is - done by calling the setup script of the - standard environment. The environment variable - stdenv points to the location of the standard - environment being used. (It wasn't specified explicitly as an - attribute in , but - mkDerivation adds it automatically.) - - - - - - Since Hello needs Perl, we have to make sure that Perl is in - the PATH. The perl environment - variable points to the location of the Perl package (since it - was passed in as an attribute to the derivation), so - $perl/bin is the - directory containing the Perl interpreter. - - - - - - Now we have to unpack the sources. The - src attribute was bound to the result of - fetching the Hello source tarball from the network, so the - src environment variable points to the location in - the Nix store to which the tarball was downloaded. After - unpacking, we cd to the resulting source - directory. - - The whole build is performed in a temporary directory - created in /tmp, by the way. This directory is - removed after the builder finishes, so there is no need to clean - up the sources afterwards. Also, the temporary directory is - always newly created, so you don't have to worry about files from - previous builds interfering with the current build. - - - - - - GNU Hello is a typical Autoconf-based package, so we first - have to run its configure script. In Nix - every package is stored in a separate location in the Nix store, - for instance - /nix/store/9a54ba97fb71b65fda531012d0443ce2-hello-2.1.1. - Nix computes this path by cryptographically hashing all attributes - of the derivation. The path is passed to the builder through the - out environment variable. So here we give - configure the parameter - --prefix=$out to cause Hello to be installed in - the expected location. - - - - - - Finally we build Hello (make) and install - it into the location specified by out - (make install). - - - - - -If you are wondering about the absence of error checking on the -result of various commands called in the builder: this is because the -shell script is evaluated with Bash's option, -which causes the script to be aborted if any command fails without an -error check. - -
- - -
Composition - -Composing GNU Hello -(<filename>all-packages.nix</filename>) - -... - -rec { - - hello = import ../applications/misc/hello/ex-1 { - inherit fetchurl stdenv perl; - }; - - perl = import ../development/interpreters/perl { - inherit fetchurl stdenv; - }; - - fetchurl = import ../build-support/fetchurl { - inherit stdenv; ... - }; - - stdenv = ...; - -} - - - -The Nix expression in is a -function; it is missing some arguments that have to be filled in -somewhere. In the Nix Packages collection this is done in the file -pkgs/top-level/all-packages.nix, where all -Nix expressions for packages are imported and called with the -appropriate arguments. shows -some fragments of -all-packages.nix. - - - - - - This file defines a set of attributes, all of which are - concrete derivations (i.e., not functions). In fact, we define a - mutually recursive set of attributes. That - is, the attributes can refer to each other. This is precisely - what we want since we want to plug the - various packages into each other. - - - - - - Here we import the Nix expression for - GNU Hello. The import operation just loads and returns the - specified Nix expression. In fact, we could just have put the - contents of in - all-packages.nix at this point. That - would be completely equivalent, but it would make the file rather - bulky. - - Note that we refer to - ../applications/misc/hello/ex-1, not - ../applications/misc/hello/ex-1/default.nix. - When you try to import a directory, Nix automatically appends - /default.nix to the file name. - - - - - - This is where the actual composition takes place. Here we - call the function imported from - ../applications/misc/hello/ex-1 with a set - containing the things that the function expects, namely - fetchurl, stdenv, and - perl. We use inherit again to use the - attributes defined in the surrounding scope (we could also have - written fetchurl = fetchurl;, etc.). - - The result of this function call is an actual derivation - that can be built by Nix (since when we fill in the arguments of - the function, what we get is its body, which is the call to - stdenv.mkDerivation in ). - - Nixpkgs has a convenience function - callPackage that imports and calls a - function, filling in any missing arguments by passing the - corresponding attribute from the Nixpkgs set, like this: - - -hello = callPackage ../applications/misc/hello/ex-1 { }; - - - If necessary, you can set or override arguments: - - -hello = callPackage ../applications/misc/hello/ex-1 { stdenv = myStdenv; }; - - - - - - - - - Likewise, we have to instantiate Perl, - fetchurl, and the standard environment. - - - - - -
- - -
Testing - -You can now try to build Hello. Of course, you could do -nix-env -f pkgs/top-level/all-packages.nix -i hello, -but you may not want to install a possibly broken package just yet. -The best way to test the package is by using the command nix-build, which builds a Nix -expression and creates a symlink named result in -the current directory: - - -$ nix-build pkgs/top-level/all-packages.nix -A hello -building path `/nix/store/632d2b22514d...-hello-2.1.1' -hello-2.1.1/ -hello-2.1.1/intl/ -hello-2.1.1/intl/ChangeLog -... - -$ ls -l result -lrwxrwxrwx ... 2006-09-29 10:43 result -> /nix/store/632d2b22514d...-hello-2.1.1 - -$ ./result/bin/hello -Hello, world! - -The option selects -the hello attribute from -all-packages.nix. This is faster than using the -symbolic package name specified by the name -attribute (which also happens to be hello) and is -unambiguous (there can be multiple packages with the symbolic name -hello, but there can be only one attribute in a set -named hello). - -nix-build registers the -./result symlink as a garbage collection root, so -unless and until you delete the ./result symlink, -the output of the build will be safely kept on your system. You can -use nix-build’s switch to give the symlink another -name. - -Nix has a transactional semantics. Once a build finishes -successfully, Nix makes a note of this in its database: it registers -that the path denoted by out is now -valid. If you try to build the derivation again, Nix -will see that the path is already valid and finish immediately. If a -build fails, either because it returns a non-zero exit code, because -Nix or the builder are killed, or because the machine crashes, then -the output paths will not be registered as valid. If you try to build -the derivation again, Nix will remove the output paths if they exist -(e.g., because the builder died half-way through make -install) and try again. Note that there is no -negative caching: Nix doesn't remember that a build -failed, and so a failed build can always be repeated. This is because -Nix cannot distinguish between permanent failures (e.g., a compiler -error due to a syntax error in the source) and transient failures -(e.g., a disk full condition). - -Nix also performs locking. If you run multiple Nix builds -simultaneously, and they try to build the same derivation, the first -Nix instance that gets there will perform the build, while the others -block (or perform other derivations if available) until the build -finishes: - - -$ nix-build pkgs/top-level/all-packages.nix -A hello -waiting for lock on `/nix/store/0h5b7hp8d4hqfrw8igvx97x1xawrjnac-hello-2.1.1x' - -So it is always safe to run multiple instances of Nix in parallel -(which isn’t the case with, say, make). - -If you have a system with multiple CPUs, you may want to have -Nix build different derivations in parallel (insofar as possible). -Just pass the option , where -N is the maximum number of jobs to be run -in parallel, or set. Typically this should be the number of -CPUs. - -
- - -
The generic builder - -Recall from that the builder -looked something like this: - - -PATH=$perl/bin:$PATH -tar xvfz $src -cd hello-* -./configure --prefix=$out -make -make install - -The builders for almost all Unix packages look like this — set up some -environment variables, unpack the sources, configure, build, and -install. For this reason the standard environment provides some Bash -functions that automate the build process. A builder using the -generic build facilities in shown in . - -Build script using the generic -build functions - -buildInputs="$perl" - -source $stdenv/setup - -genericBuild - - - - - - - The buildInputs variable tells - setup to use the indicated packages as - inputs. This means that if a package provides a - bin subdirectory, it's added to - PATH; if it has a include - subdirectory, it's added to GCC's header search path; and so - on.How does it work? setup - tries to source the file - pkg/nix-support/setup-hook - of all dependencies. These “setup hooks” can then set up whatever - environment variables they want; for instance, the setup hook for - Perl sets the PERL5LIB environment variable to - contain the lib/site_perl directories of all - inputs. - - - - - - - The function genericBuild is defined in - the file $stdenv/setup. - - - - - - The final step calls the shell function - genericBuild, which performs the steps that - were done explicitly in . The - generic builder is smart enough to figure out whether to unpack - the sources using gzip, - bzip2, etc. It can be customised in many ways; - see . - - - - - -Discerning readers will note that the -buildInputs could just as well have been set in the Nix -expression, like this: - - - buildInputs = [ perl ]; - -The perl attribute can then be removed, and the -builder becomes even shorter: - - -source $stdenv/setup -genericBuild - -In fact, mkDerivation provides a default builder -that looks exactly like that, so it is actually possible to omit the -builder for Hello entirely. - -
- - -
- - - -
The Nix expression language - -The Nix expression language is a pure, lazy, functional -language. Purity means that operations in the language don't have -side-effects (for instance, there is no variable assignment). -Laziness means that arguments to functions are evaluated only when -they are needed. Functional means that functions are -normal values that can be passed around and manipulated -in interesting ways. The language is not a full-featured, general -purpose language. Its main job is to describe packages, -compositions of packages, and the variability within -packages. - -This section presents the various features of the -language. - - -
Values - - -Simple values - -Nix has the following basic data types: - - - - - - Strings can be written in three - ways. - - The most common way is to enclose the string between double - quotes, e.g., "foo bar". Strings can span - multiple lines. The special characters " and - \ and the character sequence - ${ must be escaped by prefixing them with a - backslash (\). Newlines, carriage returns and - tabs can be written as \n, - \r and \t, - respectively. - - You can include the result of an expression into a string by - enclosing it in - ${...}, a feature - known as antiquotation. The enclosed - expression must evaluate to something that can be coerced into a - string (meaning that it must be a string, a path, or a - derivation). For instance, rather than writing - - -"--with-freetype2-library=" + freetype + "/lib" - - (where freetype is a derivation), you can - instead write the more natural - - -"--with-freetype2-library=${freetype}/lib" - - The latter is automatically translated to the former. A more - complicated example (from the Nix expression for Qt): - - -configureFlags = " - -system-zlib -system-libpng -system-libjpeg - ${if openglSupport then "-dlopen-opengl - -L${mesa}/lib -I${mesa}/include - -L${libXmu}/lib -I${libXmu}/include" else ""} - ${if threadSupport then "-thread" else "-no-thread"} -"; - - Note that Nix expressions and strings can be arbitrarily nested; - in this case the outer string contains various antiquotations that - themselves contain strings (e.g., "-thread"), - some of which in turn contain expressions (e.g., - ${mesa}). - - The second way to write string literals is as an - indented string, which is enclosed between - pairs of double single-quotes, like so: - - -'' - This is the first line. - This is the second line. - This is the third line. -'' - - This kind of string literal intelligently strips indentation from - the start of each line. To be precise, it strips from each line a - number of spaces equal to the minimal indentation of the string as - a whole (disregarding the indentation of empty lines). For - instance, the first and second line are indented two space, while - the third line is indented four spaces. Thus, two spaces are - stripped from each line, so the resulting string is - - -"This is the first line.\nThis is the second line.\n This is the third line.\n" - - - - Note that the whitespace and newline following the opening - '' is ignored if there is no non-whitespace - text on the initial line. - - Antiquotation - (${expr}) is - supported in indented strings. - - Since ${ and '' have - special meaning in indented strings, you need a way to quote them. - ${ can be escaped by prefixing it with - '' (that is, two single quotes), i.e., - ''${. '' can be escaped by - prefixing it with ', i.e., - '''. Finally, linefeed, carriage-return and - tab characters can be written as ''\n, - ''\r, ''\t. - - Indented strings are primarily useful in that they allow - multi-line string literals to follow the indentation of the - enclosing Nix expression, and that less escaping is typically - necessary for strings representing languages such as shell scripts - and configuration files because '' is much less - common than ". Example: - - -stdenv.mkDerivation { - ... - postInstall = - '' - mkdir $out/bin $out/etc - cp foo $out/bin - echo "Hello World" > $out/etc/foo.conf - ${if enableBar then "cp bar $out/bin" else ""} - ''; - ... -} - - - - - Finally, as a convenience, URIs as - defined in appendix B of RFC 2396 - can be written as is, without quotes. For - instance, the string - "http://example.org/foo.tar.bz2" - can also be written as - http://example.org/foo.tar.bz2. - - - - Integers, e.g., - 123. - - Paths, e.g., - /bin/sh or ./builder.sh. - A path must contain at least one slash to be recognised as such; for - instance, builder.sh is not a - pathIt's parsed as an expression that selects the - attribute sh from the variable - builder.. If the file name is - relative, i.e., if it does not begin with a slash, it is made - absolute at parse time relative to the directory of the Nix - expression that contained it. For instance, if a Nix expression in - /foo/bar/bla.nix refers to - ../xyzzy/fnord.nix, the absolute path is - /foo/xyzzy/fnord.nix. - - Booleans with values - true and - false. - - The null value, denoted as - null. - - - - - - - - -Lists - -Lists are formed by enclosing a whitespace-separated list of -values between square brackets. For example, - - -[ 123 ./foo.nix "abc" (f { x = y; }) ] - -defines a list of four elements, the last being the result of a call -to the function f. Note that function calls have -to be enclosed in parentheses. If they had been omitted, e.g., - - -[ 123 ./foo.nix "abc" f { x = y; } ] - -the result would be a list of five elements, the fourth one being a -function and the fifth being a set. - - - - -Sets - -Sets are really the core of the language, since ultimately the -Nix language is all about creating derivations, which are really just -sets of attributes to be passed to build scripts. - -Sets are just a list of name/value pairs (called -attributes) enclosed in curly brackets, where -each value is an arbitrary expression terminated by a semicolon. For -example: - - -{ x = 123; - text = "Hello"; - y = f { bla = 456; }; -} - -This defines a set with attributes named x, -text, y. The order of the -attributes is irrelevant. An attribute name may only occur -once. - -Attributes can be selected from a set using the -. operator. For instance, - - -{ a = "Foo"; b = "Bar"; }.a - -evaluates to "Foo". It is possible to provide a -default value in an attribute selection using the -or keyword. For example, - - -{ a = "Foo"; b = "Bar"; }.c or "Xyzzy" - -will evaluate to "Xyzzy" because there is no -c attribute in the set. - -You can use arbitrary double-quoted strings as attribute -names: - - -{ "foo ${bar}" = 123; "nix-1.0" = 456; }."foo ${bar}" - - -This will evaluate to 123 (Assuming -bar is antiquotable). In the case where an -attribute name is just a single antiquotation, the quotes can be -dropped: - - -{ foo = 123; }.${bar} or 456 - -This will evaluate to 123 if -bar evaluates to "foo" when -coerced to a string and 456 otherwise (again -assuming bar is antiquotable). - -In the special case where an attribute name inside of a set declaration -evaluates to null (which is normally an error, as -null is not antiquotable), that attribute is simply not -added to the set: - - -{ ${if foo then "bar" else null} = true; } - -This will evaluate to {} if foo -evaluates to false. - - - - - -
- - -
Language constructs - - -Recursive sets - -Recursive sets are just normal sets, but the attributes can -refer to each other. For example, - - -rec { - x = y; - y = 123; -}.x - - -evaluates to 123. Note that without -rec the binding x = y; would -refer to the variable y in the surrounding scope, -if one exists, and would be invalid if no such variable exists. That -is, in a normal (non-recursive) set, attributes are not added to the -lexical scope; in a recursive set, they are. - -Recursive sets of course introduce the danger of infinite -recursion. For example, - - -rec { - x = y; - y = x; -}.x - -does not terminateActually, Nix detects infinite -recursion in this case and aborts (infinite recursion -encountered).. - - - - -Let-expressions - -A let-expression allows you define local variables for an -expression. For instance, - - -let - x = "foo"; - y = "bar"; -in x + y - -evaluates to "foobar". - - - - - - -Inheriting attributes - -When defining a set it is often convenient to copy variables -from the surrounding lexical scope (e.g., when you want to propagate -attributes). This can be shortened using the -inherit keyword. For instance, - - -let x = 123; in -{ inherit x; - y = 456; -} - -evaluates to { x = 123; y = 456; }. (Note that -this works because x is added to the lexical scope -by the let construct.) It is also possible to -inherit attributes from another set. For instance, in this fragment -from all-packages.nix, - - - graphviz = (import ../tools/graphics/graphviz) { - inherit fetchurl stdenv libpng libjpeg expat x11 yacc; - inherit (xlibs) libXaw; - }; - - xlibs = { - libX11 = ...; - libXaw = ...; - ... - } - - libpng = ...; - libjpg = ...; - ... - -the set used in the function call to the function defined in -../tools/graphics/graphviz inherits a number of -variables from the surrounding scope (fetchurl -... yacc), but also inherits -libXaw (the X Athena Widgets) from the -xlibs (X11 client-side libraries) set. - - - - -Functions - -Functions have the following form: - - -pattern: body - -The pattern specifies what the argument of the function must look -like, and binds variables in the body to (parts of) the -argument. There are three kinds of patterns: - - - - - If a pattern is a single identifier, then the - function matches any argument. Example: - - -let negate = x: !x; - concat = x: y: x + y; -in if negate true then concat "foo" "bar" else "" - - Note that concat is a function that takes one - argument and returns a function that takes another argument. This - allows partial parameterisation (i.e., only filling some of the - arguments of a function); e.g., - - -map (concat "foo") [ "bar" "bla" "abc" ] - - evaluates to [ "foobar" "foobla" - "fooabc" ]. - - - A set pattern of the form - { name1, name2, …, nameN } matches a set - containing the listed attributes, and binds the values of those - attributes to variables in the function body. For example, the - function - - -{ x, y, z }: z + y + x - - can only be called with a set containing exactly the attributes - x, y and - z. No other attributes are allowed. If you want - to allow additional arguments, you can use an ellipsis - (...): - - -{ x, y, z, ... }: z + y + x - - This works on any set that contains at least the three named - attributes. - - It is possible to provide default values - for attributes, in which case they are allowed to be missing. A - default value is specified by writing - name ? - e, where - e is an arbitrary expression. For example, - - -{ x, y ? "foo", z ? "bar" }: z + y + x - - specifies a function that only requires an attribute named - x, but optionally accepts y - and z. - - - An @-pattern provides a means of referring - to the whole value being matched: - - -args@{ x, y, z, ... }: z + y + x + args.a - - Here args is bound to the entire argument, which - is further matched against the pattern { x, y, z, - ... }. - - - - -Note that functions do not have names. If you want to give them -a name, you can bind them to an attribute, e.g., - - -let concat = { x, y }: x + y; -in concat { x = "foo"; y = "bar"; } - - - - - - -Conditionals - -Conditionals look like this: - - -if e1 then e2 else e3 - -where e1 is an expression that should -evaluate to a Boolean value (true or -false). - - - - -Assertions - -Assertions are generally used to check that certain requirements -on or between features and dependencies hold. They look like this: - - -assert e1; e2 - -where e1 is an expression that should -evaluate to a Boolean value. If it evaluates to -true, e2 is returned; -otherwise expression evaluation is aborted and a backtrace is printed. - -Nix expression for Subversion - -{ localServer ? false -, httpServer ? false -, sslSupport ? false -, pythonBindings ? false -, javaSwigBindings ? false -, javahlBindings ? false -, stdenv, fetchurl -, openssl ? null, httpd ? null, db4 ? null, expat, swig ? null, j2sdk ? null -}: - -assert localServer -> db4 != null; -assert httpServer -> httpd != null && httpd.expat == expat; -assert sslSupport -> openssl != null && (httpServer -> httpd.openssl == openssl); -assert pythonBindings -> swig != null && swig.pythonSupport; -assert javaSwigBindings -> swig != null && swig.javaSupport; -assert javahlBindings -> j2sdk != null; - -stdenv.mkDerivation { - name = "subversion-1.1.1"; - ... - openssl = if sslSupport then openssl else null; - ... -} - - - show how assertions are -used in the Nix expression for Subversion. - - - - - This assertion states that if Subversion is to have support - for local repositories, then Berkeley DB is needed. So if the - Subversion function is called with the - localServer argument set to - true but the db4 argument - set to null, then the evaluation fails. - - - - This is a more subtle condition: if Subversion is built with - Apache (httpServer) support, then the Expat - library (an XML library) used by Subversion should be same as the - one used by Apache. This is because in this configuration - Subversion code ends up being linked with Apache code, and if the - Expat libraries do not match, a build- or runtime link error or - incompatibility might occur. - - - - This assertion says that in order for Subversion to have SSL - support (so that it can access https URLs), an - OpenSSL library must be passed. Additionally, it says that - if Apache support is enabled, then Apache's - OpenSSL should match Subversion's. (Note that if Apache support - is not enabled, we don't care about Apache's OpenSSL.) - - - - The conditional here is not really related to assertions, - but is worth pointing out: it ensures that if SSL support is - disabled, then the Subversion derivation is not dependent on - OpenSSL, even if a non-null value was passed. - This prevents an unnecessary rebuild of Subversion if OpenSSL - changes. - - - - - - - - -With-expressions - -A with-expression, - - -with e1; e2 - -introduces the set e1 into the lexical -scope of the expression e2. For instance, - - -let as = { x = "foo"; y = "bar"; }; -in with as; x + y - -evaluates to "foobar" since the -with adds the x and -y attributes of as to the -lexical scope in the expression x + y. The most -common use of with is in conjunction with the -import function. E.g., - - -with (import ./definitions.nix); ... - -makes all attributes defined in the file -definitions.nix available as if they were defined -locally in a rec-expression. - - - - -Comments - -Comments can be single-line, started with a # -character, or inline/multi-line, enclosed within /* -... */. - - - - -
- - -
Operators - - lists the operators in the -Nix expression language, in order of precedence (from strongest to -weakest binding). - - - Operators - - - - Syntax - Associativity - Description - - - - - e . - attrpath - [ or def ] - - none - Select attribute denoted by the attribute path - attrpath from set - e. (An attribute path is a - dot-separated list of attribute names.) If the attribute - doesn’t exist, return def if - provided, otherwise abort evaluation. - - - e1 e2 - left - Call function e1 with - argument e2. - - - e ? - attrpath - none - Test whether set e contains - the attribute denoted by attrpath; - return true or - false. - - - e1 ++ e2 - right - List concatenation. - - - e1 + e2 - left - String or path concatenation. - - - ! e - left - Boolean negation. - - - e1 // - e2 - right - Return a set consisting of the attributes in - e1 and - e2 (with the latter taking - precedence over the former in case of equally named - attributes). - - - e1 == - e2 - none - Equality. - - - e1 != - e2 - none - Inequality. - - - e1 && - e2 - left - Logical AND. - - - e1 || - e2 - left - Logical OR. - - - e1 -> - e2 - none - Logical implication (equivalent to - !e1 || - e2). - - - -
- -
- - -
Derivations - -The most important built-in function is -derivation, which is used to describe a single -derivation (a build action). It takes as input a set, the attributes -of which specify the inputs of the build. - - - - There must be an attribute named - system whose value must be a string specifying a - Nix platform identifier, such as "i686-linux" or - "powerpc-darwin"To figure out - your platform identifier, look at the line Checking for the - canonical Nix system name in the output of Nix's - configure script. The build - can only be performed on a machine and operating system matching the - platform identifier. (Nix can automatically forward builds for - other platforms by forwarding them to other machines; see .) - - There must be an attribute named - name whose value must be a string. This is used - as a symbolic name for the package by nix-env, - and it is appended to the output paths of the - derivation. - - There must be an attribute named - builder that identifies the program that is - executed to perform the build. It can be either a derivation or a - source (a local file reference, e.g., - ./builder.sh). - - Every attribute is passed as an environment variable - to the builder. Attribute values are translated to environment - variables as follows: - - - - Strings and integers are just passed - verbatim. - - A path (e.g., - ../foo/sources.tar) causes the referenced - file to be copied to the store; its location in the store is put - in the environment variable. The idea is that all sources - should reside in the Nix store, since all inputs to a derivation - should reside in the Nix store. - - A derivation causes that - derivation to be built prior to the present derivation; its - default output path is put in the environment - variable. - - Lists of the previous types are also allowed. - They are simply concatenated, separated by - spaces. - - true is passed as the string - 1, false and - null are passed as an empty string. - - - - - - The optional attribute args - specifies command-line arguments to be passed to the builder. It - should be a list. - - The optional attribute outputs - specifies a list of symbolic outputs of the derivation. By default, - a derivation produces a single output path, denoted as - out. However, derivations can produce multiple - output paths. This is useful because it allows outputs to be - downloaded or garbage-collected separately. For instance, imagine a - library package that provides a dynamic library, header files, and - documentation. A program that links against the library doesn’t - need the header files and documentation at runtime, and it doesn’t - need the documentation at build time. Thus, the library package - could specify: - -outputs = [ "lib" "headers" "doc" ]; - - This will cause Nix to pass environment variables - lib, headers and - doc to the builder containing the intended store - paths of each output. The builder would typically do something like - -./configure --libdir=$lib/lib --includedir=$headers/include --docdir=$doc/share/doc - - for an Autoconf-style package. You can refer to each output of a - derivation by selecting it as an attribute, e.g. - -buildInputs = [ pkg.lib pkg.headers ]; - - The first element of output determines the - default output. Thus, you could also write - -buildInputs = [ pkg pkg.headers ]; - - since pkg is equivalent to - pkg.lib. - - - -The function mkDerivation in the standard -environment is a wrapper around derivation that -adds a default value for system and always uses -Bash as the builder, to which the supplied builder is passed as a -command-line argument. See . - -The builder is executed as follows: - - - - A temporary directory is created under the directory - specified by TMPDIR (default - /tmp) where the build will take place. The - current directory is changed to this directory. - - The environment is cleared and set to the derivation - attributes, as specified above. - - In addition, the following variables are set: - - - - NIX_BUILD_TOP contains the path of - the temporary directory for this build. - - Also, TMPDIR, - TEMPDIR, TMP, TEMP - are set to point to the temporary directory. This is to prevent - the builder from accidentally writing temporary files anywhere - else. Doing so might cause interference by other - processes. - - PATH is set to - /path-not-set to prevent shells from - initialising it to their built-in default value. - - HOME is set to - /homeless-shelter to prevent programs from - using /etc/passwd or the like to find the - user's home directory, which could cause impurity. Usually, when - HOME is set, it is used as the location of the home - directory, even if it points to a non-existent - path. - - NIX_STORE is set to the path of the - top-level Nix store directory (typically, - /nix/store). - - For each output declared in - outputs, the corresponding environment variable - is set to point to the intended path in the Nix store for that - output. Each output path is a concatenation of the cryptographic - hash of all build inputs, the name attribute - and the output name. (The output name is omitted if it’s - out.) - - - - - - If an output path already exists, it is removed. - Also, locks are acquired to prevent multiple Nix instances from - performing the same build at the same time. - - A log of the combined standard output and error is - written to /nix/var/log/nix. - - The builder is executed with the arguments specified - by the attribute args. If it exits with exit - code 0, it is considered to have succeeded. - - The temporary directory is removed (unless the - option was specified). - - If the build was successful, Nix scans each output - path for references to input paths by looking for the hash parts of - the input paths. Since these are potential runtime dependencies, - Nix registers them as dependencies of the output - paths. - - After the build, Nix sets the last-modified - timestamp on all files in the build result to 1 (00:00:01 1/1/1970 - UTC), sets the group to the default group, and sets the mode of the - file to 0444 or 0555 (i.e., read-only, with execute permission - enabled if the file was originally executable). Note that possible - setuid and setgid bits are - cleared. Setuid and setgid programs are not currently supported by - Nix. This is because the Nix archives used in deployment have no - concept of ownership information, and because it makes the build - result dependent on the user performing the build. - - - - - - -
Advanced attributes - -Derivations can declare some infrequently used optional -attributes. - - - - allowedReferences - - The optional attribute - allowedReferences specifies a list of legal - references (dependencies) of the output of the builder. For - example, - - -allowedReferences = []; - - - enforces that the output of a derivation cannot have any runtime - dependencies on its inputs. To allow an output to have a runtime - dependency on itself, use "out" as a list item. - This is used in NixOS to check that generated files such as - initial ramdisks for booting Linux don’t have accidental - dependencies on other paths in the Nix store. - - - - - exportReferencesGraph - - This attribute allows builders access to the - references graph of their inputs. The attribute is a list of - inputs in the Nix store whose references graph the builder needs - to know. The value of this attribute should be a list of pairs - [ name1 - path1 name2 - path2 ... - ]. The references graph of each - pathN will be stored in a text file - nameN in the temporary build directory. - The text files have the format used by nix-store - --register-validity (with the deriver fields left - empty). For example, when the following derivation is built: - - -derivation { - ... - exportReferencesGraph = [ "libfoo-graph" libfoo ]; -}; - - - the references graph of libfoo is placed in the - file libfoo-graph in the temporary build - directory. - - exportReferencesGraph is useful for - builders that want to do something with the closure of a store - path. Examples include the builders in NixOS that generate the - initial ramdisk for booting Linux (a cpio - archive containing the closure of the boot script) and the - ISO-9660 image for the installation CD (which is populated with a - Nix store containing the closure of a bootable NixOS - configuration). - - - - - - outputHash - outputHashAlgo - outputHashMode - - These attributes declare that the derivation is a - so-called fixed-output derivation, which - means that a cryptographic hash of the output is already known in - advance. When the build of a fixed-output derivation finishes, - Nix computes the cryptographic hash of the output and compares it - to the hash declared with these attributes. If there is a - mismatch, the build fails. - - The rationale for fixed-output derivations is derivations - such as those produced by the fetchurl - function. This function downloads a file from a given URL. To - ensure that the downloaded file has not been modified, the caller - must also specify a cryptographic hash of the file. For example, - - -fetchurl { - url = http://ftp.gnu.org/pub/gnu/hello/hello-2.1.1.tar.gz; - md5 = "70c9ccf9fac07f762c24f2df2290784d"; -} - - - It sometimes happens that the URL of the file changes, e.g., - because servers are reorganised or no longer available. We then - must update the call to fetchurl, e.g., - - -fetchurl { - url = ftp://ftp.nluug.nl/pub/gnu/hello/hello-2.1.1.tar.gz; - md5 = "70c9ccf9fac07f762c24f2df2290784d"; -} - - - If a fetchurl derivation was treated like a - normal derivation, the output paths of the derivation and - all derivations depending on it would change. - For instance, if we were to change the URL of the Glibc source - distribution in Nixpkgs (a package on which almost all other - packages depend) massive rebuilds would be needed. This is - unfortunate for a change which we know cannot have a real effect - as it propagates upwards through the dependency graph. - - For fixed-output derivations, on the other hand, the name of - the output path only depends on the outputHash* - and name attributes, while all other attributes - are ignored for the purpose of computing the output path. (The - name attribute is included because it is part - of the path.) - - As an example, here is the (simplified) Nix expression for - fetchurl: - - -{ stdenv, curl }: # The curl program is used for downloading. - -{ url, md5 }: - -stdenv.mkDerivation { - name = baseNameOf (toString url); - builder = ./builder.sh; - buildInputs = [ curl ]; - - # This is a fixed-output derivation; the output must be a regular - # file with MD5 hash md5. - outputHashMode = "flat"; - outputHashAlgo = "md5"; - outputHash = md5; - - inherit url; -} - - - - - The outputHashAlgo attribute specifies - the hash algorithm used to compute the hash. It can currently be - "md5", "sha1" or - "sha256". - - The outputHashMode attribute determines - how the hash is computed. It must be one of the following two - values: - - - - "flat" - - The output must be a non-executable regular - file. If it isn’t, the build fails. The hash is simply - computed over the contents of that file (so it’s equal to what - Unix commands like md5sum or - sha1sum produce). - - This is the default. - - - - "recursive" - - The hash is computed over the NAR archive dump - of the output (i.e., the result of nix-store - --dump). In this case, the output can be - anything, including a directory tree. - - - - - - - - The outputHash attribute, finally, must - be a string containing the hash in either hexadecimal or base-32 - notation. (See the nix-hash command - for information about converting to and from base-32 - notation.) - - - - - impureEnvVars - - This attribute allows you to specify a list of - environment variables that should be passed from the environment - of the calling user to the builder. Usually, the environment is - cleared completely when the builder is executed, but with this - attribute you can allow specific environment variables to be - passed unmodified. For example, fetchurl in - Nixpkgs has the line - - -impureEnvVars = [ "http_proxy" "https_proxy" ... ]; - - - to make it use the proxy server configuration specified by the - user in the environment variables http_proxy and - friends. - - This attribute is only allowed in fixed-output derivations, where - impurities such as these are okay since (the hash of) the output - is known in advance. It is ignored for all other - derivations. - - - - - preferLocalBuild - - If this attribute is set to - true, it has two effects. First, the - derivation will always be built, not substituted, even if a - substitute is available. Second, if distributed building is - enabled, then, if possible, the derivaton will be built - locally instead of forwarded to a remote machine. This is - appropriate for trivial builders where the cost of doing a - download or remote build would exceed the cost of building - locally. - - - - - -
- - -
- - - - - - -
- - - -
The standard environment - - -The standard environment is used by passing it as an input -called stdenv to the derivation, and then doing - - -source $stdenv/setup - -at the top of the builder. - -Apart from adding the aforementioned commands to the -PATH, setup also does the -following: - - - - All input packages specified in the - buildInputs environment variable have their - /bin subdirectory added to PATH, - their /include subdirectory added to the C/C++ - header file search path, and their /lib - subdirectory added to the linker search path. This can be extended. - For instance, when the pkgconfig package is - used, the subdirectory /lib/pkgconfig of each - input is added to the PKG_CONFIG_PATH environment - variable. - - The environment variable - NIX_CFLAGS_STRIP is set so that the compiler strips - debug information from object files. This can be disabled by - setting NIX_STRIP_DEBUG to - 0. - - - - - -The setup script also exports a function -called genericBuild that knows how to build -typical Autoconf-style packages. It can be customised to perform -builds for any type of package. It is advisable to use -genericBuild since it provides facilities that -are almost always useful such as unpacking of sources, patching of -sources, nested logging, etc. - -The definitive, up-to-date documentation of the generic builder -is the source itself, which resides in -pkgs/stdenv/generic/setup.sh. - - -
Customising the generic builder - -The operation of the generic builder can be modified in many -places by setting certain variables. These hook -variables are typically set to the name of some shell -function defined by you. For instance, to perform some additional -steps after make install you would set the -postInstall variable: - - -postInstall=myPostInstall - -myPostInstall() { - mkdir $out/share/extra - cp extrafiles/* $out/share/extra -} - - - - -
- - -
Debugging failed builds - -At the beginning of each phase, the set of all shell variables -is written to the file env-vars at the top-level -build directory. This is useful for debugging: it allows you to -recreate the environment in which a build was performed. For -instance, if a build fails, then assuming you used the - flag, you can go to the output directory and -switch to the environment of the builder: - - -$ nix-build -K ./foo.nix -... fails, keeping build directory `/tmp/nix-1234-0' - -$ cd /tmp/nix-1234-0 - -$ source env-vars - -(edit some files...) - -$ make - -(execution continues with the same GCC, make, etc.) - - - -
- - -
- - - -- cgit 1.4.1