diff options
Diffstat (limited to 'doc/manual/writing-nix-expressions.xml')
| -rw-r--r-- | doc/manual/writing-nix-expressions.xml | 1918 |
1 files changed, 0 insertions, 1918 deletions
diff --git a/doc/manual/writing-nix-expressions.xml b/doc/manual/writing-nix-expressions.xml deleted file mode 100644 index c4f069893371..000000000000 --- a/doc/manual/writing-nix-expressions.xml +++ /dev/null @@ -1,1918 +0,0 @@ -<chapter xmlns="http://docbook.org/ns/docbook" - xmlns:xlink="http://www.w3.org/1999/xlink" - xml:id='chap-writing-nix-expressions' - xmlns:xi="http://www.w3.org/2001/XInclude"> - -<title>Writing Nix Expressions</title> - - -<para>This chapter shows you how to write Nix expressions, which are -the things that tell Nix how to build packages. It starts with a -simple example (a Nix expression for GNU Hello), and then moves -on to a more in-depth look at the Nix expression language.</para> - -<note><para>This chapter is mostly about the Nix expression language. -For more extensive information on adding packages to the Nix Packages -collection (such as functions in the standard environment and coding -conventions), please consult <link -xlink:href="http://nixos.org/nixpkgs/manual/">its -manual</link>.</para></note> - - -<section><title>A simple Nix expression</title> - -<para>This section shows how to add and test the <link -xlink:href='http://www.gnu.org/software/hello/hello.html'>GNU Hello -package</link> to the Nix Packages collection. Hello is a program -that prints out the text <quote>Hello, world!</quote>.</para> - -<para>To add a package to the Nix Packages collection, you generally -need to do three things: - -<orderedlist> - - <listitem><para>Write a Nix expression for the package. This is a - file that describes all the inputs involved in building the package, - such as dependencies, sources, and so on.</para></listitem> - - <listitem><para>Write a <emphasis>builder</emphasis>. This is a - shell script<footnote><para>In fact, it can be written in any - language, but typically it's a <command>bash</command> shell - script.</para></footnote> that actually builds the package from - the inputs.</para></listitem> - - <listitem><para>Add the package to the file - <filename>pkgs/top-level/all-packages.nix</filename>. The Nix - expression written in the first step is a - <emphasis>function</emphasis>; it requires other packages in order - to build it. In this step you put it all together, i.e., you call - the function with the right arguments to build the actual - package.</para></listitem> - -</orderedlist> - -</para> - - -<section><title>The Nix expression</title> - -<example xml:id='ex-hello-nix'><title>Nix expression for GNU Hello -(<filename>default.nix</filename>)</title> -<programlisting> -{ stdenv, fetchurl, perl }: <co xml:id='ex-hello-nix-co-1' /> - -stdenv.mkDerivation { <co xml:id='ex-hello-nix-co-2' /> - name = "hello-2.1.1"; <co xml:id='ex-hello-nix-co-3' /> - builder = ./builder.sh; <co xml:id='ex-hello-nix-co-4' /> - src = fetchurl { <co xml:id='ex-hello-nix-co-5' /> - url = ftp://ftp.nluug.nl/pub/gnu/hello/hello-2.1.1.tar.gz; - md5 = "70c9ccf9fac07f762c24f2df2290784d"; - }; - inherit perl; <co xml:id='ex-hello-nix-co-6' /> -}</programlisting> -</example> - -<para><xref linkend='ex-hello-nix' /> shows a Nix expression for GNU -Hello. It's actually already in the Nix Packages collection in -<filename>pkgs/applications/misc/hello/ex-1/default.nix</filename>. -It is customary to place each package in a separate directory and call -the single Nix expression in that directory -<filename>default.nix</filename>. The file has the following elements -(referenced from the figure by number): - -<calloutlist> - - <callout arearefs='ex-hello-nix-co-1'> - - <para>This states that the expression is a - <emphasis>function</emphasis> that expects to be called with three - arguments: <varname>stdenv</varname>, <varname>fetchurl</varname>, - and <varname>perl</varname>. They are needed to build Hello, but - we don't know how to build them here; that's why they are function - arguments. <varname>stdenv</varname> is a package that is used - by almost all Nix Packages packages; it provides a - <quote>standard</quote> environment consisting of the things you - would expect in a basic Unix environment: a C/C++ compiler (GCC, - to be precise), the Bash shell, fundamental Unix tools such as - <command>cp</command>, <command>grep</command>, - <command>tar</command>, etc. <varname>fetchurl</varname> is a - function that downloads files. <varname>perl</varname> is the - Perl interpreter.</para> - - <para>Nix functions generally have the form <literal>{ x, y, ..., - z }: e</literal> where <varname>x</varname>, <varname>y</varname>, - etc. are the names of the expected arguments, and where - <replaceable>e</replaceable> is the body of the function. So - here, the entire remainder of the file is the body of the - function; when given the required arguments, the body should - describe how to build an instance of the Hello package.</para> - - </callout> - - <callout arearefs='ex-hello-nix-co-2'> - - <para>So we have to build a package. Building something from - other stuff is called a <emphasis>derivation</emphasis> in Nix (as - opposed to sources, which are built by humans instead of - computers). We perform a derivation by calling - <varname>stdenv.mkDerivation</varname>. - <varname>mkDerivation</varname> is a function provided by - <varname>stdenv</varname> that builds a package from a set of - <emphasis>attributes</emphasis>. A set is just a list of - key/value pairs where each key is a string and each value is an - arbitrary Nix expression. They take the general form <literal>{ - <replaceable>name1</replaceable> = - <replaceable>expr1</replaceable>; <replaceable>...</replaceable> - <replaceable>nameN</replaceable> = - <replaceable>exprN</replaceable>; }</literal>.</para> - - </callout> - - <callout arearefs='ex-hello-nix-co-3'> - - <para>The attribute <varname>name</varname> specifies the symbolic - name and version of the package. Nix doesn't really care about - these things, but they are used by for instance <command>nix-env - -q</command> to show a <quote>human-readable</quote> name for - packages. This attribute is required by - <varname>mkDerivation</varname>.</para> - - </callout> - - <callout arearefs='ex-hello-nix-co-4'> - - <para>The attribute <varname>builder</varname> specifies the - builder. This attribute can sometimes be omitted, in which case - <varname>mkDerivation</varname> will fill in a default builder - (which does a <literal>configure; make; make install</literal>, in - essence). Hello is sufficiently simple that the default builder - would suffice, but in this case, we will show an actual builder - for educational purposes. The value - <command>./builder.sh</command> refers to the shell script shown - in <xref linkend='ex-hello-builder' />, discussed below.</para> - - </callout> - - <callout arearefs='ex-hello-nix-co-5'> - - <para>The builder has to know what the sources of the package - are. Here, the attribute <varname>src</varname> is bound to the - result of a call to the <command>fetchurl</command> function. - Given a URL and an MD5 hash of the expected contents of the file - at that URL, this function builds a derivation that downloads the - file and checks its hash. So the sources are a dependency that - like all other dependencies is built before Hello itself is - built.</para> - - <para>Instead of <varname>src</varname> any other name could have - been used, and in fact there can be any number of sources (bound - to different attributes). However, <varname>src</varname> is - customary, and it's also expected by the default builder (which we - don't use in this example).</para> - - </callout> - - <callout arearefs='ex-hello-nix-co-6'> - - <para>Since the derivation requires Perl, we have to pass the - value of the <varname>perl</varname> function argument to the - builder. All attributes in the set are actually passed as - environment variables to the builder, so declaring an attribute - - <programlisting> -perl = perl;</programlisting> - - will do the trick: it binds an attribute <varname>perl</varname> - to the function argument which also happens to be called - <varname>perl</varname>. However, it looks a bit silly, so there - is a shorter syntax. The <literal>inherit</literal> keyword - causes the specified attributes to be bound to whatever variables - with the same name happen to be in scope.</para> - - </callout> - -</calloutlist> - -</para> - -</section> - - -<section><title>The builder</title> - -<example xml:id='ex-hello-builder'><title>Build script for GNU Hello -(<filename>builder.sh</filename>)</title> -<programlisting> -source $stdenv/setup <co xml:id='ex-hello-builder-co-1' /> - -PATH=$perl/bin:$PATH <co xml:id='ex-hello-builder-co-2' /> - -tar xvfz $src <co xml:id='ex-hello-builder-co-3' /> -cd hello-* -./configure --prefix=$out <co xml:id='ex-hello-builder-co-4' /> -make <co xml:id='ex-hello-builder-co-5' /> -make install</programlisting> -</example> - -<para><xref linkend='ex-hello-builder' /> shows the builder referenced -from Hello's Nix expression (stored in -<filename>pkgs/applications/misc/hello/ex-1/builder.sh</filename>). -The builder can actually be made a lot shorter by using the -<emphasis>generic builder</emphasis> functions provided by -<varname>stdenv</varname>, but here we write out the build steps to -elucidate what a builder does. It performs the following -steps:</para> - -<calloutlist> - - <callout arearefs='ex-hello-builder-co-1'> - - <para>When Nix runs a builder, it initially completely clears the - environment (except for the attributes declared in the - derivation). For instance, the <envar>PATH</envar> variable is - empty<footnote><para>Actually, it's initialised to - <filename>/path-not-set</filename> to prevent Bash from setting it - to a default value.</para></footnote>. This is done to prevent - undeclared inputs from being used in the build process. If for - example the <envar>PATH</envar> contained - <filename>/usr/bin</filename>, then you might accidentally use - <filename>/usr/bin/gcc</filename>.</para> - - <para>So the first step is to set up the environment. This is - done by calling the <filename>setup</filename> script of the - standard environment. The environment variable - <envar>stdenv</envar> points to the location of the standard - environment being used. (It wasn't specified explicitly as an - attribute in <xref linkend='ex-hello-nix' />, but - <varname>mkDerivation</varname> adds it automatically.)</para> - - </callout> - - <callout arearefs='ex-hello-builder-co-2'> - - <para>Since Hello needs Perl, we have to make sure that Perl is in - the <envar>PATH</envar>. The <envar>perl</envar> environment - variable points to the location of the Perl package (since it - was passed in as an attribute to the derivation), so - <filename><replaceable>$perl</replaceable>/bin</filename> is the - directory containing the Perl interpreter.</para> - - </callout> - - <callout arearefs='ex-hello-builder-co-3'> - - <para>Now we have to unpack the sources. The - <varname>src</varname> attribute was bound to the result of - fetching the Hello source tarball from the network, so the - <envar>src</envar> environment variable points to the location in - the Nix store to which the tarball was downloaded. After - unpacking, we <command>cd</command> to the resulting source - directory.</para> - - <para>The whole build is performed in a temporary directory - created in <varname>/tmp</varname>, by the way. This directory is - removed after the builder finishes, so there is no need to clean - up the sources afterwards. Also, the temporary directory is - always newly created, so you don't have to worry about files from - previous builds interfering with the current build.</para> - - </callout> - - <callout arearefs='ex-hello-builder-co-4'> - - <para>GNU Hello is a typical Autoconf-based package, so we first - have to run its <filename>configure</filename> script. In Nix - every package is stored in a separate location in the Nix store, - for instance - <filename>/nix/store/9a54ba97fb71b65fda531012d0443ce2-hello-2.1.1</filename>. - Nix computes this path by cryptographically hashing all attributes - of the derivation. The path is passed to the builder through the - <envar>out</envar> environment variable. So here we give - <filename>configure</filename> the parameter - <literal>--prefix=$out</literal> to cause Hello to be installed in - the expected location.</para> - - </callout> - - <callout arearefs='ex-hello-builder-co-5'> - - <para>Finally we build Hello (<literal>make</literal>) and install - it into the location specified by <envar>out</envar> - (<literal>make install</literal>).</para> - - </callout> - -</calloutlist> - -<para>If you are wondering about the absence of error checking on the -result of various commands called in the builder: this is because the -shell script is evaluated with Bash's <option>-e</option> option, -which causes the script to be aborted if any command fails without an -error check.</para> - -</section> - - -<section><title>Composition</title> - -<example xml:id='ex-hello-composition'><title>Composing GNU Hello -(<filename>all-packages.nix</filename>)</title> -<programlisting> -... - -rec { <co xml:id='ex-hello-composition-co-1' /> - - hello = import ../applications/misc/hello/ex-1 <co xml:id='ex-hello-composition-co-2' /> { <co xml:id='ex-hello-composition-co-3' /> - inherit fetchurl stdenv perl; - }; - - perl = import ../development/interpreters/perl { <co xml:id='ex-hello-composition-co-4' /> - inherit fetchurl stdenv; - }; - - fetchurl = import ../build-support/fetchurl { - inherit stdenv; ... - }; - - stdenv = ...; - -} -</programlisting> -</example> - -<para>The Nix expression in <xref linkend='ex-hello-nix' /> is a -function; it is missing some arguments that have to be filled in -somewhere. In the Nix Packages collection this is done in the file -<filename>pkgs/top-level/all-packages.nix</filename>, where all -Nix expressions for packages are imported and called with the -appropriate arguments. <xref linkend='ex-hello-composition' /> shows -some fragments of -<filename>all-packages.nix</filename>.</para> - -<calloutlist> - - <callout arearefs='ex-hello-composition-co-1'> - - <para>This file defines a set of attributes, all of which are - concrete derivations (i.e., not functions). In fact, we define a - <emphasis>mutually recursive</emphasis> set of attributes. That - is, the attributes can refer to each other. This is precisely - what we want since we want to <quote>plug</quote> the - various packages into each other.</para> - - </callout> - - <callout arearefs='ex-hello-composition-co-2'> - - <para>Here we <emphasis>import</emphasis> the Nix expression for - GNU Hello. The import operation just loads and returns the - specified Nix expression. In fact, we could just have put the - contents of <xref linkend='ex-hello-nix' /> in - <filename>all-packages.nix</filename> at this point. That - would be completely equivalent, but it would make the file rather - bulky.</para> - - <para>Note that we refer to - <filename>../applications/misc/hello/ex-1</filename>, not - <filename>../applications/misc/hello/ex-1/default.nix</filename>. - When you try to import a directory, Nix automatically appends - <filename>/default.nix</filename> to the file name.</para> - - </callout> - - <callout arearefs='ex-hello-composition-co-3'> - - <para>This is where the actual composition takes place. Here we - <emphasis>call</emphasis> the function imported from - <filename>../applications/misc/hello/ex-1</filename> with a set - containing the things that the function expects, namely - <varname>fetchurl</varname>, <varname>stdenv</varname>, and - <varname>perl</varname>. We use inherit again to use the - attributes defined in the surrounding scope (we could also have - written <literal>fetchurl = fetchurl;</literal>, etc.).</para> - - <para>The result of this function call is an actual derivation - that can be built by Nix (since when we fill in the arguments of - the function, what we get is its body, which is the call to - <varname>stdenv.mkDerivation</varname> in <xref - linkend='ex-hello-nix' />).</para> - - <note><para>Nixpkgs has a convenience function - <function>callPackage</function> that imports and calls a - function, filling in any missing arguments by passing the - corresponding attribute from the Nixpkgs set, like this: - -<programlisting> -hello = callPackage ../applications/misc/hello/ex-1 { }; -</programlisting> - - If necessary, you can set or override arguments: - -<programlisting> -hello = callPackage ../applications/misc/hello/ex-1 { stdenv = myStdenv; }; -</programlisting> - - </para></note> - - </callout> - - <callout arearefs='ex-hello-composition-co-4'> - - <para>Likewise, we have to instantiate Perl, - <varname>fetchurl</varname>, and the standard environment.</para> - - </callout> - -</calloutlist> - -</section> - - -<section><title>Testing</title> - -<para>You can now try to build Hello. Of course, you could do -<literal>nix-env -f pkgs/top-level/all-packages.nix -i hello</literal>, -but you may not want to install a possibly broken package just yet. -The best way to test the package is by using the command <command -linkend="sec-nix-build">nix-build</command>, which builds a Nix -expression and creates a symlink named <filename>result</filename> in -the current directory: - -<screen> -$ nix-build pkgs/top-level/all-packages.nix -A hello -building path `/nix/store/632d2b22514d...-hello-2.1.1' -hello-2.1.1/ -hello-2.1.1/intl/ -hello-2.1.1/intl/ChangeLog -<replaceable>...</replaceable> - -$ ls -l result -lrwxrwxrwx ... 2006-09-29 10:43 result -> /nix/store/632d2b22514d...-hello-2.1.1 - -$ ./result/bin/hello -Hello, world!</screen> - -The <link linkend='opt-attr'><option>-A</option></link> option selects -the <literal>hello</literal> attribute from -<filename>all-packages.nix</filename>. This is faster than using the -symbolic package name specified by the <literal>name</literal> -attribute (which also happens to be <literal>hello</literal>) and is -unambiguous (there can be multiple packages with the symbolic name -<literal>hello</literal>, but there can be only one attribute in a set -named <literal>hello</literal>).</para> - -<para><command>nix-build</command> registers the -<filename>./result</filename> symlink as a garbage collection root, so -unless and until you delete the <filename>./result</filename> symlink, -the output of the build will be safely kept on your system. You can -use <command>nix-build</command>’s <option -linkend='opt-out-link'>-o</option> switch to give the symlink another -name.</para> - -<para>Nix has a transactional semantics. Once a build finishes -successfully, Nix makes a note of this in its database: it registers -that the path denoted by <envar>out</envar> is now -<quote>valid</quote>. If you try to build the derivation again, Nix -will see that the path is already valid and finish immediately. If a -build fails, either because it returns a non-zero exit code, because -Nix or the builder are killed, or because the machine crashes, then -the output paths will not be registered as valid. If you try to build -the derivation again, Nix will remove the output paths if they exist -(e.g., because the builder died half-way through <literal>make -install</literal>) and try again. Note that there is no -<quote>negative caching</quote>: Nix doesn't remember that a build -failed, and so a failed build can always be repeated. This is because -Nix cannot distinguish between permanent failures (e.g., a compiler -error due to a syntax error in the source) and transient failures -(e.g., a disk full condition).</para> - -<para>Nix also performs locking. If you run multiple Nix builds -simultaneously, and they try to build the same derivation, the first -Nix instance that gets there will perform the build, while the others -block (or perform other derivations if available) until the build -finishes: - -<screen> -$ nix-build pkgs/top-level/all-packages.nix -A hello -waiting for lock on `/nix/store/0h5b7hp8d4hqfrw8igvx97x1xawrjnac-hello-2.1.1x'</screen> - -So it is always safe to run multiple instances of Nix in parallel -(which isn’t the case with, say, <command>make</command>).</para> - -<para>If you have a system with multiple CPUs, you may want to have -Nix build different derivations in parallel (insofar as possible). -Just pass the option <link linkend='opt-max-jobs'><option>-j -<replaceable>N</replaceable></option></link>, where -<replaceable>N</replaceable> is the maximum number of jobs to be run -in parallel, or set. Typically this should be the number of -CPUs.</para> - -</section> - - -<section><title>The generic builder</title> - -<para>Recall from <xref linkend='ex-hello-builder' /> that the builder -looked something like this: - -<programlisting> -PATH=$perl/bin:$PATH -tar xvfz $src -cd hello-* -./configure --prefix=$out -make -make install</programlisting> - -The builders for almost all Unix packages look like this — set up some -environment variables, unpack the sources, configure, build, and -install. For this reason the standard environment provides some Bash -functions that automate the build process. A builder using the -generic build facilities in shown in <xref linkend='ex-hello-builder2' -/>.</para> - -<example xml:id='ex-hello-builder2'><title>Build script using the generic -build functions</title> -<programlisting> -buildInputs="$perl" <co xml:id='ex-hello-builder2-co-1' /> - -source $stdenv/setup <co xml:id='ex-hello-builder2-co-2' /> - -genericBuild <co xml:id='ex-hello-builder2-co-3' /></programlisting> -</example> - -<calloutlist> - - <callout arearefs='ex-hello-builder2-co-1'> - - <para>The <envar>buildInputs</envar> variable tells - <filename>setup</filename> to use the indicated packages as - <quote>inputs</quote>. This means that if a package provides a - <filename>bin</filename> subdirectory, it's added to - <envar>PATH</envar>; if it has a <filename>include</filename> - subdirectory, it's added to GCC's header search path; and so - on.<footnote><para>How does it work? <filename>setup</filename> - tries to source the file - <filename><replaceable>pkg</replaceable>/nix-support/setup-hook</filename> - of all dependencies. These “setup hooks” can then set up whatever - environment variables they want; for instance, the setup hook for - Perl sets the <envar>PERL5LIB</envar> environment variable to - contain the <filename>lib/site_perl</filename> directories of all - inputs.</para></footnote> - </para> - - </callout> - - <callout arearefs='ex-hello-builder2-co-2'> - - <para>The function <function>genericBuild</function> is defined in - the file <literal>$stdenv/setup</literal>.</para> - - </callout> - - <callout arearefs='ex-hello-builder2-co-3'> - - <para>The final step calls the shell function - <function>genericBuild</function>, which performs the steps that - were done explicitly in <xref linkend='ex-hello-builder' />. The - generic builder is smart enough to figure out whether to unpack - the sources using <command>gzip</command>, - <command>bzip2</command>, etc. It can be customised in many ways; - see <xref linkend='sec-standard-environment' />.</para> - - </callout> - -</calloutlist> - -<para>Discerning readers will note that the -<envar>buildInputs</envar> could just as well have been set in the Nix -expression, like this: - -<programlisting> - buildInputs = [ perl ];</programlisting> - -The <varname>perl</varname> attribute can then be removed, and the -builder becomes even shorter: - -<programlisting> -source $stdenv/setup -genericBuild</programlisting> - -In fact, <varname>mkDerivation</varname> provides a default builder -that looks exactly like that, so it is actually possible to omit the -builder for Hello entirely.</para> - -</section> - - -</section> - - - -<section><title>The Nix expression language</title> - -<para>The Nix expression language is a pure, lazy, functional -language. Purity means that operations in the language don't have -side-effects (for instance, there is no variable assignment). -Laziness means that arguments to functions are evaluated only when -they are needed. Functional means that functions are -<quote>normal</quote> values that can be passed around and manipulated -in interesting ways. The language is not a full-featured, general -purpose language. Its main job is to describe packages, -compositions of packages, and the variability within -packages.</para> - -<para>This section presents the various features of the -language.</para> - - -<section xml:id='ssec-values'><title>Values</title> - - -<simplesect><title>Simple values</title> - -<para>Nix has the following basic data types: - -<itemizedlist> - - <listitem> - - <para><emphasis>Strings</emphasis> can be written in three - ways.</para> - - <para>The most common way is to enclose the string between double - quotes, e.g., <literal>"foo bar"</literal>. Strings can span - multiple lines. The special characters <literal>"</literal> and - <literal>\</literal> and the character sequence - <literal>${</literal> must be escaped by prefixing them with a - backslash (<literal>\</literal>). Newlines, carriage returns and - tabs can be written as <literal>\n</literal>, - <literal>\r</literal> and <literal>\t</literal>, - respectively.</para> - - <para>You can include the result of an expression into a string by - enclosing it in - <literal>${<replaceable>...</replaceable>}</literal>, a feature - known as <emphasis>antiquotation</emphasis>. The enclosed - expression must evaluate to something that can be coerced into a - string (meaning that it must be a string, a path, or a - derivation). For instance, rather than writing - -<programlisting> -"--with-freetype2-library=" + freetype + "/lib"</programlisting> - - (where <varname>freetype</varname> is a derivation), you can - instead write the more natural - -<programlisting> -"--with-freetype2-library=${freetype}/lib"</programlisting> - - The latter is automatically translated to the former. A more - complicated example (from the Nix expression for <link - xlink:href='http://www.trolltech.com/products/qt'>Qt</link>): - -<programlisting> -configureFlags = " - -system-zlib -system-libpng -system-libjpeg - ${if openglSupport then "-dlopen-opengl - -L${mesa}/lib -I${mesa}/include - -L${libXmu}/lib -I${libXmu}/include" else ""} - ${if threadSupport then "-thread" else "-no-thread"} -";</programlisting> - - Note that Nix expressions and strings can be arbitrarily nested; - in this case the outer string contains various antiquotations that - themselves contain strings (e.g., <literal>"-thread"</literal>), - some of which in turn contain expressions (e.g., - <literal>${mesa}</literal>).</para> - - <para>The second way to write string literals is as an - <emphasis>indented string</emphasis>, which is enclosed between - pairs of <emphasis>double single-quotes</emphasis>, like so: - -<programlisting> -'' - This is the first line. - This is the second line. - This is the third line. -''</programlisting> - - This kind of string literal intelligently strips indentation from - the start of each line. To be precise, it strips from each line a - number of spaces equal to the minimal indentation of the string as - a whole (disregarding the indentation of empty lines). For - instance, the first and second line are indented two space, while - the third line is indented four spaces. Thus, two spaces are - stripped from each line, so the resulting string is - -<programlisting> -"This is the first line.\nThis is the second line.\n This is the third line.\n"</programlisting> - - </para> - - <para>Note that the whitespace and newline following the opening - <literal>''</literal> is ignored if there is no non-whitespace - text on the initial line.</para> - - <para>Antiquotation - (<literal>${<replaceable>expr</replaceable>}</literal>) is - supported in indented strings.</para> - - <para>Since <literal>${</literal> and <literal>''</literal> have - special meaning in indented strings, you need a way to quote them. - <literal>${</literal> can be escaped by prefixing it with - <literal>''</literal> (that is, two single quotes), i.e., - <literal>''${</literal>. <literal>''</literal> can be escaped by - prefixing it with <literal>'</literal>, i.e., - <literal>'''</literal>. Finally, linefeed, carriage-return and - tab characters can be written as <literal>''\n</literal>, - <literal>''\r</literal>, <literal>''\t</literal>.</para> - - <para>Indented strings are primarily useful in that they allow - multi-line string literals to follow the indentation of the - enclosing Nix expression, and that less escaping is typically - necessary for strings representing languages such as shell scripts - and configuration files because <literal>''</literal> is much less - common than <literal>"</literal>. Example: - -<programlisting> -stdenv.mkDerivation { - <replaceable>...</replaceable> - postInstall = - '' - mkdir $out/bin $out/etc - cp foo $out/bin - echo "Hello World" > $out/etc/foo.conf - ${if enableBar then "cp bar $out/bin" else ""} - ''; - <replaceable>...</replaceable> -} -</programlisting> - - </para> - - <para>Finally, as a convenience, <emphasis>URIs</emphasis> as - defined in appendix B of <link - xlink:href='http://www.ietf.org/rfc/rfc2396.txt'>RFC 2396</link> - can be written <emphasis>as is</emphasis>, without quotes. For - instance, the string - <literal>"http://example.org/foo.tar.bz2"</literal> - can also be written as - <literal>http://example.org/foo.tar.bz2</literal>.</para> - - </listitem> - - <listitem><para><emphasis>Integers</emphasis>, e.g., - <literal>123</literal>.</para></listitem> - - <listitem><para><emphasis>Paths</emphasis>, e.g., - <filename>/bin/sh</filename> or <filename>./builder.sh</filename>. - A path must contain at least one slash to be recognised as such; for - instance, <filename>builder.sh</filename> is not a - path<footnote><para>It's parsed as an expression that selects the - attribute <varname>sh</varname> from the variable - <varname>builder</varname>.</para></footnote>. If the file name is - relative, i.e., if it does not begin with a slash, it is made - absolute at parse time relative to the directory of the Nix - expression that contained it. For instance, if a Nix expression in - <filename>/foo/bar/bla.nix</filename> refers to - <filename>../xyzzy/fnord.nix</filename>, the absolute path is - <filename>/foo/xyzzy/fnord.nix</filename>.</para></listitem> - - <listitem><para><emphasis>Booleans</emphasis> with values - <literal>true</literal> and - <literal>false</literal>.</para></listitem> - - <listitem><para>The null value, denoted as - <literal>null</literal>.</para></listitem> - -</itemizedlist> - -</para> - -</simplesect> - - -<simplesect><title>Lists</title> - -<para>Lists are formed by enclosing a whitespace-separated list of -values between square brackets. For example, - -<programlisting> -[ 123 ./foo.nix "abc" (f { x = y; }) ]</programlisting> - -defines a list of four elements, the last being the result of a call -to the function <varname>f</varname>. Note that function calls have -to be enclosed in parentheses. If they had been omitted, e.g., - -<programlisting> -[ 123 ./foo.nix "abc" f { x = y; } ]</programlisting> - -the result would be a list of five elements, the fourth one being a -function and the fifth being a set.</para> - -</simplesect> - - -<simplesect><title>Sets</title> - -<para>Sets are really the core of the language, since ultimately the -Nix language is all about creating derivations, which are really just -sets of attributes to be passed to build scripts.</para> - -<para>Sets are just a list of name/value pairs (called -<emphasis>attributes</emphasis>) enclosed in curly brackets, where -each value is an arbitrary expression terminated by a semicolon. For -example: - -<programlisting> -{ x = 123; - text = "Hello"; - y = f { bla = 456; }; -}</programlisting> - -This defines a set with attributes named <varname>x</varname>, -<varname>text</varname>, <varname>y</varname>. The order of the -attributes is irrelevant. An attribute name may only occur -once.</para> - -<para>Attributes can be selected from a set using the -<literal>.</literal> operator. For instance, - -<programlisting> -{ a = "Foo"; b = "Bar"; }.a</programlisting> - -evaluates to <literal>"Foo"</literal>. It is possible to provide a -default value in an attribute selection using the -<literal>or</literal> keyword. For example, - -<programlisting> -{ a = "Foo"; b = "Bar"; }.c or "Xyzzy"</programlisting> - -will evaluate to <literal>"Xyzzy"</literal> because there is no -<varname>c</varname> attribute in the set.</para> - -<para>You can use arbitrary double-quoted strings as attribute -names: - -<programlisting> -{ "foo ${bar}" = 123; "nix-1.0" = 456; }."foo ${bar}" -</programlisting> - -This will evaluate to <literal>123</literal> (Assuming -<literal>bar</literal> is antiquotable). In the case where an -attribute name is just a single antiquotation, the quotes can be -dropped: - -<programlisting> -{ foo = 123; }.${bar} or 456 </programlisting> - -This will evaluate to <literal>123</literal> if -<literal>bar</literal> evaluates to <literal>"foo"</literal> when -coerced to a string and <literal>456</literal> otherwise (again -assuming <literal>bar</literal> is antiquotable).</para> - -<para>In the special case where an attribute name inside of a set declaration -evaluates to <literal>null</literal> (which is normally an error, as -<literal>null</literal> is not antiquotable), that attribute is simply not -added to the set: - -<programlisting> -{ ${if foo then "bar" else null} = true; }</programlisting> - -This will evaluate to <literal>{}</literal> if <literal>foo</literal> -evaluates to <literal>false</literal>.</para> - - -</simplesect> - - -</section> - - -<section><title>Language constructs</title> - - -<simplesect><title>Recursive sets</title> - -<para>Recursive sets are just normal sets, but the attributes can -refer to each other. For example, - -<programlisting> -rec { - x = y; - y = 123; -}.x -</programlisting> - -evaluates to <literal>123</literal>. Note that without -<literal>rec</literal> the binding <literal>x = y;</literal> would -refer to the variable <varname>y</varname> in the surrounding scope, -if one exists, and would be invalid if no such variable exists. That -is, in a normal (non-recursive) set, attributes are not added to the -lexical scope; in a recursive set, they are.</para> - -<para>Recursive sets of course introduce the danger of infinite -recursion. For example, - -<programlisting> -rec { - x = y; - y = x; -}.x</programlisting> - -does not terminate<footnote><para>Actually, Nix detects infinite -recursion in this case and aborts (<quote>infinite recursion -encountered</quote>).</para></footnote>.</para> - -</simplesect> - - -<simplesect><title>Let-expressions</title> - -<para>A let-expression allows you define local variables for an -expression. For instance, - -<programlisting> -let - x = "foo"; - y = "bar"; -in x + y</programlisting> - -evaluates to <literal>"foobar"</literal>. - -</para> - -</simplesect> - - -<simplesect><title>Inheriting attributes</title> - -<para>When defining a set it is often convenient to copy variables -from the surrounding lexical scope (e.g., when you want to propagate -attributes). This can be shortened using the -<literal>inherit</literal> keyword. For instance, - -<programlisting> -let x = 123; in -{ inherit x; - y = 456; -}</programlisting> - -evaluates to <literal>{ x = 123; y = 456; }</literal>. (Note that -this works because <varname>x</varname> is added to the lexical scope -by the <literal>let</literal> construct.) It is also possible to -inherit attributes from another set. For instance, in this fragment -from <filename>all-packages.nix</filename>, - -<programlisting> - graphviz = (import ../tools/graphics/graphviz) { - inherit fetchurl stdenv libpng libjpeg expat x11 yacc; - inherit (xlibs) libXaw; - }; - - xlibs = { - libX11 = ...; - libXaw = ...; - ... - } - - libpng = ...; - libjpg = ...; - ...</programlisting> - -the set used in the function call to the function defined in -<filename>../tools/graphics/graphviz</filename> inherits a number of -variables from the surrounding scope (<varname>fetchurl</varname> -... <varname>yacc</varname>), but also inherits -<varname>libXaw</varname> (the X Athena Widgets) from the -<varname>xlibs</varname> (X11 client-side libraries) set.</para> - -</simplesect> - - -<simplesect xml:id="ss-functions"><title>Functions</title> - -<para>Functions have the following form: - -<programlisting> -<replaceable>pattern</replaceable>: <replaceable>body</replaceable></programlisting> - -The pattern specifies what the argument of the function must look -like, and binds variables in the body to (parts of) the -argument. There are three kinds of patterns:</para> - -<itemizedlist> - - - <listitem><para>If a pattern is a single identifier, then the - function matches any argument. Example: - - <programlisting> -let negate = x: !x; - concat = x: y: x + y; -in if negate true then concat "foo" "bar" else ""</programlisting> - - Note that <function>concat</function> is a function that takes one - argument and returns a function that takes another argument. This - allows partial parameterisation (i.e., only filling some of the - arguments of a function); e.g., - - <programlisting> -map (concat "foo") [ "bar" "bla" "abc" ]</programlisting> - - evaluates to <literal>[ "foobar" "foobla" - "fooabc" ]</literal>.</para></listitem> - - - <listitem><para>A <emphasis>set pattern</emphasis> of the form - <literal>{ name1, name2, …, nameN }</literal> matches a set - containing the listed attributes, and binds the values of those - attributes to variables in the function body. For example, the - function - -<programlisting> -{ x, y, z }: z + y + x</programlisting> - - can only be called with a set containing exactly the attributes - <varname>x</varname>, <varname>y</varname> and - <varname>z</varname>. No other attributes are allowed. If you want - to allow additional arguments, you can use an ellipsis - (<literal>...</literal>): - -<programlisting> -{ x, y, z, ... }: z + y + x</programlisting> - - This works on any set that contains at least the three named - attributes.</para> - - <para>It is possible to provide <emphasis>default values</emphasis> - for attributes, in which case they are allowed to be missing. A - default value is specified by writing - <literal><replaceable>name</replaceable> ? - <replaceable>e</replaceable></literal>, where - <replaceable>e</replaceable> is an arbitrary expression. For example, - -<programlisting> -{ x, y ? "foo", z ? "bar" }: z + y + x</programlisting> - - specifies a function that only requires an attribute named - <varname>x</varname>, but optionally accepts <varname>y</varname> - and <varname>z</varname>.</para></listitem> - - - <listitem><para>An <literal>@</literal>-pattern provides a means of referring - to the whole value being matched: - -<programlisting> -args@{ x, y, z, ... }: z + y + x + args.a</programlisting> - - Here <varname>args</varname> is bound to the entire argument, which - is further matched against the pattern <literal>{ x, y, z, - ... }</literal>.</para></listitem> - - -</itemizedlist> - -<para>Note that functions do not have names. If you want to give them -a name, you can bind them to an attribute, e.g., - -<programlisting> -let concat = { x, y }: x + y; -in concat { x = "foo"; y = "bar"; }</programlisting> - -</para> - -</simplesect> - - -<simplesect><title>Conditionals</title> - -<para>Conditionals look like this: - -<programlisting> -if <replaceable>e1</replaceable> then <replaceable>e2</replaceable> else <replaceable>e3</replaceable></programlisting> - -where <replaceable>e1</replaceable> is an expression that should -evaluate to a Boolean value (<literal>true</literal> or -<literal>false</literal>).</para> - -</simplesect> - - -<simplesect><title>Assertions</title> - -<para>Assertions are generally used to check that certain requirements -on or between features and dependencies hold. They look like this: - -<programlisting> -assert <replaceable>e1</replaceable>; <replaceable>e2</replaceable></programlisting> - -where <replaceable>e1</replaceable> is an expression that should -evaluate to a Boolean value. If it evaluates to -<literal>true</literal>, <replaceable>e2</replaceable> is returned; -otherwise expression evaluation is aborted and a backtrace is printed.</para> - -<example xml:id='ex-subversion-nix'><title>Nix expression for Subversion</title> -<programlisting> -{ localServer ? false -, httpServer ? false -, sslSupport ? false -, pythonBindings ? false -, javaSwigBindings ? false -, javahlBindings ? false -, stdenv, fetchurl -, openssl ? null, httpd ? null, db4 ? null, expat, swig ? null, j2sdk ? null -}: - -assert localServer -> db4 != null; <co xml:id='ex-subversion-nix-co-1' /> -assert httpServer -> httpd != null && httpd.expat == expat; <co xml:id='ex-subversion-nix-co-2' /> -assert sslSupport -> openssl != null && (httpServer -> httpd.openssl == openssl); <co xml:id='ex-subversion-nix-co-3' /> -assert pythonBindings -> swig != null && swig.pythonSupport; -assert javaSwigBindings -> swig != null && swig.javaSupport; -assert javahlBindings -> j2sdk != null; - -stdenv.mkDerivation { - name = "subversion-1.1.1"; - ... - openssl = if sslSupport then openssl else null; <co xml:id='ex-subversion-nix-co-4' /> - ... -}</programlisting> -</example> - -<para><xref linkend='ex-subversion-nix' /> show how assertions are -used in the Nix expression for Subversion.</para> - -<calloutlist> - - <callout arearefs='ex-subversion-nix-co-1'> - <para>This assertion states that if Subversion is to have support - for local repositories, then Berkeley DB is needed. So if the - Subversion function is called with the - <varname>localServer</varname> argument set to - <literal>true</literal> but the <varname>db4</varname> argument - set to <literal>null</literal>, then the evaluation fails.</para> - </callout> - - <callout arearefs='ex-subversion-nix-co-2'> - <para>This is a more subtle condition: if Subversion is built with - Apache (<literal>httpServer</literal>) support, then the Expat - library (an XML library) used by Subversion should be same as the - one used by Apache. This is because in this configuration - Subversion code ends up being linked with Apache code, and if the - Expat libraries do not match, a build- or runtime link error or - incompatibility might occur.</para> - </callout> - - <callout arearefs='ex-subversion-nix-co-3'> - <para>This assertion says that in order for Subversion to have SSL - support (so that it can access <literal>https</literal> URLs), an - OpenSSL library must be passed. Additionally, it says that - <emphasis>if</emphasis> Apache support is enabled, then Apache's - OpenSSL should match Subversion's. (Note that if Apache support - is not enabled, we don't care about Apache's OpenSSL.)</para> - </callout> - - <callout arearefs='ex-subversion-nix-co-4'> - <para>The conditional here is not really related to assertions, - but is worth pointing out: it ensures that if SSL support is - disabled, then the Subversion derivation is not dependent on - OpenSSL, even if a non-<literal>null</literal> value was passed. - This prevents an unnecessary rebuild of Subversion if OpenSSL - changes.</para> - </callout> - -</calloutlist> - -</simplesect> - - - -<simplesect><title>With-expressions</title> - -<para>A <emphasis>with-expression</emphasis>, - -<programlisting> -with <replaceable>e1</replaceable>; <replaceable>e2</replaceable></programlisting> - -introduces the set <replaceable>e1</replaceable> into the lexical -scope of the expression <replaceable>e2</replaceable>. For instance, - -<programlisting> -let as = { x = "foo"; y = "bar"; }; -in with as; x + y</programlisting> - -evaluates to <literal>"foobar"</literal> since the -<literal>with</literal> adds the <varname>x</varname> and -<varname>y</varname> attributes of <varname>as</varname> to the -lexical scope in the expression <literal>x + y</literal>. The most -common use of <literal>with</literal> is in conjunction with the -<function>import</function> function. E.g., - -<programlisting> -with (import ./definitions.nix); ...</programlisting> - -makes all attributes defined in the file -<filename>definitions.nix</filename> available as if they were defined -locally in a <literal>rec</literal>-expression.</para> - -</simplesect> - - -<simplesect><title>Comments</title> - -<para>Comments can be single-line, started with a <literal>#</literal> -character, or inline/multi-line, enclosed within <literal>/* -... */</literal>.</para> - -</simplesect> - - -</section> - - -<section><title>Operators</title> - -<para><xref linkend='table-operators' /> lists the operators in the -Nix expression language, in order of precedence (from strongest to -weakest binding).</para> - -<table xml:id='table-operators'> - <title>Operators</title> - <tgroup cols='3'> - <thead> - <row> - <entry>Syntax</entry> - <entry>Associativity</entry> - <entry>Description</entry> - </row> - </thead> - <tbody> - <row> - <entry><replaceable>e</replaceable> <literal>.</literal> - <replaceable>attrpath</replaceable> - [ <literal>or</literal> <replaceable>def</replaceable> ] - </entry> - <entry>none</entry> - <entry>Select attribute denoted by the attribute path - <replaceable>attrpath</replaceable> from set - <replaceable>e</replaceable>. (An attribute path is a - dot-separated list of attribute names.) If the attribute - doesn’t exist, return <replaceable>def</replaceable> if - provided, otherwise abort evaluation.</entry> - </row> - <row> - <entry><replaceable>e1</replaceable> <replaceable>e2</replaceable></entry> - <entry>left</entry> - <entry>Call function <replaceable>e1</replaceable> with - argument <replaceable>e2</replaceable>.</entry> - </row> - <row> - <entry><replaceable>e</replaceable> <literal>?</literal> - <replaceable>attrpath</replaceable></entry> - <entry>none</entry> - <entry>Test whether set <replaceable>e</replaceable> contains - the attribute denoted by <replaceable>attrpath</replaceable>; - return <literal>true</literal> or - <literal>false</literal>.</entry> - </row> - <row> - <entry><replaceable>e1</replaceable> <literal>++</literal> <replaceable>e2</replaceable></entry> - <entry>right</entry> - <entry>List concatenation.</entry> - </row> - <row> - <entry><replaceable>e1</replaceable> <literal>+</literal> <replaceable>e2</replaceable></entry> - <entry>left</entry> - <entry>String or path concatenation.</entry> - </row> - <row> - <entry><literal>!</literal> <replaceable>e</replaceable></entry> - <entry>left</entry> - <entry>Boolean negation.</entry> - </row> - <row> - <entry><replaceable>e1</replaceable> <literal>//</literal> - <replaceable>e2</replaceable></entry> - <entry>right</entry> - <entry>Return a set consisting of the attributes in - <replaceable>e1</replaceable> and - <replaceable>e2</replaceable> (with the latter taking - precedence over the former in case of equally named - attributes).</entry> - </row> - <row> - <entry><replaceable>e1</replaceable> <literal>==</literal> - <replaceable>e2</replaceable></entry> - <entry>none</entry> - <entry>Equality.</entry> - </row> - <row> - <entry><replaceable>e1</replaceable> <literal>!=</literal> - <replaceable>e2</replaceable></entry> - <entry>none</entry> - <entry>Inequality.</entry> - </row> - <row> - <entry><replaceable>e1</replaceable> <literal>&&</literal> - <replaceable>e2</replaceable></entry> - <entry>left</entry> - <entry>Logical AND.</entry> - </row> - <row> - <entry><replaceable>e1</replaceable> <literal>||</literal> - <replaceable>e2</replaceable></entry> - <entry>left</entry> - <entry>Logical OR.</entry> - </row> - <row> - <entry><replaceable>e1</replaceable> <literal>-></literal> - <replaceable>e2</replaceable></entry> - <entry>none</entry> - <entry>Logical implication (equivalent to - <literal>!<replaceable>e1</replaceable> || - <replaceable>e2</replaceable></literal>).</entry> - </row> - </tbody> - </tgroup> -</table> - -</section> - - -<section xml:id="ssec-derivation"><title>Derivations</title> - -<para>The most important built-in function is -<function>derivation</function>, which is used to describe a single -derivation (a build action). It takes as input a set, the attributes -of which specify the inputs of the build.</para> - -<itemizedlist> - - <listitem xml:id="attr-system"><para>There must be an attribute named - <varname>system</varname> whose value must be a string specifying a - Nix platform identifier, such as <literal>"i686-linux"</literal> or - <literal>"powerpc-darwin"</literal><footnote><para>To figure out - your platform identifier, look at the line <quote>Checking for the - canonical Nix system name</quote> in the output of Nix's - <filename>configure</filename> script.</para></footnote> The build - can only be performed on a machine and operating system matching the - platform identifier. (Nix can automatically forward builds for - other platforms by forwarding them to other machines; see <xref - linkend='chap-distributed-builds' />.)</para></listitem> - - <listitem><para>There must be an attribute named - <varname>name</varname> whose value must be a string. This is used - as a symbolic name for the package by <command>nix-env</command>, - and it is appended to the output paths of the - derivation.</para></listitem> - - <listitem><para>There must be an attribute named - <varname>builder</varname> that identifies the program that is - executed to perform the build. It can be either a derivation or a - source (a local file reference, e.g., - <filename>./builder.sh</filename>).</para></listitem> - - <listitem><para>Every attribute is passed as an environment variable - to the builder. Attribute values are translated to environment - variables as follows: - - <itemizedlist> - - <listitem><para>Strings and integers are just passed - verbatim.</para></listitem> - - <listitem><para>A <emphasis>path</emphasis> (e.g., - <filename>../foo/sources.tar</filename>) causes the referenced - file to be copied to the store; its location in the store is put - in the environment variable. The idea is that all sources - should reside in the Nix store, since all inputs to a derivation - should reside in the Nix store.</para></listitem> - - <listitem><para>A <emphasis>derivation</emphasis> causes that - derivation to be built prior to the present derivation; its - default output path is put in the environment - variable.</para></listitem> - - <listitem><para>Lists of the previous types are also allowed. - They are simply concatenated, separated by - spaces.</para></listitem> - - <listitem><para><literal>true</literal> is passed as the string - <literal>1</literal>, <literal>false</literal> and - <literal>null</literal> are passed as an empty string. - </para></listitem> - </itemizedlist> - - </para></listitem> - - <listitem><para>The optional attribute <varname>args</varname> - specifies command-line arguments to be passed to the builder. It - should be a list.</para></listitem> - - <listitem><para>The optional attribute <varname>outputs</varname> - specifies a list of symbolic outputs of the derivation. By default, - a derivation produces a single output path, denoted as - <literal>out</literal>. However, derivations can produce multiple - output paths. This is useful because it allows outputs to be - downloaded or garbage-collected separately. For instance, imagine a - library package that provides a dynamic library, header files, and - documentation. A program that links against the library doesn’t - need the header files and documentation at runtime, and it doesn’t - need the documentation at build time. Thus, the library package - could specify: -<programlisting> -outputs = [ "lib" "headers" "doc" ]; -</programlisting> - This will cause Nix to pass environment variables - <literal>lib</literal>, <literal>headers</literal> and - <literal>doc</literal> to the builder containing the intended store - paths of each output. The builder would typically do something like -<programlisting> -./configure --libdir=$lib/lib --includedir=$headers/include --docdir=$doc/share/doc -</programlisting> - for an Autoconf-style package. You can refer to each output of a - derivation by selecting it as an attribute, e.g. -<programlisting> -buildInputs = [ pkg.lib pkg.headers ]; -</programlisting> - The first element of <varname>output</varname> determines the - <emphasis>default output</emphasis>. Thus, you could also write -<programlisting> -buildInputs = [ pkg pkg.headers ]; -</programlisting> - since <literal>pkg</literal> is equivalent to - <literal>pkg.lib</literal>.</para></listitem> - -</itemizedlist> - -<para>The function <function>mkDerivation</function> in the standard -environment is a wrapper around <function>derivation</function> that -adds a default value for <varname>system</varname> and always uses -Bash as the builder, to which the supplied builder is passed as a -command-line argument. See <xref linkend='sec-standard-environment' -/>.</para> - -<para>The builder is executed as follows: - -<itemizedlist> - - <listitem><para>A temporary directory is created under the directory - specified by <envar>TMPDIR</envar> (default - <filename>/tmp</filename>) where the build will take place. The - current directory is changed to this directory.</para></listitem> - - <listitem><para>The environment is cleared and set to the derivation - attributes, as specified above.</para></listitem> - - <listitem><para>In addition, the following variables are set: - - <itemizedlist> - - <listitem><para><envar>NIX_BUILD_TOP</envar> contains the path of - the temporary directory for this build.</para></listitem> - - <listitem><para>Also, <envar>TMPDIR</envar>, - <envar>TEMPDIR</envar>, <envar>TMP</envar>, <envar>TEMP</envar> - are set to point to the temporary directory. This is to prevent - the builder from accidentally writing temporary files anywhere - else. Doing so might cause interference by other - processes.</para></listitem> - - <listitem><para><envar>PATH</envar> is set to - <filename>/path-not-set</filename> to prevent shells from - initialising it to their built-in default value.</para></listitem> - - <listitem><para><envar>HOME</envar> is set to - <filename>/homeless-shelter</filename> to prevent programs from - using <filename>/etc/passwd</filename> or the like to find the - user's home directory, which could cause impurity. Usually, when - <envar>HOME</envar> is set, it is used as the location of the home - directory, even if it points to a non-existent - path.</para></listitem> - - <listitem><para><envar>NIX_STORE</envar> is set to the path of the - top-level Nix store directory (typically, - <filename>/nix/store</filename>).</para></listitem> - - <listitem><para>For each output declared in - <varname>outputs</varname>, the corresponding environment variable - is set to point to the intended path in the Nix store for that - output. Each output path is a concatenation of the cryptographic - hash of all build inputs, the <varname>name</varname> attribute - and the output name. (The output name is omitted if it’s - <literal>out</literal>.)</para></listitem> - - </itemizedlist> - - </para></listitem> - - <listitem><para>If an output path already exists, it is removed. - Also, locks are acquired to prevent multiple Nix instances from - performing the same build at the same time.</para></listitem> - - <listitem><para>A log of the combined standard output and error is - written to <filename>/nix/var/log/nix</filename>.</para></listitem> - - <listitem><para>The builder is executed with the arguments specified - by the attribute <varname>args</varname>. If it exits with exit - code 0, it is considered to have succeeded.</para></listitem> - - <listitem><para>The temporary directory is removed (unless the - <option>-K</option> option was specified).</para></listitem> - - <listitem><para>If the build was successful, Nix scans each output - path for references to input paths by looking for the hash parts of - the input paths. Since these are potential runtime dependencies, - Nix registers them as dependencies of the output - paths.</para></listitem> - - <listitem><para>After the build, Nix sets the last-modified - timestamp on all files in the build result to 1 (00:00:01 1/1/1970 - UTC), sets the group to the default group, and sets the mode of the - file to 0444 or 0555 (i.e., read-only, with execute permission - enabled if the file was originally executable). Note that possible - <literal>setuid</literal> and <literal>setgid</literal> bits are - cleared. Setuid and setgid programs are not currently supported by - Nix. This is because the Nix archives used in deployment have no - concept of ownership information, and because it makes the build - result dependent on the user performing the build.</para></listitem> - -</itemizedlist> - -</para> - - -<section><title>Advanced attributes</title> - -<para>Derivations can declare some infrequently used optional -attributes.</para> - -<variablelist> - - <varlistentry><term><varname>allowedReferences</varname></term> - - <listitem><para>The optional attribute - <varname>allowedReferences</varname> specifies a list of legal - references (dependencies) of the output of the builder. For - example, - -<programlisting> -allowedReferences = []; -</programlisting> - - enforces that the output of a derivation cannot have any runtime - dependencies on its inputs. To allow an output to have a runtime - dependency on itself, use <literal>"out"</literal> as a list item. - This is used in NixOS to check that generated files such as - initial ramdisks for booting Linux don’t have accidental - dependencies on other paths in the Nix store.</para></listitem> - - </varlistentry> - - <varlistentry><term><varname>allowedRequisites</varname></term> - - <listitem><para>This attribute is similar to - <varname>allowedReferences</varname>, but it specifies the legal - requisites of the whole closure, so all the dependencies - recursively. For example, - -<programlisting> -allowedReferences = [ foobar ]; -</programlisting> - - enforces that the output of a derivation cannot have any other - runtime dependency than <varname>foobar</varname>, and in addition - it enforces that <varname>foobar</varname> itself doesn't - introduce any other dependency itself.</para></listitem> - - </varlistentry> - - <varlistentry><term><varname>exportReferencesGraph</varname></term> - - <listitem><para>This attribute allows builders access to the - references graph of their inputs. The attribute is a list of - inputs in the Nix store whose references graph the builder needs - to know. The value of this attribute should be a list of pairs - <literal>[ <replaceable>name1</replaceable> - <replaceable>path1</replaceable> <replaceable>name2</replaceable> - <replaceable>path2</replaceable> <replaceable>...</replaceable> - ]</literal>. The references graph of each - <replaceable>pathN</replaceable> will be stored in a text file - <replaceable>nameN</replaceable> in the temporary build directory. - The text files have the format used by <command>nix-store - --register-validity</command> (with the deriver fields left - empty). For example, when the following derivation is built: - -<programlisting> -derivation { - ... - exportReferencesGraph = [ "libfoo-graph" libfoo ]; -}; -</programlisting> - - the references graph of <literal>libfoo</literal> is placed in the - file <filename>libfoo-graph</filename> in the temporary build - directory.</para> - - <para><varname>exportReferencesGraph</varname> is useful for - builders that want to do something with the closure of a store - path. Examples include the builders in NixOS that generate the - initial ramdisk for booting Linux (a <command>cpio</command> - archive containing the closure of the boot script) and the - ISO-9660 image for the installation CD (which is populated with a - Nix store containing the closure of a bootable NixOS - configuration).</para></listitem> - - </varlistentry> - - - <varlistentry xml:id="fixed-output-drvs"> - <term><varname>outputHash</varname></term> - <term><varname>outputHashAlgo</varname></term> - <term><varname>outputHashMode</varname></term> - - <listitem><para>These attributes declare that the derivation is a - so-called <emphasis>fixed-output derivation</emphasis>, which - means that a cryptographic hash of the output is already known in - advance. When the build of a fixed-output derivation finishes, - Nix computes the cryptographic hash of the output and compares it - to the hash declared with these attributes. If there is a - mismatch, the build fails.</para> - - <para>The rationale for fixed-output derivations is derivations - such as those produced by the <function>fetchurl</function> - function. This function downloads a file from a given URL. To - ensure that the downloaded file has not been modified, the caller - must also specify a cryptographic hash of the file. For example, - -<programlisting> -fetchurl { - url = http://ftp.gnu.org/pub/gnu/hello/hello-2.1.1.tar.gz; - md5 = "70c9ccf9fac07f762c24f2df2290784d"; -} -</programlisting> - - It sometimes happens that the URL of the file changes, e.g., - because servers are reorganised or no longer available. We then - must update the call to <function>fetchurl</function>, e.g., - -<programlisting> -fetchurl { - url = ftp://ftp.nluug.nl/pub/gnu/hello/hello-2.1.1.tar.gz; - md5 = "70c9ccf9fac07f762c24f2df2290784d"; -} -</programlisting> - - If a <function>fetchurl</function> derivation was treated like a - normal derivation, the output paths of the derivation and - <emphasis>all derivations depending on it</emphasis> would change. - For instance, if we were to change the URL of the Glibc source - distribution in Nixpkgs (a package on which almost all other - packages depend) massive rebuilds would be needed. This is - unfortunate for a change which we know cannot have a real effect - as it propagates upwards through the dependency graph.</para> - - <para>For fixed-output derivations, on the other hand, the name of - the output path only depends on the <varname>outputHash*</varname> - and <varname>name</varname> attributes, while all other attributes - are ignored for the purpose of computing the output path. (The - <varname>name</varname> attribute is included because it is part - of the path.)</para> - - <para>As an example, here is the (simplified) Nix expression for - <varname>fetchurl</varname>: - -<programlisting> -{ stdenv, curl }: # The <command>curl</command> program is used for downloading. - -{ url, md5 }: - -stdenv.mkDerivation { - name = baseNameOf (toString url); - builder = ./builder.sh; - buildInputs = [ curl ]; - - # This is a fixed-output derivation; the output must be a regular - # file with MD5 hash <varname>md5</varname>. - outputHashMode = "flat"; - outputHashAlgo = "md5"; - outputHash = md5; - - inherit url; -} -</programlisting> - - </para> - - <para>The <varname>outputHashAlgo</varname> attribute specifies - the hash algorithm used to compute the hash. It can currently be - <literal>"md5"</literal>, <literal>"sha1"</literal> or - <literal>"sha256"</literal>.</para> - - <para>The <varname>outputHashMode</varname> attribute determines - how the hash is computed. It must be one of the following two - values: - - <variablelist> - - <varlistentry><term><literal>"flat"</literal></term> - - <listitem><para>The output must be a non-executable regular - file. If it isn’t, the build fails. The hash is simply - computed over the contents of that file (so it’s equal to what - Unix commands like <command>md5sum</command> or - <command>sha1sum</command> produce).</para> - - <para>This is the default.</para></listitem> - - </varlistentry> - - <varlistentry><term><literal>"recursive"</literal></term> - - <listitem><para>The hash is computed over the NAR archive dump - of the output (i.e., the result of <link - linkend="refsec-nix-store-dump"><command>nix-store - --dump</command></link>). In this case, the output can be - anything, including a directory tree.</para></listitem> - - </varlistentry> - - </variablelist> - - </para> - - <para>The <varname>outputHash</varname> attribute, finally, must - be a string containing the hash in either hexadecimal or base-32 - notation. (See the <link - linkend="sec-nix-hash"><command>nix-hash</command> command</link> - for information about converting to and from base-32 - notation.)</para></listitem> - - </varlistentry> - - - <varlistentry><term><varname>impureEnvVars</varname></term> - - <listitem><para>This attribute allows you to specify a list of - environment variables that should be passed from the environment - of the calling user to the builder. Usually, the environment is - cleared completely when the builder is executed, but with this - attribute you can allow specific environment variables to be - passed unmodified. For example, <function>fetchurl</function> in - Nixpkgs has the line - -<programlisting> -impureEnvVars = [ "http_proxy" "https_proxy" <replaceable>...</replaceable> ]; -</programlisting> - - to make it use the proxy server configuration specified by the - user in the environment variables <envar>http_proxy</envar> and - friends.</para> - - <para>This attribute is only allowed in <link - linkend="fixed-output-drvs">fixed-output derivations</link>, where - impurities such as these are okay since (the hash of) the output - is known in advance. It is ignored for all other - derivations.</para></listitem> - - </varlistentry> - - - <varlistentry><term><varname>preferLocalBuild</varname></term> - - <listitem><para>If this attribute is set to - <literal>true</literal>, it has two effects. First, the - derivation will always be built, not substituted, even if a - substitute is available. Second, if <link - linkend="chap-distributed-builds">distributed building is - enabled</link>, then, if possible, the derivaton will be built - locally instead of forwarded to a remote machine. This is - appropriate for trivial builders where the cost of doing a - download or remote build would exceed the cost of building - locally.</para></listitem> - - </varlistentry> - -</variablelist> - -</section> - - -</section> - - - -<xi:include href="builtins.xml" /> - - -</section> - - - -<section xml:id='sec-standard-environment'><title>The standard environment</title> - - -<para>The standard environment is used by passing it as an input -called <envar>stdenv</envar> to the derivation, and then doing - -<programlisting> -source $stdenv/setup</programlisting> - -at the top of the builder.</para> - -<para>Apart from adding the aforementioned commands to the -<envar>PATH</envar>, <filename>setup</filename> also does the -following: - -<itemizedlist> - - <listitem><para>All input packages specified in the - <envar>buildInputs</envar> environment variable have their - <filename>/bin</filename> subdirectory added to <envar>PATH</envar>, - their <filename>/include</filename> subdirectory added to the C/C++ - header file search path, and their <filename>/lib</filename> - subdirectory added to the linker search path. This can be extended. - For instance, when the <command>pkgconfig</command> package is - used, the subdirectory <filename>/lib/pkgconfig</filename> of each - input is added to the <envar>PKG_CONFIG_PATH</envar> environment - variable.</para></listitem> - - <listitem><para>The environment variable - <envar>NIX_CFLAGS_STRIP</envar> is set so that the compiler strips - debug information from object files. This can be disabled by - setting <envar>NIX_STRIP_DEBUG</envar> to - <literal>0</literal>.</para></listitem> - -</itemizedlist> - -</para> - -<para>The <filename>setup</filename> script also exports a function -called <function>genericBuild</function> that knows how to build -typical Autoconf-style packages. It can be customised to perform -builds for any type of package. It is advisable to use -<function>genericBuild</function> since it provides facilities that -are almost always useful such as unpacking of sources, patching of -sources, nested logging, etc.</para> - -<para>The definitive, up-to-date documentation of the generic builder -is the source itself, which resides in -<filename>pkgs/stdenv/generic/setup.sh</filename>.</para> - - -<section><title>Customising the generic builder</title> - -<para>The operation of the generic builder can be modified in many -places by setting certain variables. These <emphasis>hook -variables</emphasis> are typically set to the name of some shell -function defined by you. For instance, to perform some additional -steps after <command>make install</command> you would set the -<varname>postInstall</varname> variable: - -<programlisting> -postInstall=myPostInstall - -myPostInstall() { - mkdir $out/share/extra - cp extrafiles/* $out/share/extra -}</programlisting> - -</para> - - -</section> - - -<section><title>Debugging failed builds</title> - -<para>At the beginning of each phase, the set of all shell variables -is written to the file <filename>env-vars</filename> at the top-level -build directory. This is useful for debugging: it allows you to -recreate the environment in which a build was performed. For -instance, if a build fails, then assuming you used the -<option>-K</option> flag, you can go to the output directory and -<quote>switch</quote> to the environment of the builder: - -<screen> -$ nix-build -K ./foo.nix -... fails, keeping build directory `/tmp/nix-1234-0' - -$ cd /tmp/nix-1234-0 - -$ source env-vars - -<lineannotation>(edit some files...)</lineannotation> - -$ make - -<lineannotation>(execution continues with the same GCC, make, etc.)</lineannotation></screen> - -</para> - -</section> - - -</section> - - -</chapter> |