diff options
Diffstat (limited to 'doc/manual')
39 files changed, 13476 insertions, 0 deletions
diff --git a/doc/manual/bugs.xml b/doc/manual/bugs.xml new file mode 100644 index 000000000000..aa87e4b57048 --- /dev/null +++ b/doc/manual/bugs.xml @@ -0,0 +1,39 @@ +<appendix xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink"> + +<title>Bugs / To-Do</title> + + +<itemizedlist> + +<listitem><para>The man-pages generated from the DocBook documentation +are ugly.</para></listitem> + +<listitem><para>Generations properly form a tree. E.g., if after +switching to generation 39, we perform an installation action, a +generation 43 is created which is a descendant of 39, not 42. So a +rollback from 43 ought to go back to 39. This is not currently +implemented; generations form a linear sequence.</para></listitem> + +<listitem><para>For security, <command>nix-push</command> manifests +should be digitally signed, and <command>nix-pull</command> should +verify the signatures. The actual NAR archives in the cache do not +need to be signed, since the manifest contains cryptographic hashes of +these files (and <filename>fetchurl.nix</filename> checks +them).</para></listitem> + +<listitem><para>It would be useful to have an option in +<command>nix-env --delete-generations</command> to remove non-current +generations older than a certain age.</para></listitem> + +<listitem><para>There should be a flexible way to change the user +environment builder. Currently, you have to replace +<filename><replaceable>prefix</replaceable>/share/nix/corepkgs/buildenv/builder.pl</filename>, +which is hard-coded into <command>nix-env</command>. Also, the +default builder should be more powerful. For instance, there should +be some way to specify priorities to resolve +collisions.</para></listitem> + +</itemizedlist> + +</appendix> diff --git a/doc/manual/build-farm.xml b/doc/manual/build-farm.xml new file mode 100644 index 000000000000..2e0d86b89f09 --- /dev/null +++ b/doc/manual/build-farm.xml @@ -0,0 +1,113 @@ +<chapter xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xml:id='chap-distributed-builds'> + +<title>Setting Up Distributed Builds</title> + +<para>Nix supports distributed builds: a local Nix installation can +forward Nix builds to other machines over the network. This allows +multiple builds to be performed in parallel (thus improving +performance) and allows Nix to perform multi-platform builds in a +semi-transparent way. For instance, if you perform a build for a +<literal>powerpc-darwin</literal> on an <literal>i686-linux</literal> +machine, Nix can automatically forward the build to a +<literal>powerpc-darwin</literal> machine, if available.</para> + +<para>You can enable distributed builds by setting the environment +variable <envar>NIX_BUILD_HOOK</envar> to point to a program that Nix +will call whenever it wants to build a derivation. The build hook +(typically a shell or Perl script) can decline the build, in which Nix +will perform it in the usual way if possible, or it can accept it, in +which case it is responsible for somehow getting the inputs of the +build to another machine, doing the build there, and getting the +results back. The details of the build hook protocol are described in +the documentation of the <link +linkend="envar-build-hook"><envar>NIX_BUILD_HOOK</envar> +variable</link>.</para> + +<example xml:id='ex-remote-systems'><title>Remote machine configuration: +<filename>remote-systems.conf</filename></title> +<programlisting> +nix@mcflurry.labs.cs.uu.nl powerpc-darwin /home/nix/.ssh/id_quarterpounder_auto 2 +nix@scratchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 1 kvm +nix@itchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 2 +nix@poochie.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 2 kvm perf +</programlisting> +</example> + +<para>Nix ships with a build hook that should be suitable for most +purposes. It uses <command>ssh</command> and +<command>nix-copy-closure</command> to copy the build inputs and +outputs and perform the remote build. To use it, you should set +<envar>NIX_BUILD_HOOK</envar> to +<filename><replaceable>prefix</replaceable>/libexec/nix/build-remote.pl</filename>. +You should also define a list of available build machines and point +the environment variable <envar>NIX_REMOTE_SYSTEMS</envar> to it. An +example configuration is shown in <xref linkend='ex-remote-systems' +/>. Each line in the file specifies a machine, with the following +bits of information: + +<orderedlist> + + <listitem><para>The name of the remote machine, with optionally the + user under which the remote build should be performed. This is + actually passed as an argument to <command>ssh</command>, so it can + be an alias defined in your + <filename>~/.ssh/config</filename>.</para></listitem> + + <listitem><para>A comma-separated list of Nix platform type + identifiers, such as <literal>powerpc-darwin</literal>. It is + possible for a machine to support multiple platform types, e.g., + <literal>i686-linux,x86_64-linux</literal>.</para></listitem> + + <listitem><para>The SSH private key to be used to log in to the + remote machine. Since builds should be non-interactive, this key + should not have a passphrase!</para></listitem> + + <listitem><para>The maximum number of builds that + <filename>build-remote.pl</filename> will execute in parallel on the + machine. Typically this should be equal to the number of CPU cores. + For instance, the machine <literal>itchy</literal> in the example + will execute up to 8 builds in parallel.</para></listitem> + + <listitem><para>The “speed factor”, indicating the relative speed of + the machine. If there are multiple machines of the right type, Nix + will prefer the fastest, taking load into account.</para></listitem> + + <listitem><para>A comma-separated list of <emphasis>supported + features</emphasis>. If a derivation has the + <varname>requiredSystemFeatures</varname> attribute, then + <filename>build-remote.pl</filename> will only perform the + derivation on a machine that has the specified features. For + instance, the attribute + +<programlisting> +requiredSystemFeatures = [ "kvm" ]; +</programlisting> + + will cause the build to be performed on a machine that has the + <literal>kvm</literal> feature (i.e., <literal>scratchy</literal> in + the example above).</para></listitem> + + <listitem><para>A comma-separated list of <emphasis>mandatory + features</emphasis>. A machine will only be used to build a + derivation if all of the machine’s mandatory features appear in the + derivation’s <varname>requiredSystemFeatures</varname> attribute. + Thus, in the example, the machine <literal>poochie</literal> will + only do derivations that have + <varname>requiredSystemFeatures</varname> set to <literal>["kvm" + "perf"]</literal> or <literal>["perf"]</literal>.</para></listitem> + +</orderedlist> + +You should also set up the environment variable +<envar>NIX_CURRENT_LOAD</envar> to point at a directory (e.g., +<filename>/var/run/nix/current-load</filename>) that +<filename>build-remote.pl</filename> uses to remember how many builds +it is currently executing remotely. It doesn't look at the actual +load on the remote machine, so if you have multiple instances of Nix +running, they should use the same <envar>NIX_CURRENT_LOAD</envar> +file. Maybe in the future <filename>build-remote.pl</filename> will +look at the actual remote load.</para> + +</chapter> diff --git a/doc/manual/builtins.xml b/doc/manual/builtins.xml new file mode 100644 index 000000000000..6a472291c363 --- /dev/null +++ b/doc/manual/builtins.xml @@ -0,0 +1,932 @@ +<section xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xml:id='ssec-builtins'> + +<title>Built-in functions</title> + + +<para>This section lists the functions and constants built into the +Nix expression evaluator. (The built-in function +<function>derivation</function> is discussed above.) Some built-ins, +such as <function>derivation</function>, are always in scope of every +Nix expression; you can just access them right away. But to prevent +polluting the namespace too much, most built-ins are not in scope. +Instead, you can access them through the <varname>builtins</varname> +built-in value, which is a set that contains all built-in functions +and values. For instance, <function>derivation</function> is also +available as <function>builtins.derivation</function>.</para> + + +<variablelist> + + + <varlistentry><term><function>abort</function> <replaceable>s</replaceable></term> + + <listitem><para>Abort Nix expression evaluation, print error + message <replaceable>s</replaceable>.</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>builtins.add</function> + <replaceable>e1</replaceable> <replaceable>e2</replaceable></term> + + <listitem><para>Return the sum of the integers + <replaceable>e1</replaceable> and + <replaceable>e2</replaceable>.</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>builtins.attrNames</function> + <replaceable>set</replaceable></term> + + <listitem><para>Return the names of the attributes in the set + <replaceable>set</replaceable> in a sorted list. For instance, + <literal>builtins.attrNames { y = 1; x = "foo"; }</literal> + evaluates to <literal>[ "x" "y" ]</literal>. There is no built-in + function <function>attrValues</function>, but you can easily + define it yourself: + +<programlisting> +attrValues = set: map (name: builtins.getAttr name set) (builtins.attrNames set);</programlisting> + + </para></listitem> + + </varlistentry> + + + <varlistentry><term><function>baseNameOf</function> <replaceable>s</replaceable></term> + + <listitem><para>Return the <emphasis>base name</emphasis> of the + string <replaceable>s</replaceable>, that is, everything following + the final slash in the string. This is similar to the GNU + <command>basename</command> command.</para></listitem> + + </varlistentry> + + + <varlistentry><term><varname>builtins</varname></term> + + <listitem><para>The set <varname>builtins</varname> contains all + the built-in functions and values. You can use + <varname>builtins</varname> to test for the availability of + features in the Nix installation, e.g., + +<programlisting> +if builtins ? getEnv then builtins.getEnv "PATH" else ""</programlisting> + + This allows a Nix expression to fall back gracefully on older Nix + installations that don’t have the desired built-in + function.</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>builtins.compareVersions</function> + <replaceable>s1</replaceable> <replaceable>s2</replaceable></term> + + <listitem><para>Compare two strings representing versions and + return <literal>-1</literal> if version + <replaceable>s1</replaceable> is older than version + <replaceable>s2</replaceable>, <literal>0</literal> if they are + the same, and <literal>1</literal> if + <replaceable>s1</replaceable> is newer than + <replaceable>s2</replaceable>. The version comparison algorithm + is the same as the one used by <link + linkend="ssec-version-comparisons"><command>nix-env + -u</command></link>.</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>builtins.concatLists</function> + <replaceable>lists</replaceable></term> + + <listitem><para>Concatenate a list of lists into a single + list.</para></listitem> + + </varlistentry> + + + <varlistentry + xml:id='builtin-currentSystem'><term><varname>builtins.currentSystem</varname></term> + + <listitem><para>The built-in value <varname>currentSystem</varname> + evaluates to the Nix platform identifier for the Nix installation + on which the expression is being evaluated, such as + <literal>"i686-linux"</literal> or + <literal>"powerpc-darwin"</literal>.</para></listitem> + + </varlistentry> + + + <!-- + <varlistentry><term><function>currentTime</function></term> + + <listitem><para>The built-in value <varname>currentTime</varname> + returns the current system time in seconds since 00:00:00 1/1/1970 + UTC. Due to the evaluation model of Nix expressions + (<emphasis>maximal laziness</emphasis>), it always yields the same + value within an execution of Nix.</para></listitem> + + </varlistentry> + --> + + + <!-- + <varlistentry><term><function>dependencyClosure</function></term> + + <listitem><para>TODO</para></listitem> + + </varlistentry> + --> + + + <varlistentry><term><function>derivation</function> + <replaceable>attrs</replaceable></term> + + <listitem><para><function>derivation</function> is described in + <xref linkend='ssec-derivation' />.</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>dirOf</function> <replaceable>s</replaceable></term> + + <listitem><para>Return the directory part of the string + <replaceable>s</replaceable>, that is, everything before the final + slash in the string. This is similar to the GNU + <command>dirname</command> command.</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>builtins.div</function> + <replaceable>e1</replaceable> <replaceable>e2</replaceable></term> + + <listitem><para>Return the quotient of the integers + <replaceable>e1</replaceable> and + <replaceable>e2</replaceable>.</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>builtins.elem</function> + <replaceable>x</replaceable> <replaceable>xs</replaceable></term> + + <listitem><para>Return <literal>true</literal> if a value equal to + <replaceable>x</replaceable> occurs in the list + <replaceable>xs</replaceable>, and <literal>false</literal> + otherwise.</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>builtins.elemAt</function> + <replaceable>xs</replaceable> <replaceable>n</replaceable></term> + + <listitem><para>Return element <replaceable>n</replaceable> from + the list <replaceable>xs</replaceable>. Elements are counted + starting from 0. A fatal error occurs in the index is out of + bounds.</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>builtins.filter</function> + <replaceable>f</replaceable> <replaceable>xs</replaceable></term> + + <listitem><para>Return a list consisting of the elements of + <replaceable>xs</replaceable> for which the function + <replaceable>f</replaceable> returns + <literal>true</literal>.</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>builtins.filterSource</function> + <replaceable>e1</replaceable> <replaceable>e2</replaceable></term> + + <listitem> + + <para>This function allows you to copy sources into the Nix + store while filtering certain files. For instance, suppose that + you want to use the directory <filename>source-dir</filename> as + an input to a Nix expression, e.g. + +<programlisting> +stdenv.mkDerivation { + ... + src = ./source-dir; +} +</programlisting> + + However, if <filename>source-dir</filename> is a Subversion + working copy, then all those annoying <filename>.svn</filename> + subdirectories will also be copied to the store. Worse, the + contents of those directories may change a lot, causing lots of + spurious rebuilds. With <function>filterSource</function> you + can filter out the <filename>.svn</filename> directories: + +<programlisting> + src = builtins.filterSource + (path: type: type != "directory" || baseNameOf path != ".svn") + ./source-dir; +</programlisting> + + </para> + + <para>Thus, the first argument <replaceable>e1</replaceable> + must be a predicate function that is called for each regular + file, directory or symlink in the source tree + <replaceable>e2</replaceable>. If the function returns + <literal>true</literal>, the file is copied to the Nix store, + otherwise it is omitted. The function is called with two + arguments. The first is the full path of the file. The second + is a string that identifies the type of the file, which is + either <literal>"regular"</literal>, + <literal>"directory"</literal>, <literal>"symlink"</literal> or + <literal>"unknown"</literal> (for other kinds of files such as + device nodes or fifos — but note that those cannot be copied to + the Nix store, so if the predicate returns + <literal>true</literal> for them, the copy will fail).</para> + + </listitem> + + </varlistentry> + + + <varlistentry><term><function>builtins.getAttr</function> + <replaceable>s</replaceable> <replaceable>set</replaceable></term> + + <listitem><para><function>getAttr</function> returns the attribute + named <replaceable>s</replaceable> from + <replaceable>set</replaceable>. Evaluation aborts if the + attribute doesn’t exist. This is a dynamic version of the + <literal>.</literal> operator, since <replaceable>s</replaceable> + is an expression rather than an identifier.</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>builtins.getEnv</function> + <replaceable>s</replaceable></term> + + <listitem><para><function>getEnv</function> returns the value of + the environment variable <replaceable>s</replaceable>, or an empty + string if the variable doesn’t exist. This function should be + used with care, as it can introduce all sorts of nasty environment + dependencies in your Nix expression.</para> + + <para><function>getEnv</function> is used in Nix Packages to + locate the file <filename>~/.nixpkgs/config.nix</filename>, which + contains user-local settings for Nix Packages. (That is, it does + a <literal>getEnv "HOME"</literal> to locate the user’s home + directory.)</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>builtins.hasAttr</function> + <replaceable>s</replaceable> <replaceable>set</replaceable></term> + + <listitem><para><function>hasAttr</function> returns + <literal>true</literal> if <replaceable>set</replaceable> has an + attribute named <replaceable>s</replaceable>, and + <literal>false</literal> otherwise. This is a dynamic version of + the <literal>?</literal> operator, since + <replaceable>s</replaceable> is an expression rather than an + identifier.</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>builtins.hashString</function> + <replaceable>type</replaceable> <replaceable>s</replaceable></term> + + <listitem><para>Return a base-16 representation of the + cryptographic hash of string <replaceable>s</replaceable>. The + hash algorithm specified by <replaceable>type</replaceable> must + be one of <literal>"md5"</literal>, <literal>"sha1"</literal> or + <literal>"sha256"</literal>.</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>builtins.head</function> + <replaceable>list</replaceable></term> + + <listitem><para>Return the first element of a list; abort + evaluation if the argument isn’t a list or is an empty list. You + can test whether a list is empty by comparing it with + <literal>[]</literal>.</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>import</function> + <replaceable>path</replaceable></term> + + <listitem><para>Load, parse and return the Nix expression in the + file <replaceable>path</replaceable>. If <replaceable>path + </replaceable> is a directory, the file <filename>default.nix + </filename> in that directory is loaded. Evaluation aborts if the + file doesn’t exist or contains an incorrect Nix expression. + <function>import</function> implements Nix’s module system: you + can put any Nix expression (such as a set or a function) in a + separate file, and use it from Nix expressions in other + files.</para> + + <para>A Nix expression loaded by <function>import</function> must + not contain any <emphasis>free variables</emphasis> (identifiers + that are not defined in the Nix expression itself and are not + built-in). Therefore, it cannot refer to variables that are in + scope at the call site. For instance, if you have a calling + expression + +<programlisting> +rec { + x = 123; + y = import ./foo.nix; +}</programlisting> + + then the following <filename>foo.nix</filename> will give an + error: + +<programlisting> +x + 456</programlisting> + + since <varname>x</varname> is not in scope in + <filename>foo.nix</filename>. If you want <varname>x</varname> + to be available in <filename>foo.nix</filename>, you should pass + it as a function argument: + +<programlisting> +rec { + x = 123; + y = import ./foo.nix x; +}</programlisting> + + and + +<programlisting> +x: x + 456</programlisting> + + (The function argument doesn’t have to be called + <varname>x</varname> in <filename>foo.nix</filename>; any name + would work.)</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>builtins.intersectAttrs</function> + <replaceable>e1</replaceable> <replaceable>e2</replaceable></term> + + <listitem><para>Return a set consisting of the attributes in the + set <replaceable>e2</replaceable> that also exist in the set + <replaceable>e1</replaceable>.</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>builtins.isAttrs</function> + <replaceable>e</replaceable></term> + + <listitem><para>Return <literal>true</literal> if + <replaceable>e</replaceable> evaluates to a set, and + <literal>false</literal> otherwise.</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>builtins.isList</function> + <replaceable>e</replaceable></term> + + <listitem><para>Return <literal>true</literal> if + <replaceable>e</replaceable> evaluates to a list, and + <literal>false</literal> otherwise.</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>builtins.isFunction</function> + <replaceable>e</replaceable></term> + + <listitem><para>Return <literal>true</literal> if + <replaceable>e</replaceable> evaluates to a function, and + <literal>false</literal> otherwise.</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>builtins.isString</function> + <replaceable>e</replaceable></term> + + <listitem><para>Return <literal>true</literal> if + <replaceable>e</replaceable> evaluates to a string, and + <literal>false</literal> otherwise.</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>builtins.isInt</function> + <replaceable>e</replaceable></term> + + <listitem><para>Return <literal>true</literal> if + <replaceable>e</replaceable> evaluates to an int, and + <literal>false</literal> otherwise.</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>builtins.isBool</function> + <replaceable>e</replaceable></term> + + <listitem><para>Return <literal>true</literal> if + <replaceable>e</replaceable> evaluates to a bool, and + <literal>false</literal> otherwise.</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>isNull</function> + <replaceable>e</replaceable></term> + + <listitem><para>Return <literal>true</literal> if + <replaceable>e</replaceable> evaluates to <literal>null</literal>, + and <literal>false</literal> otherwise.</para> + + <warning><para>This function is <emphasis>deprecated</emphasis>; + just write <literal>e == null</literal> instead.</para></warning> + + </listitem> + + </varlistentry> + + + <varlistentry><term><function>builtins.length</function> + <replaceable>e</replaceable></term> + + <listitem><para>Return the length of the list + <replaceable>e</replaceable>.</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>builtins.lessThan</function> + <replaceable>e1</replaceable> <replaceable>e2</replaceable></term> + + <listitem><para>Return <literal>true</literal> if the integer + <replaceable>e1</replaceable> is less than the integer + <replaceable>e2</replaceable>, and <literal>false</literal> + otherwise. Evaluation aborts if either + <replaceable>e1</replaceable> or <replaceable>e2</replaceable> + does not evaluate to an integer.</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>builtins.listToAttrs</function> + <replaceable>e</replaceable></term> + + <listitem><para>Construct a set from a list specifying the names + and values of each attribute. Each element of the list should be + a set consisting of a string-valued attribute + <varname>name</varname> specifying the name of the attribute, and + an attribute <varname>value</varname> specifying its value. + Example: + +<programlisting> +builtins.listToAttrs + [ { name = "foo"; value = 123; } + { name = "bar"; value = 456; } + ] +</programlisting> + + evaluates to + +<programlisting> +{ foo = 123; bar = 456; } +</programlisting> + + </para></listitem> + + </varlistentry> + + <varlistentry><term><function>map</function> + <replaceable>f</replaceable> <replaceable>list</replaceable></term> + + <listitem><para>Apply the function <replaceable>f</replaceable> to + each element in the list <replaceable>list</replaceable>. For + example, + +<programlisting> +map (x: "foo" + x) [ "bar" "bla" "abc" ]</programlisting> + + evaluates to <literal>[ "foobar" "foobla" "fooabc" + ]</literal>.</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>builtins.mul</function> + <replaceable>e1</replaceable> <replaceable>e2</replaceable></term> + + <listitem><para>Return the product of the integers + <replaceable>e1</replaceable> and + <replaceable>e2</replaceable>.</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>builtins.parseDrvName</function> + <replaceable>s</replaceable></term> + + <listitem><para>Split the string <replaceable>s</replaceable> into + a package name and version. The package name is everything up to + but not including the first dash followed by a digit, and the + version is everything following that dash. The result is returned + in a set <literal>{ name, version }</literal>. Thus, + <literal>builtins.parseDrvName "nix-0.12pre12876"</literal> + returns <literal>{ name = "nix"; version = "0.12pre12876"; + }</literal>.</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>builtins.pathExists</function> + <replaceable>path</replaceable></term> + + <listitem><para>Return <literal>true</literal> if the path + <replaceable>path</replaceable> exists, and + <literal>false</literal> otherwise. One application of this + function is to conditionally include a Nix expression containing + user configuration: + +<programlisting> +let + fileName = builtins.getEnv "CONFIG_FILE"; + config = + if fileName != "" && builtins.pathExists (builtins.toPath fileName) + then import (builtins.toPath fileName) + else { someSetting = false; }; <lineannotation># default configuration</lineannotation> +in config.someSetting</programlisting> + + (Note that <envar>CONFIG_FILE</envar> must be an absolute path for + this to work.)</para></listitem> + + </varlistentry> + + + <!-- + <varlistentry><term><function>relativise</function></term> + + <listitem><para>TODO</para></listitem> + + </varlistentry> + --> + + + <varlistentry><term><function>builtins.readFile</function> + <replaceable>path</replaceable></term> + + <listitem><para>Return the contents of the file + <replaceable>path</replaceable> as a string.</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>removeAttrs</function> + <replaceable>set</replaceable> <replaceable>list</replaceable></term> + + <listitem><para>Remove the attributes listed in + <replaceable>list</replaceable> from + <replaceable>set</replaceable>. The attributes don’t have to + exist in <replaceable>set</replaceable>. For instance, + +<screen> +removeAttrs { x = 1; y = 2; z = 3; } [ "a" "x" "z" ]</screen> + + evaluates to <literal>{ y = 2; }</literal>.</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>builtins.stringLength</function> + <replaceable>e</replaceable></term> + + <listitem><para>Return the length of the string + <replaceable>e</replaceable>. If <replaceable>e</replaceable> is + not a string, evaluation is aborted.</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>builtins.sub</function> + <replaceable>e1</replaceable> <replaceable>e2</replaceable></term> + + <listitem><para>Return the difference between the integers + <replaceable>e1</replaceable> and + <replaceable>e2</replaceable>.</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>builtins.substring</function> + <replaceable>start</replaceable> <replaceable>len</replaceable> + <replaceable>s</replaceable></term> + + <listitem><para>Return the substring of + <replaceable>s</replaceable> from character position + <replaceable>start</replaceable> (zero-based) up to but not + including <replaceable>start + len</replaceable>. If + <replaceable>start</replaceable> is greater than the length of the + string, an empty string is returned, and if <replaceable>start + + len</replaceable> lies beyond the end of the string, only the + substring up to the end of the string is returned. + <replaceable>start</replaceable> must be + non-negative.</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>builtins.tail</function> + <replaceable>list</replaceable></term> + + <listitem><para>Return the second to last elements of a list; + abort evaluation if the argument isn’t a list or is an empty + list.</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>throw</function> + <replaceable>s</replaceable></term> + + <listitem><para>Throw an error message + <replaceable>s</replaceable>. This usually aborts Nix expression + evaluation, but in <command>nix-env -qa</command> and other + commands that try to evaluate a set of derivations to get + information about those derivations, a derivation that throws an + error is silently skipped (which is not the case for + <function>abort</function>).</para></listitem> + + </varlistentry> + + + <varlistentry + xml:id='builtin-toFile'><term><function>builtins.toFile</function> + <replaceable>name</replaceable> <replaceable>s</replaceable></term> + + <listitem><para>Store the string <replaceable>s</replaceable> in a + file in the Nix store and return its path. The file has suffix + <replaceable>name</replaceable>. This file can be used as an + input to derivations. One application is to write builders + “inline”. For instance, the following Nix expression combines + <xref linkend='ex-hello-nix' /> and <xref + linkend='ex-hello-builder' /> into one file: + +<programlisting> +{ stdenv, fetchurl, perl }: + +stdenv.mkDerivation { + name = "hello-2.1.1"; + + builder = builtins.toFile "builder.sh" " + source $stdenv/setup + + PATH=$perl/bin:$PATH + + tar xvfz $src + cd hello-* + ./configure --prefix=$out + make + make install + "; + + src = fetchurl { + url = http://nix.cs.uu.nl/dist/tarballs/hello-2.1.1.tar.gz; + md5 = "70c9ccf9fac07f762c24f2df2290784d"; + }; + inherit perl; +}</programlisting> + + </para> + + <para>It is even possible for one file to refer to another, e.g., + +<programlisting> + builder = let + configFile = builtins.toFile "foo.conf" " + # This is some dummy configuration file. + <replaceable>...</replaceable> + "; + in builtins.toFile "builder.sh" " + source $stdenv/setup + <replaceable>...</replaceable> + cp ${configFile} $out/etc/foo.conf + ";</programlisting> + + Note that <literal>${configFile}</literal> is an antiquotation + (see <xref linkend='ssec-values' />), so the result of the + expression <literal>configFile</literal> (i.e., a path like + <filename>/nix/store/m7p7jfny445k...-foo.conf</filename>) will be + spliced into the resulting string.</para> + + <para>It is however <emphasis>not</emphasis> allowed to have files + mutually referring to each other, like so: + +<programlisting> +let + foo = builtins.toFile "foo" "...${bar}..."; + bar = builtins.toFile "bar" "...${foo}..."; +in foo</programlisting> + + This is not allowed because it would cause a cyclic dependency in + the computation of the cryptographic hashes for + <varname>foo</varname> and <varname>bar</varname>.</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>builtins.toJSON</function> <replaceable>e</replaceable></term> + + <listitem><para>Return a string containing a JSON representation + of <replaceable>e</replaceable>. Strings, integers, booleans, + nulls and lists are mapped to their JSON equivalents. Sets + (except derivations) are represented as objects. Derivations are + translated to a JSON string containing the derivation’s output + path. Paths are copied to the store and represented as a JSON + string of the resulting store path.</para></listitem> + + </varlistentry> + + <varlistentry><term><function>builtins.toPath</function> <replaceable>s</replaceable></term> + + <listitem><para>Convert the string value + <replaceable>s</replaceable> into a path value. The string + <replaceable>s</replaceable> must represent an absolute path + (i.e., must start with <literal>/</literal>). The path need not + exist. The resulting path is canonicalised, e.g., + <literal>builtins.toPath "//foo/xyzzy/../bar/"</literal> returns + <literal>/foo/bar</literal>.</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>toString</function> <replaceable>e</replaceable></term> + + <listitem><para>Convert the expression + <replaceable>e</replaceable> to a string. + <replaceable>e</replaceable> can be a string (in which case + <function>toString</function> is a no-op) or a path (e.g., + <literal>toString /foo/bar</literal> yields + <literal>"/foo/bar"</literal>.</para></listitem> + + </varlistentry> + + + <varlistentry xml:id='builtin-toXML'><term><function>builtins.toXML</function> <replaceable>e</replaceable></term> + + <listitem><para>Return a string containing an XML representation + of <replaceable>e</replaceable>. The main application for + <function>toXML</function> is to communicate information with the + builder in a more structured format than plain environment + variables.</para> + + <!-- TODO: more formally describe the schema of the XML + representation --> + + <para><xref linkend='ex-toxml' /> shows an example where this is + the case. The builder is supposed to generate the configuration + file for a <link xlink:href='http://jetty.mortbay.org/'>Jetty + servlet container</link>. A servlet container contains a number + of servlets (<filename>*.war</filename> files) each exported under + a specific URI prefix. So the servlet configuration is a list of + sets containing the <varname>path</varname> and + <varname>war</varname> of the servlet (<xref + linkend='ex-toxml-co-servlets' />). This kind of information is + difficult to communicate with the normal method of passing + information through an environment variable, which just + concatenates everything together into a string (which might just + work in this case, but wouldn’t work if fields are optional or + contain lists themselves). Instead the Nix expression is + converted to an XML representation with + <function>toXML</function>, which is unambiguous and can easily be + processed with the appropriate tools. For instance, in the + example an XSLT stylesheet (<xref linkend='ex-toxml-co-stylesheet' + />) is applied to it (<xref linkend='ex-toxml-co-apply' />) to + generate the XML configuration file for the Jetty server. The XML + representation produced from <xref linkend='ex-toxml-co-servlets' + /> by <function>toXML</function> is shown in <xref + linkend='ex-toxml-result' />.</para> + + <para>Note that <xref linkend='ex-toxml' /> uses the <function + linkend='builtin-toFile'>toFile</function> built-in to write the + builder and the stylesheet “inline” in the Nix expression. The + path of the stylesheet is spliced into the builder at + <literal>xsltproc ${stylesheet} + <replaceable>...</replaceable></literal>.</para> + + <example xml:id='ex-toxml'><title>Passing information to a builder + using <function>toXML</function></title> + +<programlisting><![CDATA[ +{ stdenv, fetchurl, libxslt, jira, uberwiki }: + +stdenv.mkDerivation (rec { + name = "web-server"; + + buildInputs = [ libxslt ]; + + builder = builtins.toFile "builder.sh" " + source $stdenv/setup + mkdir $out + echo $servlets | xsltproc ${stylesheet} - > $out/server-conf.xml]]> <co xml:id='ex-toxml-co-apply' /> <![CDATA[ + "; + + stylesheet = builtins.toFile "stylesheet.xsl"]]> <co xml:id='ex-toxml-co-stylesheet' /> <![CDATA[ + "<?xml version='1.0' encoding='UTF-8'?> + <xsl:stylesheet xmlns:xsl='http://www.w3.org/1999/XSL/Transform' version='1.0'> + <xsl:template match='/'> + <Configure> + <xsl:for-each select='/expr/list/attrs'> + <Call name='addWebApplication'> + <Arg><xsl:value-of select=\"attr[@name = 'path']/string/@value\" /></Arg> + <Arg><xsl:value-of select=\"attr[@name = 'war']/path/@value\" /></Arg> + </Call> + </xsl:for-each> + </Configure> + </xsl:template> + </xsl:stylesheet> + "; + + servlets = builtins.toXML []]> <co xml:id='ex-toxml-co-servlets' /> <![CDATA[ + { path = "/bugtracker"; war = jira + "/lib/atlassian-jira.war"; } + { path = "/wiki"; war = uberwiki + "/uberwiki.war"; } + ]; +})]]></programlisting> + + </example> + + <example xml:id='ex-toxml-result'><title>XML representation produced by + <function>toXML</function></title> + +<programlisting><![CDATA[<?xml version='1.0' encoding='utf-8'?> +<expr> + <list> + <attrs> + <attr name="path"> + <string value="/bugtracker" /> + </attr> + <attr name="war"> + <path value="/nix/store/d1jh9pasa7k2...-jira/lib/atlassian-jira.war" /> + </attr> + </attrs> + <attrs> + <attr name="path"> + <string value="/wiki" /> + </attr> + <attr name="war"> + <path value="/nix/store/y6423b1yi4sx...-uberwiki/uberwiki.war" /> + </attr> + </attrs> + </list> +</expr>]]></programlisting> + + </example> + + </listitem> + + </varlistentry> + + + <varlistentry><term><function>builtins.trace</function> + <replaceable>e1</replaceable> <replaceable>e2</replaceable></term> + + <listitem><para>Evaluate <replaceable>e1</replaceable> and print its + abstract syntax representation on standard error. Then return + <replaceable>e2</replaceable>. This function is useful for + debugging.</para></listitem> + + </varlistentry> + + + <varlistentry><term><function>builtins.typeOf</function> + <replaceable>e</replaceable></term> + + <listitem><para>Return a string representing the type of the value + <replaceable>e</replaceable>, namely <literal>"int"</literal>, + <literal>"bool"</literal>, <literal>"string"</literal>, + <literal>"path"</literal>, <literal>"null"</literal>, + <literal>"set"</literal>, <literal>"list"</literal> or + <literal>"lambda"</literal>.</para></listitem> + + </varlistentry> + + +</variablelist> + + +</section> diff --git a/doc/manual/conf-file.xml b/doc/manual/conf-file.xml new file mode 100644 index 000000000000..c832108fed06 --- /dev/null +++ b/doc/manual/conf-file.xml @@ -0,0 +1,460 @@ +<refentry xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + xml:id="sec-conf-file"> + +<refmeta> + <refentrytitle>nix.conf</refentrytitle> + <manvolnum>5</manvolnum> + <refmiscinfo class="source">Nix</refmiscinfo> + <refmiscinfo class="version"><xi:include href="version.txt" parse="text"/></refmiscinfo> +</refmeta> + +<refnamediv> + <refname>nix.conf</refname> + <refpurpose>Nix configuration file</refpurpose> +</refnamediv> + +<refsection><title>Description</title> + +<para>A number of persistent settings of Nix are stored in the file +<filename><replaceable>sysconfdir</replaceable>/nix/nix.conf</filename>. +This file is a list of <literal><replaceable>name</replaceable> = +<replaceable>value</replaceable></literal> pairs, one per line. +Comments start with a <literal>#</literal> character. Here is an example +configuration file:</para> + +<programlisting> +gc-keep-outputs = true # Nice for developers +gc-keep-derivations = true # Idem +env-keep-derivations = false +</programlisting> + +<para>You can override settings using the <option>--option</option> +flag, e.g. <literal>--option gc-keep-outputs false</literal>.</para> + +<para>The following settings are currently available: + +<variablelist> + + + <varlistentry xml:id="conf-gc-keep-outputs"><term><literal>gc-keep-outputs</literal></term> + + <listitem><para>If <literal>true</literal>, the garbage collector + will keep the outputs of non-garbage derivations. If + <literal>false</literal> (default), outputs will be deleted unless + they are GC roots themselves (or reachable from other roots).</para> + + <para>In general, outputs must be registered as roots separately. + However, even if the output of a derivation is registered as a + root, the collector will still delete store paths that are used + only at build time (e.g., the C compiler, or source tarballs + downloaded from the network). To prevent it from doing so, set + this option to <literal>true</literal>.</para></listitem> + + </varlistentry> + + + <varlistentry xml:id="conf-gc-keep-derivations"><term><literal>gc-keep-derivations</literal></term> + + <listitem><para>If <literal>true</literal> (default), the garbage + collector will keep the derivations from which non-garbage store + paths were built. If <literal>false</literal>, they will be + deleted unless explicitly registered as a root (or reachable from + other roots).</para> + + <para>Keeping derivation around is useful for querying and + traceability (e.g., it allows you to ask with what dependencies or + options a store path was built), so by default this option is on. + Turn it off to safe a bit of disk space (or a lot if + <literal>gc-keep-outputs</literal> is also turned on).</para></listitem> + + </varlistentry> + + + <varlistentry><term><literal>env-keep-derivations</literal></term> + + <listitem><para>If <literal>false</literal> (default), derivations + are not stored in Nix user environments. That is, the derivation + any build-time-only dependencies may be garbage-collected.</para> + + <para>If <literal>true</literal>, when you add a Nix derivation to + a user environment, the path of the derivation is stored in the + user environment. Thus, the derivation will not be + garbage-collected until the user environment generation is deleted + (<command>nix-env --delete-generations</command>). To prevent + build-time-only dependencies from being collected, you should also + turn on <literal>gc-keep-outputs</literal>.</para> + + <para>The difference between this option and + <literal>gc-keep-derivations</literal> is that this one is + “sticky”: it applies to any user environment created while this + option was enabled, while <literal>gc-keep-derivations</literal> + only applies at the moment the garbage collector is + run.</para></listitem> + + </varlistentry> + + + <varlistentry xml:id="conf-build-max-jobs"><term><literal>build-max-jobs</literal></term> + + <listitem><para>This option defines the maximum number of jobs + that Nix will try to build in parallel. The default is + <literal>1</literal>. You should generally set it to the number + of CPUs in your system (e.g., <literal>2</literal> on an Athlon 64 + X2). It can be overridden using the <option + linkend='opt-max-jobs'>--max-jobs</option> (<option>-j</option>) + command line switch.</para></listitem> + + </varlistentry> + + + <varlistentry xml:id="conf-build-cores"><term><literal>build-cores</literal></term> + + <listitem><para>Sets the value of the + <envar>NIX_BUILD_CORES</envar> environment variable in the + invocation of builders. Builders can use this variable at their + discretion to control the maximum amount of parallelism. For + instance, in Nixpkgs, if the derivation attribute + <varname>enableParallelBuilding</varname> is set to + <literal>true</literal>, the builder passes the + <option>-j<replaceable>N</replaceable></option> flag to GNU Make. + It can be overridden using the <option + linkend='opt-cores'>--cores</option> command line switch and + defaults to <literal>1</literal>. The value <literal>0</literal> + means that the builder should use all available CPU cores in the + system.</para></listitem> + + </varlistentry> + + + <varlistentry xml:id="conf-build-max-silent-time"><term><literal>build-max-silent-time</literal></term> + + <listitem> + + <para>This option defines the maximum number of seconds that a + builder can go without producing any data on standard output or + standard error. This is useful (for instance in an automated + build system) to catch builds that are stuck in an infinite + loop, or to catch remote builds that are hanging due to network + problems. It can be overridden using the <option + linkend="opt-max-silent-time">--max-silent-time</option> command + line switch.</para> + + <para>The value <literal>0</literal> means that there is no + timeout. This is also the default.</para> + + </listitem> + + </varlistentry> + + + <varlistentry xml:id="conf-build-timeout"><term><literal>build-timeout</literal></term> + + <listitem> + + <para>This option defines the maximum number of seconds that a + builder can run. This is useful (for instance in an automated + build system) to catch builds that are stuck in an infinite loop + but keep writing to their standard output or standard error. It + can be overridden using the <option + linkend="opt-timeout">--timeout</option> command line + switch.</para> + + <para>The value <literal>0</literal> means that there is no + timeout. This is also the default.</para> + + </listitem> + + </varlistentry> + + + <varlistentry xml:id="conf-build-max-log-size"><term><literal>build-max-log-size</literal></term> + + <listitem> + + <para>This option defines the maximum number of bytes that a + builder can write to its stdout/stderr. If the builder exceeds + this limit, it’s killed. A value of <literal>0</literal> (the + default) means that there is no limit.</para> + + </listitem> + + </varlistentry> + + + <varlistentry xml:id="conf-build-users-group"><term><literal>build-users-group</literal></term> + + <listitem><para>This options specifies the Unix group containing + the Nix build user accounts. In multi-user Nix installations, + builds should not be performed by the Nix account since that would + allow users to arbitrarily modify the Nix store and database by + supplying specially crafted builders; and they cannot be performed + by the calling user since that would allow him/her to influence + the build result.</para> + + <para>Therefore, if this option is non-empty and specifies a valid + group, builds will be performed under the user accounts that are a + member of the group specified here (as listed in + <filename>/etc/group</filename>). Those user accounts should not + be used for any other purpose!</para> + + <para>Nix will never run two builds under the same user account at + the same time. This is to prevent an obvious security hole: a + malicious user writing a Nix expression that modifies the build + result of a legitimate Nix expression being built by another user. + Therefore it is good to have as many Nix build user accounts as + you can spare. (Remember: uids are cheap.)</para> + + <para>The build users should have permission to create files in + the Nix store, but not delete them. Therefore, + <filename>/nix/store</filename> should be owned by the Nix + account, its group should be the group specified here, and its + mode should be <literal>1775</literal>.</para> + + <para>If the build users group is empty, builds will be performed + under the uid of the Nix process (that is, the uid of the caller + if <envar>NIX_REMOTE</envar> is empty, the uid under which the Nix + daemon runs if <envar>NIX_REMOTE</envar> is + <literal>daemon</literal>). Obviously, this should not be used in + multi-user settings with untrusted users.</para> + + </listitem> + + </varlistentry> + + + <varlistentry><term><literal>build-use-chroot</literal></term> + + <listitem><para>If set to <literal>true</literal>, builds will be + performed in a <emphasis>chroot environment</emphasis>, i.e., the + build will be isolated from the normal file system hierarchy and + will only see the Nix store, the temporary build directory, and + the directories configured with the <link + linkend='conf-build-chroot-dirs'><literal>build-chroot-dirs</literal> + option</link> (such as <filename>/proc</filename> and + <filename>/dev</filename>). This is useful to prevent undeclared + dependencies on files in directories such as + <filename>/usr/bin</filename>.</para> + + <para>The use of a chroot requires that Nix is run as root (but + you can still use the <link + linkend='conf-build-users-group'>“build users” feature</link> to + perform builds under different users than root). Currently, + chroot builds only work on Linux because Nix uses “bind mounts” to + make the Nix store and other directories available inside the + chroot.</para> + + </listitem> + + </varlistentry> + + + <varlistentry xml:id="conf-build-chroot-dirs"><term><literal>build-chroot-dirs</literal></term> + + <listitem><para>When builds are performed in a chroot environment, + Nix will mount some directories from the normal file system + hierarchy inside the chroot. These are the Nix store, the + temporary build directory (usually + <filename>/tmp/nix-build-<replaceable>drvname</replaceable>-<replaceable>number</replaceable></filename>), + the <literal>/proc</literal> filesystem, and the directories + listed here. The default is <literal>/dev /dev/pts</literal>, + since these contain files needed by many builds (such as + <filename>/dev/null</filename>). You can use the syntax + <literal><replaceable>target</replaceable>=<replaceable>source</replaceable></literal> + to mount a path in a different location in the chroot; for + instance, <literal>/bin=/nix-bin</literal> will mount the + directory <literal>/nix-bin</literal> as <literal>/bin</literal> + inside the chroot.</para></listitem> + + </varlistentry> + + + <varlistentry><term><literal>build-use-substitutes</literal></term> + + <listitem><para>If set to <literal>true</literal> (default), Nix + will use binary substitutes if available. This option can be + disabled to force building from source.</para></listitem> + + </varlistentry> + + + <varlistentry><term><literal>build-fallback</literal></term> + + <listitem><para>If set to <literal>true</literal>, Nix will fall + back to building from source if a binary substitute fails. This + is equivalent to the <option>--fallback</option> flag. The + default is <literal>false</literal>.</para></listitem> + + </varlistentry> + + + <varlistentry><term><literal>build-cache-failures</literal></term> + + <listitem><para>If set to <literal>true</literal>, Nix will + “cache” build failures, meaning that it will remember (in its + database) that a derivation previously failed. If you then try to + build the derivation again, Nix will immediately fail rather than + perform the build again. Failures in fixed-output derivations + (such as <function>fetchurl</function> calls) are never cached. + The “failed” status of a derivation can be cleared using + <command>nix-store --clear-failed-paths</command>. By default, + failure caching is disabled.</para></listitem> + + </varlistentry> + + + <varlistentry><term><literal>build-keep-log</literal></term> + + <listitem><para>If set to <literal>true</literal> (the default), + Nix will write the build log of a derivation (i.e. the standard + output and error of its builder) to the directory + <filename>/nix/var/log/nix/drvs</filename>. The build log can be + retrieved using the command <command>nix-store -l + <replaceable>path</replaceable></command>.</para></listitem> + + </varlistentry> + + + <varlistentry><term><literal>build-compress-log</literal></term> + + <listitem><para>If set to <literal>true</literal> (the default), + build logs written to <filename>/nix/var/log/nix/drvs</filename> + will be compressed on the fly using bzip2. Otherwise, they will + not be compressed.</para></listitem> + + </varlistentry> + + + <varlistentry><term><literal>use-binary-caches</literal></term> + + <listitem><para>If set to <literal>true</literal> (the default), + Nix will check the binary caches specified by + <option>binary-caches</option> and related options to obtain + binary substitutes.</para></listitem> + + </varlistentry> + + + <varlistentry><term><literal>binary-caches</literal></term> + + <listitem><para>A list of URLs of binary caches, separated by + whitespace. The default is + <literal>http://cache.nixos.org</literal>.</para></listitem> + + </varlistentry> + + + <varlistentry><term><literal>binary-caches-files</literal></term> + + <listitem><para>A list of names of files that will be read to + obtain additional binary cache URLs. The default is + <literal>/nix/var/nix/profiles/per-user/<replaceable>username</replaceable>/channels/binary-caches/*</literal>. + Note that when you’re using the Nix daemon, + <replaceable>username</replaceable> is always equal to + <literal>root</literal>, so Nix will only use the binary caches + provided by the channels installed by root. Do not set this + option to read files created by untrusted users!</para></listitem> + + </varlistentry> + + + <varlistentry><term><literal>trusted-binary-caches</literal></term> + + <listitem><para>A list of URLs of binary caches, separated by + whitespace. These are not used by default, but can be enabled by + users of the Nix daemon by specifying <literal>--option + binary-caches <replaceable>urls</replaceable></literal> on the + command line. Unprivileged users are only allowed to pass a + subset of the URLs listed in <literal>binary-caches</literal> and + <literal>trusted-binary-caches</literal>.</para></listitem> + + </varlistentry> + + + <varlistentry><term><literal>extra-binary-caches</literal></term> + + <listitem><para>Additional binary caches appended to those + specified in <option>binary-caches</option> and + <option>binary-caches-files</option>. When used by unprivileged + users, untrusted binary caches (i.e. those not listed in + <option>trusted-binary-caches</option>) are silently + ignored.</para></listitem> + + </varlistentry> + + + <varlistentry><term><literal>binary-caches-parallel-connections</literal></term> + + <listitem><para>The maximum number of parallel HTTP connections + used by the binary cache substituter to get NAR info files. This + number should be high to minimise latency. It defaults to + 150.</para></listitem> + + </varlistentry> + + + <varlistentry><term><literal>force-manifest</literal></term> + + <listitem><para>If this option is set to <literal>false</literal> + (default) and a Nix channel provides both a manifest and a binary + cache, only the binary cache will be used. If set to + <literal>true</literal>, the manifest will be fetched as well. + This is useful if you want to use binary patches (which are + currently not supported by binary caches).</para></listitem> + + </varlistentry> + + + <varlistentry><term><literal>system</literal></term> + + <listitem><para>This option specifies the canonical Nix system + name of the current installation, such as + <literal>i686-linux</literal> or + <literal>powerpc-darwin</literal>. Nix can only build derivations + whose <literal>system</literal> attribute equals the value + specified here. In general, it never makes sense to modify this + value from its default, since you can use it to ‘lie’ about the + platform you are building on (e.g., perform a Mac OS build on a + Linux machine; the result would obviously be wrong). It only + makes sense if the Nix binaries can run on multiple platforms, + e.g., ‘universal binaries’ that run on <literal>powerpc-darwin</literal> and + <literal>i686-darwin</literal>.</para> + + <para>It defaults to the canonical Nix system name detected by + <filename>configure</filename> at build time.</para></listitem> + + </varlistentry> + + + <varlistentry><term><literal>fsync-metadata</literal></term> + + <listitem><para>If set to <literal>true</literal>, changes to the + Nix store metadata (in <filename>/nix/var/nix/db</filename>) are + synchronously flushed to disk. This improves robustness in case + of system crashes, but reduces performance. The default is + <literal>true</literal>.</para></listitem> + + </varlistentry> + + + <varlistentry><term><literal>auto-optimise-store</literal></term> + + <listitem><para>If set to <literal>true</literal>, Nix + automatically detects files in the store that have identical + contents, and replaces them with hard links to a single copy. + This saves disk space. If set to <literal>false</literal> (the + default), you can still run <command>nix-store + --optimise</command> to get rid of duplicate + files.</para></listitem> + + </varlistentry> + + +</variablelist> + +</para> + +</refsection> + +</refentry> diff --git a/doc/manual/env-common.xml b/doc/manual/env-common.xml new file mode 100644 index 000000000000..9e97e58f5784 --- /dev/null +++ b/doc/manual/env-common.xml @@ -0,0 +1,347 @@ +<section xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xml:id="sec-common-env"> + +<title>Common environment variables</title> + + +<para>Most Nix commands interpret the following environment variables:</para> + +<variablelist xml:id="env-common"> + + +<varlistentry><term><envar>NIX_PATH</envar></term> + + <listitem> + + <para>A colon-separated list of directories used to look up Nix + expressions enclosed in angle brackets (i.e., + <literal><<replaceable>path</replaceable>></literal>). For + instance, the value + + <screen> +/home/eelco/Dev:/etc/nixos</screen> + + will cause Nix to look for paths relative to + <filename>/home/eelco/Dev</filename> and + <filename>/etc/nixos</filename>, in that order. It is also + possible to match paths against a prefix. For example, the value + + <screen> +nixpkgs=/home/eelco/Dev/nixpkgs-branch:/etc/nixos</screen> + + will cause Nix to search for + <literal><nixpkgs/<replaceable>path</replaceable>></literal> in + <filename>/home/eelco/Dev/nixpkgs-branch/<replaceable>path</replaceable></filename> + and + <filename>/etc/nixos/nixpkgs/<replaceable>path</replaceable></filename>. + </para> + + <para>The search path can be extended using the + <option>-I</option> option, which takes precedence over + <envar>NIX_PATH</envar>.</para></listitem> + +</varlistentry> + + +<varlistentry><term><envar>NIX_IGNORE_SYMLINK_STORE</envar></term> + + <listitem> + + <para>Normally, the Nix store directory (typically + <filename>/nix/store</filename>) is not allowed to contain any + symlink components. This is to prevent “impure” builds. Builders + sometimes “canonicalise” paths by resolving all symlink components. + Thus, builds on different machines (with + <filename>/nix/store</filename> resolving to different locations) + could yield different results. This is generally not a problem, + except when builds are deployed to machines where + <filename>/nix/store</filename> resolves differently. If you are + sure that you’re not going to do that, you can set + <envar>NIX_IGNORE_SYMLINK_STORE</envar> to <envar>1</envar>.</para> + + <para>Note that if you’re symlinking the Nix store so that you can + put it on another file system than the root file system, on Linux + you’re better off using <literal>bind</literal> mount points, e.g., + + <screen> +$ mkdir /nix +$ mount -o bind /mnt/otherdisk/nix /nix</screen> + + Consult the <citerefentry><refentrytitle>mount</refentrytitle> + <manvolnum>8</manvolnum></citerefentry> manual page for details.</para> + + </listitem> + +</varlistentry> + + +<varlistentry><term><envar>NIX_STORE_DIR</envar></term> + + <listitem><para>Overrides the location of the Nix store (default + <filename><replaceable>prefix</replaceable>/store</filename>).</para></listitem> + +</varlistentry> + + +<varlistentry><term><envar>NIX_DATA_DIR</envar></term> + + <listitem><para>Overrides the location of the Nix static data + directory (default + <filename><replaceable>prefix</replaceable>/share</filename>).</para></listitem> + +</varlistentry> + + +<varlistentry><term><envar>NIX_LOG_DIR</envar></term> + + <listitem><para>Overrides the location of the Nix log directory + (default <filename><replaceable>prefix</replaceable>/log/nix</filename>).</para></listitem> + +</varlistentry> + + +<varlistentry><term><envar>NIX_STATE_DIR</envar></term> + + <listitem><para>Overrides the location of the Nix state directory + (default <filename><replaceable>prefix</replaceable>/var/nix</filename>).</para></listitem> + +</varlistentry> + + +<varlistentry><term><envar>NIX_DB_DIR</envar></term> + + <listitem><para>Overrides the location of the Nix database (default + <filename><replaceable>$NIX_STATE_DIR</replaceable>/db</filename>, i.e., + <filename><replaceable>prefix</replaceable>/var/nix/db</filename>).</para></listitem> + +</varlistentry> + + +<varlistentry><term><envar>NIX_CONF_DIR</envar></term> + + <listitem><para>Overrides the location of the Nix configuration + directory (default + <filename><replaceable>prefix</replaceable>/etc/nix</filename>).</para></listitem> + +</varlistentry> + + +<varlistentry><term><envar>NIX_LOG_TYPE</envar></term> + + <listitem><para>Equivalent to the <link + linkend="opt-log-type"><option>--log-type</option> + option</link>.</para></listitem> + +</varlistentry> + + +<varlistentry><term><envar>TMPDIR</envar></term> + + <listitem><para>Use the specified directory to store temporary + files. In particular, this includes temporary build directories; + these can take up substantial amounts of disk space. The default is + <filename>/tmp</filename>.</para></listitem> + +</varlistentry> + + +<varlistentry xml:id="envar-build-hook"><term><envar>NIX_BUILD_HOOK</envar></term> + + <listitem> + + <para>Specifies the location of the <emphasis>build hook</emphasis>, + which is a program (typically some script) that Nix will call + whenever it wants to build a derivation. This is used to implement + distributed builds<phrase condition="manual"> (see <xref + linkend="chap-distributed-builds" />)</phrase>.</para> + + <!-- + The protocol by + which the calling Nix process and the build hook communicate is as + follows. + + <para>The build hook is called with the following command-line + arguments: + + <orderedlist> + + <listitem><para>A boolean value <literal>0</literal> or + <literal>1</literal> specifying whether Nix can locally execute + more builds, as per the <link + linkend="opt-max-jobs"><option>- -max-jobs</option> option</link>. + The purpose of this argument is to allow the hook to not have to + maintain bookkeeping for the local machine.</para></listitem> + + <listitem><para>The Nix platform identifier for the local machine + (e.g., <literal>i686-linux</literal>).</para></listitem> + + <listitem><para>The Nix platform identifier for the derivation, + i.e., its <link linkend="attr-system"><varname>system</varname> + attribute</link>.</para></listitem> + + <listitem><para>The store path of the derivation.</para></listitem> + + </orderedlist> + + </para> + + <para>On the basis of this information, and whatever persistent + state the build hook keeps about other machines and their current + load, it has to decide what to do with the build. It should print + out on standard error one of the following responses (terminated by + a newline, <literal>"\n"</literal>): + + <variablelist> + + <varlistentry><term><literal># decline</literal></term> + + <listitem><para>The build hook is not willing or able to perform + the build; the calling Nix process should do the build itself, + if possible.</para></listitem> + + </varlistentry> + + <varlistentry><term><literal># postpone</literal></term> + + <listitem><para>The build hook cannot perform the build now, but + can do so in the future (e.g., because all available build slots + on remote machines are in use). The calling Nix process should + postpone this build until at least one currently running build + has terminated.</para></listitem> + + </varlistentry> + + <varlistentry><term><literal># accept</literal></term> + + <listitem><para>The build hook has accepted the + build.</para></listitem> + + </varlistentry> + + </variablelist> + + </para> + + <para>After sending <literal># accept</literal>, the hook should + read one line from standard input, which will be the string + <literal>okay</literal>. It can then proceed with the build. + Before sending <literal>okay</literal>, Nix will store in the hook’s + current directory a number of text files that contain information + about the derivation: + + <variablelist> + + <varlistentry><term><filename>inputs</filename></term> + + <listitem><para>The set of store paths that are inputs to the + build process (one per line). These have to be copied + <emphasis>to</emphasis> the remote machine (in addition to the + store derivation itself).</para></listitem> + + </varlistentry> + + <varlistentry><term><filename>outputs</filename></term> + + <listitem><para>The set of store paths that are outputs of the + derivation (one per line). These have to be copied + <emphasis>from</emphasis> the remote machine if the build + succeeds.</para></listitem> + + </varlistentry> + + <varlistentry><term><filename>references</filename></term> + + <listitem><para>The reference graph of the inputs, in the format + accepted by the command <command>nix-store + - -register-validity</command>. It is necessary to run this + command on the remote machine after copying the inputs to inform + Nix on the remote machine that the inputs are valid + paths.</para></listitem> + + </varlistentry> + + </variablelist> + + </para> + + <para>The hook should copy the inputs to the remote machine, + register the validity of the inputs, perform the remote build, and + copy the outputs back to the local machine. An exit code other than + <literal>0</literal> indicates that the hook has failed. An exit + code equal to 100 means that the remote build failed (as opposed to, + e.g., a network error).</para> + --> + + </listitem> + + +</varlistentry> + + +<varlistentry xml:id="envar-remote"><term><envar>NIX_REMOTE</envar></term> + + <listitem><para>This variable should be set to + <literal>daemon</literal> if you want to use the Nix daemon to + execute Nix operations. This is necessary in <link + linkend="ssec-multi-user">multi-user Nix installations</link>. + Otherwise, it should be left unset.</para></listitem> + +</varlistentry> + + +<varlistentry xml:id="envar-other-stores"><term><envar>NIX_OTHER_STORES</envar></term> + + <listitem><para>This variable contains the paths of remote Nix + installations from which packages can be copied, separated by colons. + <phrase condition="manual">See <xref linkend="sec-sharing-packages" + /> for details.</phrase> Each path should be the + <filename>/nix</filename> directory of a remote Nix installation + (i.e., not the <filename>/nix/store</filename> directory). The + paths are subject to globbing, so you can set it so something like + <literal>/var/run/nix/remote-stores/*/nix</literal> and mount + multiple remote filesystems in + <literal>/var/run/nix/remote-stores</literal>.</para> + + <para>Note that if you’re building through the <link + linkend="sec-nix-daemon">Nix daemon</link>, the only setting for + this variable that matters is the one that the + <command>nix-daemon</command> process uses. So if you want to + change it, you have to restart the daemon.</para></listitem> + +</varlistentry> + + +<varlistentry><term><envar>NIX_SHOW_STATS</envar></term> + + <listitem><para>If set to <literal>1</literal>, Nix will print some + evaluation statistics, such as the number of values + allocated.</para></listitem> + +</varlistentry> + + +<varlistentry><term><envar>NIX_COUNT_CALLS</envar></term> + + <listitem><para>If set to <literal>1</literal>, Nix will print how + often functions were called during Nix expression evaluation. This + is useful for profiling your Nix expressions.</para></listitem> + +</varlistentry> + + +<varlistentry><term><envar>GC_INITIAL_HEAP_SIZE</envar></term> + + <listitem><para>If Nix has been configured to use the Boehm garbage + collector, this variable sets the initial size of the heap in bytes. + It defaults to 384 MiB. Setting it to a low value reduces memory + consumption, but will increase runtime due to the overhead of + garbage collection.</para></listitem> + +</varlistentry> + + +</variablelist> + + +</section> diff --git a/doc/manual/figures/user-environments.png b/doc/manual/figures/user-environments.png new file mode 100644 index 000000000000..1f781cf23ce5 --- /dev/null +++ b/doc/manual/figures/user-environments.png Binary files differdiff --git a/doc/manual/figures/user-environments.sxd b/doc/manual/figures/user-environments.sxd new file mode 100644 index 000000000000..bc661b6406fb --- /dev/null +++ b/doc/manual/figures/user-environments.sxd Binary files differdiff --git a/doc/manual/glossary.xml b/doc/manual/glossary.xml new file mode 100644 index 000000000000..d74940c90b30 --- /dev/null +++ b/doc/manual/glossary.xml @@ -0,0 +1,179 @@ +<appendix xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink"> + +<title>Glossary</title> + + +<glosslist> + + +<glossentry xml:id="gloss-derivation"><glossterm>derivation</glossterm> + + <glossdef><para>A description of a build action. The result of a + derivation is a store object. Derivations are typically specified + in Nix expressions using the <link + linkend="ssec-derivation"><function>derivation</function> + primitive</link>. These are translated into low-level + <emphasis>store derivations</emphasis> (implicitly by + <command>nix-env</command> and <command>nix-build</command>, or + explicitly by <command>nix-instantiate</command>).</para></glossdef> + +</glossentry> + + +<glossentry><glossterm>store</glossterm> + + <glossdef><para>The location in the file system where store objects + live. Typically <filename>/nix/store</filename>.</para></glossdef> + +</glossentry> + + +<glossentry><glossterm>store path</glossterm> + + <glossdef><para>The location in the file system of a store object, + i.e., an immediate child of the Nix store + directory.</para></glossdef> + +</glossentry> + + +<glossentry><glossterm>store object</glossterm> + + <glossdef><para>A file that is an immediate child of the Nix store + directory. These can be regular files, but also entire directory + trees. Store objects can be sources (objects copied from outside of + the store), derivation outputs (objects produced by running a build + action), or derivations (files describing a build + action).</para></glossdef> + +</glossentry> + + +<glossentry xml:id="gloss-substitute"><glossterm>substitute</glossterm> + + <glossdef><para>A substitute is a command invocation stored in the + Nix database that describes how to build a store object, bypassing + normal the build mechanism (i.e., derivations). Typically, the + substitute builds the store object by downloading a pre-built + version of the store object from some server.</para></glossdef> + +</glossentry> + + +<glossentry><glossterm>purity</glossterm> + + <glossdef><para>The assumption that equal Nix derivations when run + always produce the same output. This cannot be guaranteed in + general (e.g., a builder can rely on external inputs such as the + network or the system time) but the Nix model assumes + it.</para></glossdef> + +</glossentry> + + +<glossentry><glossterm>Nix expression</glossterm> + + <glossdef><para>A high-level description of software packages and + compositions thereof. Deploying software using Nix entails writing + Nix expressions for your packages. Nix expressions are translated + to derivations that are stored in the Nix store. These derivations + can then be built.</para></glossdef> + +</glossentry> + + +<glossentry xml:id="gloss-reference"><glossterm>reference</glossterm> + + <glossdef><para>A store path <varname>P</varname> is said to have a + reference to a store path <varname>Q</varname> if the store object + at <varname>P</varname> contains the path <varname>Q</varname> + somewhere. This implies than an execution involving + <varname>P</varname> potentially needs <varname>Q</varname> to be + present. The <emphasis>references</emphasis> of a store path are + the set of store paths to which it has a reference.</para></glossdef> + +</glossentry> + + +<glossentry xml:id="gloss-closure"><glossterm>closure</glossterm> + + <glossdef><para>The closure of a store path is the set of store + paths that are directly or indirectly “reachable” from that store + path; that is, it’s the closure of the path under the <link + linkend="gloss-reference">references</link> relation. For instance, + if the store object at path <varname>P</varname> contains a + reference to path <varname>Q</varname>, then <varname>Q</varname> is + in the closure of <varname>P</varname>. For correct deployment it + is necessary to deploy whole closures, since otherwise at runtime + files could be missing. The command <command>nix-store + -qR</command> prints out closures of store paths.</para></glossdef> + +</glossentry> + + +<glossentry xml:id="gloss-output-path"><glossterm>output path</glossterm> + + <glossdef><para>A store path produced by a derivation.</para></glossdef> + +</glossentry> + + +<glossentry xml:id="gloss-deriver"><glossterm>deriver</glossterm> + + <glossdef><para>The deriver of an <link + linkend="gloss-output-path">output path</link> is the store + derivation that built it.</para></glossdef> + +</glossentry> + + +<glossentry xml:id="gloss-validity"><glossterm>validity</glossterm> + + <glossdef><para>A store path is considered + <emphasis>valid</emphasis> if it exists in the file system, is + listed in the Nix database as being valid, and if all paths in its + closure are also valid.</para></glossdef> + +</glossentry> + + +<glossentry xml:id="gloss-user-env"><glossterm>user environment</glossterm> + + <glossdef><para>An automatically generated store object that + consists of a set of symlinks to “active” applications, i.e., other + store paths. These are generated automatically by <link + linkend="sec-nix-env"><command>nix-env</command></link>. See <xref + linkend="sec-profiles" />.</para> + + </glossdef> + +</glossentry> + + +<glossentry xml:id="gloss-profile"><glossterm>profile</glossterm> + + <glossdef><para>A symlink to the current <link + linkend="gloss-user-env">user environment</link> of a user, e.g., + <filename>/nix/var/nix/profiles/default</filename>.</para></glossdef> + +</glossentry> + + +<glossentry xml:id="gloss-nar"><glossterm>NAR</glossterm> + + <glossdef><para>A <emphasis>N</emphasis>ix + <emphasis>AR</emphasis>chive. This is a serialisation of a path in + the Nix store. It can contain regular files, directories and + symbolic links. NARs are generated and unpacked using + <command>nix-store --dump</command> and <command>nix-store + --restore</command>.</para></glossdef> + +</glossentry> + + + +</glosslist> + + +</appendix> diff --git a/doc/manual/hacking.xml b/doc/manual/hacking.xml new file mode 100644 index 000000000000..f8478c9093e6 --- /dev/null +++ b/doc/manual/hacking.xml @@ -0,0 +1,41 @@ +<appendix xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xml:id="chap-hacking"> + +<title>Hacking</title> + +<para>This section provides some notes on how to hack on Nix. To get +the latest version of Nix from GitHub: +<screen> +$ git clone git://github.com/NixOS/nix.git +$ cd hydra +</screen> +</para> + +<para>To build it and its dependencies: +<screen> +$ nix-build release.nix -A build.x86_64-linux +</screen> +</para> + +<para>To build all dependencies and start a shell in which all +environment variables are set up so that those dependencies can be +found: +<screen> +$ ./dev-shell +</screen> +To build Nix itself in this shell: +<screen> +[nix-shell]$ ./bootstrap +[nix-shell]$ configurePhase +[nix-shell]$ make +</screen> +To test it: +<screen> +[nix-shell]$ make install +[nix-shell]$ make installcheck +</screen> + +</para> + +</appendix> diff --git a/doc/manual/installation.xml b/doc/manual/installation.xml new file mode 100644 index 000000000000..a136d3b1129e --- /dev/null +++ b/doc/manual/installation.xml @@ -0,0 +1,475 @@ +<?xml version="1.0" encoding="utf-8"?> +<chapter xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xml:id="chap-installation"> + +<title>Installation</title> + + +<section><title>Supported platforms</title> + +<para>Nix is currently supported on the following platforms: + +<itemizedlist> + + <listitem><para>Linux (particularly on x86, x86_64, and + PowerPC).</para></listitem> + + <listitem><para>Mac OS X.</para></listitem> + + <listitem><para>FreeBSD (only tested on Intel).</para></listitem> + + <!-- + <listitem><para>Windows through <link + xlink:href="http://www.cygwin.com/">Cygwin</link>.</para> + + <warning><para>On Cygwin, Nix <emphasis>must</emphasis> be installed + on an NTFS partition. It will not work correctly on a FAT + partition.</para></warning> + + </listitem> + --> + +</itemizedlist> + +</para> + +<para>Nix is pretty portable, so it should work on most other Unix +platforms as well.</para> + +</section> + + +<section><title>Installing a binary distribution</title> + +<para>The easiest way to install Nix is to use a binary package. +Binary packages of the latest stable release are available for Fedora, +Debian, Ubuntu, Mac OS X and various other systems from the <link +xlink:href="http://nixos.org/nix/download.html">Nix homepage</link>. +You can also get builds of the latest development release from our +<link +xlink:href="http://hydra.nixos.org/job/nix/trunk/release/latest-finished#tabs-constituents">continuous +build system</link>.</para> + +<para>For Fedora, RPM packages are available. These can be installed +or upgraded using <command>rpm -U</command>. For example, + +<screen> +$ rpm -U nix-1.0-1.i386.rpm</screen> + +</para> + +<para>For Debian and Ubuntu, you can download a Deb package and +install it like this: + +<screen> +$ dpkg -i nix_1.0-1_amd64.deb</screen> + +</para> + +<para>For other platforms, including Mac OS X (Darwin), FreeBSD and +other Linux distributions, you can download a binary tarball. It +contains Nix and all its dependencies. You should unpack it in the +root directory, then run <command>nix-finish-install</command>: + +<screen> +$ cd / +$ tar xfj nix-1.1-x86_64-darwin.tar.bz2 +$ nix-finish-install +</screen> + +After this you can delete +<filename>/usr/bin/nix-finish-install</filename>.</para> + +<para>If you plan to use Nix from a single non-root user account, it’s +probably convenient to change the ownership of the entire Nix store +and database to that user account. In that case, install as follows: + +<screen> +alice$ cd / +alice$ sudo tar xfj nix-1.1-x86_64-darwin.tar.bz2 +alice$ sudo chown -R alice /nix +alice$ nix-finish-install +</screen> + +</para> + +<para>Nix can be uninstalled using <command>rpm -e nix</command> or +<command>dpkg -r nix</command> on RPM- and Dpkg-based systems, +respectively. After this you should manually remove the Nix store and +other auxiliary data, if desired: + +<screen> +$ rm -rf /nix</screen> + +</para> + +</section> + + +<section><title>Installing Nix from source</title> + +<para>If no binary package is available, you can download and compile +a source distribution.</para> + +<section><title>Prerequisites</title> + +<itemizedlist> + + <listitem><para>GNU Make.</para></listitem> + + <listitem><para>A fairly recent version of GCC/G++. Version 2.95 + and higher should work. Clang will also work.</para></listitem> + + <listitem><para>Perl 5.8 or higher.</para></listitem> + + <listitem><para><command>pkg-config</command> to locate + dependencies. If your distribution does not provide it, you can get + it from <link + xlink:href="http://www.freedesktop.org/wiki/Software/pkg-config" + />.</para></listitem> + + <listitem><para>The bzip2 compressor program and the + <literal>libbz2</literal> library. Thus you must have bzip2 + installed, including development headers and libraries. If your + distribution does not provide these, you can obtain bzip2 from <link + xlink:href="http://www.bzip.org/"/>.</para></listitem> + + <listitem><para>The SQLite embedded database library, version 3.6.19 + or higher. If your distribution does not provide it, please install + it from <link xlink:href="http://www.sqlite.org/" />.</para></listitem> + + <listitem><para>The Perl DBI and DBD::SQLite libraries, which are + available from <link + xlink:href="http://search.cpan.org/">CPAN</link> if your + distribution does not provide them.</para></listitem> + + <listitem><para>The <link + xlink:href="http://www.hpl.hp.com/personal/Hans_Boehm/gc/">Boehm + garbage collector</link> to reduce the evaluator’s memory + consumption (optional). To enable it, install + <literal>pkgconfig</literal> and the Boehm garbage collector, and + pass the flag <option>--enable-gc</option> to + <command>configure</command>.</para></listitem> + + <listitem><para>The <command>xmllint</command> and + <command>xsltproc</command> programs to build this manual and the + man-pages. These are part of the <literal>libxml2</literal> and + <literal>libxslt</literal> packages, respectively. You also need + the <link + xlink:href="http://docbook.sourceforge.net/projects/xsl/">DocBook + XSL stylesheets</link> and optionally the <link + xlink:href="http://www.docbook.org/schemas/5x"> DocBook 5.0 RELAX NG + schemas</link>. Note that these are only required if you modify the + manual sources or when you are building from the Git + repository.</para></listitem> + + <listitem><para>Recent versions of Bison and Flex to build the + parser. (This is because Nix needs GLR support in Bison and + reentrancy support in Flex.) For Bison, you need version 2.6, which + can be obtained from the <link + xlink:href="ftp://alpha.gnu.org/pub/gnu/bison">GNU FTP + server</link>. For Flex, you need version 2.5.35, which is + available on <link + xlink:href="http://lex.sourceforge.net/">SourceForge</link>. + Slightly older versions may also work, but ancient versions like the + ubiquitous 2.5.4a won't. Note that these are only required if you + modify the parser or when you are building from the Git + repository.</para></listitem> + +</itemizedlist> + +</section> + + +<section><title>Obtaining a source distribution</title> + +<para>The source tarball of the most recent stable release can be +downloaded from the <link +xlink:href="http://nixos.org/nix/download.html">Nix homepage</link>. +You can also grab the <link +xlink:href="http://hydra.nixos.org/job/nix/trunk/release/latest-finished#tabs-constituents">most +recent development release</link>.</para> + +<para>Alternatively, the most recent sources of Nix can be obtained +from its <link +xlink:href="https://github.com/NixOS/nix">Git +repository</link>. For example, the following command will check out +the latest revision into a directory called +<filename>nix</filename>:</para> + +<screen> +$ git clone https://github.com/NixOS/nix</screen> + +<para>Likewise, specific releases can be obtained from the <link +xlink:href="https://github.com/NixOS/nix/tags">tags</link> of the +repository.</para> + +</section> + + +<section><title>Building Nix from source</title> + +<para>After unpacking or checking out the Nix sources, issue the +following commands: + +<screen> +$ ./configure <replaceable>options...</replaceable> +$ make +$ make install</screen> + +Nix requires GNU Make so you may need to invoke +<command>gmake</command> instead.</para> + +<para>When building from the Git repository, these should be preceded +by the command: + +<screen> +$ ./bootstrap.sh</screen> + +</para> + +<para>The installation path can be specified by passing the +<option>--prefix=<replaceable>prefix</replaceable></option> to +<command>configure</command>. The default installation directory is +<filename>/usr/local</filename>. You can change this to any location +you like. You must have write permission to the +<replaceable>prefix</replaceable> path.</para> + +<para>Nix keeps its <emphasis>store</emphasis> (the place where +packages are stored) in <filename>/nix/store</filename> by default. +This can be changed using +<option>--with-store-dir=<replaceable>path</replaceable></option>.</para> + +<warning><para>It is best <emphasis>not</emphasis> to change the Nix +store from its default, since doing so makes it impossible to use +pre-built binaries from the standard Nixpkgs channels — that is, all +packages will need to be built from source.</para></warning> + +<para>Nix keeps state (such as its database and log files) in +<filename>/nix/var</filename> by default. This can be changed using +<option>--localstatedir=<replaceable>path</replaceable></option>.</para> + +<para>If you want to rebuild the documentation, pass the full path to +the DocBook RELAX NG schemas and to the DocBook XSL stylesheets using +the +<option>--with-docbook-rng=<replaceable>path</replaceable></option> +and +<option>--with-docbook-xsl=<replaceable>path</replaceable></option> +options.</para> + +</section> + + +</section> + + +<!-- TODO: should be updated +<section><title>Upgrading Nix through Nix</title> + +<para>You can install the latest stable version of Nix through Nix +itself by subscribing to the channel <link +xlink:href="http://nixos.org/releases/nix/channels/nix-stable" />, +or the latest unstable version by subscribing to the channel <link +xlink:href="http://nixos.org/releases/nix/channels/nix-unstable" />. +You can also do a <link linkend="sec-one-click">one-click +installation</link> by clicking on the package links at <link +xlink:href="http://nixos.org/releases/full-index-nix.html" />.</para> + +</section> +--> + + +<section><title>Security</title> + +<para>Nix has two basic security models. First, it can be used in +“single-user mode”, which is similar to what most other package +management tools do: there is a single user (typically <systemitem +class="username">root</systemitem>) who performs all package +management operations. All other users can then use the installed +packages, but they cannot perform package management operations +themselves.</para> + +<para>Alternatively, you can configure Nix in “multi-user mode”. In +this model, all users can perform package management operations — for +instance, every user can install software without requiring root +privileges. Nix ensures that this is secure. For instance, it’s not +possible for one user to overwrite a package used by another user with +a Trojan horse.</para> + + +<section><title>Single-user mode</title> + +<para>In single-user mode, all Nix operations that access the database +in <filename><replaceable>prefix</replaceable>/var/nix/db</filename> +or modify the Nix store in +<filename><replaceable>prefix</replaceable>/store</filename> must be +performed under the user ID that owns those directories. This is +typically <systemitem class="username">root</systemitem>. (If you +install from RPM packages, that’s in fact the default ownership.) +However, on single-user machines, it is often convenient to +<command>chown</command> those directories to your normal user account +so that you don’t have to <command>su</command> to <systemitem +class="username">root</systemitem> all the time.</para> + +</section> + + +<section xml:id="ssec-multi-user"><title>Multi-user mode</title> + +<para>To allow a Nix store to be shared safely among multiple users, +it is important that users are not able to run builders that modify +the Nix store or database in arbitrary ways, or that interfere with +builds started by other users. If they could do so, they could +install a Trojan horse in some package and compromise the accounts of +other users.</para> + +<para>To prevent this, the Nix store and database are owned by some +privileged user (usually <literal>root</literal>) and builders are +executed under special user accounts (usually named +<literal>nixbld1</literal>, <literal>nixbld2</literal>, etc.). When a +unprivileged user runs a Nix command, actions that operate on the Nix +store (such as builds) are forwarded to a <emphasis>Nix +daemon</emphasis> running under the owner of the Nix store/database +that performs the operation.</para> + +<note><para>Multi-user mode has one important limitation: only +<systemitem class="username">root</systemitem> can run <command +linkend="sec-nix-pull">nix-pull</command> to register the availability +of pre-built binaries. However, those registrations are shared by all +users, so they still get the benefit from <command>nix-pull</command>s +done by <systemitem class="username">root</systemitem>.</para></note> + + +<section><title>Setting up the build users</title> + +<para>The <emphasis>build users</emphasis> are the special UIDs under +which builds are performed. They should all be members of the +<emphasis>build users group</emphasis> (usually called +<literal>nixbld</literal>). This group should have no other members. +The build users should not be members of any other group.</para> + +<para>Here is a typical <filename>/etc/group</filename> definition of +the build users group with 10 build users: + +<programlisting> +nixbld:!:30000:nixbld1,nixbld2,nixbld3,nixbld4,nixbld5,nixbld6,nixbld7,nixbld8,nixbld9,nixbld10 +</programlisting> + +In this example the <literal>nixbld</literal> group has UID 30000, but +of course it can be anything that doesn’t collide with an existing +group.</para> + +<para>Here is the corresponding part of +<filename>/etc/passwd</filename>: + +<programlisting> +nixbld1:x:30001:65534:Nix build user 1:/var/empty:/noshell +nixbld2:x:30002:65534:Nix build user 2:/var/empty:/noshell +nixbld3:x:30003:65534:Nix build user 3:/var/empty:/noshell +... +nixbld10:x:30010:65534:Nix build user 10:/var/empty:/noshell +</programlisting> + +The home directory of the build users should not exist or should be an +empty directory to which they do not have write access.</para> + +<para>The build users should have write access to the Nix store, but +they should not have the right to delete files. Thus the Nix store’s +group should be the build users group, and it should have the sticky +bit turned on (like <filename>/tmp</filename>): + +<screen> +$ chown root.nixbld /nix/store +$ chmod 1775 /nix/store +</screen> + +</para> + +<para>Finally, you should tell Nix to use the build users by +specifying the build users group in the <link +linkend="conf-build-users-group"><literal>build-users-group</literal> +option</link> in the <link linkend="sec-conf-file">Nix configuration +file</link> (usually <literal>/etc/nix/nix.conf</literal>): + +<programlisting> +build-users-group = nixbld +</programlisting> + +</para> + +</section> + + +<section><title>Running the daemon</title> + +<para>The <link linkend="sec-nix-daemon">Nix daemon</link> should be +started as follows (as <literal>root</literal>): + +<screen> +$ nix-daemon</screen> + +You’ll want to put that line somewhere in your system’s boot +scripts.</para> + +<para>To let unprivileged users use the daemon, they should set the +<link linkend="envar-remote"><envar>NIX_REMOTE</envar> environment +variable</link> to <literal>daemon</literal>. So you should put a +line like + +<programlisting> +export NIX_REMOTE=daemon</programlisting> + +into the users’ login scripts.</para> + +</section> + + +<section><title>Restricting access</title> + +<para>To limit which users can perform Nix operations, you can use the +permissions on the directory +<filename>/nix/var/nix/daemon-socket</filename>. For instance, if you +want to restrict the use of Nix to the members of a group called +<literal>nix-users</literal>, do + +<screen> +$ chgrp nix-users /nix/var/nix/daemon-socket +$ chmod ug=rwx,o= /nix/var/nix/daemon-socket +</screen> + +This way, users who are not in the <literal>nix-users</literal> group +cannot connect to the Unix domain socket +<filename>/nix/var/nix/daemon-socket/socket</filename>, so they cannot +perform Nix operations.</para> + +</section> + + +</section> <!-- end of multi-user --> + + +</section> <!-- end of security --> + + +<section><title>Using Nix</title> + +<para>To use Nix, some environment variables should be set. In +particular, <envar>PATH</envar> should contain the directories +<filename><replaceable>prefix</replaceable>/bin</filename> and +<filename>~/.nix-profile/bin</filename>. The first directory contains +the Nix tools themselves, while <filename>~/.nix-profile</filename> is +a symbolic link to the current <emphasis>user environment</emphasis> +(an automatically generated package consisting of symlinks to +installed packages). The simplest way to set the required environment +variables is to include the file +<filename><replaceable>prefix</replaceable>/etc/profile.d/nix.sh</filename> +in your <filename>~/.profile</filename> (or similar), like this:</para> + +<screen> +source <replaceable>prefix</replaceable>/etc/profile.d/nix.sh</screen> + +</section> + + +</chapter> diff --git a/doc/manual/introduction.xml b/doc/manual/introduction.xml new file mode 100644 index 000000000000..21b1df1564e5 --- /dev/null +++ b/doc/manual/introduction.xml @@ -0,0 +1,337 @@ +<chapter xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xml:id="chap-introduction"> + +<title>Introduction</title> + + +<section><title>About Nix</title> + +<para>Nix is a <emphasis>purely functional package manager</emphasis>. +This means that it treats packages like values in purely functional +programming languages such as Haskell — they are built by functions +that don’t have side-effects, and they never change after they have +been built. Nix stores packages in the <emphasis>Nix +store</emphasis>, usually the directory +<filename>/nix/store</filename>, where each package has its own unique +subdirectory such as + +<programlisting> +/nix/store/nlc4z5y1hm8w9s8vm6m1f5hy962xjmp5-firefox-12.0 +</programlisting> + +where <literal>nlc4z5…</literal> is a unique identifier for the +package that captures all its dependencies (it’s a cryptographic hash +of the package’s build dependency graph). This enables many powerful +features.</para> + + +<simplesect><title>Multiple versions</title> + +<para>You can have multiple versions or variants of a package +installed at the same time. This is especially important when +different applications have dependencies on different versions of the +same package — it prevents the “DLL hell”. Because of the hashing +scheme, different versions of a package end up in different paths in +the Nix store, so they don’t interfere with each other.</para> + +<para>An important consequence is that operations like upgrading or +uninstalling an application cannot break other applications, since +these operations never “destructively” update or delete files that are +used by other packages.</para> + +</simplesect> + + +<simplesect><title>Complete dependencies</title> + +<para>Nix helps you make sure that package dependency specifications +are complete. In general, when you’re making a package for a package +management system like RPM, you have to specify for each package what +its dependencies are, but there are no guarantees that this +specification is complete. If you forget a dependency, then the +package will build and work correctly on <emphasis>your</emphasis> +machine if you have the dependency installed, but not on the end +user's machine if it's not there.</para> + +<para>Since Nix on the other hand doesn’t install packages in “global” +locations like <filename>/usr/bin</filename> but in package-specific +directories, the risk of incomplete dependencies is greatly reduced. +This is because tools such as compilers don’t search in per-packages +directories such as +<filename>/nix/store/5lbfaxb722zp…-openssl-0.9.8d/include</filename>, +so if a package builds correctly on your system, this is because you +specified the dependency explicitly.</para> + +<para>Runtime dependencies are found by scanning binaries for the hash +parts of Nix store paths (such as <literal>r8vvq9kq…</literal>). This +sounds risky, but it works extremely well.</para> + +</simplesect> + + +<simplesect><title>Multi-user support</title> + +<para>Nix has multi-user support. This means that non-privileged +users can securely install software. Each user can have a different +<emphasis>profile</emphasis>, a set of packages in the Nix store that +appear in the user’s <envar>PATH</envar>. If a user installs a +package that another user has already installed previously, the +package won’t be built or downloaded a second time. At the same time, +it is not possible for one user to inject a Trojan horse into a +package that might be used by another user.</para> + +<!-- +<para>More details can be found in Section 3 of our <a +href="docs/papers.html#securesharing">ASE 2005 paper</a>.</para> +--> + +</simplesect> + + +<simplesect><title>Atomic upgrades and rollbacks</title> + +<para>Since package management operations never overwrite packages in +the Nix store but just add new versions in different paths, they are +<emphasis>atomic</emphasis>. So during a package upgrade, there is no +time window in which the package has some files from the old version +and some files from the new version — which would be bad because a +program might well crash if it’s started during that period.</para> + +<para>And since package aren’t overwritten, the old versions are still +there after an upgrade. This means that you can <emphasis>roll +back</emphasis> to the old version:</para> + +<screen> +$ nix-env --upgrade <replaceable>some-packages</replaceable> +$ nix-env --rollback +</screen> + +</simplesect> + + +<simplesect><title>Garbage collection</title> + +<para>When you uninstall a package like this… + +<screen> +$ nix-env --uninstall firefox +</screen> + +the package isn’t deleted from the system right away (after all, you +might want to do a rollback, or it might be in the profiles of other +users). Instead, unused packages can be deleted safely by running the +<emphasis>garbage collector</emphasis>: + +<screen> +$ nix-collect-garbage +</screen> + +This deletes all packages that aren’t in use by any user profile or by +a currently running program.</para> + +</simplesect> + + +<simplesect><title>Functional package language</title> + +<para>Packages are built from <emphasis>Nix expressions</emphasis>, +which is a simple functional language. A Nix expression describes +everything that goes into a package build action (a “derivation”): +other packages, sources, the build script, environment variables for +the build script, etc. Nix tries very hard to ensure that Nix +expressions are <emphasis>deterministic</emphasis>: building a Nix +expression twice should yield the same result.</para> + +<para>Because it’s a functional language, it’s easy to support +building variants of a package: turn the Nix expression into a +function and call it any number of times with the appropriate +arguments. Due to the hashing scheme, variants don’t conflict with +each other in the Nix store.</para> + +</simplesect> + + +<simplesect><title>Transparent source/binary deployment</title> + +<para>Nix expressions generally describe how to build a package from +source, so an installation action like + +<screen> +$ nix-env --install firefox +</screen> + +<emphasis>could</emphasis> cause quite a bit of build activity, as not +only Firefox but also all its dependencies (all the way up to the C +library and the compiler) would have to built, at least if they are +not already in the Nix store. This is a <emphasis>source deployment +model</emphasis>. For most users, building from source is not very +pleasant as it takes far too long. However, Nix can automatically +skip building from source and download a pre-built binary instead if +it knows about it. <emphasis>Nix channels</emphasis> provide Nix +expressions along with pre-built binaries.</para> + +<!-- +<para>source deployment model (like <a +href="http://www.gentoo.org/">Gentoo</a>) and a binary model (like +RPM)</para> +--> + +</simplesect> + + +<simplesect><title>Binary patching</title> + +<para>In addition to downloading binaries automatically if they’re +available, Nix can download binary deltas that patch an existing +package in the Nix store into a new version. This speeds up +upgrades.</para> + +</simplesect> + + +<simplesect><title>Nix Packages collection</title> + +<para>We provide a large set of Nix expressions containing hundreds of +existing Unix packages, the <emphasis>Nix Packages +collection</emphasis> (Nixpkgs).</para> + +</simplesect> + + +<simplesect><title>Service deployment</title> + +<para>Nix can be used not only for rolling out packages, but also +complete <emphasis>configurations</emphasis> of services. This is +done by treating all the static bits of a service (such as software +packages, configuration files, control scripts, static web pages, +etc.) as “packages” that can be built by Nix expressions. As a +result, all the features above apply to services as well: for +instance, you can roll back a web server configuration if a +configuration change turns out to be undesirable, you can easily have +multiple instances of a service (e.g., a test and production server), +and because the whole service is built in a purely functional way from +a Nix expression, it is repeatable so you can easily reproduce the +service on another machine.</para> + +<!-- +<para>You can read more about this in our <a +href="docs/papers.html#servicecm">SCM-12 paper</a>.</para> +--> + +</simplesect> + + +<simplesect><title>Portability</title> + +<para>Nix should run on most Unix systems, including Linux, FreeBSD and +Mac OS X.<!-- It is also supported on Windows using Cygwin.--></para> + +</simplesect> + + +<simplesect><title>NixOS</title> + +<para>NixOS is a Linux distribution based on Nix. It uses Nix not +just for package management but also to manage the system +configuration (e.g., to build configuration files in +<filename>/etc</filename>). This means, among other things, that it’s +possible to easily roll back the entire configuration of the system to +an earlier state. Also, users can install software without root +privileges. For more information and downloads, see the <link +xlink:href="http://nixos.org/">NixOS homepage</link>.</para> + +</simplesect> + + +<!-- other features: + +- build farms +- reproducibility (Nix expressions allows whole configuration to be rebuilt) + +--> + +</section> + + +<section><title>About us</title> + +<para>Nix was originally developed at the <link +xlink:href="http://www.cs.uu.nl/">Department of Information and +Computing Sciences</link>, Utrecht University by the <link +xlink:href="http://www.cs.uu.nl/wiki/Trace/WebHome">TraCE +project</link> (2003-2008). The project was funded by the Software +Engineering Research Program <link +xlink:href="http://www.jacquard.nl/">Jacquard</link> to improve the +support for variability in software systems. Further funding was +provided by the NIRICT LaQuSo Build Farm project. Development is +currently supported by <link +xlink:href="http://www.logicblox.com/">LogicBlox</link>.</para> + +</section> + + +<section><title>About this manual</title> + +<para>This manual tells you how to install and use Nix and how to +write Nix expressions for software not already in the Nix Packages +collection. It also discusses some advanced topics, such as setting +up distributed multi-platform building.</para> + +</section> + + +<section><title>License</title> + +<para>Nix is free software; you can redistribute it and/or modify it +under the terms of the <link +xlink:href="http://www.gnu.org/licenses/lgpl.html">GNU Lesser General +Public License</link> as published by the <link +xlink:href="http://www.fsf.org/">Free Software Foundation</link>; +either version 2.1 of the License, or (at your option) any later +version. Nix is distributed in the hope that it will be useful, but +WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +Lesser General Public License for more details.</para> + +</section> + + +<section><title>More information</title> + +<para>Some background information on Nix can be found in a number of +papers. The ICSE 2004 paper <citetitle +xlink:href='http://www.st.ewi.tudelft.nl/~dolstra/pubs/immdsd-icse2004-final.pdf'>Imposing +a Memory Management Discipline on Software Deployment</citetitle> +discusses the hashing mechanism used to ensure reliable dependency +identification and non-interference between different versions and +variants of packages. The LISA 2004 paper <citetitle +xlink:href='http://www.st.ewi.tudelft.nl/~dolstra/pubs/nspfssd-lisa2004-final.pdf'>Nix: +A Safe and Policy-Free System for Software Deployment</citetitle> +gives a more general discussion of Nix from a system-administration +perspective. The CBSE 2005 paper <citetitle +xlink:href='http://www.st.ewi.tudelft.nl/~dolstra/pubs/eupfcdm-cbse2005-final.pdf'>Efficient +Upgrading in a Purely Functional Component Deployment Model +</citetitle> is about transparent patch deployment in Nix. The SCM-12 +paper <citetitle +xlink:href='http://www.st.ewi.tudelft.nl/~dolstra/pubs/servicecm-scm12-final.pdf'> +Service Configuration Management</citetitle> shows how services (e.g., +web servers) can be deployed and managed through Nix. An overview of +NixOS is given in the JFP article <citetitle +xlink:href="http://www.st.ewi.tudelft.nl/~dolstra/pubs/nixos-jfp-final.pdf">NixOS: +A Purely Functional Linux Distribution</citetitle>. The Nix homepage +has <link xlink:href="http://nixos.org/docs/papers.html">an up-to-date +list of Nix-related papers</link>.</para> + +<para>Nix is the subject of Eelco Dolstra’s PhD thesis <citetitle +xlink:href="http://igitur-archive.library.uu.nl/dissertations/2006-0118-200031/index.htm">The +Purely Functional Software Deployment Model</citetitle>, which +contains most of the papers listed above.</para> + +<para>Nix has a homepage at <link +xlink:href="http://nixos.org/"/>.</para> + +</section> + + +</chapter> diff --git a/doc/manual/local.mk b/doc/manual/local.mk new file mode 100644 index 000000000000..4342215b0e44 --- /dev/null +++ b/doc/manual/local.mk @@ -0,0 +1,86 @@ +XSLTPROC = $(xsltproc) --nonet $(xmlflags) \ + --param section.autolabel 1 \ + --param section.label.includes.component.label 1 \ + --param html.stylesheet \'style.css\' \ + --param xref.with.number.and.title 1 \ + --param toc.section.depth 3 \ + --param admon.style \'\' \ + --param callout.graphics.extension \'.gif\' \ + --param contrib.inline.enabled 0 + +MANUAL_SRCS := $(wildcard $(d)/*.xml) + + +# Do XInclude processing / RelaxNG validation +$(d)/manual.xmli: $(d)/manual.xml $(MANUAL_SRCS) $(d)/version.txt + $(trace-gen) $(xmllint) --nonet --xinclude $< -o $@.tmp + @mv $@.tmp $@ + +$(d)/version.txt: + $(trace-gen) echo -n $(PACKAGE_VERSION) > $@ + +# Note: RelaxNG validation requires xmllint >= 2.7.4. +$(d)/manual.is-valid: $(d)/manual.xmli + $(trace-gen) $(XSLTPROC) --novalid --stringparam profile.condition manual \ + $(docbookxsl)/profiling/profile.xsl $< 2> /dev/null | \ + $(xmllint) --nonet --noout --relaxng $(docbookrng)/docbook.rng - + @touch $@ + +clean-files += $(d)/manual.xmli $(d)/version.txt $(d)/manual.is-valid + +dist-files += $(d)/manual.xmli $(d)/version.txt $(d)/manual.is-valid + + +# Generate man pages. +man-pages := $(foreach n, \ + nix-env.1 nix-build.1 nix-shell.1 nix-store.1 nix-instantiate.1 \ + nix-collect-garbage.1 nix-push.1 nix-pull.1 \ + nix-prefetch-url.1 nix-channel.1 \ + nix-install-package.1 nix-hash.1 nix-copy-closure.1 \ + nix.conf.5 nix-daemon.8, \ + $(d)/$(n)) + +$(man-pages): $(d)/manual.xmli $(d)/manual.is-valid + $(trace-gen) $(XSLTPROC) --stringparam profile.condition manpage \ + $(docbookxsl)/profiling/profile.xsl $< 2> /dev/null | \ + (cd doc/manual && $(XSLTPROC) $(docbookxsl)/manpages/docbook.xsl -) + +clean-files += $(d)/*.1 $(d)/*.5 $(d)/*.8 + +dist-files += $(man-pages) + + +# Generate the HTML manual. +$(d)/manual.html: $(d)/manual.xml $(MANUAL_SRCS) $(d)/manual.is-valid + $(trace-gen) $(XSLTPROC) --xinclude --stringparam profile.condition manual \ + $(docbookxsl)/profiling/profile.xsl $< | \ + $(XSLTPROC) --output $@ $(docbookxsl)/html/docbook.xsl - + +$(foreach file, $(d)/manual.html $(d)/style.css, $(eval $(call install-data-in, $(file), $(docdir)/manual))) + +$(foreach file, $(wildcard $(d)/figures/*.png), $(eval $(call install-data-in, $(file), $(docdir)/manual/figures))) + +$(foreach file, $(wildcard $(docbookxsl)/images/callouts/*.gif), $(eval $(call install-data-in, $(file), $(docdir)/manual/images/callouts))) + +$(eval $(call install-symlink, manual.html, $(docdir)/manual/index.html)) + +all: $(d)/manual.html + +clean-files += $(d)/manual.html + +dist-files += $(d)/manual.html + + +# Generate the PDF manual. +$(d)/manual.pdf: $(d)/manual.xml $(MANUAL_SRCS) $(d)/manual.is-valid + $(trace-gen) if test "$(dblatex)" != ""; then \ + cd doc/manual && $(XSLTPROC) --xinclude --stringparam profile.condition manual \ + $(docbookxsl)/profiling/profile.xsl manual.xml | \ + $(dblatex) -o manual.pdf $(dblatex_opts) -; \ + else \ + echo "Please install dblatex and rerun configure."; \ + exit 1; \ + fi + +clean-files += $(d)/manual.pdf + diff --git a/doc/manual/manual.xml b/doc/manual/manual.xml new file mode 100644 index 000000000000..671ca9d65c1a --- /dev/null +++ b/doc/manual/manual.xml @@ -0,0 +1,84 @@ +<book xmlns="http://docbook.org/ns/docbook" + xmlns:xi="http://www.w3.org/2001/XInclude"> + + <info> + + <title>Nix User's Guide</title> + + <edition>Version <xi:include href="version.txt" parse="text" /></edition> + + <author> + <personname> + <firstname>Eelco</firstname> + <surname>Dolstra</surname> + </personname> + <affiliation> + <orgname>LogicBlox</orgname> + </affiliation> + <contrib>Author</contrib> + </author> + + <copyright> + <year>2004-2013</year> + <holder>Eelco Dolstra</holder> + </copyright> + + <date>July 2013</date> + + </info> + + + <xi:include href="introduction.xml" /> + <xi:include href="quick-start.xml" /> + <xi:include href="installation.xml" /> + <xi:include href="package-management.xml" /> + <xi:include href="writing-nix-expressions.xml" /> + <xi:include href="build-farm.xml" /> + + + <appendix> + <title>Command Reference</title> + <xi:include href="opt-common.xml" /> + <xi:include href="env-common.xml" /> + + <section> + <title>Main commands</title> + <xi:include href="nix-env.xml" /> + <xi:include href="nix-instantiate.xml" /> + <xi:include href="nix-store.xml" /> + </section> + + <section> + <title>Utilities</title> + <xi:include href="nix-build.xml" /> + <xi:include href="nix-shell.xml" /> + <xi:include href="nix-channel.xml" /> + <xi:include href="nix-collect-garbage.xml" /> + <xi:include href="nix-copy-closure.xml" /> + <xi:include href="nix-hash.xml" /> + <xi:include href="nix-install-package.xml" /> + <xi:include href="nix-prefetch-url.xml" /> + <xi:include href="nix-pull.xml" /> + <xi:include href="nix-push.xml" /> + <xi:include href="nix-daemon.xml" /> + </section> + + <section> + <title>Files</title> + <xi:include href="conf-file.xml" /> + </section> + + </appendix> + + <xi:include href="troubleshooting.xml" /> + <!-- <xi:include href="bugs.xml" /> --> + <xi:include href="glossary.xml" /> + <xi:include href="hacking.xml" /> + + <appendix> + <title>Nix Release Notes</title> + <xi:include href="release-notes.xml" + xpointer="xmlns(x=http://docbook.org/ns/docbook)xpointer(x:article/x:section)" /> + </appendix> + +</book> diff --git a/doc/manual/nix-build.xml b/doc/manual/nix-build.xml new file mode 100644 index 000000000000..969faf9d1694 --- /dev/null +++ b/doc/manual/nix-build.xml @@ -0,0 +1,179 @@ +<refentry xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + xml:id="sec-nix-build"> + +<refmeta> + <refentrytitle>nix-build</refentrytitle> + <manvolnum>1</manvolnum> + <refmiscinfo class="source">Nix</refmiscinfo> + <refmiscinfo class="version"><xi:include href="version.txt" parse="text"/></refmiscinfo> +</refmeta> + +<refnamediv> + <refname>nix-build</refname> + <refpurpose>build a Nix expression</refpurpose> +</refnamediv> + +<refsynopsisdiv> + <cmdsynopsis> + <command>nix-build</command> + <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="opt-common-syn.xml#xmlns(db=http://docbook.org/ns/docbook)xpointer(/db:nop/*)" /> + <arg><option>--arg</option> <replaceable>name</replaceable> <replaceable>value</replaceable></arg> + <arg><option>--argstr</option> <replaceable>name</replaceable> <replaceable>value</replaceable></arg> + <arg> + <group choice='req'> + <arg choice='plain'><option>--attr</option></arg> + <arg choice='plain'><option>-A</option></arg> + </group> + <replaceable>attrPath</replaceable> + </arg> + <arg><option>--drv-link</option> <replaceable>drvlink</replaceable></arg> + <arg><option>--add-drv-link</option></arg> + <arg><option>--no-out-link</option></arg> + <arg> + <group choice='req'> + <arg choice='plain'><option>--out-link</option></arg> + <arg choice='plain'><option>-o</option></arg> + </group> + <replaceable>outlink</replaceable> + </arg> + <arg choice='plain' rep='repeat'><replaceable>paths</replaceable></arg> + </cmdsynopsis> +</refsynopsisdiv> + +<refsection><title>Description</title> + +<para>The <command>nix-build</command> command builds the derivations +described by the Nix expressions in <replaceable>paths</replaceable>. +If the build succeeds, it places a symlink to the result in the +current directory. The symlink is called <filename>result</filename>. +If there are multiple Nix expressions, or the Nix expressions evaluate +to multiple derivations, multiple sequentially numbered symlinks are +created (<filename>result</filename>, <filename>result-2</filename>, +and so on).</para> + +<para>If no <replaceable>paths</replaceable> are specified, then +<command>nix-build</command> will use <filename>default.nix</filename> +in the current directory, if it exists.</para> + +<para><command>nix-build</command> is essentially a wrapper around +<link +linkend="sec-nix-instantiate"><command>nix-instantiate</command></link> +(to translate a high-level Nix expression to a low-level store +derivation) and <link +linkend="rsec-nix-store-realise"><command>nix-store +--realise</command></link> (to build the store derivation).</para> + +<warning><para>The result of the build is automatically registered as +a root of the Nix garbage collector. This root disappears +automatically when the <filename>result</filename> symlink is deleted +or renamed. So don’t rename the symlink.</para></warning> + +</refsection> + + +<refsection><title>Options</title> + +<para>All options not listed here are passed to <command>nix-store +--realise</command>, except for <option>--arg</option> and +<option>--attr</option> / <option>-A</option> which are passed to +<command>nix-instantiate</command>. <phrase condition="manual">See +also <xref linkend="sec-common-options" />.</phrase></para> + +<variablelist> + + <varlistentry><term><option>--drv-link</option> <replaceable>drvlink</replaceable></term> + + <listitem><para>Add a symlink named + <replaceable>drvlink</replaceable> to the store derivation + produced by <command>nix-instantiate</command>. The derivation is + a root of the garbage collector until the symlink is deleted or + renamed. If there are multiple derivations, numbers are suffixed + to <replaceable>drvlink</replaceable> to distinguish between + them.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--add-drv-link</option></term> + + <listitem><para>Shorthand for <option>--drv-link</option> + <filename>./derivation</filename>.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--no-out-link</option></term> + + <listitem><para>Do not create a symlink to the output path. Note + that as a result the output does not become a root of the garbage + collector, and so might be deleted by <command>nix-store + --gc</command>.</para></listitem> + + </varlistentry> + + <varlistentry xml:id='opt-out-link'><term><option>--out-link</option> / + <option>-o</option> <replaceable>outlink</replaceable></term> + + <listitem><para>Change the name of the symlink to the output path + created from <filename>result</filename> to + <replaceable>outlink</replaceable>.</para></listitem> + + </varlistentry> + +</variablelist> + +<para>The following common options are supported:</para> + +<variablelist condition="manpage"> + <xi:include href="opt-common.xml#xmlns(db=http://docbook.org/ns/docbook)xpointer(//db:variablelist[@xml:id='opt-common']/*)" /> +</variablelist> + +</refsection> + + +<refsection><title>Examples</title> + +<screen> +$ nix-build '<nixpkgs>' -A firefox +store derivation is /nix/store/qybprl8sz2lc...-firefox-1.5.0.7.drv +/nix/store/d18hyl92g30l...-firefox-1.5.0.7 + +$ ls -l result +lrwxrwxrwx <replaceable>...</replaceable> result -> /nix/store/d18hyl92g30l...-firefox-1.5.0.7 + +$ ls ./result/bin/ +firefox firefox-config</screen> + +<para>If a derivation has multiple outputs, +<command>nix-build</command> will build the default (first) output. +You can also build all outputs: +<screen> +$ nix-build '<nixpkgs>' -A openssl.all +</screen> +This will create a symlink for each output named +<filename>result-<replaceable>outputname</replaceable></filename>. +The suffix is omitted if the output name is <literal>out</literal>. +So if <literal>openssl</literal> has outputs <literal>out</literal>, +<literal>bin</literal> and <literal>man</literal>, +<command>nix-build</command> will create symlinks +<literal>result</literal>, <literal>result-bin</literal> and +<literal>result-man</literal>. It’s also possible to build a specific +output: +<screen> +$ nix-build '<nixpkgs>' -A openssl.man +</screen> +This will create a symlink <literal>result-man</literal>.</para> + +</refsection> + + +<refsection condition="manpage"><title>Environment variables</title> + +<variablelist> + <xi:include href="env-common.xml#xmlns(db=http://docbook.org/ns/docbook)xpointer(//db:variablelist[@xml:id='env-common']/*)" /> +</variablelist> + +</refsection> + + +</refentry> diff --git a/doc/manual/nix-channel.xml b/doc/manual/nix-channel.xml new file mode 100644 index 000000000000..2c4e1151b0d4 --- /dev/null +++ b/doc/manual/nix-channel.xml @@ -0,0 +1,112 @@ +<refentry xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + xml:id="sec-nix-channel"> + +<refmeta> + <refentrytitle>nix-channel</refentrytitle> + <manvolnum>1</manvolnum> + <refmiscinfo class="source">Nix</refmiscinfo> + <refmiscinfo class="version"><xi:include href="version.txt" parse="text"/></refmiscinfo> +</refmeta> + +<refnamediv> + <refname>nix-channel</refname> + <refpurpose>manage Nix channels</refpurpose> +</refnamediv> + +<refsynopsisdiv> + <cmdsynopsis> + <command>nix-channel</command> + <group choice='req'> + <arg choice='plain'><option>--add</option> <replaceable>url</replaceable> <arg choice='opt'><replaceable>name</replaceable></arg></arg> + <arg choice='plain'><option>--remove</option> <replaceable>url</replaceable></arg> + <arg choice='plain'><option>--list</option></arg> + <arg choice='plain'><option>--update</option> <arg rep='repeat'><replaceable>names</replaceable></arg></arg> + </group> + </cmdsynopsis> +</refsynopsisdiv> + +<refsection><title>Description</title> + +<para>A Nix channel is mechanism that allows you to automatically stay +up-to-date with a set of pre-built Nix expressions. A Nix channel is +just a URL that points to a place containing a set of Nix expressions +and a <command>nix-push</command> manifest. <phrase +condition="manual">See also <xref linkend="sec-channels" +/>.</phrase></para> + +<para>This command has the following operations: + +<variablelist> + + <varlistentry><term><option>--add</option> <replaceable>url</replaceable> [<replaceable>name</replaceable>]</term> + + <listitem><para>Adds a channel named + <replaceable>name</replaceable> with URL + <replaceable>url</replaceable> to the list of subscribed channels. + If <replaceable>name</replaceable> is omitted, it defaults to the + last component of <replaceable>url</replaceable>, with the + suffixes <literal>-stable</literal> or + <literal>-unstable</literal> removed.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--remove</option> <replaceable>name</replaceable></term> + + <listitem><para>Removes the channel named + <replaceable>name</replaceable> from the list of subscribed + channels.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--list</option></term> + + <listitem><para>Prints the names and URLs of all subscribed + channels on standard output.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--update</option> [<replaceable>names</replaceable>…]</term> + + <listitem><para>Downloads the Nix expressions of all subscribed + channels (or only those included in + <replaceable>names</replaceable> if specified), makes them the + default for <command>nix-env</command> operations (by symlinking + them from the directory <filename>~/.nix-defexpr</filename>), and + performs a <command>nix-pull</command> on the manifests of all + channels to make pre-built binaries available.</para></listitem> + + </varlistentry> + +</variablelist> + +</para> + +<para>Note that <option>--add</option> does not automatically perform +an update.</para> + +<para>The list of subscribed channels is stored in +<filename>~/.nix-channels</filename>.</para> + +<para>A channel consists of two elements: a bzipped Tar archive +containing the Nix expressions, and a manifest created by +<command>nix-push</command>. These must be stored under +<literal><replaceable>url</replaceable>/nixexprs.tar.bz2</literal> and +<literal><replaceable>url</replaceable>/MANIFEST</literal>, +respectively.</para> + +</refsection> + +<refsection><title>Examples</title> + +<para>To subscribe to the Nixpkgs channel and install the GNU Hello package:</para> + +<screen> +$ nix-channel --add http://nixos.org/channels/nixpkgs-unstable +$ nix-channel --update +$ nix-env -iA nixpkgs.hello</screen> + +</refsection> + +</refentry> diff --git a/doc/manual/nix-collect-garbage.xml b/doc/manual/nix-collect-garbage.xml new file mode 100644 index 000000000000..a13e365a4e00 --- /dev/null +++ b/doc/manual/nix-collect-garbage.xml @@ -0,0 +1,60 @@ +<refentry xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + xml:id="sec-nix-collect-garbage"> + +<refmeta> + <refentrytitle>nix-collect-garbage</refentrytitle> + <manvolnum>1</manvolnum> + <refmiscinfo class="source">Nix</refmiscinfo> + <refmiscinfo class="version"><xi:include href="version.txt" parse="text"/></refmiscinfo> +</refmeta> + +<refnamediv> + <refname>nix-collect-garbage</refname> + <refpurpose>delete unreachable store paths</refpurpose> +</refnamediv> + +<refsynopsisdiv> + <cmdsynopsis> + <command>nix-collect-garbage</command> + <arg><option>--delete-old</option></arg> + <arg><option>-d</option></arg> + <group choice='opt'> + <arg choice='plain'><option>--print-roots</option></arg> + <arg choice='plain'><option>--print-live</option></arg> + <arg choice='plain'><option>--print-dead</option></arg> + <arg choice='plain'><option>--delete</option></arg> + </group> + <arg><option>--dry-run</option></arg> + </cmdsynopsis> +</refsynopsisdiv> + +<refsection><title>Description</title> + +<para>The command <command>nix-collect-garbage</command> is mostly an +alias of <link linkend="rsec-nix-store-gc"><command>nix-store +--gc</command></link>, that is, it deletes all unreachable paths in +the Nix store to clean up your system. However, it provides an +additional option <option>-d</option> (<option>--delete-old</option>) +that deletes all old generations of all profiles in +<filename>/nix/var/nix/profiles</filename> by invoking +<literal>nix-env --delete-generations old</literal> on all profiles. +Of course, this makes rollbacks to previous configurations +impossible.</para> + +</refsection> + +<refsection><title>Example</title> + +<para>To delete from the Nix store everything that is not used by the +current generations of each profile, do + +<screen> +$ nix-collect-garbage -d</screen> + +</para> + +</refsection> + +</refentry> diff --git a/doc/manual/nix-copy-closure.xml b/doc/manual/nix-copy-closure.xml new file mode 100644 index 000000000000..30b3aaee3dcf --- /dev/null +++ b/doc/manual/nix-copy-closure.xml @@ -0,0 +1,188 @@ +<refentry xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + xml:id="sec-nix-copy-closure"> + +<refmeta> + <refentrytitle>nix-copy-closure</refentrytitle> + <manvolnum>1</manvolnum> + <refmiscinfo class="source">Nix</refmiscinfo> + <refmiscinfo class="version"><xi:include href="version.txt" parse="text"/></refmiscinfo> +</refmeta> + +<refnamediv> + <refname>nix-copy-closure</refname> + <refpurpose>copy a closure to or from a remote machine via SSH</refpurpose> +</refnamediv> + +<refsynopsisdiv> + <cmdsynopsis> + <command>nix-copy-closure</command> + <group> + <arg choice='plain'><option>--to</option></arg> + <arg choice='plain'><option>--from</option></arg> + </group> + <arg><option>--sign</option></arg> + <arg><option>--gzip</option></arg> + <arg><option>--bzip2</option></arg> + <arg><option>--xz</option></arg> + <arg><option>--show-progress</option></arg> + <arg><option>--include-outputs</option></arg> + <arg><option>--use-substitutes</option></arg> + <arg><option>-s</option></arg> + <arg choice='plain'> + <replaceable>user@</replaceable><replaceable>machine</replaceable> + </arg> + <arg choice='plain'><replaceable>paths</replaceable></arg> + </cmdsynopsis> +</refsynopsisdiv> + + +<refsection><title>Description</title> + +<para><command>nix-copy-closure</command> gives you an easy and +efficient way to exchange software between machines. Given one or +more Nix store paths <replaceable>paths</replaceable> on the local +machine, <command>nix-copy-closure</command> computes the closure of +those paths (i.e. all their dependencies in the Nix store), and copies +all paths in the closure to the remote machine via the +<command>ssh</command> (Secure Shell) command. With the +<option>--from</option>, the direction is reversed: +the closure of <replaceable>paths</replaceable> on a remote machine is +copied to the Nix store on the local machine.</para> + +<para>This command is efficient because it only sends the store paths +that are missing on the target machine.</para> + +<para>Since <command>nix-copy-closure</command> calls +<command>ssh</command>, you may be asked to type in the appropriate +password or passphrase. In fact, you may be asked +<emphasis>twice</emphasis> because <command>nix-copy-closure</command> +currently connects twice to the remote machine, first to get the set +of paths missing on the target machine, and second to send the dump of +those paths. If this bothers you, use +<command>ssh-agent</command>.</para> + + +<refsection><title>Options</title> + +<variablelist> + + <varlistentry><term><option>--to</option></term> + + <listitem><para>Copy the closure of + <replaceable>paths</replaceable> from the local Nix store to the + Nix store on <replaceable>machine</replaceable>. This is the + default.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--from</option></term> + + <listitem><para>Copy the closure of + <replaceable>paths</replaceable> from the Nix store on + <replaceable>machine</replaceable> to the local Nix + store.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--sign</option></term> + + <listitem><para>Let the sending machine cryptographically sign the + dump of each path with the key in + <filename><replaceable>sysconfdir</replaceable>/nix/signing-key.sec</filename>. + If the user on the target machine does not have direct access to + the Nix store (i.e., if the target machine has a multi-user Nix + installation), then the target machine will check the dump against + <filename><replaceable>sysconfdir</replaceable>/nix/signing-key.pub</filename> + before unpacking it in its Nix store. This allows secure sharing + of store paths between untrusted users on two machines, provided + that there is a trust relation between the Nix installations on + both machines (namely, they have matching public/secret + keys).</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--gzip</option> / <option>--bzip2</option> / <option>--xz</option></term> + + <listitem><para>Compress the dump of each path with respectively + <command>gzip</command>, <command>bzip2</command> or + <command>xz</command> before sending it. The corresponding + decompression program must be installed on the target + machine.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--show-progress</option></term> + + <listitem><para>Show the progress of each path's transfer as it's made. + This requires the <command>pv</command> utility to be in <envar>PATH</envar>.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--include-outputs</option></term> + + <listitem><para>Also copy the outputs of store derivations + included in the closure.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--use-substitutes</option> / <option>-s</option></term> + + <listitem><para>Attempt to download missing paths on the target + machine using Nix’s substitute mechanism. Any paths that cannot + be substituted on the target are still copied normally from the + source. This is useful, for instance, if the connection between + the source and target machine is slow, but the connection between + the target machine and <literal>nixos.org</literal> (the default + binary cache server) is fast.</para></listitem> + + </varlistentry> + +</variablelist> + +</refsection> + + +<refsection><title>Environment variables</title> + +<variablelist> + + <varlistentry><term><envar>NIX_SSHOPTS</envar></term> + + <listitem><para>Additional options to be passed to + <command>ssh</command> on the command line.</para></listitem> + + </varlistentry> + +</variablelist> + +</refsection> + + +<refsection><title>Examples</title> + +<para>Copy Firefox with all its dependencies to a remote machine: + +<screen> +$ nix-copy-closure --to alice@itchy.labs $(type -tP firefox)</screen> + +</para> + +<para>Copy Subversion from a remote machine and then install it into a +user environment: + +<screen> +$ nix-copy-closure --from alice@itchy.labs \ + /nix/store/0dj0503hjxy5mbwlafv1rsbdiyx1gkdy-subversion-1.4.4 +$ nix-env -i /nix/store/0dj0503hjxy5mbwlafv1rsbdiyx1gkdy-subversion-1.4.4 +</screen> + +</para> + +</refsection> + + +</refsection> + +</refentry> diff --git a/doc/manual/nix-daemon.xml b/doc/manual/nix-daemon.xml new file mode 100644 index 000000000000..c68605fd6566 --- /dev/null +++ b/doc/manual/nix-daemon.xml @@ -0,0 +1,34 @@ +<refentry xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + xml:id="sec-nix-daemon"> + +<refmeta> + <refentrytitle>nix-daemon</refentrytitle> + <manvolnum>8</manvolnum> + <refmiscinfo class="source">Nix</refmiscinfo> + <refmiscinfo class="version"><xi:include href="version.txt" parse="text"/></refmiscinfo> +</refmeta> + +<refnamediv> + <refname>nix-daemon</refname> + <refpurpose>Nix multi-user support daemon</refpurpose> +</refnamediv> + +<refsynopsisdiv> + <cmdsynopsis> + <command>nix-daemon</command> + </cmdsynopsis> +</refsynopsisdiv> + + +<refsection><title>Description</title> + +<para>The Nix daemon is necessary in multi-user Nix installations. It +performs build actions and other operations on the Nix store on behalf +of unprivileged users.</para> + + +</refsection> + +</refentry> diff --git a/doc/manual/nix-env.xml b/doc/manual/nix-env.xml new file mode 100644 index 000000000000..5f87093ac163 --- /dev/null +++ b/doc/manual/nix-env.xml @@ -0,0 +1,1287 @@ +<refentry xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + xml:id="sec-nix-env"> + +<refmeta> + <refentrytitle>nix-env</refentrytitle> + <manvolnum>1</manvolnum> + <refmiscinfo class="source">Nix</refmiscinfo> + <refmiscinfo class="version"><xi:include href="version.txt" parse="text"/></refmiscinfo> +</refmeta> + +<refnamediv> + <refname>nix-env</refname> + <refpurpose>manipulate or query Nix user environments</refpurpose> +</refnamediv> + +<refsynopsisdiv> + <cmdsynopsis> + <command>nix-env</command> + <xi:include href="opt-common-syn.xml#xmlns(db=http://docbook.org/ns/docbook)xpointer(/db:nop/*)" /> + <arg><option>--arg</option> <replaceable>name</replaceable> <replaceable>value</replaceable></arg> + <arg><option>--argstr</option> <replaceable>name</replaceable> <replaceable>value</replaceable></arg> + <arg> + <group choice='req'> + <arg choice='plain'><option>--file</option></arg> + <arg choice='plain'><option>-f</option></arg> + </group> + <replaceable>path</replaceable> + </arg> + <arg> + <group choice='req'> + <arg choice='plain'><option>--profile</option></arg> + <arg choice='plain'><option>-p</option></arg> + </group> + <replaceable>path</replaceable> + </arg> + <arg> + <arg choice='plain'><option>--system-filter</option></arg> + <replaceable>system</replaceable> + </arg> + <arg><option>--dry-run</option></arg> + <arg choice='plain'><replaceable>operation</replaceable></arg> + <arg rep='repeat'><replaceable>options</replaceable></arg> + <arg rep='repeat'><replaceable>arguments</replaceable></arg> + </cmdsynopsis> +</refsynopsisdiv> + + +<refsection><title>Description</title> + +<para>The command <command>nix-env</command> is used to manipulate Nix +user environments. User environments are sets of software packages +available to a user at some point in time. In other words, they are a +synthesised view of the programs available in the Nix store. There +may be many user environments: different users can have different +environments, and individual users can switch between different +environments.</para> + +<para><command>nix-env</command> takes exactly one +<emphasis>operation</emphasis> flag which indicates the subcommand to +be performed. These are documented below.</para> + +</refsection> + + + +<!--######################################################################--> + +<refsection><title>Common options</title> + +<para>This section lists the options that are common to all +operations. These options are allowed for every subcommand, though +they may not always have an effect. <phrase condition="manual">See +also <xref linkend="sec-common-options" />.</phrase></para> + +<variablelist> + + <varlistentry><term><option>--file</option></term> + <term><option>-f</option></term> + + <listitem><para>Specifies the Nix expression (designated below as + the <emphasis>active Nix expression</emphasis>) used by the + <option>--install</option>, <option>--upgrade</option>, and + <option>--query --available</option> operations to obtain + derivations. The default is + <filename>~/.nix-defexpr</filename>.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--profile</option></term> + <term><option>-p</option></term> + + <listitem><para>Specifies the profile to be used by those + operations that operate on a profile (designated below as the + <emphasis>active profile</emphasis>). A profile is a sequence of + user environments called <emphasis>generations</emphasis>, one of + which is the <emphasis>current + generation</emphasis>.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--dry-run</option></term> + + <listitem><para>For the <option>--install</option>, + <option>--upgrade</option>, <option>--uninstall</option>, + <option>--switch-generation</option>, + <option>--delete-generations</option> and + <option>--rollback</option> operations, this flag will cause + <command>nix-env</command> to print what + <emphasis>would</emphasis> be done if this flag had not been + specified, without actually doing it.</para> + + <para><option>--dry-run</option> also prints out which paths will + be <link linkend="gloss-substitute">substituted</link> (i.e., + downloaded) and which paths will be built from source (because no + substitute is available).</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--system-filter</option> <replaceable>system</replaceable></term> + + <listitem><para>By default, operations such as <option>--query + --available</option> show derivations matching any platform. This + option allows you to use derivations for the specified platform + <replaceable>system</replaceable>.</para></listitem> + + </varlistentry> + +</variablelist> + +<variablelist condition="manpage"> + <xi:include href="opt-common.xml#xmlns(db=http://docbook.org/ns/docbook)xpointer(//db:variablelist[@xml:id='opt-common']/*)" /> +</variablelist> + +</refsection> + + + +<!--######################################################################--> + +<refsection><title>Files</title> + +<variablelist> + + <varlistentry><term><filename>~/.nix-defexpr</filename></term> + + <listitem><para>A directory that contains the default Nix + expressions used by the <option>--install</option>, + <option>--upgrade</option>, and <option>--query + --available</option> operations to obtain derivations. The + <option>--file</option> option may be used to override this + default.</para> + + <para>The Nix expressions in this directory are combined into a + single set, with each file as an attribute that has the name of + the file. Thus, if <filename>~/.nix-defexpr</filename> contains + two files, <filename>foo</filename> and <filename>bar</filename>, + then the default Nix expression will essentially be + +<programlisting> +{ + foo = import ~/.nix-defexpr/foo; + bar = import ~/.nix-defexpr/bar; +}</programlisting> + + </para> + + <para>The command <command>nix-channel</command> places symlinks + to the downloaded Nix expressions from each subscribed channel in + this directory.</para> + + </listitem> + + </varlistentry> + + <varlistentry><term><filename>~/.nix-profile</filename></term> + + <listitem><para>A symbolic link to the user's current profile. By + default, this symlink points to + <filename><replaceable>prefix</replaceable>/var/nix/profiles/default</filename>. + The <envar>PATH</envar> environment variable should include + <filename>~/.nix-profile/bin</filename> for the user environment + to be visible to the user.</para></listitem> + + </varlistentry> + +</variablelist> + +</refsection> + + + +<!--######################################################################--> + +<refsection xml:id="rsec-nix-env-install"><title>Operation <option>--install</option></title> + +<refsection><title>Synopsis</title> + +<cmdsynopsis> + <command>nix-env</command> + <group choice='req'> + <arg choice='plain'><option>--install</option></arg> + <arg choice='plain'><option>-i</option></arg> + </group> + <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="opt-inst-syn.xml#xmlns(db=http://docbook.org/ns/docbook)xpointer(/db:nop/*)" /> + <group choice='opt'> + <arg choice='plain'><option>--preserve-installed</option></arg> + <arg choice='plain'><option>-P</option></arg> + </group> + <group choice='opt'> + <arg choice='plain'><option>--remove-all</option></arg> + <arg choice='plain'><option>-r</option></arg> + </group> + <arg choice='plain' rep='repeat'><replaceable>args</replaceable></arg> +</cmdsynopsis> + +</refsection> + + +<refsection><title>Description</title> + +<para>The install operation creates a new user environment, based on +the current generation of the active profile, to which a set of store +paths described by <replaceable>args</replaceable> is added. The +arguments <replaceable>args</replaceable> map to store paths in a +number of possible ways: + +<itemizedlist> + + <listitem><para>By default, <replaceable>args</replaceable> is a set + of derivation names denoting derivations in the active Nix + expression. These are realised, and the resulting output paths are + installed. Currently installed derivations with a name equal to the + name of a derivation being added are removed unless the option + <option>--preserve-installed</option> is + specified.</para> + + <para>If there are multiple derivations matching a name in + <replaceable>args</replaceable> that have the same name (e.g., + <literal>gcc-3.3.6</literal> and <literal>gcc-4.1.1</literal>), then + the derivation with the highest <emphasis>priority</emphasis> is + used. A derivation can define a priority by declaring the + <varname>meta.priority</varname> attribute. This attribute should + be a number, with a higher value denoting a lower priority. The + default priority is <literal>0</literal>.</para> + + <para>If there are multiple matching derivations with the same + priority, then the derivation with the highest version will be + installed.</para> + + <para>You can force the installation of multiple derivations with + the same name by being specific about the versions. For instance, + <literal>nix-env -i gcc-3.3.6 gcc-4.1.1</literal> will install both + version of GCC (and will probably cause a user environment + conflict!).</para></listitem> + + <listitem><para>If <link + linkend='opt-attr'><option>--attr</option></link> + (<option>-A</option>) is specified, the arguments are + <emphasis>attribute paths</emphasis> that select attributes from the + top-level Nix expression. This is faster than using derivation + names and unambiguous. To find out the attribute paths of available + packages, use <literal>nix-env -qaP '*'</literal>.</para></listitem> + + <listitem><para>If <option>--from-profile</option> + <replaceable>path</replaceable> is given, + <replaceable>args</replaceable> is a set of names denoting installed + store paths in the profile <replaceable>path</replaceable>. This is + an easy way to copy user environment elements from one profile to + another.</para></listitem> + + <listitem><para>If <option>--from-expression</option> is given, + <replaceable>args</replaceable> are Nix <link + linkend="ss-functions">functions</link> that are called with the + active Nix expression as their single argument. The derivations + returned by those function calls are installed. This allows + derivations to be specified in an unambiguous way, which is necessary + if there are multiple derivations with the same + name.</para></listitem> + + <listitem><para>If <replaceable>args</replaceable> are store + derivations, then these are <link + linkend="rsec-nix-store-realise">realised</link>, and the resulting + output paths are installed.</para></listitem> + + <listitem><para>If <replaceable>args</replaceable> are store paths + that are not store derivations, then these are <link + linkend="rsec-nix-store-realise">realised</link> and + installed.</para></listitem> + +</itemizedlist> + +</para> + +</refsection> + + +<refsection><title>Flags</title> + +<variablelist> + + <varlistentry><term><option>--prebuild-only</option> / <option>-b</option></term> + + <listitem><para>Use only derivations for which a substitute is + registered, i.e., there is a pre-built binary available that can + be downloaded in lieu of building the derivation. Thus, no + packages will be built from source.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--preserve-installed</option></term> + <term><option>-P</option></term> + + <listitem><para>Do not remove derivations with a name matching one + of the derivations being installed. Usually, trying to have two + versions of the same package installed in the same generation of a + profile will lead to an error in building the generation, due to + file name clashes between the two versions. However, this is not + the case for all packages.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--remove-all</option></term> + <term><option>-r</option></term> + + <listitem><para>Remove all previously installed packages first. + This is equivalent to running <literal>nix-env -e '*'</literal> + first, except that everything happens in a single + transaction.</para></listitem> + + </varlistentry> + +</variablelist> + +</refsection> + + +<refsection xml:id='refsec-nix-env-install-examples'><title>Examples</title> + +<para>To install a specific version of <command>gcc</command> from the +active Nix expression: + +<screen> +$ nix-env --install gcc-3.3.2 +installing `gcc-3.3.2' +uninstalling `gcc-3.1'</screen> + +Note the the previously installed version is removed, since +<option>--preserve-installed</option> was not specified.</para> + +<para>To install an arbitrary version: + +<screen> +$ nix-env --install gcc +installing `gcc-3.3.2'</screen> + +</para> + +<para>To install using a specific attribute: + +<screen> +$ nix-env -i -A gcc40mips +$ nix-env -i -A xorg.xorgserver</screen> + +</para> + +<para>To install all derivations in the Nix expression <filename>foo.nix</filename>: + +<screen> +$ nix-env -f ~/foo.nix -i '*'</screen> + +</para> + +<para>To copy the store path with symbolic name <literal>gcc</literal> +from another profile: + +<screen> +$ nix-env -i --from-profile /nix/var/nix/profiles/foo -i gcc</screen> + +</para> + +<para>To install a specific store derivation (typically created by +<command>nix-instantiate</command>): + +<screen> +$ nix-env -i /nix/store/fibjb1bfbpm5mrsxc4mh2d8n37sxh91i-gcc-3.4.3.drv</screen> + +</para> + +<para>To install a specific output path: + +<screen> +$ nix-env -i /nix/store/y3cgx0xj1p4iv9x0pnnmdhr8iyg741vk-gcc-3.4.3</screen> + +</para> + +<para>To install from a Nix expression specified on the command-line: + +<screen> +$ nix-env -f ./foo.nix -i -E \ + 'f: (f {system = "i686-linux";}).subversionWithJava'</screen> + +I.e., this evaluates to <literal>(f: (f {system = +"i686-linux";}).subversionWithJava) (import ./foo.nix)</literal>, thus +selecting the <literal>subversionWithJava</literal> attribute from the +set returned by calling the function defined in +<filename>./foo.nix</filename>.</para> + +<para>A dry-run tells you which paths will be downloaded or built from +source: + +<screen> +$ nix-env -f pkgs/top-level/all-packages.nix -i f-spot --dry-run +(dry run; not doing anything) +installing `f-spot-0.0.10' +the following derivations will be built: + /nix/store/0g63jv9aagwbgci4nnzs2dkxqz84kdja-libgnomeprintui-2.12.1.tar.bz2.drv + /nix/store/0gfarvxq6sannsdw8a1ir40j1ys2mqb4-ORBit2-2.14.2.tar.bz2.drv + /nix/store/0i9gs5zc04668qiy60ga2rc16abkj7g8-sqlite-2.8.17.drv + <replaceable>...</replaceable> +the following paths will be substituted: + /nix/store/8zbipvm4gp9jfqh9nnk1n3bary1a37gs-perl-XML-Parser-2.34 + /nix/store/b8a2bg7gnyvvvjjibp4axg9x1hzkw36c-mono-1.1.4 + <replaceable>...</replaceable></screen> + +</para> + +</refsection> + +</refsection> + + + +<!--######################################################################--> + +<refsection xml:id="rsec-nix-env-upgrade"><title>Operation <option>--upgrade</option></title> + +<refsection><title>Synopsis</title> + +<cmdsynopsis> + <command>nix-env</command> + <group choice='req'> + <arg choice='plain'><option>--upgrade</option></arg> + <arg choice='plain'><option>-u</option></arg> + </group> + <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="opt-inst-syn.xml#xmlns(db=http://docbook.org/ns/docbook)xpointer(/db:nop/*)" /> + <group choice='opt'> + <arg choice='plain'><option>--lt</option></arg> + <arg choice='plain'><option>--leq</option></arg> + <arg choice='plain'><option>--eq</option></arg> + <arg choice='plain'><option>--always</option></arg> + </group> + <arg choice='plain' rep='repeat'><replaceable>args</replaceable></arg> +</cmdsynopsis> + +</refsection> + +<refsection><title>Description</title> + +<para>The upgrade operation creates a new user environment, based on +the current generation of the active profile, in which all store paths +are replaced for which there are newer versions in the set of paths +described by <replaceable>args</replaceable>. Paths for which there +are no newer versions are left untouched; this is not an error. It is +also not an error if an element of <replaceable>args</replaceable> +matches no installed derivations.</para> + +<para>For a description of how <replaceable>args</replaceable> is +mapped to a set of store paths, see <link +linkend="rsec-nix-env-install"><option>--install</option></link>. If +<replaceable>args</replaceable> describes multiple store paths with +the same symbolic name, only the one with the highest version is +installed.</para> + +</refsection> + +<refsection><title>Flags</title> + +<variablelist> + + <varlistentry><term><option>--lt</option></term> + + <listitem><para>Only upgrade a derivation to newer versions. This + is the default.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--leq</option></term> + + <listitem><para>In addition to upgrading to newer versions, also + “upgrade” to derivations that have the same version. Version are + not a unique identification of a derivation, so there may be many + derivations that have the same version. This flag may be useful + to force “synchronisation” between the installed and available + derivations.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--eq</option></term> + + <listitem><para><emphasis>Only</emphasis> “upgrade” to derivations + that have the same version. This may not seem very useful, but it + actually is, e.g., when there is a new release of Nixpkgs and you + want to replace installed applications with the same versions + built against newer dependencies (to reduce the number of + dependencies floating around on your system).</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--always</option></term> + + <listitem><para>In addition to upgrading to newer versions, also + “upgrade” to derivations that have the same or a lower version. + I.e., derivations may actually be downgraded depending on what is + available in the active Nix expression.</para></listitem> + + </varlistentry> + +</variablelist> + +<para>For the other flags, see <option +linkend="rsec-nix-env-install">--install</option>.</para> + +</refsection> + +<refsection><title>Examples</title> + +<screen> +$ nix-env --upgrade gcc +upgrading `gcc-3.3.1' to `gcc-3.4' + +$ nix-env -u gcc-3.3.2 --always <lineannotation>(switch to a specific version)</lineannotation> +upgrading `gcc-3.4' to `gcc-3.3.2' + +$ nix-env --upgrade pan +<lineannotation>(no upgrades available, so nothing happens)</lineannotation> + +$ nix-env -u '*' <lineannotation>(try to upgrade everything)</lineannotation> +upgrading `hello-2.1.2' to `hello-2.1.3' +upgrading `mozilla-1.2' to `mozilla-1.4'</screen> + +</refsection> + +<refsection xml:id="ssec-version-comparisons"><title>Versions</title> + +<para>The upgrade operation determines whether a derivation +<varname>y</varname> is an upgrade of a derivation +<varname>x</varname> by looking at their respective +<literal>name</literal> attributes. The names (e.g., +<literal>gcc-3.3.1</literal> are split into two parts: the package +name (<literal>gcc</literal>), and the version +(<literal>3.3.1</literal>). The version part starts after the first +dash not following by a letter. <varname>x</varname> is considered an +upgrade of <varname>y</varname> if their package names match, and the +version of <varname>y</varname> is higher that that of +<varname>x</varname>.</para> + +<para>The versions are compared by splitting them into contiguous +components of numbers and letters. E.g., <literal>3.3.1pre5</literal> +is split into <literal>[3, 3, 1, "pre", 5]</literal>. These lists are +then compared lexicographically (from left to right). Corresponding +components <varname>a</varname> and <varname>b</varname> are compared +as follows. If they are both numbers, integer comparison is used. If +<varname>a</varname> is an empty string and <varname>b</varname> is a +number, <varname>a</varname> is considered less than +<varname>b</varname>. The special string component +<literal>pre</literal> (for <emphasis>pre-release</emphasis>) is +considered to be less than other components. String components are +considered less than number components. Otherwise, they are compared +lexicographically (i.e., using case-sensitive string comparison).</para> + +<para>This is illustrated by the following examples: + +<screen> +1.0 < 2.3 +2.1 < 2.3 +2.3 = 2.3 +2.5 > 2.3 +3.1 > 2.3 +2.3.1 > 2.3 +2.3.1 > 2.3a +2.3pre1 < 2.3 +2.3pre3 < 2.3pre12 +2.3a < 2.3c +2.3pre1 < 2.3c +2.3pre1 < 2.3q</screen> + +</para> + +</refsection> + +</refsection> + + + +<!--######################################################################--> + +<refsection><title>Operation <option>--uninstall</option></title> + +<refsection><title>Synopsis</title> + +<cmdsynopsis> + <command>nix-env</command> + <group choice='req'> + <arg choice='plain'><option>--uninstall</option></arg> + <arg choice='plain'><option>-e</option></arg> + </group> + <arg choice='plain' rep='repeat'><replaceable>drvnames</replaceable></arg> +</cmdsynopsis> +</refsection> + +<refsection><title>Description</title> + +<para>The uninstall operation creates a new user environment, based on +the current generation of the active profile, from which the store +paths designated by the symbolic names +<replaceable>names</replaceable> are removed.</para> + +</refsection> + +<refsection><title>Examples</title> + +<screen> +$ nix-env --uninstall gcc +$ nix-env -e '*' <lineannotation>(remove everything)</lineannotation></screen> + +</refsection> + +</refsection> + + + +<!--######################################################################--> + +<refsection xml:id="rsec-nix-env-set-flag"><title>Operation <option>--set-flag</option></title> + +<refsection><title>Synopsis</title> + +<cmdsynopsis> + <command>nix-env</command> + <arg choice='plain'><option>--set-flag</option></arg> + <arg choice='plain'><replaceable>name</replaceable></arg> + <arg choice='plain'><replaceable>value</replaceable></arg> + <arg choice='plain' rep='repeat'><replaceable>drvnames</replaceable></arg> +</cmdsynopsis> +</refsection> + +<refsection><title>Description</title> + +<para>The <option>--set-flag</option> operation allows meta attributes +of installed packages to be modified. There are several attributes +that can be usefully modified, because they affect the behaviour of +<command>nix-env</command> or the user environment build +script: + +<itemizedlist> + + <listitem><para><varname>priority</varname> can be changed to + resolve filename clashes. The user environment build script uses + the <varname>meta.priority</varname> attribute of derivations to + resolve filename collisions between packages. Lower priority values + denote a higher priority. For instance, the GCC wrapper package and + the Binutils package in Nixpkgs both have a file + <filename>bin/ld</filename>, so previously if you tried to install + both you would get a collision. Now, on the other hand, the GCC + wrapper declares a higher priority than Binutils, so the former’s + <filename>bin/ld</filename> is symlinked in the user + environment.</para></listitem> + + <listitem><para><varname>keep</varname> can be set to + <literal>true</literal> to prevent the package from being upgraded + or replaced. This is useful if you want to hang on to an older + version of a package.</para></listitem> + + <listitem><para><varname>active</varname> can be set to + <literal>false</literal> to “disable” the package. That is, no + symlinks will be generated to the files of the package, but it + remains part of the profile (so it won’t be garbage-collected). It + can be set back to <literal>true</literal> to re-enable the + package.</para></listitem> + +</itemizedlist> + +</para> + +</refsection> + +<refsection><title>Examples</title> + +<para>To prevent the currently installed Firefox from being upgraded: + +<screen> +$ nix-env --set-flag keep true firefox</screen> + +After this, <command>nix-env -u</command> will ignore Firefox.</para> + +<para>To disable the currently installed Firefox, then install a new +Firefox while the old remains part of the profile: + +<screen> +$ nix-env -q \* +firefox-2.0.0.9 <lineannotation>(the current one)</lineannotation> + +$ nix-env --preserve-installed -i firefox-2.0.0.11 +installing `firefox-2.0.0.11' +building path(s) `/nix/store/myy0y59q3ig70dgq37jqwg1j0rsapzsl-user-environment' +collision between `/nix/store/<replaceable>...</replaceable>-firefox-2.0.0.11/bin/firefox' + and `/nix/store/<replaceable>...</replaceable>-firefox-2.0.0.9/bin/firefox'. +<lineannotation>(i.e., can’t have two active at the same time)</lineannotation> + +$ nix-env --set-flag active false firefox +setting flag on `firefox-2.0.0.9' + +$ nix-env --preserve-installed -i firefox-2.0.0.11 +installing `firefox-2.0.0.11' + +$ nix-env -q \* +firefox-2.0.0.11 <lineannotation>(the enabled one)</lineannotation> +firefox-2.0.0.9 <lineannotation>(the disabled one)</lineannotation></screen> + +</para> + +<para>To make files from <literal>binutils</literal> take precedence +over files from <literal>gcc</literal>: + +<screen> +$ nix-env --set-flag priority 5 binutils +$ nix-env --set-flag priority 10 gcc</screen> + +</para> + +</refsection> + +</refsection> + + + +<!--######################################################################--> + +<refsection><title>Operation <option>--query</option></title> + +<refsection><title>Synopsis</title> + +<cmdsynopsis> + <command>nix-env</command> + <group choice='req'> + <arg choice='plain'><option>--query</option></arg> + <arg choice='plain'><option>-q</option></arg> + </group> + <group choice='opt'> + <arg choice='plain'><option>--installed</option></arg> + <arg choice='plain'><option>--available</option></arg> + <arg choice='plain'><option>-a</option></arg> + </group> + + <sbr /> + + <arg> + <group choice='req'> + <arg choice='plain'><option>--status</option></arg> + <arg choice='plain'><option>-s</option></arg> + </group> + </arg> + <arg> + <group choice='req'> + <arg choice='plain'><option>--attr-path</option></arg> + <arg choice='plain'><option>-P</option></arg> + </group> + </arg> + <arg><option>--no-name</option></arg> + <arg> + <group choice='req'> + <arg choice='plain'><option>--compare-versions</option></arg> + <arg choice='plain'><option>-c</option></arg> + </group> + </arg> + <arg><option>--system</option></arg> + <arg><option>--drv-path</option></arg> + <arg><option>--out-path</option></arg> + <arg><option>--description</option></arg> + <arg><option>--meta</option></arg> + + <sbr /> + + <arg><option>--xml</option></arg> + <arg> + <group choice='req'> + <arg choice='plain'><option>--prebuilt-only</option></arg> + <arg choice='plain'><option>-b</option></arg> + </group> + </arg> + + <arg> + <group choice='req'> + <arg choice='plain'><option>--attr</option></arg> + <arg choice='plain'><option>-A</option></arg> + </group> + <replaceable>attribute-path</replaceable> + </arg> + + <sbr /> + + <arg choice='plain' rep='repeat'><replaceable>names</replaceable></arg> +</cmdsynopsis> + +</refsection> + + +<refsection><title>Description</title> + +<para>The query operation displays information about either the store +paths that are installed in the current generation of the active +profile (<option>--installed</option>), or the derivations that are +available for installation in the active Nix expression +(<option>--available</option>). It only prints information about +derivations whose symbolic name matches one of +<replaceable>names</replaceable>. The wildcard <literal>*</literal> +shows all derivations.</para> + +<para>The derivations are sorted by their <literal>name</literal> +attributes.</para> + +</refsection> + + +<refsection><title>Source selection</title> + +<para>The following flags specify the set of things on which the query +operates.</para> + +<variablelist> + + <varlistentry><term><option>--installed</option></term> + + <listitem><para>The query operates on the store paths that are + installed in the current generation of the active profile. This + is the default.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--available</option></term> + <term><option>-a</option></term> + + <listitem><para>The query operates on the derivations that are + available in the active Nix expression.</para></listitem> + + </varlistentry> + +</variablelist> + +</refsection> + + +<refsection><title>Queries</title> + +<para>The following flags specify what information to display about +the selected derivations. Multiple flags may be specified, in which +case the information is shown in the order given here. Note that the +name of the derivation is shown unless <option>--no-name</option> is +specified.</para> + +<!-- TODO: fix the terminology here; i.e., derivations, store paths, +user environment elements, etc. --> + +<variablelist> + + <varlistentry><term><option>--xml</option></term> + + <listitem><para>Print the result in an XML representation suitable + for automatic processing by other tools. The root element is + called <literal>items</literal>, which contains a + <literal>item</literal> element for each available or installed + derivation. The fields discussed below are all stored in + attributes of the <literal>item</literal> + elements.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--prebuild-only</option> / <option>-b</option></term> + + <listitem><para>Show only derivations for which a substitute is + registered, i.e., there is a pre-built binary available that can + be downloaded in lieu of building the derivation. Thus, this + shows all packages that probably can be installed + quickly.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--status</option></term> + <term><option>-s</option></term> + + <listitem><para>Print the <emphasis>status</emphasis> of the + derivation. The status consists of three characters. The first + is <literal>I</literal> or <literal>-</literal>, indicating + whether the derivation is currently installed in the current + generation of the active profile. This is by definition the case + for <option>--installed</option>, but not for + <option>--available</option>. The second is <literal>P</literal> + or <literal>-</literal>, indicating whether the derivation is + present on the system. This indicates whether installation of an + available derivation will require the derivation to be built. The + third is <literal>S</literal> or <literal>-</literal>, indicating + whether a substitute is available for the + derivation.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--attr-path</option></term> + <term><option>-P</option></term> + + <listitem><para>Print the <emphasis>attribute path</emphasis> of + the derivation, which can be used to unambiguously select it using + the <link linkend="opt-attr"><option>--attr</option> option</link> + available in commands that install derivations like + <literal>nix-env --install</literal>.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--no-name</option></term> + + <listitem><para>Suppress printing of the <literal>name</literal> + attribute of each derivation.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--compare-versions</option> / + <option>-c</option></term> + + <listitem><para>Compare installed versions to available versions, + or vice versa (if <option>--available</option> is given). This is + useful for quickly seeing whether upgrades for installed + packages are available in a Nix expression. A column is added + with the following meaning: + + <variablelist> + + <varlistentry><term><literal><</literal> <replaceable>version</replaceable></term> + + <listitem><para>A newer version of the package is available + or installed.</para></listitem> + + </varlistentry> + + <varlistentry><term><literal>=</literal> <replaceable>version</replaceable></term> + + <listitem><para>At most the same version of the package is + available or installed.</para></listitem> + + </varlistentry> + + <varlistentry><term><literal>></literal> <replaceable>version</replaceable></term> + + <listitem><para>Only older versions of the package are + available or installed.</para></listitem> + + </varlistentry> + + <varlistentry><term><literal>- ?</literal></term> + + <listitem><para>No version of the package is available or + installed.</para></listitem> + + </varlistentry> + + </variablelist> + + </para></listitem> + + </varlistentry> + + <varlistentry><term><option>--system</option></term> + + <listitem><para>Print the <literal>system</literal> attribute of + the derivation.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--drv-path</option></term> + + <listitem><para>Print the path of the store + derivation.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--out-path</option></term> + + <listitem><para>Print the output path of the + derivation.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--description</option></term> + + <listitem><para>Print a short (one-line) description of the + derivation, if available. The description is taken from the + <literal>meta.description</literal> attribute of the + derivation.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--meta</option></term> + + <listitem><para>Print all of the meta-attributes of the + derivation. This option is only available with + <option>--xml</option>.</para></listitem> + + </varlistentry> + +</variablelist> + +</refsection> + + +<refsection><title>Examples</title> + +<screen> +$ nix-env -q '*' <lineannotation>(show installed derivations)</lineannotation> +bison-1.875c +docbook-xml-4.2 +firefox-1.0.4 +MPlayer-1.0pre7 +ORBit2-2.8.3 +... + +$ nix-env -qa '*' <lineannotation>(show available derivations)</lineannotation> +firefox-1.0.7 +GConf-2.4.0.1 +MPlayer-1.0pre7 +ORBit2-2.8.3 +... + +$ nix-env -qas '*' <lineannotation>(show status of available derivations)</lineannotation> +-P- firefox-1.0.7 <lineannotation>(not installed but present)</lineannotation> +--S GConf-2.4.0.1 <lineannotation>(not present, but there is a substitute for fast installation)</lineannotation> +--S MPlayer-1.0pre3 <lineannotation>(i.e., this is not the installed MPlayer, even though the version is the same!)</lineannotation> +IP- ORBit2-2.8.3 <lineannotation>(installed and by definition present)</lineannotation> +... + +<lineannotation>(show available derivations in the Nix expression <!-- !!! <filename>-->foo.nix<!-- </filename> -->)</lineannotation> +$ nix-env -f ./foo.nix -qa '*' +foo-1.2.3 + +$ nix-env -qc '*' <lineannotation>(compare installed versions to what’s available)</lineannotation> +<replaceable>...</replaceable> +acrobat-reader-7.0 - ? <lineannotation>(package is not available at all)</lineannotation> +autoconf-2.59 = 2.59 <lineannotation>(same version)</lineannotation> +firefox-1.0.4 < 1.0.7 <lineannotation>(a more recent version is available)</lineannotation> +<replaceable>...</replaceable> + +<lineannotation>(show info about a specific package, in XML)</lineannotation> +$ nix-env -qa --xml --description firefox +<![CDATA[<?xml version='1.0' encoding='utf-8'?> +<items> + <item attrPath="0.0.firefoxWrapper" + description="Mozilla Firefox - the browser, reloaded (with various plugins)" + name="firefox-1.5.0.7" system="i686-linux" /> +</items>]]></screen> + +</refsection> + +</refsection> + + + +<!--######################################################################--> + +<refsection><title>Operation <option>--switch-profile</option></title> + +<refsection><title>Synopsis</title> + +<cmdsynopsis> + <command>nix-env</command> + <group choice='req'> + <arg choice='plain'><option>--switch-profile</option></arg> + <arg choice='plain'><option>-S</option></arg> + </group> + <arg choice='req'><replaceable>path</replaceable></arg> +</cmdsynopsis> + +</refsection> + + +<refsection><title>Description</title> + +<para>This operation makes <replaceable>path</replaceable> the current +profile for the user. That is, the symlink +<filename>~/.nix-profile</filename> is made to point to +<replaceable>path</replaceable>.</para> + +</refsection> + +<refsection><title>Examples</title> + +<screen> +$ nix-env -S ~/my-profile</screen> + +</refsection> + +</refsection> + + + +<!--######################################################################--> + +<refsection><title>Operation <option>--list-generations</option></title> + +<refsection><title>Synopsis</title> + +<cmdsynopsis> + <command>nix-env</command> + <arg choice='plain'><option>--list-generations</option></arg> +</cmdsynopsis> + +</refsection> + + +<refsection><title>Description</title> + +<para>This operation print a list of all the currently existing +generations for the active profile. These may be switched to using +the <option>--switch-generation</option> operation. It also prints +the creation date of the generation, and indicates the current +generation.</para> + +</refsection> + + +<refsection><title>Examples</title> + +<screen> +$ nix-env --list-generations + 95 2004-02-06 11:48:24 + 96 2004-02-06 11:49:01 + 97 2004-02-06 16:22:45 + 98 2004-02-06 16:24:33 (current)</screen> + +</refsection> + +</refsection> + + + +<!--######################################################################--> + +<refsection><title>Operation <option>--delete-generations</option></title> + +<refsection><title>Synopsis</title> + +<cmdsynopsis> + <command>nix-env</command> + <arg choice='plain'><option>--delete-generations</option></arg> + <arg choice='plain' rep='repeat'><replaceable>generations</replaceable></arg> +</cmdsynopsis> + +</refsection> + + +<refsection><title>Description</title> + +<para>This operation deletes the specified generations of the current +profile. The generations can be a list of generation numbers, or the +special value <literal>old</literal> to delete all non-current +generations. Periodically deleting old generations is important to +make garbage collection effective.</para> + +</refsection> + +<refsection><title>Examples</title> + +<screen> +$ nix-env --delete-generations 3 4 8 + +$ nix-env -p other_profile --delete-generations old</screen> + +</refsection> + +</refsection> + + + +<!--######################################################################--> + +<refsection><title>Operation <option>--switch-generation</option></title> + +<refsection><title>Synopsis</title> + +<cmdsynopsis> + <command>nix-env</command> + <group choice='req'> + <arg choice='plain'><option>--switch-generation</option></arg> + <arg choice='plain'><option>-G</option></arg> + </group> + <arg choice='req'><replaceable>generation</replaceable></arg> +</cmdsynopsis> + +</refsection> + + +<refsection><title>Description</title> + +<para>This operation makes generation number +<replaceable>generation</replaceable> the current generation of the +active profile. That is, if the +<filename><replaceable>profile</replaceable></filename> is the path to +the active profile, then the symlink +<filename><replaceable>profile</replaceable></filename> is made to +point to +<filename><replaceable>profile</replaceable>-<replaceable>generation</replaceable>-link</filename>, +which is in turn a symlink to the actual user environment in the Nix +store.</para> + +<para>Switching will fail if the specified generation does not exist.</para> + +</refsection> + + +<refsection><title>Examples</title> + +<screen> +$ nix-env -G 42 +switching from generation 50 to 42</screen> + +</refsection> + +</refsection> + + + +<!--######################################################################--> + +<refsection><title>Operation <option>--rollback</option></title> + +<refsection><title>Synopsis</title> + +<cmdsynopsis> + <command>nix-env</command> + <arg choice='plain'><option>--rollback</option></arg> +</cmdsynopsis> + +</refsection> + +<refsection><title>Description</title> + +<para>This operation switches to the “previous” generation of the +active profile, that is, the highest numbered generation lower than +the current generation, if it exists. It is just a convenience +wrapper around <option>--list-generations</option> and +<option>--switch-generation</option>.</para> + +</refsection> + + +<refsection><title>Examples</title> + +<screen> +$ nix-env --rollback +switching from generation 92 to 91 + +$ nix-env --rollback +error: no generation older than the current (91) exists</screen> + +</refsection> + +</refsection> + + +<refsection condition="manpage"><title>Environment variables</title> + +<variablelist> + + <varlistentry><term><envar>NIX_PROFILE</envar></term> + + <listitem><para>Location of the Nix profile. Defaults to the + target of the symlink <filename>~/.nix-profile</filename>, if it + exists, or <filename>/nix/var/nix/profiles/default</filename> + otherwise.</para></listitem> + + </varlistentry> + + <xi:include href="env-common.xml#xmlns(db=http://docbook.org/ns/docbook)xpointer(//db:variablelist[@xml:id='env-common']/*)" /> +</variablelist> + +</refsection> + + +</refentry> diff --git a/doc/manual/nix-hash.xml b/doc/manual/nix-hash.xml new file mode 100644 index 000000000000..af4e361ff8d4 --- /dev/null +++ b/doc/manual/nix-hash.xml @@ -0,0 +1,164 @@ +<refentry xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + xml:id="sec-nix-hash"> + +<refmeta> + <refentrytitle>nix-hash</refentrytitle> + <manvolnum>1</manvolnum> + <refmiscinfo class="source">Nix</refmiscinfo> + <refmiscinfo class="version"><xi:include href="version.txt" parse="text"/></refmiscinfo> +</refmeta> + +<refnamediv> + <refname>nix-hash</refname> + <refpurpose>compute the cryptographic hash of a path</refpurpose> +</refnamediv> + +<refsynopsisdiv> + <cmdsynopsis> + <command>nix-hash</command> + <arg><option>--flat</option></arg> + <arg><option>--base32</option></arg> + <arg><option>--truncate</option></arg> + <arg><option>--type</option> <replaceable>hashAlgo</replaceable></arg> + <arg choice='plain' rep='repeat'><replaceable>path</replaceable></arg> + </cmdsynopsis> + <cmdsynopsis> + <command>nix-hash</command> + <arg choice='plain'><option>--to-base16</option></arg> + <arg choice='plain' rep='repeat'><replaceable>hash</replaceable></arg> + </cmdsynopsis> + <cmdsynopsis> + <command>nix-hash</command> + <arg choice='plain'><option>--to-base32</option></arg> + <arg choice='plain' rep='repeat'><replaceable>hash</replaceable></arg> + </cmdsynopsis> +</refsynopsisdiv> + + +<refsection><title>Description</title> + +<para>The command <command>nix-hash</command> computes the +cryptographic hash of the contents of each +<replaceable>path</replaceable> and prints it on standard output. By +default, it computes an MD5 hash, but other hash algorithms are +available as well. The hash is printed in hexadecimal.</para> + +<para>The hash is computed over a <emphasis>serialisation</emphasis> +of each path: a dump of the file system tree rooted at the path. This +allows directories and symlinks to be hashed as well as regular files. +The dump is in the <emphasis>NAR format</emphasis> produced by <link +linkend="refsec-nix-store-dump"><command>nix-store</command> +<option>--dump</option></link>. Thus, <literal>nix-hash +<replaceable>path</replaceable></literal> yields the same +cryptographic hash as <literal>nix-store --dump +<replaceable>path</replaceable> | md5sum</literal>.</para> + +</refsection> + + +<refsection><title>Options</title> + +<variablelist> + + <varlistentry><term><option>--flat</option></term> + + <listitem><para>Print the cryptographic hash of the contents of + each regular file <replaceable>path</replaceable>. That is, do + not compute the hash over the dump of + <replaceable>path</replaceable>. The result is identical to that + produced by the GNU commands <command>md5sum</command> and + <command>sha1sum</command>.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--base32</option></term> + + <listitem><para>Print the hash in a base-32 representation rather + than hexadecimal. This base-32 representation is more compact and + can be used in Nix expressions (such as in calls to + <function>fetchurl</function>).</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--truncate</option></term> + + <listitem><para>Truncate hashes longer than 160 bits (such as + SHA-256) to 160 bits.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--type</option> <replaceable>hashAlgo</replaceable></term> + + <listitem><para>Use the specified cryptographic hash algorithm, + which can be one of <literal>md5</literal>, + <literal>sha1</literal>, and + <literal>sha256</literal>.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--to-base16</option></term> + + <listitem><para>Don’t hash anything, but convert the base-32 hash + representation <replaceable>hash</replaceable> to + hexadecimal.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--to-base32</option></term> + + <listitem><para>Don’t hash anything, but convert the hexadecimal + hash representation <replaceable>hash</replaceable> to + base-32.</para></listitem> + + </varlistentry> + +</variablelist> + +</refsection> + + +<refsection><title>Examples</title> + +<para>Computing hashes: + +<screen> +$ mkdir test +$ echo "hello" > test/world + +$ nix-hash test/ <lineannotation>(MD5 hash; default)</lineannotation> +8179d3caeff1869b5ba1744e5a245c04 + +$ nix-store --dump test/ | md5sum <lineannotation>(for comparison)</lineannotation> +8179d3caeff1869b5ba1744e5a245c04 - + +$ nix-hash --type sha1 test/ +e4fd8ba5f7bbeaea5ace89fe10255536cd60dab6 + +$ nix-hash --type sha1 --base32 test/ +nvd61k9nalji1zl9rrdfmsmvyyjqpzg4 + +$ nix-hash --type sha256 --flat test/ +error: reading file `test/': Is a directory + +$ nix-hash --type sha256 --flat test/world +5891b5b522d5df086d0ff0b110fbd9d21bb4fc7163af34d08286a2e846f6be03</screen> + +</para> + +<para>Converting between hexadecimal and base-32: + +<screen> +$ nix-hash --type sha1 --to-base32 e4fd8ba5f7bbeaea5ace89fe10255536cd60dab6 +nvd61k9nalji1zl9rrdfmsmvyyjqpzg4 + +$ nix-hash --type sha1 --to-base16 nvd61k9nalji1zl9rrdfmsmvyyjqpzg4 +e4fd8ba5f7bbeaea5ace89fe10255536cd60dab6</screen> + +</para> + +</refsection> + + +</refentry> diff --git a/doc/manual/nix-install-package.xml b/doc/manual/nix-install-package.xml new file mode 100644 index 000000000000..5a081cd9a612 --- /dev/null +++ b/doc/manual/nix-install-package.xml @@ -0,0 +1,198 @@ +<refentry xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + xml:id="sec-nix-install-package"> + +<refmeta> + <refentrytitle>nix-install-package</refentrytitle> + <manvolnum>1</manvolnum> + <refmiscinfo class="source">Nix</refmiscinfo> + <refmiscinfo class="version"><xi:include href="version.txt" parse="text"/></refmiscinfo> +</refmeta> + +<refnamediv> + <refname>nix-install-package</refname> + <refpurpose>install a Nix Package file</refpurpose> +</refnamediv> + +<refsynopsisdiv> + <cmdsynopsis> + <command>nix-install-package</command> + <arg><option>--non-interactive</option></arg> + <arg> + <group choice='req'> + <arg choice='plain'><option>--profile</option></arg> + <arg choice='plain'><option>-p</option></arg> + </group> + <replaceable>path</replaceable> + </arg> + <sbr /> + <group choice='req'> + <arg choice='req'> + <option>--url</option> + <arg choice='plain'><replaceable>url</replaceable></arg> + </arg> + <arg choice='req'> + <arg choice='plain'><replaceable>file</replaceable></arg> + </arg> + </group> + </cmdsynopsis> +</refsynopsisdiv> + + +<refsection><title>Description</title> + +<para>The command <command>nix-install-package</command> interactively +installs a Nix Package file (<filename>*.nixpkg</filename>), which is +a small file that contains a store path to be installed along with the +URL of a <link linkend="sec-nix-push"><command>nix-push</command> +manifest</link>. The Nix Package file is either +<replaceable>file</replaceable>, or automatically downloaded from +<replaceable>url</replaceable> if the <option>--url</option> switch is +used.</para> + +<para><command>nix-install-package</command> is used in <link +linkend="sec-one-click">one-click installs</link> to download and +install pre-built binary packages with all necessary dependencies. +<command>nix-install-package</command> is intended to be associated +with the MIME type <literal>application/nix-package</literal> in a web +browser so that it is invoked automatically when you click on +<filename>*.nixpkg</filename> files. When invoked, it restarts itself +in a terminal window (since otherwise it would be invisible when run +from a browser), asks the user to confirm whether to install the +package, and if so downloads and installs the package into the user’s +current profile.</para> + +<para>To obtain a window, <command>nix-install-package</command> tries +to restart itself with <command>xterm</command>, +<command>konsole</command> and +<command>gnome-terminal</command>.</para> + +</refsection> + + +<refsection><title>Options</title> + +<variablelist> + + <varlistentry><term><option>--non-interactive</option></term> + + <listitem><para>Do not open a new terminal window and do not ask + for confirmation.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--profile</option></term> + <term><option>-p</option></term> + + <listitem><para>Install the package into the specified profile + rather than the user’s current profile.</para></listitem> + + </varlistentry> + +</variablelist> + +</refsection> + + +<refsection><title>Examples</title> + +<para>To install <filename>subversion-1.4.0.nixpkg</filename> into the +user’s current profile, without any prompting: + +<screen> +$ nix-install-package --non-interactive subversion-1.4.0.nixpkg</screen> + +</para> + +<para>To install the same package from some URL into a different +profile: + +<screen> +$ nix-install-package --non-interactive -p /nix/var/nix/profiles/eelco \ + --url http://nix.cs.uu.nl/dist/nix/nixpkgs-0.10pre6622/pkgs/subversion-1.4.0-i686-linux.nixpkg</screen> + +</para> + +</refsection> + + +<refsection><title>Format of <literal>nixpkg</literal> files</title> + +<para>A Nix Package file consists of a single line with the following +format: + +<screen> +NIXPKG1 <replaceable>manifestURL</replaceable> <replaceable>name</replaceable> <replaceable>system</replaceable> <replaceable>drvPath</replaceable> <replaceable>outPath</replaceable></screen> + +The elemens are as follows: + +<variablelist> + + <varlistentry><term><literal>NIXPKG1</literal></term> + + <listitem><para>The version of the Nix Package + file.</para></listitem> + + </varlistentry> + + <varlistentry><term><replaceable>manifestURL</replaceable></term> + + <listitem><para>The manifest to be pulled by + <command>nix-pull</command>. The manifest must contain + <replaceable>outPath</replaceable>.</para></listitem> + + </varlistentry> + + <varlistentry><term><replaceable>name</replaceable></term> + + <listitem><para>The symbolic name and version of the + package.</para></listitem> + + </varlistentry> + + <varlistentry><term><replaceable>system</replaceable></term> + + <listitem><para>The platform identifier of the platform for which + this binary package is intended.</para></listitem> + + </varlistentry> + + <varlistentry><term><replaceable>drvPath</replaceable></term> + + <listitem><para>The path in the Nix store of the derivation from + which <replaceable>outPath</replaceable> was built. Not currently + used.</para></listitem> + + </varlistentry> + + <varlistentry><term><replaceable>outPath</replaceable></term> + + <listitem><para>The path in the Nix store of the package. After + <command>nix-install-package</command> has obtained the manifest + from <replaceable>manifestURL</replaceable>, it performs a + <literal>nix-env -i</literal> <replaceable>outPath</replaceable> + to install the binary package.</para></listitem> + + </varlistentry> + +</variablelist> + +</para> + +<para>An example follows: + +<screen> +NIXPKG1 http://.../nixpkgs-0.10pre6622/MANIFEST subversion-1.4.0 i686-darwin \ + /nix/store/4kh60jkp...-subversion-1.4.0.drv \ + /nix/store/nkw7wpgb...-subversion-1.4.0</screen> + +(The line breaks (<literal>\</literal>) are for presentation purposes +and not part of the actual file.) + +</para> + +</refsection> + + +</refentry> diff --git a/doc/manual/nix-instantiate.xml b/doc/manual/nix-instantiate.xml new file mode 100644 index 000000000000..618b61f159dc --- /dev/null +++ b/doc/manual/nix-instantiate.xml @@ -0,0 +1,229 @@ +<refentry xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + xml:id="sec-nix-instantiate"> + +<refmeta> + <refentrytitle>nix-instantiate</refentrytitle> + <manvolnum>1</manvolnum> + <refmiscinfo class="source">Nix</refmiscinfo> + <refmiscinfo class="version"><xi:include href="version.txt" parse="text"/></refmiscinfo> +</refmeta> + +<refnamediv> + <refname>nix-instantiate</refname> + <refpurpose>instantiate store derivations from Nix expressions</refpurpose> +</refnamediv> + +<refsynopsisdiv> + <cmdsynopsis> + <command>nix-instantiate</command> + <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="opt-common-syn.xml#xmlns(db=http://docbook.org/ns/docbook)xpointer(/db:nop/*)" /> + <arg><option>--arg</option> <replaceable>name</replaceable> <replaceable>value</replaceable></arg> + <arg> + <group choice='req'> + <arg choice='plain'><option>--attr</option></arg> + <arg choice='plain'><option>-A</option></arg> + </group> + <replaceable>attrPath</replaceable> + </arg> + <arg><option>--add-root</option> <replaceable>path</replaceable></arg> + <arg><option>--indirect</option></arg> + <arg> + <group choice='req'> + <arg choice='plain'><option>--parse-only</option></arg> + <arg choice='plain'> + <option>--eval-only</option> + <arg><option>--strict</option></arg> + </arg> + <arg choice='plain'><option>--find-file</option></arg> + </group> + <arg><option>--xml</option></arg> + </arg> + <arg choice='plain' rep='repeat'><replaceable>files</replaceable></arg> + </cmdsynopsis> +</refsynopsisdiv> + + +<refsection><title>Description</title> + +<para>The command <command>nix-instantiate</command> generates <link +linkend="gloss-derivation">store derivations</link> from (high-level) +Nix expressions. It loads and evaluates the Nix expressions in each +of <replaceable>files</replaceable> (which defaults to +<replaceable>./default.nix</replaceable>). Each top-level expression +should evaluate to a derivation, a list of derivations, or a set of +derivations. The paths of the resulting store derivations are printed +on standard output.</para> + +<para>If <replaceable>files</replaceable> is the character +<literal>-</literal>, then a Nix expression will be read from standard +input.</para> + +<para>Most users and developers don’t need to use this command +(<command>nix-env</command> and <command>nix-build</command> perform +store derivation instantiation from Nix expressions automatically). +It is most commonly used for implementing new deployment +policies.</para> + +<para condition="manual">See also <xref linkend="sec-common-options" +/> for a list of common options.</para> + +</refsection> + + +<refsection><title>Options</title> + +<variablelist> + + <varlistentry> + <term><option>--add-root</option> <replaceable>path</replaceable></term> + <term><option>--indirect</option></term> + + <listitem><para>See the <link linkend="opt-add-root">corresponding + options</link> in <command>nix-store</command>.</para></listitem> + + </varlistentry> + + + <varlistentry><term><option>--parse-only</option></term> + + <listitem><para>Just parse the input files, and print their + abstract syntax trees on standard output in ATerm + format.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--eval-only</option></term> + + <listitem><para>Just parse and evaluate the input files, and print + the resulting values on standard output. No instantiation of + store derivations takes place.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--find-file</option></term> + + <listitem><para>Look up the given files in Nix’s search path (as + specified by the <envar>NIX_PATH</envar> environment variable). + If found, print the corresponding absolute paths on standard + output. For instance, if <envar>NIX_PATH</envar> is + <literal>nixpkgs=/home/alice/nixpkgs</literal>, then + <literal>nix-instantiate --find-file nixpkgs/default.nix</literal> + will print + <literal>/home/alice/nixpkgs/default.nix</literal>.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--xml</option></term> + + <listitem><para>When used with <option>--parse-only</option> and + <option>--eval-only</option>, print the resulting expression as an + XML representation of the abstract syntax tree rather than as an + ATerm. The schema is the same as that used by the <link + linkend="builtin-toXML"><function>toXML</function> + built-in</link>.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--strict</option></term> + + <listitem><para>When used with <option>--eval-only</option>, + recursively evaluate list elements and attributes. Normally, such + sub-expressions are left unevaluated (since the Nix expression + language is lazy).</para> + + <warning><para>This option can cause non-termination, because lazy + data structures can be infinitely large.</para></warning> + + </listitem> + + </varlistentry> + +</variablelist> + +<variablelist condition="manpage"> + <xi:include href="opt-common.xml#xmlns(db=http://docbook.org/ns/docbook)xpointer(//db:variablelist[@xml:id='opt-common']/*)" /> +</variablelist> + +</refsection> + + +<refsection><title>Examples</title> + +<para>Instantiating store derivations from a Nix expression, and +building them using <command>nix-store</command>: + +<screen> +$ nix-instantiate test.nix <lineannotation>(instantiate)</lineannotation> +/nix/store/cigxbmvy6dzix98dxxh9b6shg7ar5bvs-perl-BerkeleyDB-0.26.drv + +$ nix-store -r $(nix-instantiate test.nix) <lineannotation>(build)</lineannotation> +<replaceable>...</replaceable> +/nix/store/qhqk4n8ci095g3sdp93x7rgwyh9rdvgk-perl-BerkeleyDB-0.26 <lineannotation>(output path)</lineannotation> + +$ ls -l /nix/store/qhqk4n8ci095g3sdp93x7rgwyh9rdvgk-perl-BerkeleyDB-0.26 +dr-xr-xr-x 2 eelco users 4096 1970-01-01 01:00 lib +...</screen> + +</para> + +<para>Parsing and evaluating Nix expressions: + +<screen> +$ echo '"foo" + "bar"' | nix-instantiate --parse-only - +OpPlus(Str("foo"),Str("bar")) + +$ echo '"foo" + "bar"' | nix-instantiate --eval-only - +Str("foobar") + +$ echo '"foo" + "bar"' | nix-instantiate --eval-only --xml - +<![CDATA[<?xml version='1.0' encoding='utf-8'?> +<expr> + <string value="foobar" /> +</expr>]]></screen> + +</para> + +<para>The difference between non-strict and strict evaluation: + +<screen> +$ echo 'rec { x = "foo"; y = x; }' | nix-instantiate --eval-only --xml - +<replaceable>...</replaceable><![CDATA[ + <attr name="x"> + <string value="foo" /> + </attr> + <attr name="y"> + <unevaluated /> + </attr>]]> +<replaceable>...</replaceable></screen> + +Note that <varname>y</varname> is left unevaluated (the XML +representation doesn’t attempt to show non-normal forms). + +<screen> +$ echo 'rec { x = "foo"; y = x; }' | nix-instantiate --eval-only --xml --strict - +<replaceable>...</replaceable><![CDATA[ + <attr name="x"> + <string value="foo" /> + </attr> + <attr name="y"> + <string value="foo" /> + </attr>]]> +<replaceable>...</replaceable></screen> + +</para> + +</refsection> + + +<refsection condition="manpage"><title>Environment variables</title> + +<variablelist> + <xi:include href="env-common.xml#xmlns(db=http://docbook.org/ns/docbook)xpointer(//db:variablelist[@xml:id='env-common']/*)" /> +</variablelist> + +</refsection> + + +</refentry> diff --git a/doc/manual/nix-lang-ref.xml b/doc/manual/nix-lang-ref.xml new file mode 100644 index 000000000000..86273ac3d016 --- /dev/null +++ b/doc/manual/nix-lang-ref.xml @@ -0,0 +1,182 @@ +<appendix> + <title>Nix Language Reference</title> + + <sect1> + <title>Grammar</title> + + <productionset> + <title>Expressions</title> + + <production id="nix.expr"> + <lhs>Expr</lhs> + <rhs> + <nonterminal def="#nix.expr_function" /> + </rhs> + </production> + + <production id="nix.expr_function"> + <lhs>ExprFunction</lhs> + <rhs> + '{' <nonterminal def="#nix.formals" /> '}' ':' <nonterminal def="#nix.expr_function" /> + <sbr />| + <nonterminal def="#nix.expr_assert" /> + </rhs> + </production> + + <production id="nix.expr_assert"> + <lhs>ExprAssert</lhs> + <rhs> + 'assert' <nonterminal def="#nix.expr" /> ';' <nonterminal def="#nix.expr_assert" /> + <sbr />| + <nonterminal def="#nix.expr_if" /> + </rhs> + </production> + + <production id="nix.expr_if"> + <lhs>ExprIf</lhs> + <rhs> + 'if' <nonterminal def="#nix.expr" /> 'then' <nonterminal def="#nix.expr" /> + 'else' <nonterminal def="#nix.expr" /> + <sbr />| + <nonterminal def="#nix.expr_op" /> + </rhs> + </production> + + <production id="nix.expr_op"> + <lhs>ExprOp</lhs> + <rhs> + '!' <nonterminal def="#nix.expr_op" /> + <sbr />| + <nonterminal def="#nix.expr_op" /> '==' <nonterminal def="#nix.expr_op" /> + <sbr />| + <nonterminal def="#nix.expr_op" /> '!=' <nonterminal def="#nix.expr_op" /> + <sbr />| + <nonterminal def="#nix.expr_op" /> '&&' <nonterminal def="#nix.expr_op" /> + <sbr />| + <nonterminal def="#nix.expr_op" /> '||' <nonterminal def="#nix.expr_op" /> + <sbr />| + <nonterminal def="#nix.expr_op" /> '->' <nonterminal def="#nix.expr_op" /> + <sbr />| + <nonterminal def="#nix.expr_op" /> '//' <nonterminal def="#nix.expr_op" /> + <sbr />| + <nonterminal def="#nix.expr_op" /> '~' <nonterminal def="#nix.expr_op" /> + <sbr />| + <nonterminal def="#nix.expr_op" /> '?' <nonterminal def="#nix.id" /> + <sbr />| + <nonterminal def="#nix.expr_app" /> + </rhs> + </production> + + <production id="nix.expr_app"> + <lhs>ExprApp</lhs> + <rhs> + <nonterminal def="#nix.expr_app" /> '.' <nonterminal def="#nix.expr_select" /> + <sbr />| + <nonterminal def="#nix.expr_select" /> + </rhs> + </production> + + <production id="nix.expr_select"> + <lhs>ExprSelect</lhs> + <rhs> + <nonterminal def="#nix.expr_select" /> <nonterminal def="#nix.id" /> + <sbr />| + <nonterminal def="#nix.expr_simple" /> + </rhs> + </production> + + <production id="nix.expr_simple"> + <lhs>ExprSimple</lhs> + <rhs> + <nonterminal def="#nix.id" /> | + <nonterminal def="#nix.int" /> | + <nonterminal def="#nix.str" /> | + <nonterminal def="#nix.path" /> | + <nonterminal def="#nix.uri" /> + <sbr />| + 'true' | 'false' | 'null' + <sbr />| + '(' <nonterminal def="#nix.expr" /> ')' + <sbr />| + '{' <nonterminal def="#nix.bind" />* '}' + <sbr />| + 'let' '{' <nonterminal def="#nix.bind" />* '}' + <sbr />| + 'rec' '{' <nonterminal def="#nix.bind" />* '}' + <sbr />| + '[' <nonterminal def="#nix.expr_select" />* ']' + </rhs> + </production> + + <production id="nix.bind"> + <lhs>Bind</lhs> + <rhs> + <nonterminal def="#nix.id" /> '=' <nonterminal def="#nix.expr" /> ';' + <sbr />| + 'inherit' ('(' <nonterminal def="#nix.expr" /> ')')? <nonterminal def="#nix.id" />* ';' + </rhs> + </production> + + <production id="nix.formals"> + <lhs>Formals</lhs> + <rhs> + <nonterminal def="#nix.formal" /> ',' <nonterminal def="#nix.formals" /> + | <nonterminal def="#nix.formal" /> + </rhs> + </production> + + <production id="nix.formal"> + <lhs>Formal</lhs> + <rhs> + <nonterminal def="#nix.id" /> + <sbr />| + <nonterminal def="#nix.id" /> '?' <nonterminal def="#nix.expr" /> + </rhs> + </production> + + </productionset> + + <productionset> + <title>Terminals</title> + + <production id="nix.id"> + <lhs>Id</lhs> + <rhs>[a-zA-Z\_][a-zA-Z0-9\_\']*</rhs> + </production> + + <production id="nix.int"> + <lhs>Int</lhs> + <rhs>[0-9]+</rhs> + </production> + + <production id="nix.str"> + <lhs>Str</lhs> + <rhs>\"[^\n\"]*\"</rhs> + </production> + + <production id="nix.path"> + <lhs>Path</lhs> + <rhs>[a-zA-Z0-9\.\_\-\+]*(\/[a-zA-Z0-9\.\_\-\+]+)+</rhs> + </production> + + <production id="nix.uri"> + <lhs>Uri</lhs> + <rhs>[a-zA-Z][a-zA-Z0-9\+\-\.]*\:[a-zA-Z0-9\%\/\?\:\@\&\=\+\$\,\-\_\.\!\~\*\']+</rhs> + </production> + + <production id="nix.ws"> + <lhs>Whitespace</lhs> + <rhs> + [ \t\n]+ + <sbr />| + \#[^\n]* + <sbr />| + \/\*(.|\n)*\*\/ + </rhs> + </production> + + </productionset> + + </sect1> + +</appendix> diff --git a/doc/manual/nix-prefetch-url.xml b/doc/manual/nix-prefetch-url.xml new file mode 100644 index 000000000000..c416e675b05c --- /dev/null +++ b/doc/manual/nix-prefetch-url.xml @@ -0,0 +1,93 @@ +<refentry xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + xml:id="sec-nix-prefetch-url"> + +<refmeta> + <refentrytitle>nix-prefetch-url</refentrytitle> + <manvolnum>1</manvolnum> + <refmiscinfo class="source">Nix</refmiscinfo> + <refmiscinfo class="version"><xi:include href="version.txt" parse="text"/></refmiscinfo> +</refmeta> + +<refnamediv> + <refname>nix-prefetch-url</refname> + <refpurpose>copy a file from a URL into the store and print its hash</refpurpose> +</refnamediv> + +<refsynopsisdiv> + <cmdsynopsis> + <command>nix-prefetch-url</command> + <arg><option>--type</option> <replaceable>hashAlgo</replaceable></arg> + <arg choice='plain'><replaceable>url</replaceable></arg> + <arg><replaceable>hash</replaceable></arg> + </cmdsynopsis> +</refsynopsisdiv> + + +<refsection><title>Description</title> + +<para>The command <command>nix-prefetch-url</command> downloads the +file referenced by the URL <replaceable>url</replaceable>, prints its +cryptographic hash, and copies it into the Nix store. The file name +in the store is +<filename><replaceable>hash</replaceable>-<replaceable>baseName</replaceable></filename>, +where <replaceable>baseName</replaceable> is everything following the +final slash in <replaceable>url</replaceable>.</para> + +<para>This command is just a convenience for Nix expression writers. +Often a Nix expression fetches some source distribution from the +network using the <literal>fetchurl</literal> expression contained in +Nixpkgs. However, <literal>fetchurl</literal> requires a +cryptographic hash. If you don't know the hash, you would have to +download the file first, and then <literal>fetchurl</literal> would +download it again when you build your Nix expression. Since +<literal>fetchurl</literal> uses the same name for the downloaded file +as <command>nix-prefetch-url</command>, the redundant download can be +avoided.</para> + +<para>If <replaceable>hash</replaceable> is specified, then a download +is not performed if the Nix store already contains a file with the +same hash and base name. Otherwise, the file is downloaded, and an +error if signaled if the actual hash of the file does not match the +specified hash.</para> + +<para>This command prints the hash on standard output. Additionally, +if the environment variable <envar>PRINT_PATH</envar> is set, the path +of the downloaded file in the Nix store is also printed.</para> + +</refsection> + + +<refsection><title>Options</title> + +<variablelist> + + <varlistentry><term><option>--type</option> <replaceable>hashAlgo</replaceable></term> + + <listitem><para>Use the specified cryptographic hash algorithm, + which can be one of <literal>md5</literal>, + <literal>sha1</literal>, and + <literal>sha256</literal>.</para></listitem> + + </varlistentry> + +</variablelist> + +</refsection> + + +<refsection><title>Examples</title> + +<screen> +$ nix-prefetch-url ftp://ftp.nluug.nl/pub/gnu/make/make-3.80.tar.bz2 +0bbd1df101bc0294d440471e50feca71 + +$ PRINT_PATH=1 nix-prefetch-url ftp://ftp.nluug.nl/pub/gnu/make/make-3.80.tar.bz2 +0bbd1df101bc0294d440471e50feca71 +/nix/store/wvyz8ifdn7wyz1p3pqyn0ra45ka2l492-make-3.80.tar.bz2</screen> + +</refsection> + + +</refentry> diff --git a/doc/manual/nix-pull.xml b/doc/manual/nix-pull.xml new file mode 100644 index 000000000000..8e4a505e1d25 --- /dev/null +++ b/doc/manual/nix-pull.xml @@ -0,0 +1,50 @@ +<refentry xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + xml:id="sec-nix-pull"> + +<refmeta> + <refentrytitle>nix-pull</refentrytitle> + <manvolnum>1</manvolnum> + <refmiscinfo class="source">Nix</refmiscinfo> + <refmiscinfo class="version"><xi:include href="version.txt" parse="text"/></refmiscinfo> +</refmeta> + +<refnamediv> + <refname>nix-pull</refname> + <refpurpose>pull substitutes from a network cache</refpurpose> +</refnamediv> + +<refsynopsisdiv> + <cmdsynopsis> + <command>nix-pull</command> + <arg choice='plain'><replaceable>url</replaceable></arg> + </cmdsynopsis> +</refsynopsisdiv> + + +<refsection><title>Description</title> + +<para>The command <command>nix-pull</command> obtains a list of +pre-built store paths from the URL <replaceable>url</replaceable>, and +for each of these store paths, registers a substitute derivation that +downloads and unpacks it into the Nix store. This is used to speed up +installations: if you attempt to install something that has already +been built and stored into the network cache, Nix can transparently +re-use the pre-built store paths.</para> + +<para>The file at <replaceable>url</replaceable> must be compatible +with the files created by <replaceable>nix-push</replaceable>.</para> + +</refsection> + + +<refsection><title>Examples</title> + +<screen> +$ nix-pull http://nix.cs.uu.nl/dist/nix/nixpkgs-0.5pre753/MANIFEST</screen> + +</refsection> + + +</refentry> diff --git a/doc/manual/nix-push.xml b/doc/manual/nix-push.xml new file mode 100644 index 000000000000..e789bbf7d352 --- /dev/null +++ b/doc/manual/nix-push.xml @@ -0,0 +1,398 @@ +<refentry xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + xml:id="sec-nix-push"> + +<refmeta> + <refentrytitle>nix-push</refentrytitle> + <manvolnum>1</manvolnum> + <refmiscinfo class="source">Nix</refmiscinfo> + <refmiscinfo class="version"><xi:include href="version.txt" parse="text"/></refmiscinfo> +</refmeta> + +<refnamediv> + <refname>nix-push</refname> + <refpurpose>generate a binary cache</refpurpose> +</refnamediv> + +<refsynopsisdiv> + <cmdsynopsis> + <command>nix-push</command> + <arg choice='plain'><option>--dest</option> <replaceable>dest-dir</replaceable></arg> + <arg><option>--bzip2</option></arg> + <arg><option>--none</option></arg> + <arg><option>--force</option></arg> + <arg><option>--link</option></arg> + <arg><option>--manifest</option></arg> + <arg><option>--manifest-path</option> <replaceable>filename</replaceable></arg> + <arg><option>--url-prefix</option> <replaceable>url</replaceable></arg> + <arg choice='plain' rep='repeat'><replaceable>paths</replaceable></arg> + </cmdsynopsis> +</refsynopsisdiv> + + +<refsection><title>Description</title> + +<para>The command <command>nix-push</command> produces a +<emphasis>binary cache</emphasis>, a directory containing compressed +Nix archives (NARs) plus some metadata of the closure of the specified +store paths. This directory can then be made available through a web +server to other Nix installations, allowing them to skip building from +source and instead download binaries from the cache +automatically.</para> + +<para><command>nix-push</command> performs the following actions. + +<orderedlist> + + <listitem><para>Each path in <replaceable>paths</replaceable> is + built (using <link + linkend='rsec-nix-store-realise'><command>nix-store + --realise</command></link>).</para></listitem> + + <listitem><para>All paths in the closure of + <replaceable>paths</replaceable> are determined (using + <command>nix-store --query --requisites + --include-outputs</command>). Note that since the + <option>--include-outputs</option> flag is used, if + <replaceable>paths</replaceable> includes a store derivation, you + get a combined source/binary distribution (e.g., source tarballs + will be included).</para></listitem> + + <listitem><para>All store paths determined in the previous step are + packaged into a NAR (using <command>nix-store --dump</command>) and + compressed using <command>xz</command> or <command>bzip2</command>. + The resulting files have the extension <filename>.nar.xz</filename> + or <filename>.nar.bz2</filename>. Also for each store path, Nix + generates a file with extension <filename>.narinfo</filename> + containing metadata such as the references, cryptographic hash and + size of each path.</para></listitem> + + <listitem><para>Optionally, a single <emphasis>manifest</emphasis> + file is created that contains the same metadata as the + <filename>.narinfo</filename> files. This is for compatibility with + Nix versions prior to 1.2 (see <command>nix-pull</command> for + details).</para></listitem> + + <listitem><para>A file named <option>nix-cache-info</option> is + placed in the destination directory. The existence of this file + marks the directory as a binary cache.</para></listitem> + +</orderedlist> + +</para> + +</refsection> + + +<refsection><title>Options</title> + +<variablelist> + + <varlistentry><term><option>--dest</option> <replaceable>dest-dir</replaceable></term> + + <listitem><para>Set the destination directory to + <replaceable>dir</replaceable>, which is created if it does not + exist. This flag is required.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--bzip2</option></term> + + <listitem><para>Compress NARs using <command>bzip2</command> + instead of <command>xz -9</command>. The latter compresses about + 30% better on typical archives, decompresses about twice as fast, + but compresses a lot slower and is not supported by Nix prior to + version 1.2.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--none</option></term> + + <listitem><para>Do not compress NARs.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--force</option></term> + + <listitem><para>Overwrite <filename>.narinfo</filename> files if + they already exist.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--link</option></term> + + <listitem><para>By default, NARs are generated in the Nix store + and then copied to <replaceable>dest-dir</replaceable>. If this + option is given, hard links are used instead. This only works if + <replaceable>dest-dir</replaceable> is on the same filesystem as + the Nix store.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--manifest</option></term> + + <listitem><para>Force the generation of a manifest suitable for + use by <command>nix-pull</command>. The manifest is stored as + <filename><replaceable>dest-dir</replaceable>/MANIFEST</filename>.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--manifest-path</option> <replaceable>filename</replaceable></term> + + <listitem><para>Like <option>--manifest</option>, but store the + manifest in <replaceable>filename</replaceable>.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--url-prefix</option> <replaceable>url</replaceable></term> + + <listitem><para>Manifests are expected to contain the absolute + URLs of NARs. For generating these URLs, the prefix + <replaceable>url</replaceable> is used. It defaults to + <uri>file://<replaceable>dest-dir</replaceable></uri>.</para></listitem> + + </varlistentry> + +</variablelist> + +</refsection> + + +<refsection><title>Examples</title> + +<para>To add the closure of Thunderbird to a binary cache: + +<screen> +$ nix-push --dest /tmp/cache $(nix-build -A thunderbird) +</screen> + +Assuming that <filename>/tmp/cache</filename> is exported by a web +server as <uri>http://example.org/cache</uri>, you can then use this +cache on another machine to speed up the installation of Thunderbird: + +<screen> +$ nix-build -A thunderbird --option binary-caches http://example.org/cache +</screen> + +Alternatively, you could add <literal>binary-caches = +http://example.org/cache</literal> to +<filename>nix.conf</filename>.</para> + +<para>To also include build-time dependencies (such as source +tarballs): + +<screen> +$ nix-push --dest /tmp/cache $(nix-instantiate -A thunderbird) +</screen> + +</para> + +<para>To generate a manifest suitable for <command>nix-pull</command>: + +<screen> +$ nix-push --dest /tmp/cache $(nix-build -A thunderbird) --manifest +</screen> + +On another machine you can then do: + +<screen> +$ nix-pull http://example.org/cache +</screen> + +to cause the binaries to be used by subsequent Nix operations.</para> + +</refsection> + + +<refsection><title>Binary cache format and operation</title> + +<para>A binary cache with URL <replaceable>url</replaceable> only +denotes a valid binary cache if the file +<uri><replaceable>url</replaceable>/nix-cache-info</uri> exists. If +this file does not exist (or cannot be downloaded), the cache is +ignored. If it does exist, it must be a text file containing cache +properties. Here’s an example: + +<screen> +StoreDir: /nix/store +WantMassQuery: 1 +Priority: 10 +</screen> + +The properties that are currently supported are: + +<variablelist> + + <varlistentry><term><literal>StoreDir</literal></term> + + <listitem><para>The path of the Nix store to which this binary + cache applies. Binaries are not relocatable — a binary built for + <filename>/nix/store</filename> won’t generally work in + <filename>/home/alice/store</filename> — so to prevent binaries + from being used in a wrong store, a binary cache is only used if + its <literal>StoreDir</literal> matches the local Nix + configuration. The default is + <filename>/nix/store</filename>.</para></listitem> + + </varlistentry> + + <varlistentry><term><literal>WantMassQuery</literal></term> + + <listitem><para>Query operations such as <command>nix-env + -qas</command> can cause thousands of cache queries, and thus + thousands of HTTP requests, to determine which packages are + available in binary form. While these requests are small, not + every server may appreciate a potential onslaught of queries. If + <literal>WantMassQuery</literal> is set to <literal>0</literal> + (default), “mass queries” such as <command>nix-env -qas</command> + will skip this cache. Thus a package may appear not to have a + binary substitute. However, the binary will still be used when + you actually install the package. If + <literal>WantMassQuery</literal> is set to <literal>1</literal>, + mass queries will use this cache.</para></listitem> + + </varlistentry> + + <varlistentry><term><literal>Priority</literal></term> + + <listitem><para>Each binary cache has a priority (defaulting to + 50). Binary caches are checked for binaries in order of ascending + priority; thus a higher number denotes a lower priority. The + binary cache <uri>http://cache.nixos.org</uri> has priority + 40.</para></listitem> + + </varlistentry> + +</variablelist> + +</para> + +<para>Every time Nix needs to build some store path +<replaceable>p</replaceable>, it will check each configured binary +cache to see if it has a NAR file for <replaceable>p</replaceable>, +until it finds one. If no cache has a NAR, Nix will fall back to +building the path from source (if applicable). To see if a cache with +URL <replaceable>url</replaceable> has a binary for +<replaceable>p</replaceable>, Nix fetches +<replaceable>url/h</replaceable>, where <replaceable>h</replaceable> +is the hash part of <replaceable>p</replaceable>. Thus, if we have a +cache <uri>http://cache.nixos.org</uri> and we want to obtain +the store path +<screen> +/nix/store/a8922c0h87iilxzzvwn2hmv8x210aqb9-glibc-2.7 +</screen> +then Nix will attempt to fetch +<screen> +http://cache.nixos.org/a8922c0h87iilxzzvwn2hmv8x210aqb9.narinfo +</screen> +(Commands such as <command>nix-env -qas</command> will issue an HTTP +HEAD request, since it only needs to know if the +<filename>.narinfo</filename> file exists.) The +<filename>.narinfo</filename> file is a simple text file that looks +like this: + +<screen> +StorePath: /nix/store/a8922c0h87iilxzzvwn2hmv8x210aqb9-glibc-2.7 +URL: nar/0zzjpdz46mdn74v09m053yczlz4am038g8r74iy8w43gx8801h70.nar.bz2 +Compression: bzip2 +FileHash: sha256:0zzjpdz46mdn74v09m053yczlz4am038g8r74iy8w43gx8801h70 +FileSize: 24473768 +NarHash: sha256:0s491y1h9hxj5ghiizlxk7ax6jwbha00zwn7lpyd5xg5bhf60vzg +NarSize: 109521136 +References: 2ma2k0ys8knh4an48n28vigcmc2z8773-linux-headers-2.6.23.16 ... +Deriver: 7akyyc87ka32xwmqza9dvyg5pwx3j212-glibc-2.7.drv +</screen> + +The fields are as follows: + +<variablelist> + + <varlistentry><term><literal>StorePath</literal></term> + + <listitem><para>The full store path, including the name part + (e.g., <literal>glibc-2.7</literal>). It must match the + requested store path.</para></listitem> + + </varlistentry> + + <varlistentry><term><literal>URL</literal></term> + + <listitem><para>The URL of the NAR, relative to the binary cache + URL.</para></listitem> + + </varlistentry> + + <varlistentry><term><literal>Compression</literal></term> + + <listitem><para>The compression method; either + <literal>xz</literal> or + <literal>bzip2</literal>.</para></listitem> + + </varlistentry> + + <varlistentry><term><literal>FileHash</literal></term> + + <listitem><para>The SHA-256 hash of the compressed + NAR.</para></listitem> + + </varlistentry> + + <varlistentry><term><literal>FileSize</literal></term> + + <listitem><para>The size of the compressed NAR.</para></listitem> + + </varlistentry> + + <varlistentry><term><literal>NarHash</literal></term> + + <listitem><para>The SHA-256 hash of the uncompressed NAR. This is + equal to the hash of the store path as returned by + <command>nix-store -q --hash + <replaceable>p</replaceable></command>.</para></listitem> + + </varlistentry> + + <varlistentry><term><literal>NarSize</literal></term> + + <listitem><para>The size of the uncompressed NAR.</para></listitem> + + </varlistentry> + + <varlistentry><term><literal>References</literal></term> + + <listitem><para>The references of the store path, without the Nix + store prefix.</para></listitem> + + </varlistentry> + + <varlistentry><term><literal>Deriver</literal></term> + + <listitem><para>The deriver of the store path, without the Nix + store prefix. This field is optional.</para></listitem> + + </varlistentry> + + <varlistentry><term><literal>System</literal></term> + + <listitem><para>The Nix platform type of this binary, if known. + This field is optional.</para></listitem> + + </varlistentry> + +</variablelist> + +</para> + +<para>Thus, in our example, after recursively ensuring that the +references exist (e.g., +<filename>/nix/store/2ma2k0ys8knh4an48n28vigcmc2z8773-linux-headers-2.6.23.16</filename>), +Nix will fetch <screen> +http://cache.nixos.org/nar/0zzjpdz46mdn74v09m053yczlz4am038g8r74iy8w43gx8801h70.nar.bz2 +</screen> and decompress and unpack it to +<filename>/nix/store/a8922c0h87iilxzzvwn2hmv8x210aqb9-glibc-2.7</filename>.</para> + +</refsection> + + +</refentry> diff --git a/doc/manual/nix-shell.xml b/doc/manual/nix-shell.xml new file mode 100644 index 000000000000..866b3413b967 --- /dev/null +++ b/doc/manual/nix-shell.xml @@ -0,0 +1,142 @@ +<refentry xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + xml:id="sec-nix-shell"> + +<refmeta> + <refentrytitle>nix-shell</refentrytitle> + <manvolnum>1</manvolnum> + <refmiscinfo class="source">Nix</refmiscinfo> + <refmiscinfo class="version"><xi:include href="version.txt" parse="text"/></refmiscinfo> +</refmeta> + +<refnamediv> + <refname>nix-shell</refname> + <refpurpose>start an interactive shell based on a Nix expression</refpurpose> +</refnamediv> + +<refsynopsisdiv> + <cmdsynopsis> + <command>nix-shell</command> + <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="opt-common-syn.xml#xmlns(db=http://docbook.org/ns/docbook)xpointer(/db:nop/*)" /> + <arg><option>--arg</option> <replaceable>name</replaceable> <replaceable>value</replaceable></arg> + <arg><option>--argstr</option> <replaceable>name</replaceable> <replaceable>value</replaceable></arg> + <arg> + <group choice='req'> + <arg choice='plain'><option>--attr</option></arg> + <arg choice='plain'><option>-A</option></arg> + </group> + <replaceable>attrPath</replaceable> + </arg> + <arg><option>--command</option> <replaceable>cmd</replaceable></arg> + <arg><option>--exclude</option> <replaceable>regexp</replaceable></arg> + <arg><option>--pure</option></arg> + <arg choice='plain' rep='repeat'><replaceable>paths</replaceable></arg> + </cmdsynopsis> +</refsynopsisdiv> + +<refsection><title>Description</title> + +<para>The command <command>nix-shell</command> will build +the dependencies of the specified derivation, but not the derivation +itself. It will then start an interactive shell in which all +environment variables defined by the derivation have been set to their +corresponding values, and the script <literal>$stdenv/setup</literal> +has been sourced. This is useful for reproducing the environment of a +derivation for development.</para> + +</refsection> + + +<refsection><title>Options</title> + +<para>All options not listed here are passed to <command>nix-store +--realise</command>, except for <option>--arg</option> and +<option>--attr</option> / <option>-A</option> which are passed to +<command>nix-instantiate</command>. <phrase condition="manual">See +also <xref linkend="sec-common-options" />.</phrase></para> + +<variablelist> + + <varlistentry><term><option>--command</option> <replaceable>cmd</replaceable></term> + + <listitem><para>In the environment of the derivation, run the + shell command <replaceable>cmd</replaceable> instead of starting + an interactive shell. However, if you end the shell command with + <literal>return</literal>, you still get an interactive shell. + This can be useful for doing any additional + initialisation.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--exclude</option> <replaceable>regexp</replaceable></term> + + <listitem><para>Do not build any dependencies whose store path + matches the regular expression <replaceable>regexp</replaceable>. + This option may be specified multiple times.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--pure</option></term> + + <listitem><para>If this flag is specified, the environment is + almost entirely cleared before the interactive shell is started, + so you get an environment that more closely corresponds to the + “real” Nix build. A few variables, in particular + <envar>HOME</envar>, <envar>USER</envar> and + <envar>DISPLAY</envar>, are retained. Note that + <filename>~/.bashrc</filename> and (depending on your Bash + installation) <filename>/etc/bashrc</filename> are still sourced, + so any variables set there will affect the interactive + shell.</para></listitem> + + </varlistentry> + +</variablelist> + +<para>The following common options are supported:</para> + +<variablelist condition="manpage"> + <xi:include href="opt-common.xml#xmlns(db=http://docbook.org/ns/docbook)xpointer(//db:variablelist[@xml:id='opt-common']/*)" /> +</variablelist> + +</refsection> + + +<refsection><title>Examples</title> + +<para>To build the dependencies of the package Pan, and start an +interactive shell in which to build it: + +<screen> +$ nix-shell '<nixpkgs>' -A pan +$ unpackPhase +$ cd pan-* +$ configurePhase +$ buildPhase +$ ./pan/gui/pan +</screen> + +To clear the environment first, and do some additional automatic +initialisation of the interactive shell: + +<screen> +$ nix-shell '<nixpkgs>' -A pan --pure \ + --command 'export NIX_DEBUG=1; export NIX_CORES=8; return' +</screen> + +</para> + +</refsection> + + +<refsection condition="manpage"><title>Environment variables</title> + +<variablelist> + <xi:include href="env-common.xml#xmlns(db=http://docbook.org/ns/docbook)xpointer(//db:variablelist[@xml:id='env-common']/*)" /> +</variablelist> + +</refsection> + + +</refentry> diff --git a/doc/manual/nix-store.xml b/doc/manual/nix-store.xml new file mode 100644 index 000000000000..63ddef28e1b9 --- /dev/null +++ b/doc/manual/nix-store.xml @@ -0,0 +1,1342 @@ +<refentry xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + xml:id="sec-nix-store"> + +<refmeta> + <refentrytitle>nix-store</refentrytitle> + <manvolnum>1</manvolnum> + <refmiscinfo class="source">Nix</refmiscinfo> + <refmiscinfo class="version"><xi:include href="version.txt" parse="text"/></refmiscinfo> +</refmeta> + +<refnamediv> + <refname>nix-store</refname> + <refpurpose>manipulate or query the Nix store</refpurpose> +</refnamediv> + +<refsynopsisdiv> + <cmdsynopsis> + <command>nix-store</command> + <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="opt-common-syn.xml#xmlns(db=http://docbook.org/ns/docbook)xpointer(/db:nop/*)" /> + <arg><option>--add-root</option> <replaceable>path</replaceable></arg> + <arg><option>--indirect</option></arg> + <arg choice='plain'><replaceable>operation</replaceable></arg> + <arg rep='repeat'><replaceable>options</replaceable></arg> + <arg rep='repeat'><replaceable>arguments</replaceable></arg> + </cmdsynopsis> +</refsynopsisdiv> + + +<refsection><title>Description</title> + +<para>The command <command>nix-store</command> performs primitive +operations on the Nix store. You generally do not need to run this +command manually.</para> + +<para><command>nix-store</command> takes exactly one +<emphasis>operation</emphasis> flag which indicates the subcommand to +be performed. These are documented below.</para> + +</refsection> + + + +<!--######################################################################--> + +<refsection><title>Common options</title> + +<para>This section lists the options that are common to all +operations. These options are allowed for every subcommand, though +they may not always have an effect. <phrase condition="manual">See +also <xref linkend="sec-common-options" /> for a list of common +options.</phrase></para> + +<variablelist> + + <varlistentry xml:id="opt-add-root"><term><option>--add-root</option> <replaceable>path</replaceable></term> + + <listitem><para>Causes the result of a realisation + (<option>--realise</option> and <option>--force-realise</option>) + to be registered as a root of the garbage collector<phrase + condition="manual"> (see <xref linkend="ssec-gc-roots" + />)</phrase>. The root is stored in + <replaceable>path</replaceable>, which must be inside a directory + that is scanned for roots by the garbage collector (i.e., + typically in a subdirectory of + <filename>/nix/var/nix/gcroots/</filename>) + <emphasis>unless</emphasis> the <option>--indirect</option> flag + is used.</para> + + <para>If there are multiple results, then multiple symlinks will + be created by sequentially numbering symlinks beyond the first one + (e.g., <filename>foo</filename>, <filename>foo-2</filename>, + <filename>foo-3</filename>, and so on).</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--indirect</option></term> + + <listitem> + + <para>In conjunction with <option>--add-root</option>, this option + allows roots to be stored <emphasis>outside</emphasis> of the GC + roots directory. This is useful for commands such as + <command>nix-build</command> that place a symlink to the build + result in the current directory; such a build result should not be + garbage-collected unless the symlink is removed.</para> + + <para>The <option>--indirect</option> flag causes a uniquely named + symlink to <replaceable>path</replaceable> to be stored in + <filename>/nix/var/nix/gcroots/auto/</filename>. For instance, + + <screen> +$ nix-store --add-root /home/eelco/bla/result --indirect -r <replaceable>...</replaceable> + +$ ls -l /nix/var/nix/gcroots/auto +lrwxrwxrwx 1 ... 2005-03-13 21:10 dn54lcypm8f8... -> /home/eelco/bla/result + +$ ls -l /home/eelco/bla/result +lrwxrwxrwx 1 ... 2005-03-13 21:10 /home/eelco/bla/result -> /nix/store/1r11343n6qd4...-f-spot-0.0.10</screen> + + Thus, when <filename>/home/eelco/bla/result</filename> is removed, + the GC root in the <filename>auto</filename> directory becomes a + dangling symlink and will be ignored by the collector.</para> + + <warning><para>Note that it is not possible to move or rename + indirect GC roots, since the symlink in the + <filename>auto</filename> directory will still point to the old + location.</para></warning> + + </listitem> + + </varlistentry> + +</variablelist> + +<variablelist condition="manpage"> + <xi:include href="opt-common.xml#xmlns(db=http://docbook.org/ns/docbook)xpointer(//db:variablelist[@xml:id='opt-common']/*)" /> +</variablelist> + +</refsection> + + + +<!--######################################################################--> + +<refsection xml:id='rsec-nix-store-realise'><title>Operation <option>--realise</option></title> + +<refsection><title>Synopsis</title> + +<cmdsynopsis> + <command>nix-store</command> + <group choice='req'> + <arg choice='plain'><option>--realise</option></arg> + <arg choice='plain'><option>-r</option></arg> + </group> + <arg choice='plain' rep='repeat'><replaceable>paths</replaceable></arg> + <arg><option>--dry-run</option></arg> +</cmdsynopsis> + +</refsection> + +<refsection><title>Description</title> + +<para>The operation <option>--realise</option> essentially “builds” +the specified store paths. Realisation is a somewhat overloaded term: + +<itemizedlist> + + <listitem><para>If the store path is a + <emphasis>derivation</emphasis>, realisation ensures that the output + paths of the derivation are <link + linkend="gloss-validity">valid</link> (i.e., the output path and its + closure exist in the file system). This can be done in several + ways. First, it is possible that the outputs are already valid, in + which case we are done immediately. Otherwise, there may be <link + linkend="gloss-substitute">substitutes</link> that produce the + outputs (e.g., by downloading them). Finally, the outputs can be + produced by performing the build action described by the + derivation.</para></listitem> + + <listitem><para>If the store path is not a derivation, realisation + ensures that the specified path is valid (i.e., it and its closure + exist in the file system). If the path is already valid, we are + done immediately. Otherwise, the path and any missing paths in its + closure may be produced through substitutes. If there are no + (successful) subsitutes, realisation fails.</para></listitem> + +</itemizedlist> + +</para> + +<para>The output path of each derivation is printed on standard +output. (For non-derivations argument, the argument itself is +printed.)</para> + +<para>The following flags are available:</para> + +<variablelist> + + <varlistentry><term><option>--dry-run</option></term> + + <listitem><para>Print on standard error a description of what + packages would be built or downloaded, without actually performing + the operation.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--ignore-unknown</option></term> + + <listitem><para>If a non-derivation path does not have a + substitute, then silently ignore it.</para></listitem> + + </varlistentry> + +</variablelist> + +</refsection> + + +<refsection><title>Examples</title> + +<para>This operation is typically used to build store derivations +produced by <link +linkend="sec-nix-instantiate"><command>nix-instantiate</command></link>: + +<screen> +$ nix-store -r $(nix-instantiate ./test.nix) +/nix/store/31axcgrlbfsxzmfff1gyj1bf62hvkby2-aterm-2.3.1</screen> + +This is essentially what <link +linkend="sec-nix-build"><command>nix-build</command></link> does.</para> + +</refsection> + + +</refsection> + + + +<!--######################################################################--> + +<refsection xml:id='rsec-nix-store-gc'><title>Operation <option>--gc</option></title> + +<refsection><title>Synopsis</title> + +<cmdsynopsis> + <command>nix-store</command> + <arg choice='plain'><option>--gc</option></arg> + <group> + <arg choice='plain'><option>--print-roots</option></arg> + <arg choice='plain'><option>--print-live</option></arg> + <arg choice='plain'><option>--print-dead</option></arg> + <arg choice='plain'><option>--delete</option></arg> + </group> + <arg><option>--max-freed</option> <replaceable>bytes</replaceable></arg> +</cmdsynopsis> + +</refsection> + +<refsection><title>Description</title> + +<para>Without additional flags, the operation <option>--gc</option> +performs a garbage collection on the Nix store. That is, all paths in +the Nix store not reachable via file system references from a set of +“roots”, are deleted.</para> + +<para>The following suboperations may be specified:</para> + +<variablelist> + + <varlistentry><term><option>--print-roots</option></term> + + <listitem><para>This operation prints on standard output the set + of roots used by the garbage collector. What constitutes a root + is described in <xref linkend="ssec-gc-roots" + />.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--print-live</option></term> + + <listitem><para>This operation prints on standard output the set + of “live” store paths, which are all the store paths reachable + from the roots. Live paths should never be deleted, since that + would break consistency — it would become possible that + applications are installed that reference things that are no + longer present in the store.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--print-dead</option></term> + + <listitem><para>This operation prints out on standard output the + set of “dead” store paths, which is just the opposite of the set + of live paths: any path in the store that is not live (with + respect to the roots) is dead.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--delete</option></term> + + <listitem><para>This operation performs an actual garbage + collection. All dead paths are removed from the + store. This is the default.</para></listitem> + + </varlistentry> + +</variablelist> + +<para>By default, all unreachable paths are deleted. The following +options control what gets deleted and in what order: + +<variablelist> + + <varlistentry><term><option>--max-freed</option> <replaceable>bytes</replaceable></term> + + <listitem><para>Keep deleting paths until at least + <replaceable>bytes</replaceable> bytes have been + deleted, then stop.</para></listitem> + + </varlistentry> + +</variablelist> + +</para> + +<para>The behaviour of the collector is also influenced by the <link +linkend="conf-gc-keep-outputs"><literal>gc-keep-outputs</literal></link> +and <link +linkend="conf-gc-keep-derivations"><literal>gc-keep-derivations</literal></link> +variables in the Nix configuration file.</para> + +<para>With <option>--delete</option>, the collector prints the total +number of freed bytes when it finishes (or when it is interrupted). +With <option>--print-dead</option>, it prints the number of bytes that +would be freed.</para> + +</refsection> + + +<refsection><title>Examples</title> + +<para>To delete all unreachable paths, just do: + +<screen> +$ nix-store --gc +deleting `/nix/store/kq82idx6g0nyzsp2s14gfsc38npai7lf-cairo-1.0.4.tar.gz.drv' +<replaceable>...</replaceable> +8825586 bytes freed (8.42 MiB)</screen> + +</para> + +<para>To delete at least 100 MiBs of unreachable paths: + +<screen> +$ nix-store --gc --max-freed $((100 * 1024 * 1024))</screen> + +</para> + +</refsection> + + +</refsection> + + + +<!--######################################################################--> + +<refsection><title>Operation <option>--delete</option></title> + +<refsection><title>Synopsis</title> + +<cmdsynopsis> + <command>nix-store</command> + <arg choice='plain'><option>--delete</option></arg> + <arg><option>--ignore-liveness</option></arg> + <arg choice='plain' rep='repeat'><replaceable>paths</replaceable></arg> +</cmdsynopsis> + +</refsection> + +<refsection><title>Description</title> + +<para>The operation <option>--delete</option> deletes the store paths +<replaceable>paths</replaceable> from the Nix store, but only if it is +safe to do so; that is, when the path is not reachable from a root of +the garbage collector. This means that you can only delete paths that +would also be deleted by <literal>nix-store --gc</literal>. Thus, +<literal>--delete</literal> is a more targeted version of +<literal>--gc</literal>.</para> + +<para>With the option <option>--ignore-liveness</option>, reachability +from the roots is ignored. However, the path still won’t be deleted +if there are other paths in the store that refer to it (i.e., depend +on it).</para> + +</refsection> + +<refsection><title>Example</title> + +<screen> +$ nix-store --delete /nix/store/zq0h41l75vlb4z45kzgjjmsjxvcv1qk7-mesa-6.4 +0 bytes freed (0.00 MiB) +error: cannot delete path `/nix/store/zq0h41l75vlb4z45kzgjjmsjxvcv1qk7-mesa-6.4' since it is still alive</screen> + +</refsection> + +</refsection> + + + +<!--######################################################################--> + +<refsection xml:id='refsec-nix-store-query'><title>Operation <option>--query</option></title> + +<refsection><title>Synopsis</title> + +<cmdsynopsis> + <command>nix-store</command> + <group choice='req'> + <arg choice='plain'><option>--query</option></arg> + <arg choice='plain'><option>-q</option></arg> + </group> + <group choice='req'> + <arg choice='plain'><option>--outputs</option></arg> + <arg choice='plain'><option>--requisites</option></arg> + <arg choice='plain'><option>-R</option></arg> + <arg choice='plain'><option>--references</option></arg> + <arg choice='plain'><option>--referrers</option></arg> + <arg choice='plain'><option>--referrers-closure</option></arg> + <arg choice='plain'><option>--deriver</option></arg> + <arg choice='plain'><option>--deriver</option></arg> + <arg choice='plain'><option>--graph</option></arg> + <arg choice='plain'><option>--tree</option></arg> + <arg choice='plain'><option>--binding</option> <replaceable>name</replaceable></arg> + <arg choice='plain'><option>--hash</option></arg> + <arg choice='plain'><option>--size</option></arg> + <arg choice='plain'><option>--roots</option></arg> + </group> + <arg><option>--use-output</option></arg> + <arg><option>-u</option></arg> + <arg><option>--force-realise</option></arg> + <arg><option>-f</option></arg> + <arg choice='plain' rep='repeat'><replaceable>paths</replaceable></arg> +</cmdsynopsis> + +</refsection> + + +<refsection><title>Description</title> + +<para>The operation <option>--query</option> displays various bits of +information about the store paths . The queries are described below. At +most one query can be specified. The default query is +<option>--outputs</option>.</para> + +<para>The paths <replaceable>paths</replaceable> may also be symlinks +from outside of the Nix store, to the Nix store. In that case, the +query is applied to the target of the symlink.</para> + + +</refsection> + + +<refsection><title>Common query options</title> + +<variablelist> + + <varlistentry><term><option>--use-output</option></term> + <term><option>-u</option></term> + + <listitem><para>For each argument to the query that is a store + derivation, apply the query to the output path of the derivation + instead.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--force-realise</option></term> + <term><option>-f</option></term> + + <listitem><para>Realise each argument to the query first (see + <link linkend="rsec-nix-store-realise"><command>nix-store + --realise</command></link>).</para></listitem> + + </varlistentry> + +</variablelist> + +</refsection> + + +<refsection xml:id='nixref-queries'><title>Queries</title> + +<variablelist> + + <varlistentry><term><option>--outputs</option></term> + + <listitem><para>Prints out the <link + linkend="gloss-output-path">output paths</link> of the store + derivations <replaceable>paths</replaceable>. These are the paths + that will be produced when the derivation is + built.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--requisites</option></term> + <term><option>-R</option></term> + + <listitem><para>Prints out the <link + linkend="gloss-closure">closure</link> of the store path + <replaceable>paths</replaceable>.</para> + + <para>This query has one option:</para> + + <variablelist> + + <varlistentry><term><option>--include-outputs</option></term> + + <listitem><para>Also include the output path of store + derivations, and their closures.</para></listitem> + + </varlistentry> + + </variablelist> + + <para>This query can be used to implement various kinds of + deployment. A <emphasis>source deployment</emphasis> is obtained + by distributing the closure of a store derivation. A + <emphasis>binary deployment</emphasis> is obtained by distributing + the closure of an output path. A <emphasis>cache + deployment</emphasis> (combined source/binary deployment, + including binaries of build-time-only dependencies) is obtained by + distributing the closure of a store derivation and specifying the + option <option>--include-outputs</option>.</para> + + </listitem> + + </varlistentry> + + <varlistentry><term><option>--references</option></term> + + <listitem><para>Prints the set of <link + linkend="gloss-reference">references</link> of the store paths + <replaceable>paths</replaceable>, that is, their immediate + dependencies. (For <emphasis>all</emphasis> dependencies, use + <option>--requisites</option>.)</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--referrers</option></term> + + <listitem><para>Prints the set of <emphasis>referrers</emphasis> of + the store paths <replaceable>paths</replaceable>, that is, the + store paths currently existing in the Nix store that refer to one + of <replaceable>paths</replaceable>. Note that contrary to the + references, the set of referrers is not constant; it can change as + store paths are added or removed.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--referrers-closure</option></term> + + <listitem><para>Prints the closure of the set of store paths + <replaceable>paths</replaceable> under the referrers relation; that + is, all store paths that directly or indirectly refer to one of + <replaceable>paths</replaceable>. These are all the path currently + in the Nix store that are dependent on + <replaceable>paths</replaceable>.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--deriver</option></term> + + <listitem><para>Prints the <link + linkend="gloss-deriver">deriver</link> of the store paths + <replaceable>paths</replaceable>. If the path has no deriver + (e.g., if it is a source file), or if the deriver is not known + (e.g., in the case of a binary-only deployment), the string + <literal>unknown-deriver</literal> is printed.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--graph</option></term> + + <listitem><para>Prints the references graph of the store paths + <replaceable>paths</replaceable> in the format of the + <command>dot</command> tool of AT&T's <link + xlink:href="http://www.graphviz.org/">Graphviz package</link>. + This can be used to visualise dependency graphs. To obtain a + build-time dependency graph, apply this to a store derivation. To + obtain a runtime dependency graph, apply it to an output + path.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--tree</option></term> + + <listitem><para>Prints the references graph of the store paths + <replaceable>paths</replaceable> as a nested ASCII tree. + References are ordered by descending closure size; this tends to + flatten the tree, making it more readable. The query only + recurses into a store path when it is first encountered; this + prevents a blowup of the tree representation of the + graph.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--binding</option> <replaceable>name</replaceable></term> + + <listitem><para>Prints the value of the attribute + <replaceable>name</replaceable> (i.e., environment variable) of + the store derivations <replaceable>paths</replaceable>. It is an + error for a derivation to not have the specified + attribute.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--hash</option></term> + + <listitem><para>Prints the SHA-256 hash of the contents of the + store paths <replaceable>paths</replaceable> (that is, the hash of + the output of <command>nix-store --dump</command> on the given + paths). Since the hash is stored in the Nix database, this is a + fast operation.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--size</option></term> + + <listitem><para>Prints the size in bytes of the contents of the + store paths <replaceable>paths</replaceable> — to be precise, the + size of the output of <command>nix-store --dump</command> on the + given paths. Note that the actual disk space required by the + store paths may be higher, especially on filesystems with large + cluster sizes.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--roots</option></term> + + <listitem><para>Prints the garbage collector roots that point, + directly or indirectly, at the store paths + <replaceable>paths</replaceable>.</para></listitem> + + </varlistentry> + +</variablelist> + +</refsection> + + +<refsection><title>Examples</title> + +<para>Print the closure (runtime dependencies) of the +<command>svn</command> program in the current user environment: + +<screen> +$ nix-store -qR $(which svn) +/nix/store/5mbglq5ldqld8sj57273aljwkfvj22mc-subversion-1.1.4 +/nix/store/9lz9yc6zgmc0vlqmn2ipcpkjlmbi51vv-glibc-2.3.4 +<replaceable>...</replaceable></screen> + +</para> + +<para>Print the build-time dependencies of <command>svn</command>: + +<screen> +$ nix-store -qR $(nix-store -qd $(which svn)) +/nix/store/02iizgn86m42q905rddvg4ja975bk2i4-grep-2.5.1.tar.bz2.drv +/nix/store/07a2bzxmzwz5hp58nf03pahrv2ygwgs3-gcc-wrapper.sh +/nix/store/0ma7c9wsbaxahwwl04gbw3fcd806ski4-glibc-2.3.4.drv +<replaceable>... lots of other paths ...</replaceable></screen> + +The difference with the previous example is that we ask the closure of +the derivation (<option>-qd</option>), not the closure of the output +path that contains <command>svn</command>.</para> + +<para>Show the build-time dependencies as a tree: + +<screen> +$ nix-store -q --tree $(nix-store -qd $(which svn)) +/nix/store/7i5082kfb6yjbqdbiwdhhza0am2xvh6c-subversion-1.1.4.drv ++---/nix/store/d8afh10z72n8l1cr5w42366abiblgn54-builder.sh ++---/nix/store/fmzxmpjx2lh849ph0l36snfj9zdibw67-bash-3.0.drv +| +---/nix/store/570hmhmx3v57605cqg9yfvvyh0nnb8k8-bash +| +---/nix/store/p3srsbd8dx44v2pg6nbnszab5mcwx03v-builder.sh +<replaceable>...</replaceable></screen> + +</para> + +<para>Show all paths that depend on the same OpenSSL library as +<command>svn</command>: + +<screen> +$ nix-store -q --referrers $(nix-store -q --binding openssl $(nix-store -qd $(which svn))) +/nix/store/23ny9l9wixx21632y2wi4p585qhva1q8-sylpheed-1.0.0 +/nix/store/5mbglq5ldqld8sj57273aljwkfvj22mc-subversion-1.1.4 +/nix/store/dpmvp969yhdqs7lm2r1a3gng7pyq6vy4-subversion-1.1.3 +/nix/store/l51240xqsgg8a7yrbqdx1rfzyv6l26fx-lynx-2.8.5</screen> + +</para> + +<para>Show all paths that directly or indirectly depend on the Glibc +(C library) used by <command>svn</command>: + +<screen> +$ nix-store -q --referrers-closure $(ldd $(which svn) | grep /libc.so | awk '{print $3}') +/nix/store/034a6h4vpz9kds5r6kzb9lhh81mscw43-libgnomeprintui-2.8.2 +/nix/store/15l3yi0d45prm7a82pcrknxdh6nzmxza-gawk-3.1.4 +<replaceable>...</replaceable></screen> + +Note that <command>ldd</command> is a command that prints out the +dynamic libraries used by an ELF executable.</para> + +<para>Make a picture of the runtime dependency graph of the current +user environment: + +<screen> +$ nix-store -q --graph ~/.nix-profile | dot -Tps > graph.ps +$ gv graph.ps</screen> + +</para> + +<para>Show every garbage collector root that points to a store path +that depends on <command>svn</command>: + +<screen> +$ nix-store -q --roots $(which svn) +/nix/var/nix/profiles/default-81-link +/nix/var/nix/profiles/default-82-link +/nix/var/nix/profiles/per-user/eelco/profile-97-link +</screen> + +</para> + +</refsection> + + +</refsection> + + + +<!--######################################################################--> + +<!-- +<refsection xml:id="rsec-nix-store-reg-val"><title>Operation <option>-XXX-register-validity</option></title> + +<refsection><title>Synopsis</title> + +<cmdsynopsis> + <command>nix-store</command> + <arg choice='plain'><option>-XXX-register-validity</option></arg> +</cmdsynopsis> +</refsection> + +<refsection><title>Description</title> + +<para>TODO</para> + +</refsection> + +</refsection> +--> + + + +<!--######################################################################--> + +<refsection><title>Operation <option>--add</option></title> + +<refsection><title>Synopsis</title> + +<cmdsynopsis> + <command>nix-store</command> + <arg choice='plain'><option>--add</option></arg> + <arg choice='plain' rep='repeat'><replaceable>paths</replaceable></arg> +</cmdsynopsis> + +</refsection> + +<refsection><title>Description</title> + +<para>The operation <option>--add</option> adds the specified paths to +the Nix store. It prints the resulting paths in the Nix store on +standard output.</para> + +</refsection> + +<refsection><title>Example</title> + +<screen> +$ nix-store --add ./foo.c +/nix/store/m7lrha58ph6rcnv109yzx1nk1cj7k7zf-foo.c</screen> + +</refsection> + +</refsection> + + + +<!--######################################################################--> + +<refsection xml:id='refsec-nix-store-verify'><title>Operation <option>--verify</option></title> + +<refsection> + <title>Synopsis</title> + <cmdsynopsis> + <command>nix-store</command> + <arg choice='plain'><option>--verify</option></arg> + <arg><option>--check-contents</option></arg> + <arg><option>--repair</option></arg> + </cmdsynopsis> +</refsection> + +<refsection><title>Description</title> + +<para>The operation <option>--verify</option> verifies the internal +consistency of the Nix database, and the consistency between the Nix +database and the Nix store. Any inconsistencies encountered are +automatically repaired. Inconsistencies are generally the result of +the Nix store or database being modified by non-Nix tools, or of bugs +in Nix itself.</para> + +<para>This operation has the following options: + +<variablelist> + + <varlistentry><term><option>--check-contents</option></term> + + <listitem><para>Checks that the contents of every valid store path + has not been altered by computing a SHA-256 hash of the contents + and comparing it with the hash stored in the Nix database at build + time. Paths that have been modified are printed out. For large + stores, <option>--check-contents</option> is obviously quite + slow.</para></listitem> + + </varlistentry> + + <varlistentry><term><option>--repair</option></term> + + <listitem><para>If any valid path is missing from the store, or + (if <option>--check-contents</option> is given) the contents of a + valid path has been modified, then try to repair the path by + redownloading it. See <command>nix-store --repair-path</command> + for details.</para></listitem> + + </varlistentry> + +</variablelist> + +</para> + +</refsection> + + +</refsection> + + +<!--######################################################################--> + +<refsection><title>Operation <option>--verify-path</option></title> + +<refsection> + <title>Synopsis</title> + <cmdsynopsis> + <command>nix-store</command> + <arg choice='plain'><option>--verify-path</option></arg> + <arg choice='plain' rep='repeat'><replaceable>paths</replaceable></arg> + </cmdsynopsis> +</refsection> + +<refsection><title>Description</title> + +<para>The operation <option>--verify-path</option> compares the +contents of the given store paths to their cryptographic hashes stored +in Nix’s database. For every changed path, it prints a warning +message. The exit status is 0 if no path has changed, and 1 +otherwise.</para> + +</refsection> + +<refsection><title>Example</title> + +<para>To verify the integrity of the <command>svn</command> command and all its dependencies: + +<screen> +$ nix-store --verify-path $(nix-store -qR $(which svn)) +</screen> + +</para> + +</refsection> + +</refsection> + + +<!--######################################################################--> + +<refsection><title>Operation <option>--repair-path</option></title> + +<refsection> + <title>Synopsis</title> + <cmdsynopsis> + <command>nix-store</command> + <arg choice='plain'><option>--repair-path</option></arg> + <arg choice='plain' rep='repeat'><replaceable>paths</replaceable></arg> + </cmdsynopsis> +</refsection> + +<refsection><title>Description</title> + +<para>The operation <option>--repair-path</option> attempts to +“repair” the specified paths by redownloading them using the available +substituters. If no substitutes are available, then repair is not +possible.</para> + +<warning><para>During repair, there is a very small time window during +which the old path (if it exists) is moved out of the way and replaced +with the new path. If repair is interrupted in between, then the +system may be left in a broken state (e.g., if the path contains a +critical system component like the GNU C Library).</para></warning> + +</refsection> + +<refsection><title>Example</title> + +<screen> +$ nix-store --verify-path /nix/store/dj7a81wsm1ijwwpkks3725661h3263p5-glibc-2.13 +path `/nix/store/dj7a81wsm1ijwwpkks3725661h3263p5-glibc-2.13' was modified! + expected hash `2db57715ae90b7e31ff1f2ecb8c12ec1cc43da920efcbe3b22763f36a1861588', + got `481c5aa5483ebc97c20457bb8bca24deea56550d3985cda0027f67fe54b808e4' + +$ nix-store --repair-path /nix/store/dj7a81wsm1ijwwpkks3725661h3263p5-glibc-2.13 +fetching path `/nix/store/d7a81wsm1ijwwpkks3725661h3263p5-glibc-2.13'... +… +</screen> + +</refsection> + +</refsection> + + +<!--######################################################################--> + +<refsection xml:id='refsec-nix-store-dump'><title>Operation <option>--dump</option></title> + +<refsection> + <title>Synopsis</title> + <cmdsynopsis> + <command>nix-store</command> + <arg choice='plain'><option>--dump</option></arg> + <arg choice='plain'><replaceable>path</replaceable></arg> + </cmdsynopsis> +</refsection> + +<refsection><title>Description</title> + +<para>The operation <option>--dump</option> produces a NAR (Nix +ARchive) file containing the contents of the file system tree rooted +at <replaceable>path</replaceable>. The archive is written to +standard output.</para> + +<para>A NAR archive is like a TAR or Zip archive, but it contains only +the information that Nix considers important. For instance, +timestamps are elided because all files in the Nix store have their +timestamp set to 0 anyway. Likewise, all permissions are left out +except for the execute bit, because all files in the Nix store have +644 or 755 permission.</para> + +<para>Also, a NAR archive is <emphasis>canonical</emphasis>, meaning +that “equal” paths always produce the same NAR archive. For instance, +directory entries are always sorted so that the actual on-disk order +doesn’t influence the result. This means that the cryptographic hash +of a NAR dump of a path is usable as a fingerprint of the contents of +the path. Indeed, the hashes of store paths stored in Nix’s database +(see <link linkend="refsec-nix-store-query"><literal>nix-store -q +--hash</literal></link>) are SHA-256 hashes of the NAR dump of each +store path.</para> + +<para>NAR archives support filenames of unlimited length and 64-bit +file sizes. They can contain regular files, directories, and symbolic +links, but not other types of files (such as device nodes).</para> + +<para>A Nix archive can be unpacked using <literal>nix-store +--restore</literal>.</para> + +</refsection> + + +</refsection> + + +<!--######################################################################--> + +<refsection><title>Operation <option>--restore</option></title> + +<refsection> + <title>Synopsis</title> + <cmdsynopsis> + <command>nix-store</command> + <arg choice='plain'><option>--restore</option></arg> + <arg choice='plain'><replaceable>path</replaceable></arg> + </cmdsynopsis> +</refsection> + +<refsection><title>Description</title> + +<para>The operation <option>--restore</option> unpacks a NAR archive +to <replaceable>path</replaceable>, which must not already exist. The +archive is read from standard input.</para> + +</refsection> + + +</refsection> + + +<!--######################################################################--> + +<refsection xml:id='refsec-nix-store-export'><title>Operation <option>--export</option></title> + +<refsection> + <title>Synopsis</title> + <cmdsynopsis> + <command>nix-store</command> + <arg choice='plain'><option>--export</option></arg> + <arg choice='plain' rep='repeat'><replaceable>paths</replaceable></arg> + </cmdsynopsis> +</refsection> + +<refsection><title>Description</title> + +<para>The operation <option>--export</option> writes a serialisation +of the specified store paths to standard output in a format that can +be imported into another Nix store with <command +linkend="refsec-nix-store-import">nix-store --import</command>. This +is like <command linkend="refsec-nix-store-dump">nix-store +--dump</command>, except that the NAR archive produced by that command +doesn’t contain the necessary meta-information to allow it to be +imported into another Nix store (namely, the set of references of the +path).</para> + +<para>This command does not produce a <emphasis>closure</emphasis> of +the specified paths, so if a store path references other store paths +that are missing in the target Nix store, the import will fail. To +copy a whole closure, do something like + +<screen> +$ nix-store --export $(nix-store -qR <replaceable>paths</replaceable>) > out</screen> + +</para> + +<para>For an example of how <option>--export</option> and +<option>--import</option> can be used, see the source of the <command +linkend="sec-nix-copy-closure">nix-copy-closure</command> +command.</para> + +</refsection> + + +</refsection> + + +<!--######################################################################--> + +<refsection xml:id='refsec-nix-store-import'><title>Operation <option>--import</option></title> + +<refsection> + <title>Synopsis</title> + <cmdsynopsis> + <command>nix-store</command> + <arg choice='plain'><option>--import</option></arg> + </cmdsynopsis> +</refsection> + +<refsection><title>Description</title> + +<para>The operation <option>--export</option> reads a serialisation of +a set of store paths produced by <command +linkend="refsec-nix-store-export">nix-store --import</command> from +standard input and adds those store paths to the Nix store. Paths +that already exist in the Nix store are ignored. If a path refers to +another path that doesn’t exist in the Nix store, the import +fails.</para> + +</refsection> + + +</refsection> + + +<!--######################################################################--> + +<refsection><title>Operation <option>--optimise</option></title> + +<refsection> + <title>Synopsis</title> + <cmdsynopsis> + <command>nix-store</command> + <arg choice='plain'><option>--optimise</option></arg> + </cmdsynopsis> +</refsection> + +<refsection><title>Description</title> + +<para>The operation <option>--optimise</option> reduces Nix store disk +space usage by finding identical files in the store and hard-linking +them to each other. It typically reduces the size of the store by +something like 25-35%. Only regular files and symlinks are +hard-linked in this manner. Files are considered identical when they +have the same NAR archive serialisation: that is, regular files must +have the same contents and permission (executable or non-executable), +and symlinks must have the same contents.</para> + +<para>After completion, or when the command is interrupted, a report +on the achieved savings is printed on standard error.</para> + +<para>Use <option>-vv</option> or <option>-vvv</option> to get some +progress indication.</para> + +</refsection> + +<refsection><title>Example</title> + +<screen> +$ nix-store --optimise +hashing files in `/nix/store/qhqx7l2f1kmwihc9bnxs7rc159hsxnf3-gcc-4.1.1' +<replaceable>...</replaceable> +541838819 bytes (516.74 MiB) freed by hard-linking 54143 files; +there are 114486 files with equal contents out of 215894 files in total +</screen> + +</refsection> + + +</refsection> + + +<!--######################################################################--> + +<refsection><title>Operation <option>--read-log</option></title> + +<refsection> + <title>Synopsis</title> + <cmdsynopsis> + <command>nix-store</command> + <group choice='req'> + <arg choice='plain'><option>--read-log</option></arg> + <arg choice='plain'><option>-l</option></arg> + </group> + <arg choice='plain' rep='repeat'><replaceable>paths</replaceable></arg> + </cmdsynopsis> +</refsection> + +<refsection><title>Description</title> + +<para>The operation <option>--read-log</option> prints the build log +of the specified store paths on standard output. The build log is +whatever the builder of a derivation wrote to standard output and +standard error. If a store path is not a derivation, the deriver of +the store path is used.</para> + +<para>Build logs are kept in +<filename>/nix/var/log/nix/drvs</filename>. However, there is no +guarantee that a build log is available for any particular store +path. For instance, if the path was downloaded as a pre-built binary +through a substitute, then the log is unavailable.</para> + +</refsection> + +<refsection><title>Example</title> + +<screen> +$ nix-store -l $(which ktorrent) +building /nix/store/dhc73pvzpnzxhdgpimsd9sw39di66ph1-ktorrent-2.2.1 +unpacking sources +unpacking source archive /nix/store/p8n1jpqs27mgkjw07pb5269717nzf5f8-ktorrent-2.2.1.tar.gz +ktorrent-2.2.1/ +ktorrent-2.2.1/NEWS +<replaceable>...</replaceable> +</screen> + +</refsection> + + +</refsection> + + +<!--######################################################################--> + +<refsection><title>Operation <option>--dump-db</option></title> + +<refsection> + <title>Synopsis</title> + <cmdsynopsis> + <command>nix-store</command> + <arg choice='plain'><option>--dump-db</option></arg> + </cmdsynopsis> +</refsection> + +<refsection><title>Description</title> + +<para>The operation <option>--dump-db</option> writes a dump of the +Nix database to standard output. It can be loaded into an empty Nix +store using <option>--load-db</option>. This is useful for making +backups and when migrating to different database schemas.</para> + +</refsection> + +</refsection> + + +<!--######################################################################--> + +<refsection><title>Operation <option>--load-db</option></title> + +<refsection> + <title>Synopsis</title> + <cmdsynopsis> + <command>nix-store</command> + <arg choice='plain'><option>--load-db</option></arg> + </cmdsynopsis> +</refsection> + +<refsection><title>Description</title> + +<para>The operation <option>--load-db</option> reads a dump of the Nix +database created by <option>--dump-db</option> from standard input and +loads it into the Nix database.</para> + +</refsection> + +</refsection> + + +<!--######################################################################--> + +<refsection><title>Operation <option>--print-env</option></title> + +<refsection> + <title>Synopsis</title> + <cmdsynopsis> + <command>nix-store</command> + <arg choice='plain'><option>--print-env</option></arg> + <arg choice='plain'><replaceable>drvpath</replaceable></arg> + </cmdsynopsis> +</refsection> + +<refsection><title>Description</title> + +<para>The operation <option>--print-env</option> prints out the +environment of a derivation in a format that can be evaluated by a +shell. The command line arguments of the builder are placed in the +variable <envar>_args</envar>.</para> + +</refsection> + +<refsection><title>Example</title> + +<screen> +$ nix-store --print-env $(nix-instantiate '<nixpkgs>' -A firefox) +<replaceable>…</replaceable> +export src; src='/nix/store/plpj7qrwcz94z2psh6fchsi7s8yihc7k-firefox-12.0.source.tar.bz2' +export stdenv; stdenv='/nix/store/7c8asx3yfrg5dg1gzhzyq2236zfgibnn-stdenv' +export system; system='x86_64-linux' +export _args; _args='-e /nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25c-default-builder.sh' +</screen> + +</refsection> + +</refsection> + + +<!--######################################################################--> + +<refsection><title>Operation <option>--query-failed-paths</option></title> + +<refsection> + <title>Synopsis</title> + <cmdsynopsis> + <command>nix-store</command> + <arg choice='plain'><option>--query-failed-paths</option></arg> + </cmdsynopsis> +</refsection> + +<refsection><title>Description</title> + +<para>If build failure caching is enabled through the +<literal>build-cache-failures</literal> configuration option, the +operation <option>--query-failed-paths</option> will print out all +store paths that have failed to build.</para> + +</refsection> + +<refsection><title>Example</title> + +<screen> +$ nix-store --query-failed-paths +/nix/store/000zi5dcla86l92jn1g997jb06sidm7x-perl-PerlMagick-6.59 +/nix/store/0011iy7sfwbc1qj5a1f6ifjnbcdail8a-haskell-gitit-ghc7.0.4-0.8.1 +/nix/store/001c0yn1hkh86gprvrb46cxnz3pki7q3-gamin-0.1.10 +<replaceable>…</replaceable> +</screen> + +</refsection> + +</refsection> + + +<!--######################################################################--> + +<refsection><title>Operation <option>--clear-failed-paths</option></title> + +<refsection> + <title>Synopsis</title> + <cmdsynopsis> + <command>nix-store</command> + <arg choice='plain'><option>--clear-failed-paths</option></arg> + <arg choice='plain' rep='repeat'><replaceable>paths</replaceable></arg> + </cmdsynopsis> +</refsection> + +<refsection><title>Description</title> + +<para>If build failure caching is enabled through the +<literal>build-cache-failures</literal> configuration option, the +operation <option>--clear-failed-paths</option> clears the “failed” +state of the given store paths, allowing them to be built again. This +is useful if the failure was actually transient (e.g. because the disk +was full).</para> + +<para>If a path denotes a derivation, its output paths are cleared. +You can provide the argument <literal>*</literal> to clear all store +paths.</para> + +</refsection> + +<refsection><title>Example</title> + +<screen> +$ nix-store --clear-failed-paths /nix/store/000zi5dcla86l92jn1g997jb06sidm7x-perl-PerlMagick-6.59 +$ nix-store --clear-failed-paths * +</screen> + +</refsection> + +</refsection> + + +<!--######################################################################--> + +<refsection condition="manpage"><title>Environment variables</title> + +<variablelist> + <xi:include href="env-common.xml#xmlns(db=http://docbook.org/ns/docbook)xpointer(//db:variablelist[@xml:id='env-common']/*)" /> +</variablelist> + +</refsection> + + +</refentry> diff --git a/doc/manual/opt-common-syn.xml b/doc/manual/opt-common-syn.xml new file mode 100644 index 000000000000..d65f4009ee6e --- /dev/null +++ b/doc/manual/opt-common-syn.xml @@ -0,0 +1,47 @@ +<nop xmlns="http://docbook.org/ns/docbook"> + +<arg><option>--help</option></arg> +<arg><option>--version</option></arg> +<arg rep='repeat'><option>--verbose</option></arg> +<arg rep='repeat'><option>-v</option></arg> +<arg><option>--no-build-output</option></arg> +<arg><option>-Q</option></arg> +<arg> + <group choice='req'> + <arg choice='plain'><option>--max-jobs</option></arg> + <arg choice='plain'><option>-j</option></arg> + </group> + <replaceable>number</replaceable> +</arg> +<arg> + <option>--cores</option> + <replaceable>number</replaceable> +</arg> +<arg> + <option>--max-silent-time</option> + <replaceable>number</replaceable> +</arg> +<arg> + <option>--timeout</option> + <replaceable>number</replaceable> +</arg> +<arg><option>--keep-going</option></arg> +<arg><option>-k</option></arg> +<arg><option>--keep-failed</option></arg> +<arg><option>-K</option></arg> +<arg><option>--fallback</option></arg> +<arg><option>--readonly-mode</option></arg> +<arg><option>--log-type</option> <replaceable>type</replaceable></arg> +<arg><option>--show-trace</option></arg> +<arg> + <option>-I</option> + <replaceable>path</replaceable> +</arg> +<arg> + <option>--option</option> + <replaceable>name</replaceable> + <replaceable>value</replaceable> +</arg> +<sbr /> + +</nop> diff --git a/doc/manual/opt-common.xml b/doc/manual/opt-common.xml new file mode 100644 index 000000000000..a877b536a5c7 --- /dev/null +++ b/doc/manual/opt-common.xml @@ -0,0 +1,379 @@ +<section xmlns="http://docbook.org/ns/docbook" xml:id="sec-common-options"> + +<title>Common options</title> + + +<para>Most Nix commands accept the following command-line options:</para> + +<variablelist xml:id="opt-common"> + +<varlistentry><term><option>--help</option></term> + + <listitem><para>Prints out a summary of the command syntax and + exits.</para></listitem> + +</varlistentry> + + +<varlistentry><term><option>--version</option></term> + + <listitem><para>Prints out the Nix version number on standard output + and exits.</para></listitem> +</varlistentry> + + +<varlistentry><term><option>--verbose</option></term> + <term><option>-v</option></term> + + <listitem> + + <para>Increases the level of verbosity of diagnostic messages + printed on standard error. For each Nix operation, the information + printed on standard output is well-defined; any diagnostic + information is printed on standard error, never on standard + output.</para> + + <para>This option may be specified repeatedly. Currently, the + following verbosity levels exist:</para> + + <variablelist> + + <varlistentry><term>0</term> + <listitem><para>“Errors only”: only print messages + explaining why the Nix invocation failed.</para></listitem> + </varlistentry> + + <varlistentry><term>1</term> + <listitem><para>“Informational”: print + <emphasis>useful</emphasis> messages about what Nix is doing. + This is the default.</para></listitem> + </varlistentry> + + <varlistentry><term>2</term> + <listitem><para>“Talkative”: print more informational + messages.</para></listitem> + </varlistentry> + + <varlistentry><term>3</term> + <listitem><para>“Chatty”: print even more + informational messages.</para></listitem> + </varlistentry> + + <varlistentry><term>4</term> + <listitem><para>“Debug”: print debug + information.</para></listitem> + </varlistentry> + + <varlistentry><term>5</term> + <listitem><para>“Vomit”: print vast amounts of debug + information.</para></listitem> + </varlistentry> + + </variablelist> + + </listitem> + +</varlistentry> + + +<varlistentry><term><option>--no-build-output</option></term> + <term><option>-Q</option></term> + + <listitem><para>By default, output written by builders to standard + output and standard error is echoed to the Nix command's standard + error. This option suppresses this behaviour. Note that the + builder's standard output and error are always written to a log file + in + <filename><replaceable>prefix</replaceable>/nix/var/log/nix</filename>.</para></listitem> + +</varlistentry> + + +<varlistentry xml:id="opt-max-jobs"><term><option>--max-jobs</option></term> + <term><option>-j</option></term> + + <listitem><para>Sets the maximum number of build jobs that Nix will + perform in parallel to the specified number. The default is + specified by the <link + linkend='conf-build-max-jobs'><literal>build-max-jobs</literal></link> + configuration setting, which itself defaults to + <literal>1</literal>. A higher value is useful on SMP systems or to + exploit I/O latency.</para></listitem> + +</varlistentry> + + +<varlistentry xml:id="opt-cores"><term><option>--cores</option></term> + + <listitem><para>Sets the value of the <envar>NIX_BUILD_CORES</envar> + environment variable in the invocation of builders. Builders can + use this variable at their discretion to control the maximum amount + of parallelism. For instance, in Nixpkgs, if the derivation + attribute <varname>enableParallelBuilding</varname> is set to + <literal>true</literal>, the builder passes the + <option>-j<replaceable>N</replaceable></option> flag to GNU Make. + It defaults to the value of the <link + linkend='conf-build-cores'><literal>build-cores</literal></link> + configuration setting, if set, or <literal>1</literal> otherwise. + The value <literal>0</literal> means that the builder should use all + available CPU cores in the system.</para></listitem> + +</varlistentry> + + +<varlistentry xml:id="opt-max-silent-time"><term><option>--max-silent-time</option></term> + + <listitem><para>Sets the maximum number of seconds that a builder + can go without producing any data on standard output or standard + error. The default is specified by the <link + linkend='conf-build-max-silent-time'><literal>build-max-silent-time</literal></link> + configuration setting. <literal>0</literal> means no + time-out.</para></listitem> + +</varlistentry> + +<varlistentry xml:id="opt-timeout"><term><option>--timeout</option></term> + + <listitem><para>Sets the maximum number of seconds that a builder + can run. The default is specified by the <link + linkend='conf-build-timeout'><literal>build-timeout</literal></link> + configuration setting. <literal>0</literal> means no + timeout.</para></listitem> + +</varlistentry> + +<varlistentry><term><option>--keep-going</option></term> + <term><option>-k</option></term> + + <listitem><para>Keep going in case of failed builds, to the + greatest extent possible. That is, if building an input of some + derivation fails, Nix will still build the other inputs, but not the + derivation itself. Without this option, Nix stops if any build + fails (except for builds of substitutes), possibly killing builds in + progress (in case of parallel or distributed builds).</para></listitem> + +</varlistentry> + + +<varlistentry><term><option>--keep-failed</option></term> + <term><option>-K</option></term> + + <listitem><para>Specifies that in case of a build failure, the + temporary directory (usually in <filename>/tmp</filename>) in which + the build takes place should not be deleted. The path of the build + directory is printed as an informational message. + </para> + </listitem> +</varlistentry> + + +<varlistentry><term><option>--fallback</option></term> + + <listitem> + + <para>Whenever Nix attempts to build a derivation for which + substitutes are known for each output path, but realising the output + paths through the substitutes fails, fall back on building the + derivation.</para> + + <para>The most common scenario in which this is useful is when we + have registered substitutes in order to perform binary distribution + from, say, a network repository. If the repository is down, the + realisation of the derivation will fail. When this option is + specified, Nix will build the derivation instead. Thus, + installation from binaries falls back on installation from source. + This option is not the default since it is generally not desirable + for a transient failure in obtaining the substitutes to lead to a + full build from source (with the related consumption of + resources).</para> + + </listitem> + +</varlistentry> + + +<varlistentry><term><option>--readonly-mode</option></term> + + <listitem><para>When this option is used, no attempt is made to open + the Nix database. Most Nix operations do need database access, so + those operations will fail.</para></listitem> + +</varlistentry> + + +<varlistentry xml:id="opt-log-type"><term><option>--log-type</option> +<replaceable>type</replaceable></term> + + <listitem> + + <para>This option determines how the output written to standard + error is formatted. Nix’s diagnostic messages are typically + <emphasis>nested</emphasis>. For instance, when tracing Nix + expression evaluation (<command>nix-env -vvvvv</command>, messages + from subexpressions are nested inside their parent expressions. Nix + builder output is also often nested. For instance, the Nix Packages + generic builder nests the various build tasks (unpack, configure, + compile, etc.), and the GNU Make in <literal>stdenv-linux</literal> + has been patched to provide nesting for recursive Make + invocations.</para> + + <para><replaceable>type</replaceable> can be one of the + following: + + <variablelist> + + <varlistentry><term><literal>pretty</literal></term> + + <listitem><para>Pretty-print the output, indicating different + nesting levels using spaces. This is the + default.</para></listitem> + + </varlistentry> + + <varlistentry><term><literal>escapes</literal></term> + + <listitem><para>Indicate nesting using escape codes that can be + interpreted by the <command>nix-log2xml</command> tool in the + Nix source distribution. The resulting XML file can be fed into + the <command>log2html.xsl</command> stylesheet to create an HTML + file that can be browsed interactively, using JavaScript to + expand and collapse parts of the output.</para></listitem> + + </varlistentry> + + <varlistentry><term><literal>flat</literal></term> + + <listitem><para>Remove all nesting.</para></listitem> + + </varlistentry> + + </variablelist> + + </para> + + </listitem> + +</varlistentry> + + +<varlistentry><term><option>--arg</option> <replaceable>name</replaceable> <replaceable>value</replaceable></term> + + <listitem><para>This option is accepted by + <command>nix-env</command>, <command>nix-instantiate</command> and + <command>nix-build</command>. When evaluating Nix expressions, the + expression evaluator will automatically try to call functions that + it encounters. It can automatically call functions for which every + argument has a <link linkend='ss-functions'>default value</link> + (e.g., <literal>{ <replaceable>argName</replaceable> ? + <replaceable>defaultValue</replaceable> }: + <replaceable>...</replaceable></literal>). With + <option>--arg</option>, you can also call functions that have + arguments without a default value (or override a default value). + That is, if the evaluator encounters a function with an argument + named <replaceable>name</replaceable>, it will call it with value + <replaceable>value</replaceable>.</para> + + <para>For instance, the file + <literal>pkgs/top-level/all-packages.nix</literal> in Nixpkgs is + actually a function: + +<programlisting> +{ # The system (e.g., `i686-linux') for which to build the packages. + system ? builtins.currentSystem + <replaceable>...</replaceable> +}: <replaceable>...</replaceable></programlisting> + + So if you call this Nix expression (e.g., when you do + <literal>nix-env -i <replaceable>pkgname</replaceable></literal>), + the function will be called automatically using the value <link + linkend='builtin-currentSystem'><literal>builtins.currentSystem</literal></link> + for the <literal>system</literal> argument. You can override this + using <option>--arg</option>, e.g., <literal>nix-env -i + <replaceable>pkgname</replaceable> --arg system + \"i686-freebsd\"</literal>. (Note that since the argument is a Nix + string literal, you have to escape the quotes.)</para></listitem> + +</varlistentry> + + +<varlistentry><term><option>--argstr</option> <replaceable>name</replaceable> <replaceable>value</replaceable></term> + + <listitem><para>This option is like <option>--arg</option>, only the + value is not a Nix expression but a string. So instead of + <literal>--arg system \"i686-linux\"</literal> (the outer quotes are + to keep the shell happy) you can say <literal>--argstr system + i686-linux</literal>.</para></listitem> + +</varlistentry> + + +<varlistentry xml:id="opt-attr"><term><option>--attr</option> / <option>-A</option> +<replaceable>attrPath</replaceable></term> + + <listitem><para>In <command>nix-env</command>, + <command>nix-instantiate</command> and <command>nix-build</command>, + <option>--attr</option> allows you to select an attribute from the + top-level Nix expression being evaluated. The <emphasis>attribute + path</emphasis> <replaceable>attrPath</replaceable> is a sequence of + attribute names separated by dots. For instance, given a top-level + Nix expression <replaceable>e</replaceable>, the attribute path + <literal>xorg.xorgserver</literal> would cause the expression + <literal><replaceable>e</replaceable>.xorg.xorgserver</literal> to + be used. See <link + linkend='refsec-nix-env-install-examples'><command>nix-env + --install</command></link> for some concrete examples.</para> + + <para>In addition to attribute names, you can also specify array + indices. For instance, the attribute path + <literal>foo.3.bar</literal> selects the <literal>bar</literal> + attribute of the fourth element of the array in the + <literal>foo</literal> attribute of the top-level + expression.</para></listitem> + +</varlistentry> + + +<varlistentry><term><option>--show-trace</option></term> + + <listitem><para>Causes Nix to print out a stack trace in case of Nix + expression evaluation errors.</para></listitem> + +</varlistentry> + + +<varlistentry><term><option>-I</option> <replaceable>path</replaceable></term> + + <listitem><para>Add a path to the Nix expression search path. This + option may be given multiple times. See the <envar>NIX_PATH</envar> + environment variable for information on the semantics of the Nix + search path. Paths added through <option>-I</option> take + precedence over <envar>NIX_PATH</envar>.</para></listitem> + +</varlistentry> + + +<varlistentry><term><option>--option</option> <replaceable>name</replaceable> <replaceable>value</replaceable></term> + + <listitem><para>Set the Nix configuration option + <replaceable>name</replaceable> to <replaceable>value</replaceable>. + This overrides settings in the Nix configuration file (see + <citerefentry><refentrytitle>nix.conf</refentrytitle><manvolnum>5</manvolnum></citerefentry>).</para></listitem> + +</varlistentry> + + +<varlistentry><term><option>--repair</option></term> + + <listitem><para>Fix corrupted or missing store paths by + redownloading or rebuilding them. Note that this is slow because it + requires computing a cryptographic hash of the contents of every + path in the closure of the build. Also note the warning under + <command>nix-store --repair-path</command>.</para></listitem> + +</varlistentry> + + +</variablelist> + + +</section> diff --git a/doc/manual/opt-inst-syn.xml b/doc/manual/opt-inst-syn.xml new file mode 100644 index 000000000000..e8c3f1ec6f04 --- /dev/null +++ b/doc/manual/opt-inst-syn.xml @@ -0,0 +1,22 @@ +<nop xmlns="http://docbook.org/ns/docbook"> + + <arg> + <group choice='req'> + <arg choice='plain'><option>--prebuilt-only</option></arg> + <arg choice='plain'><option>-b</option></arg> + </group> + </arg> + + <arg> + <group choice='req'> + <arg choice='plain'><option>--attr</option></arg> + <arg choice='plain'><option>-A</option></arg> + </group> + </arg> + + <arg><option>--from-expression</option></arg> + <arg><option>-E</option></arg> + + <arg><option>--from-profile</option> <replaceable>path</replaceable></arg> + +</nop> diff --git a/doc/manual/package-management.xml b/doc/manual/package-management.xml new file mode 100644 index 000000000000..e1d24b14772c --- /dev/null +++ b/doc/manual/package-management.xml @@ -0,0 +1,591 @@ +<chapter xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xml:id='chap-package-management'> + +<title>Package Management</title> + + +<para>This chapter discusses how to do package management with Nix, +i.e., how to obtain, install, upgrade, and erase packages. This is +the “user’s” perspective of the Nix system — people +who want to <emphasis>create</emphasis> packages should consult +<xref linkend='chap-writing-nix-expressions' />.</para> + + +<section><title>Basic package management</title> + +<para>The main command for package management is <link +linkend="sec-nix-env"><command>nix-env</command></link>. You can use +it to install, upgrade, and erase packages, and to query what +packages are installed or are available for installation.</para> + +<para>In Nix, different users can have different “views” +on the set of installed applications. That is, there might be lots of +applications present on the system (possibly in many different +versions), but users can have a specific selection of those active — +where “active” just means that it appears in a directory +in the user’s <envar>PATH</envar>. Such a view on the set of +installed applications is called a <emphasis>user +environment</emphasis>, which is just a directory tree consisting of +symlinks to the files of the active applications. </para> + +<para>Components are installed from a set of <emphasis>Nix +expressions</emphasis> that tell Nix how to build those packages, +including, if necessary, their dependencies. There is a collection of +Nix expressions called the Nix Package collection that contains +packages ranging from basic development stuff such as GCC and Glibc, +to end-user applications like Mozilla Firefox. (Nix is however not +tied to the Nix Package collection; you could write your own Nix +expressions based on it, or completely new ones.) You can download +the latest version from <link +xlink:href='http://nixos.org/nixpkgs/download.html' />.</para> + +<para>Assuming that you have downloaded and unpacked a release of Nix +Packages, you can view the set of available packages in the release: + +<screen> +$ nix-env -qaf nixpkgs-<replaceable>version</replaceable> '*' +ant-blackdown-1.4.2 +aterm-2.2 +bash-3.0 +binutils-2.15 +bison-1.875d +blackdown-1.4.2 +bzip2-1.0.2 +...</screen> + +where <literal>nixpkgs-<replaceable>version</replaceable></literal> is +where you’ve unpacked the release. The flag <option>-q</option> +specifies a query operation; <option>-a</option> means that you want +to show the “available” (i.e., installable) packages, as opposed to +the installed packages; and <option>-f</option> +<filename>nixpkgs-<replaceable>version</replaceable></filename> +specifies the source of the packages. The argument +<literal>'*'</literal> shows all installable packages. (The quotes are +necessary to prevent shell expansion.) You can also select specific +packages by name: + +<screen> +$ nix-env -qaf nixpkgs-<replaceable>version</replaceable> gcc +gcc-3.4.6 +gcc-4.0.3 +gcc-4.1.1</screen> + +</para> + +<para>It is also possible to see the <emphasis>status</emphasis> of +available packages, i.e., whether they are installed into the user +environment and/or present in the system: + +<screen> +$ nix-env -qasf nixpkgs-<replaceable>version</replaceable> '*' +... +-PS bash-3.0 +--S binutils-2.15 +IPS bison-1.875d +...</screen> + +The first character (<literal>I</literal>) indicates whether the +package is installed in your current user environment. The second +(<literal>P</literal>) indicates whether it is present on your system +(in which case installing it into your user environment would be a +very quick operation). The last one (<literal>S</literal>) indicates +whether there is a so-called <emphasis>substitute</emphasis> for the +package, which is Nix’s mechanism for doing binary deployment. It +just means that Nix knows that it can fetch a pre-built package from +somewhere (typically a network server) instead of building it +locally.</para> + +<para>So now that we have a set of Nix expressions we can build the +packages contained in them. This is done using <literal>nix-env +-i</literal>. For instance, + +<screen> +$ nix-env -f nixpkgs-<replaceable>version</replaceable> -i subversion</screen> + +will install the package called <literal>subversion</literal> (which +is, of course, the <link +xlink:href='http://subversion.tigris.org/'>Subversion version +management system</link>).</para> + +<para>When you do this for the first time, Nix will start building +Subversion and all its dependencies. This will take quite a while — +typically an hour or two on modern machines. Fortunately, there is a +faster way (so do a Ctrl-C on that install operation!): you just need +to tell Nix that pre-built binaries of all those packages are +available somewhere. This is done using the +<command>nix-pull</command> command, which must be supplied with a URL +containing a <emphasis>manifest</emphasis> describing what binaries +are available. This URL should correspond to the Nix Packages release +that you’re using. For instance, if you obtained a release from <link +xlink:href='http://nixos.org/releases/nixpkgs/nixpkgs-0.12pre11712-4lrp7j8x' +/>, then you should do: + +<screen> +$ nix-pull http://nixos.org/releases/nixpkgs/nixpkgs-0.12pre11712-4lrp7j8x/MANIFEST</screen> + +If you then issue the installation command, it should start +downloading binaries from <systemitem +class='fqdomainname'>nixos.org</systemitem>, instead of building +them from source. This might still take a while since all +dependencies must be downloaded, but on a reasonably fast connection +such as a DSL line it’s on the order of a few minutes.</para> + +<para>Naturally, packages can also be uninstalled: + +<screen> +$ nix-env -e subversion</screen> + +</para> + +<para>Upgrading to a new version is just as easy. If you have a new +release of Nix Packages, you can do: + +<screen> +$ nix-env -f nixpkgs-<replaceable>version</replaceable> -u subversion</screen> + +This will <emphasis>only</emphasis> upgrade Subversion if there is a +“newer” version in the new set of Nix expressions, as +defined by some pretty arbitrary rules regarding ordering of version +numbers (which generally do what you’d expect of them). To just +unconditionally replace Subversion with whatever version is in the Nix +expressions, use <parameter>-i</parameter> instead of +<parameter>-u</parameter>; <parameter>-i</parameter> will remove +whatever version is already installed.</para> + +<para>You can also upgrade all packages for which there are newer +versions: + +<screen> +$ nix-env -f nixpkgs-<replaceable>version</replaceable> -u '*'</screen> + +</para> + +<para>Sometimes it’s useful to be able to ask what +<command>nix-env</command> would do, without actually doing it. For +instance, to find out what packages would be upgraded by +<literal>nix-env -u '*'</literal>, you can do + +<screen> +$ nix-env ... -u '*' --dry-run +(dry run; not doing anything) +upgrading `libxslt-1.1.0' to `libxslt-1.1.10' +upgrading `graphviz-1.10' to `graphviz-1.12' +upgrading `coreutils-5.0' to `coreutils-5.2.1'</screen> + +</para> + +</section> + + +<section xml:id="sec-profiles"><title>Profiles</title> + +<para>Profiles and user environments are Nix’s mechanism for +implementing the ability to allow different users to have different +configurations, and to do atomic upgrades and rollbacks. To +understand how they work, it’s useful to know a bit about how Nix +works. In Nix, packages are stored in unique locations in the +<emphasis>Nix store</emphasis> (typically, +<filename>/nix/store</filename>). For instance, a particular version +of the Subversion package might be stored in a directory +<filename>/nix/store/dpmvp969yhdqs7lm2r1a3gng7pyq6vy4-subversion-1.1.3/</filename>, +while another version might be stored in +<filename>/nix/store/5mq2jcn36ldlmh93yj1n8s9c95pj7c5s-subversion-1.1.2</filename>. +The long strings prefixed to the directory names are cryptographic +hashes<footnote><para>160-bit truncations of SHA-256 hashes encoded in +a base-32 notation, to be precise.</para></footnote> of +<emphasis>all</emphasis> inputs involved in building the package — +sources, dependencies, compiler flags, and so on. So if two +packages differ in any way, they end up in different locations in +the file system, so they don’t interfere with each other. <xref +linkend='fig-user-environments' /> shows a part of a typical Nix +store.</para> + +<figure xml:id='fig-user-environments'><title>User environments</title> + <mediaobject> + <imageobject> + <imagedata fileref='figures/user-environments.png' format='PNG' /> + </imageobject> + </mediaobject> +</figure> + +<para>Of course, you wouldn’t want to type + +<screen> +$ /nix/store/dpmvp969yhdq...-subversion-1.1.3/bin/svn</screen> + +every time you want to run Subversion. Of course we could set up the +<envar>PATH</envar> environment variable to include the +<filename>bin</filename> directory of every package we want to use, +but this is not very convenient since changing <envar>PATH</envar> +doesn’t take effect for already existing processes. The solution Nix +uses is to create directory trees of symlinks to +<emphasis>activated</emphasis> packages. These are called +<emphasis>user environments</emphasis> and they are packages +themselves (though automatically generated by +<command>nix-env</command>), so they too reside in the Nix store. For +instance, in <xref linkend='fig-user-environments' /> the user +environment <filename>/nix/store/0c1p5z4kda11...-user-env</filename> +contains a symlink to just Subversion 1.1.2 (arrows in the figure +indicate symlinks). This would be what we would obtain if we had done + +<screen> +$ nix-env -i subversion</screen> + +on a set of Nix expressions that contained Subversion 1.1.2.</para> + +<para>This doesn’t in itself solve the problem, of course; you +wouldn’t want to type +<filename>/nix/store/0c1p5z4kda11...-user-env/bin/svn</filename> +either. That’s why there are symlinks outside of the store that point +to the user environments in the store; for instance, the symlinks +<filename>default-42-link</filename> and +<filename>default-43-link</filename> in the example. These are called +<emphasis>generations</emphasis> since every time you perform a +<command>nix-env</command> operation, a new user environment is +generated based on the current one. For instance, generation 43 was +created from generation 42 when we did + +<screen> +$ nix-env -i subversion mozilla</screen> + +on a set of Nix expressions that contained Mozilla and a new version +of Subversion.</para> + +<para>Generations are grouped together into +<emphasis>profiles</emphasis> so that different users don’t interfere +with each other if they don’t want to. For example: + +<screen> +$ ls -l /nix/var/nix/profiles/ +... +lrwxrwxrwx 1 eelco ... default-42-link -> /nix/store/0c1p5z4kda11...-user-env +lrwxrwxrwx 1 eelco ... default-43-link -> /nix/store/3aw2pdyx2jfc...-user-env +lrwxrwxrwx 1 eelco ... default -> default-43-link</screen> + +This shows a profile called <filename>default</filename>. The file +<filename>default</filename> itself is actually a symlink that points +to the current generation. When we do a <command>nix-env</command> +operation, a new user environment and generation link are created +based on the current one, and finally the <filename>default</filename> +symlink is made to point at the new generation. This last step is +atomic on Unix, which explains how we can do atomic upgrades. (Note +that the building/installing of new packages doesn’t interfere in +any way with old packages, since they are stored in different +locations in the Nix store.)</para> + +<para>If you find that you want to undo a <command>nix-env</command> +operation, you can just do + +<screen> +$ nix-env --rollback</screen> + +which will just make the current generation link point at the previous +link. E.g., <filename>default</filename> would be made to point at +<filename>default-42-link</filename>. You can also switch to a +specific generation: + +<screen> +$ nix-env --switch-generation 43</screen> + +which in this example would roll forward to generation 43 again. You +can also see all available generations: + +<screen> +$ nix-env --list-generations</screen></para> + +<para>Actually, there is another level of indirection not shown in the +figure above. You generally wouldn’t have +<filename>/nix/var/nix/profiles/<replaceable>some-profile</replaceable>/bin</filename> +in your <envar>PATH</envar>. Rather, there is a symlink +<filename>~/.nix-profile</filename> that points to your current +profile. This means that you should put +<filename>~/.nix-profile/bin</filename> in your <envar>PATH</envar> +(and indeed, that’s what the initialisation script +<filename>/nix/etc/profile.d/nix.sh</filename> does). This makes it +easier to switch to a different profile. You can do that using the +command <command>nix-env --switch-profile</command>: + +<screen> +$ nix-env --switch-profile /nix/var/nix/profiles/my-profile + +$ nix-env --switch-profile /nix/var/nix/profiles/default</screen> + +These commands switch to the <filename>my-profile</filename> and +default profile, respectively. If the profile doesn’t exist, it will +be created automatically. You should be careful about storing a +profile in another location than the <filename>profiles</filename> +directory, since otherwise it might not be used as a root of the +garbage collector (see <xref linkend='sec-garbage-collection' +/>).</para> + +<para>All <command>nix-env</command> operations work on the profile +pointed to by <command>~/.nix-profile</command>, but you can override +this using the <option>--profile</option> option (abbreviation +<option>-p</option>): + +<screen> +$ nix-env -p /nix/var/nix/profiles/other-profile -i subversion</screen> + +This will <emphasis>not</emphasis> change the +<command>~/.nix-profile</command> symlink.</para> + +</section> + + +<section xml:id='sec-garbage-collection'><title>Garbage collection</title> + +<para><command>nix-env</command> operations such as upgrades +(<option>-u</option>) and uninstall (<option>-e</option>) never +actually delete packages from the system. All they do (as shown +above) is to create a new user environment that no longer contains +symlinks to the “deleted” packages.</para> + +<para>Of course, since disk space is not infinite, unused packages +should be removed at some point. You can do this by running the Nix +garbage collector. It will remove from the Nix store any package +not used (directly or indirectly) by any generation of any +profile.</para> + +<para>Note however that as long as old generations reference a +package, it will not be deleted. After all, we wouldn’t be able to +do a rollback otherwise. So in order for garbage collection to be +effective, you should also delete (some) old generations. Of course, +this should only be done if you are certain that you will not need to +roll back.</para> + +<para>To delete all old (non-current) generations of your current +profile: + +<screen> +$ nix-env --delete-generations old</screen> + +Instead of <literal>old</literal> you can also specify a list of +generations, e.g., + +<screen> +$ nix-env --delete-generations 10 11 14</screen> + +</para> + +<para>After removing appropriate old generations you can run the +garbage collector as follows: + +<screen> +$ nix-store --gc</screen> + +If you are feeling uncertain, you can also first view what files would +be deleted: + +<screen> +$ nix-store --gc --print-dead</screen> + +Likewise, the option <option>--print-live</option> will show the paths +that <emphasis>won’t</emphasis> be deleted.</para> + +<para>There is also a convenient little utility +<command>nix-collect-garbage</command>, which when invoked with the +<option>-d</option> (<option>--delete-old</option>) switch deletes all +old generations of all profiles in +<filename>/nix/var/nix/profiles</filename>. So + +<screen> +$ nix-collect-garbage -d</screen> + +is a quick and easy way to clean up your system.</para> + + + + +<section xml:id="ssec-gc-roots"><title>Garbage collector roots</title> + +<para>The roots of the garbage collector are all store paths to which +there are symlinks in the directory +<filename><replaceable>prefix</replaceable>/nix/var/nix/gcroots</filename>. +For instance, the following command makes the path +<filename>/nix/store/d718ef...-foo</filename> a root of the collector: + +<screen> +$ ln -s /nix/store/d718ef...-foo /nix/var/nix/gcroots/bar</screen> + +That is, after this command, the garbage collector will not remove +<filename>/nix/store/d718ef...-foo</filename> or any of its +dependencies.</para> + +<para>Subdirectories of +<filename><replaceable>prefix</replaceable>/nix/var/nix/gcroots</filename> +are also searched for symlinks. Symlinks to non-store paths are +followed and searched for roots, but symlinks to non-store paths +<emphasis>inside</emphasis> the paths reached in that way are not +followed to prevent infinite recursion.</para> + +</section> + +</section> + + +<section xml:id="sec-channels"><title>Channels</title> + +<para>If you want to stay up to date with a set of packages, it’s not +very convenient to manually download the latest set of Nix expressions +for those packages, use <command>nix-pull</command> to register +pre-built binaries (if available), and upgrade using +<command>nix-env</command>. Fortunately, there’s a better way: +<emphasis>Nix channels</emphasis>.</para> + +<para>A Nix channel is just a URL that points to a place that contains +a set of Nix expressions and a manifest. Using the command <link +linkend="sec-nix-channel"><command>nix-channel</command></link> you +can automatically stay up to date with whatever is available at that +URL.</para> + +<para>You can “subscribe” to a channel using +<command>nix-channel --add</command>, e.g., + +<screen> +$ nix-channel --add http://nixos.org/channels/nixpkgs-unstable</screen> + +subscribes you to a channel that always contains that latest version +of the Nix Packages collection. (Instead of +<literal>nixpkgs-unstable</literal> you could also subscribe to +<literal>nixpkgs-stable</literal>, which should have a higher level of +stability, but right now is just outdated.) Subscribing really just +means that the URL is added to the file +<filename>~/.nix-channels</filename>. Right now there is no command +to “unsubscribe”; you should just edit that file manually +and delete the offending URL.</para> + +<para>To obtain the latest Nix expressions available in a channel, do + +<screen> +$ nix-channel --update</screen> + +This downloads the Nix expressions in every channel (downloaded from +<literal><replaceable>url</replaceable>/nixexprs.tar.bz2</literal>) +and registers any available pre-built binaries in every channel +(by <command>nix-pull</command>ing +<literal><replaceable>url</replaceable>/MANIFEST</literal>). It also +makes the union of each channel’s Nix expressions the default for +<command>nix-env</command> operations. Consequently, you can then say + +<screen> +$ nix-env -u '*'</screen> + +to upgrade all packages in your profile to the latest versions +available in the subscribed channels.</para> + +</section> + + +<section xml:id="sec-one-click"><title>One-click installs</title> + +<para>Often, when you want to install a specific package (e.g., from +the <link +xlink:href="http://nixos.org/nixpkgs/">Nix +Packages collection</link>), subscribing to a channel is a bit +cumbersome. And channels don’t help you at all if you want to install +an older version of a package than the one provided by the current +contents of the channel, or a package that has been removed from the +channel. That’s when <emphasis>one-click installs</emphasis> come in +handy: you can just go to the web page that contains the package, +click on it, and it will be installed with all the necessary +dependencies.</para> + +<para>For instance, you can go to <link +xlink:href="http://hydra.nixos.org/jobset/nixpkgs/trunk/channel/latest" +/> and click on any link for the individual packages for your +platform. The first time you do this, your browser will ask what to +do with <literal>application/nix-package</literal> files. You should +open them with <filename>/nix/bin/nix-install-package</filename>. +This will open a window that asks you to confirm that you want to +install the package. When you answer <literal>Y</literal>, the +package and all its dependencies will be installed. This is a binary +deployment mechanism — you get packages pre-compiled for the selected +platform type.</para> + +<para>You can also install <literal>application/nix-package</literal> +files from the command line directly. See <xref +linkend='sec-nix-install-package' /> for details.</para> + +</section> + + +<section xml:id="sec-sharing-packages"><title>Sharing packages between machines</title> + +<para>Sometimes you want to copy a package from one machine to +another. Or, you want to install some packages and you know that +another machine already has some or all of those packages or their +dependencies. In that case there are mechanisms to quickly copy +packages between machines.</para> + +<para>The command <command +linkend="sec-nix-copy-closure">nix-copy-closure</command> copies a Nix +store path along with all its dependencies to or from another machine +via the SSH protocol. It doesn’t copy store paths that are already +present on the target machine. For example, the following command +copies Firefox with all its dependencies: + +<screen> +$ nix-copy-closure --to alice@itchy.example.org $(type -p firefox)</screen> + +See <xref linkend='sec-nix-copy-closure' /> for details.</para> + +<para>With <command linkend='refsec-nix-store-export'>nix-store +--export</command> and <command +linkend='refsec-nix-store-import'>nix-store --import</command> you can +write the closure of a store path (that is, the path and all its +dependencies) to a file, and then unpack that file into another Nix +store. For example, + +<screen> +$ nix-store --export $(nix-store -qR $(type -p firefox)) > firefox.closure</screen> + +writes the closure of Firefox to a file. You can then copy this file +to another machine and install the closure: + +<screen> +$ nix-store --import < firefox.closure</screen> + +Any store paths in the closure that are already present in the target +store are ignored. It is also possible to pipe the export into +another command, e.g. to copy and install a closure directly to/on +another machine: + +<screen> +$ nix-store --export $(nix-store -qR $(type -p firefox)) | bzip2 | \ + ssh alice@itchy.example.org "bunzip2 | nix-store --import"</screen> + +But note that <command>nix-copy-closure</command> is generally more +efficient in this example because it only copies paths that are not +already present in the target Nix store.</para> + +<para>Finally, if you can mount the Nix store of a remote machine in +your local filesystem, Nix can copy paths from the remote Nix store to +the local Nix store <emphasis>on demand</emphasis>. For instance, +suppose that you mount a remote machine containing a Nix store via +<command +xlink:href="http://fuse.sourceforge.net/sshfs.html">sshfs</command>: + +<screen> +$ sshfs alice@itchy.example.org:/ /mnt</screen> + +You should then set the <envar>NIX_OTHER_STORES</envar> environment +variable to tell Nix about this remote Nix store: + +<screen> +$ export NIX_OTHER_STORES=/mnt/nix</screen> + +Then if you do any Nix operation, e.g. + +<screen> +$ nix-env -i firefox</screen> + +and Nix has to build a path that it sees is already present in +<filename>/mnt/nix</filename>, then it will just copy from there +instead of building it from source.</para> + + +</section> + + +</chapter> diff --git a/doc/manual/quick-start.xml b/doc/manual/quick-start.xml new file mode 100644 index 000000000000..779b9b9b8546 --- /dev/null +++ b/doc/manual/quick-start.xml @@ -0,0 +1,145 @@ +<chapter xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xml:id="chap-quick-start"> + +<title>Quick Start</title> + + +<para>This chapter is for impatient people who don't like reading +documentation. For more in-depth information you are kindly referred +to the following chapters.</para> + +<orderedlist> + +<listitem><para>Download a source tarball or RPM or Debian/Ubuntu +package from <link xlink:href='http://nixos.org/'/>. Build source +distributions using the regular sequence: + +<screen> +$ tar xvfj nix-<replaceable>version</replaceable>.tar.bz2 +$ cd nix-<replaceable>version</replaceable> +$ ./configure +$ make +$ make install <lineannotation>(as root)</lineannotation></screen> + +This will install the Nix binaries in <filename>/usr/local</filename> +and keep the Nix store and other state in <filename>/nix</filename>. +You can change the former by specifying +<option>--prefix=<replaceable>path</replaceable></option>. The +location of the store can be changed using +<option>--with-store-dir=<replaceable>path</replaceable></option>. +However, you shouldn't change the store location, if at all possible, +since that will make it impossible to use pre-built binaries from the +Nixpkgs channel and other channels. The location of the state can be +changed using +<option>--localstatedir=<replaceable>path</replaceable>.</option></para></listitem> + +<listitem><para>You should add +<filename><replaceable>prefix</replaceable>/etc/profile.d/nix.sh</filename> +to your <filename>~/.profile</filename> (or some other login +file).</para></listitem> + +<listitem><para>Subscribe to the Nix Packages channel. + +<screen> +$ nix-channel --add http://nixos.org/channels/nixpkgs-unstable</screen> + +</para></listitem> + +<listitem><para>Download the latest Nix expressions available in the channel. +<screen> +$ nix-channel --update</screen> + +Note that this in itself doesn't download any packages, it just +downloads the Nix expressions that build them and stores them +somewhere (under <filename>~/.nix-defexpr</filename>, in case you're +curious). Also, it registers the fact that pre-built binaries are +available remotely.</para></listitem> + +<listitem><para>See what installable packages are currently available +in the channel: + +<screen> +$ nix-env -qa \* +docbook-xml-4.2 +firefox-1.0pre-PR-0.10.1 +hello-2.1.1 +libxslt-1.1.0 +<replaceable>...</replaceable></screen> + +</para></listitem> + +<listitem><para>Install some packages from the channel: + +<screen> +$ nix-env -i hello firefox <replaceable>...</replaceable> </screen> + +This should download pre-built packages; it should not build them +locally (if it does, something went wrong).</para></listitem> + +<listitem><para>Test that they work: + +<screen> +$ which hello +/home/eelco/.nix-profile/bin/hello +$ hello +Hello, world! +$ firefox +<lineannotation>(read Slashdot or something)</lineannotation></screen> + +</para></listitem> + +<listitem><para>Uninstall a package: + +<screen> +$ nix-env -e hello</screen> + +</para></listitem> + +<listitem><para>To keep up-to-date with the channel, do: + +<screen> +$ nix-channel --update +$ nix-env -u '*'</screen> + +The latter command will upgrade each installed package for which there +is a “newer” version (as determined by comparing the version +numbers).</para></listitem> + +<listitem><para>You can also install specific packages directly from +your web browser. For instance, you can go to <link +xlink:href="http://hydra.nixos.org/jobset/nixpkgs/trunk/channel/latest" +/> and click on any link for the individual packages for your +platform. Associate <literal>application/nix-package</literal> with +the program <command>nix-install-package</command>. A window should +appear asking you whether it’s okay to install the package. Say +<literal>Y</literal>. The package and all its dependencies will be +installed.</para></listitem> + +<listitem><para>If you're unhappy with the result of a +<command>nix-env</command> action (e.g., an upgraded package turned +out not to work properly), you can go back: + +<screen> +$ nix-env --rollback</screen> + +</para></listitem> + +<listitem><para>You should periodically run the Nix garbage collector +to get rid of unused packages, since uninstalls or upgrades don't +actually delete them: + +<screen> +$ nix-collect-garbage -d</screen> + +<!-- +The first command deletes old “generations” of your profile (making +rollbacks impossible, but also making the packages in those old +generations available for garbage collection), while the second +command actually deletes them.--> + +</para></listitem> + +</orderedlist> + +</chapter> diff --git a/doc/manual/quote-literals.xsl b/doc/manual/quote-literals.xsl new file mode 100644 index 000000000000..03971bff90dc --- /dev/null +++ b/doc/manual/quote-literals.xsl @@ -0,0 +1,44 @@ +<?xml version="1.0"?> + +<xsl:stylesheet + version="1.0" + xmlns:xsl="http://www.w3.org/1999/XSL/Transform" + xmlns:str="http://exslt.org/strings" + extension-element-prefixes="str"> + + <xsl:output method="xml"/> + + <xsl:template match="function|command|literal|varname|filename|option|quote">`<xsl:apply-templates/>'</xsl:template> + + <xsl:template match="token"><xsl:text> </xsl:text><xsl:apply-templates /><xsl:text> +</xsl:text></xsl:template> + + <xsl:template match="screen|programlisting"> + <screen><xsl:apply-templates select="str:split(., '
')" /></screen> + </xsl:template> + + <xsl:template match="section[following::section]"> + <section> + <xsl:apply-templates /> + <screen><xsl:text> + </xsl:text></screen> + </section> + </xsl:template> + + <xsl:template match="*"> + <xsl:element name="{name(.)}" namespace="{namespace-uri(.)}"> + <xsl:copy-of select="namespace::*" /> + <xsl:for-each select="@*"> + <xsl:attribute name="{name(.)}" namespace="{namespace-uri(.)}"> + <xsl:value-of select="."/> + </xsl:attribute> + </xsl:for-each> + <xsl:apply-templates/> + </xsl:element> + </xsl:template> + + <xsl:template match="text()"> + <xsl:value-of select="translate(., '‘’“”—', concat("`'", '""-'))" /> + </xsl:template> + +</xsl:stylesheet> diff --git a/doc/manual/release-notes.xml b/doc/manual/release-notes.xml new file mode 100644 index 000000000000..37bb730f4063 --- /dev/null +++ b/doc/manual/release-notes.xml @@ -0,0 +1,2272 @@ +<article xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xml:id="sec-relnotes"> + +<title>Nix Release Notes</title> + + +<!--==================================================================--> + +<section xml:id="ssec-relnotes-1.7"><title>Release 1.7 (TBA)</title> + +<para>This release has the following changes:</para> + +<itemizedlist> + + <listitem> + <para>New built-in function: <function>builtins.toJSON</function>, + which returns a JSON representation of a value.</para> + </listitem> + + <listitem><para><command>nix-setuid-helper</command> is + gone.</para></listitem> + +</itemizedlist> + +</section> + + +<!--==================================================================--> + +<section xml:id="ssec-relnotes-1.6.1"><title>Release 1.6.1 (October 28, 2013)</title> + +<para>This is primarily a bug fix release. Changes of interest +are:</para> + +<itemizedlist> + + <listitem> + <para>Nix 1.6 accidentally changed the semantics of antiquoted + paths in strings, such as <literal>"${/foo}/bar"</literal>. This + release reverts to the Nix 1.5.3 behaviour.</para> + </listitem> + + <listitem> + <para>Previously, Nix optimised expressions such as + <literal>"${<replaceable>expr</replaceable>}"</literal> to + <replaceable>expr</replaceable>. Thus it neither checked whether + <replaceable>expr</replaceable> could be coerced to a string, nor + applied such coercions. This meant that + <literal>"${123}"</literal> evaluatued to <literal>123</literal>, + and <literal>"${./foo}"</literal> evaluated to + <literal>./foo</literal> (even though + <literal>"${./foo} "</literal> evaluates to + <literal>"/nix/store/<replaceable>hash</replaceable>-foo "</literal>). + Nix now checks the type of antiquoted expressions and + applies coercions.</para> + </listitem> + + <listitem> + <para>Nix now shows the exact position of undefined variables. In + particular, undefined variable errors in a <literal>with</literal> + previously didn't show <emphasis>any</emphasis> position + information, so this makes it a lot easier to fix such + errors.</para> + </listitem> + + <listitem> + <para>Undefined variables are now treated consistently. + Previously, the <function>tryEval</function> function would catch + undefined variables inside a <literal>with</literal> but not + outside. Now <function>tryEval</function> never catches undefined + variables.</para> + </listitem> + + <listitem> + <para>Bash completion in <command>nix-shell</command> now works + correctly.</para> + </listitem> + + <listitem> + <para>Stack traces are less verbose: they no longer show calls to + builtin functions and only show a single line for each derivation + on the call stack.</para> + </listitem> + + <listitem> + <para>New built-in function: <function>builtins.typeOf</function>, + which returns the type of its argument as a string.</para> + </listitem> + +</itemizedlist> + +</section> + + +<!--==================================================================--> + +<section xml:id="ssec-relnotes-1.6.0"><title>Release 1.6 (September 10, 2013)</title> + +<para>In addition to the usual bug fixes, this release has several new +features:</para> + +<itemizedlist> + + <listitem> + <para>The command <command>nix-build --run-env</command> has been + renamed to <command>nix-shell</command>.</para> + </listitem> + + <listitem> + <para><command>nix-shell</command> now sources + <filename>$stdenv/setup</filename> <emphasis>inside</emphasis> the + interactive shell, rather than in a parent shell. This ensures + that shell functions defined by <literal>stdenv</literal> can be + used in the interactive shell.</para> + </listitem> + + <listitem> + <para><command>nix-shell</command> has a new flag + <option>--pure</option> to clear the environment, so you get an + environment that more closely corresponds to the “real” Nix build. + </para> + </listitem> + + <listitem> + <para><command>nix-shell</command> now sets the shell prompt + (<envar>PS1</envar>) to ensure that Nix shells are distinguishable + from your regular shells.</para> + </listitem> + + <listitem> + <para><command>nix-env</command> no longer requires a + <literal>*</literal> argument to match all packages, so + <literal>nix-env -qa</literal> is equivalent to <literal>nix-env + -qa '*'</literal>.</para> + </listitem> + + <listitem> + <para><command>nix-env -i</command> has a new flag + <option>--remove-all</option> (<option>-r</option>) to remove all + previous packages from the profile. This makes it easier to do + declarative package management similar to NixOS’s + <option>environment.systemPackages</option>. For instance, if you + have a specification <filename>my-packages.nix</filename> like this: + +<programlisting> +with import <nixpkgs> {}; +[ thunderbird + geeqie + ... +] +</programlisting> + + then after any change to this file, you can run: + +<screen> +$ nix-env -f my-packages.nix -ir +</screen> + + to update your profile to match the specification.</para> + </listitem> + + <listitem> + <para>The ‘<literal>with</literal>’ language construct is now more + lazy. It only evaluates its argument if a variable might actually + refer to an attribute in the argument. For instance, this now + works: + +<programlisting> +let + pkgs = with pkgs; { foo = "old"; bar = foo; } // overrides; + overrides = { foo = "new"; }; +in pkgs.bar +</programlisting> + + This evaluates to <literal>"new"</literal>, while previously it + gave an “infinite recursion” error.</para> + </listitem> + + <listitem> + <para>Nix now has proper integer arithmetic operators. For + instance, you can write <literal>x + y</literal> instead of + <literal>builtins.add x y</literal>, or <literal>x < + y</literal> instead of <literal>builtins.lessThan x y</literal>. + The comparison operators also work on strings.</para> + </listitem> + + <listitem> + <para>On 64-bit systems, Nix integers are now 64 bits rather than + 32 bits.</para> + </listitem> + + <listitem> + <para>When using the Nix daemon, the <command>nix-daemon</command> + worker process now runs on the same CPU as the client, on systems + that support setting CPU affinity. This gives a significant speedup + on some systems.</para> + </listitem> + + <listitem> + <para>If a stack overflow occurs in the Nix evaluator, you now get + a proper error message (rather than “Segmentation fault”) on some + systems.</para> + </listitem> + + <listitem> + <para>In addition to directories, you can now bind-mount regular + files in chroots through the (now misnamed) option + <option>build-chroot-dirs</option>.</para> + </listitem> + +</itemizedlist> + +<para>This release has contributions from Domen Kožar, Eelco Dolstra, +Florian Friesdorf, Gergely Risko, Ivan Kozik, Ludovic Courtès and Shea +Levy.</para> + +</section> + + +<!--==================================================================--> + +<section xml:id="ssec-relnotes-1.5.3"><title>Release 1.5.3 (June 17, 2013)</title> + +<para>This is primarily a bug fix release. The following changes are +noteworthy:</para> + +<itemizedlist> + + <listitem> + <para>Yet another security bug involving hard links to files + outside the store was fixed. This bug only affected multi-user + installations that do not have hard link restrictions + enabled. (NixOS is thus not vulnerable.)</para> + </listitem> + + <listitem> + <para>The default binary cache URL has changed from + <uri>http://nixos.org/binary-cache</uri> to + <uri>http://cache.nixos.org</uri>. The latter is hosted on Amazon + CloudFront (courtesy of <link + xlink:href="http://www.logicblox.com/">LogicBlox</link>) and + should provide better performance for users in both Europe and + North America.</para> + </listitem> + + <listitem> + <para>The binary cache substituter now prints a warning message if + fetching information from the cache takes more than five seconds. + Thus network or server problems no longer make Nix appear to just + hang.</para> + </listitem> + + <listitem> + <para>Stack traces now show function names, e.g. +<screen> +while evaluating `concatMapStrings' at `<replaceable>...</replaceable>/nixpkgs/pkgs/lib/strings.nix:18:25': +</screen> + Also, if a function is called with an unexpected argument, Nix + now shows the name of the argument. + </para> + </listitem> + +</itemizedlist> + +</section> + + +<!--==================================================================--> + +<section xml:id="ssec-relnotes-1.5.2"><title>Release 1.5.2 (May 13, 2013)</title> + +<para>This is primarily a bug fix release. It has contributions from +Eelco Dolstra, Lluís Batlle i Rossell and Shea Levy.</para> + +</section> + + +<!--==================================================================--> + +<section xml:id="ssec-relnotes-1.5.1"><title>Release 1.5.1 (February 28, 2013)</title> + +<para>The bug fix to the bug fix had a bug itself, of course. But +this time it will work for sure!</para> + +</section> + + +<!--==================================================================--> + +<section xml:id="ssec-relnotes-1.5"><title>Release 1.5 (February 27, 2013)</title> + +<para>This is a brown paper bag release to fix a regression introduced +by the hard link security fix in 1.4.</para> + +</section> + + +<!--==================================================================--> + +<section xml:id="ssec-relnotes-1.4"><title>Release 1.4 (February 26, 2013)</title> + +<para>This release fixes a security bug in multi-user operation. It +was possible for derivations to cause the mode of files outside of the +Nix store to be changed to 444 (read-only but world-readable) by +creating hard links to those files (<link +xlink:href="https://github.com/NixOS/nix/commit/5526a282b5b44e9296e61e07d7d2626a79141ac4">details</link>).</para> + +<para>There are also the following improvements:</para> + +<itemizedlist> + + <listitem><para>New built-in function: + <function>builtins.hashString</function>.</para></listitem> + + <listitem><para>Build logs are now stored in + <filename>/nix/var/log/nix/drvs/<replaceable>XX</replaceable>/</filename>, + where <replaceable>XX</replaceable> is the first two characters of + the derivation. This is useful on machines that keep a lot of build + logs (such as Hydra servers).</para></listitem> + + <listitem><para>The function <function>corepkgs/fetchurl</function> + can now make the downloaded file executable. This will allow + getting rid of all bootstrap binaries in the Nixpkgs source + tree.</para></listitem> + + <listitem><para>Language change: The expression <literal>"${./path} + ..."</literal> now evaluates to a string instead of a + path.</para></listitem> + +</itemizedlist> + +</section> + + +<!--==================================================================--> + +<section xml:id="ssec-relnotes-1.3"><title>Release 1.3 (January 4, 2013)</title> + +<para>This is primarily a bug fix release. When this version is first +run on Linux, it removes any immutable bits from the Nix store and +increases the schema version of the Nix store. (The previous release +removed support for setting the immutable bit; this release clears any +remaining immutable bits to make certain operations more +efficient.)</para> + +<para>This release has contributions from Eelco Dolstra and Stuart +Pernsteiner.</para> + +</section> + + +<!--==================================================================--> + +<section xml:id="ssec-relnotes-1.2"><title>Release 1.2 (December 6, 2012)</title> + +<para>This release has the following improvements and changes:</para> + +<itemizedlist> + + <listitem> + <para>Nix has a new binary substituter mechanism: the + <emphasis>binary cache</emphasis>. A binary cache contains + pre-built binaries of Nix packages. Whenever Nix wants to build a + missing Nix store path, it will check a set of binary caches to + see if any of them has a pre-built binary of that path. The + configuration setting <option>binary-caches</option> contains a + list of URLs of binary caches. For instance, doing +<screen> +$ nix-env -i thunderbird --option binary-caches http://cache.nixos.org +</screen> + will install Thunderbird and its dependencies, using the available + pre-built binaries in <uri>http://cache.nixos.org</uri>. + The main advantage over the old “manifest”-based method of getting + pre-built binaries is that you don’t have to worry about your + manifest being in sync with the Nix expressions you’re installing + from; i.e., you don’t need to run <command>nix-pull</command> to + update your manifest. It’s also more scalable because you don’t + need to redownload a giant manifest file every time. + </para> + + <para>A Nix channel can provide a binary cache URL that will be + used automatically if you subscribe to that channel. If you use + the Nixpkgs or NixOS channels + (<uri>http://nixos.org/channels</uri>) you automatically get the + cache <uri>http://cache.nixos.org</uri>.</para> + + <para>Binary caches are created using <command>nix-push</command>. + For details on the operation and format of binary caches, see the + <command>nix-push</command> manpage. More details are provided in + <link xlink:href="http://lists.science.uu.nl/pipermail/nix-dev/2012-September/009826.html">this + nix-dev posting</link>.</para> + </listitem> + + <listitem> + <para>Multiple output support should now be usable. A derivation + can declare that it wants to produce multiple store paths by + saying something like +<programlisting> +outputs = [ "lib" "headers" "doc" ]; +</programlisting> + This will cause Nix to pass the intended store path of each output + to the builder through the environment variables + <literal>lib</literal>, <literal>headers</literal> and + <literal>doc</literal>. Other packages can refer to a specific + output by referring to + <literal><replaceable>pkg</replaceable>.<replaceable>output</replaceable></literal>, + e.g. +<programlisting> +buildInputs = [ pkg.lib pkg.headers ]; +</programlisting> + If you install a package with multiple outputs using + <command>nix-env</command>, each output path will be symlinked + into the user environment.</para> + </listitem> + + <listitem> + <para>Dashes are now valid as part of identifiers and attribute + names.</para> + </listitem> + + <listitem> + <para>The new operation <command>nix-store --repair-path</command> + allows corrupted or missing store paths to be repaired by + redownloading them. <command>nix-store --verify --check-contents + --repair</command> will scan and repair all paths in the Nix + store. Similarly, <command>nix-env</command>, + <command>nix-build</command>, <command>nix-instantiate</command> + and <command>nix-store --realise</command> have a + <option>--repair</option> flag to detect and fix bad paths by + rebuilding or redownloading them.</para> + </listitem> + + <listitem> + <para>Nix no longer sets the immutable bit on files in the Nix + store. Instead, the recommended way to guard the Nix store + against accidental modification on Linux is to make it a read-only + bind mount, like this: + +<screen> +$ mount --bind /nix/store /nix/store +$ mount -o remount,ro,bind /nix/store +</screen> + + Nix will automatically make <filename>/nix/store</filename> + writable as needed (using a private mount namespace) to allow + modifications.</para> + </listitem> + + <listitem> + <para>Store optimisation (replacing identical files in the store + with hard links) can now be done automatically every time a path + is added to the store. This is enabled by setting the + configuration option <literal>auto-optimise-store</literal> to + <literal>true</literal> (disabled by default).</para> + </listitem> + + <listitem> + <para>Nix now supports <command>xz</command> compression for NARs + in addition to <command>bzip2</command>. It compresses about 30% + better on typical archives and decompresses about twice as + fast.</para> + </listitem> + + <listitem> + <para>Basic Nix expression evaluation profiling: setting the + environment variable <envar>NIX_COUNT_CALLS</envar> to + <literal>1</literal> will cause Nix to print how many times each + primop or function was executed.</para> + </listitem> + + <listitem> + <para>New primops: <varname>concatLists</varname>, + <varname>elem</varname>, <varname>elemAt</varname> and + <varname>filter</varname>.</para> + </listitem> + + <listitem> + <para>The command <command>nix-copy-closure</command> has a new + flag <option>--use-substitutes</option> (<option>-s</option>) to + download missing paths on the target machine using the substitute + mechanism.</para> + </listitem> + + <listitem> + <para>The command <command>nix-worker</command> has been renamed + to <command>nix-daemon</command>. Support for running the Nix + worker in “slave” mode has been removed.</para> + </listitem> + + <listitem> + <para>The <option>--help</option> flag of every Nix command now + invokes <command>man</command>.</para> + </listitem> + + <listitem> + <para>Chroot builds are now supported on systemd machines.</para> + </listitem> + +</itemizedlist> + +<para>This release has contributions from Eelco Dolstra, Florian +Friesdorf, Mats Erik Andersson and Shea Levy.</para> + +</section> + + +<!--==================================================================--> + +<section xml:id="ssec-relnotes-1.1"><title>Release 1.1 (July 18, 2012)</title> + +<para>This release has the following improvements:</para> + +<itemizedlist> + + <listitem> + <para>On Linux, when doing a chroot build, Nix now uses various + namespace features provided by the Linux kernel to improve + build isolation. Namely: + <itemizedlist> + <listitem><para>The private network namespace ensures that + builders cannot talk to the outside world (or vice versa): each + build only sees a private loopback interface. This also means + that two concurrent builds can listen on the same port (e.g. as + part of a test) without conflicting with each + other.</para></listitem> + <listitem><para>The PID namespace causes each build to start as + PID 1. Processes outside of the chroot are not visible to those + on the inside. On the other hand, processes inside the chroot + <emphasis>are</emphasis> visible from the outside (though with + different PIDs).</para></listitem> + <listitem><para>The IPC namespace prevents the builder from + communicating with outside processes using SysV IPC mechanisms + (shared memory, message queues, semaphores). It also ensures + that all IPC objects are destroyed when the builder + exits.</para></listitem> + <listitem><para>The UTS namespace ensures that builders see a + hostname of <literal>localhost</literal> rather than the actual + hostname.</para></listitem> + <listitem><para>The private mount namespace was already used by + Nix to ensure that the bind-mounts used to set up the chroot are + cleaned up automatically.</para></listitem> + </itemizedlist> + </para> + </listitem> + + <listitem> + <para>Build logs are now compressed using + <command>bzip2</command>. The command <command>nix-store + -l</command> decompresses them on the fly. This can be disabled + by setting the option <literal>build-compress-log</literal> to + <literal>false</literal>.</para> + </listitem> + + <listitem> + <para>The creation of build logs in + <filename>/nix/var/log/nix/drvs</filename> can be disabled by + setting the new option <literal>build-keep-log</literal> to + <literal>false</literal>. This is useful, for instance, for Hydra + build machines.</para> + </listitem> + + <listitem> + <para>Nix now reserves some space in + <filename>/nix/var/nix/db/reserved</filename> to ensure that the + garbage collector can run successfully if the disk is full. This + is necessary because SQLite transactions fail if the disk is + full.</para> + </listitem> + + <listitem> + <para>Added a basic <function>fetchurl</function> function. This + is not intended to replace the <function>fetchurl</function> in + Nixpkgs, but is useful for bootstrapping; e.g., it will allow us + to get rid of the bootstrap binaries in the Nixpkgs source tree + and download them instead. You can use it by doing + <literal>import <nix/fetchurl.nix> { url = + <replaceable>url</replaceable>; sha256 = + "<replaceable>hash</replaceable>"; }</literal>. (Shea Levy)</para> + </listitem> + + <listitem> + <para>Improved RPM spec file. (Michel Alexandre Salim)</para> + </listitem> + + <listitem> + <para>Support for on-demand socket-based activation in the Nix + daemon with <command>systemd</command>.</para> + </listitem> + + <listitem> + <para>Added a manpage for + <citerefentry><refentrytitle>nix.conf</refentrytitle><manvolnum>5</manvolnum></citerefentry>.</para> + </listitem> + + <listitem> + <para>When using the Nix daemon, the <option>-s</option> flag in + <command>nix-env -qa</command> is now much faster.</para> + </listitem> + +</itemizedlist> + +</section> + + +<!--==================================================================--> + +<section xml:id="ssec-relnotes-1.0"><title>Release 1.0 (May 11, 2012)</title> + +<para>There have been numerous improvements and bug fixes since the +previous release. Here are the most significant:</para> + +<itemizedlist> + + <listitem> + <para>Nix can now optionally use the Boehm garbage collector. + This significantly reduces the Nix evaluator’s memory footprint, + especially when evaluating large NixOS system configurations. It + can be enabled using the <option>--enable-gc</option> configure + option.</para> + </listitem> + + <listitem> + <para>Nix now uses SQLite for its database. This is faster and + more flexible than the old <emphasis>ad hoc</emphasis> format. + SQLite is also used to cache the manifests in + <filename>/nix/var/nix/manifests</filename>, resulting in a + significant speedup.</para> + </listitem> + + <listitem> + <para>Nix now has an search path for expressions. The search path + is set using the environment variable <envar>NIX_PATH</envar> and + the <option>-I</option> command line option. In Nix expressions, + paths between angle brackets are used to specify files that must + be looked up in the search path. For instance, the expression + <literal><nixpkgs/default.nix></literal> looks for a file + <filename>nixpkgs/default.nix</filename> relative to every element + in the search path.</para> + </listitem> + + <listitem> + <para>The new command <command>nix-build --run-env</command> + builds all dependencies of a derivation, then starts a shell in an + environment containing all variables from the derivation. This is + useful for reproducing the environment of a derivation for + development.</para> + </listitem> + + <listitem> + <para>The new command <command>nix-store --verify-path</command> + verifies that the contents of a store path have not + changed.</para> + </listitem> + + <listitem> + <para>The new command <command>nix-store --print-env</command> + prints out the environment of a derivation in a format that can be + evaluated by a shell.</para> + </listitem> + + <listitem> + <para>Attribute names can now be arbitrary strings. For instance, + you can write <literal>{ "foo-1.2" = …; "bla bla" = …; }."bla + bla"</literal>.</para> + </listitem> + + <listitem> + <para>Attribute selection can now provide a default value using + the <literal>or</literal> operator. For instance, the expression + <literal>x.y.z or e</literal> evaluates to the attribute + <literal>x.y.z</literal> if it exists, and <literal>e</literal> + otherwise.</para> + </listitem> + + <listitem> + <para>The right-hand side of the <literal>?</literal> operator can + now be an attribute path, e.g., <literal>attrs ? + a.b.c</literal>.</para> + </listitem> + + <listitem> + <para>On Linux, Nix will now make files in the Nix store immutable + on filesystems that support it. This prevents accidental + modification of files in the store by the root user.</para> + </listitem> + + <listitem> + <para>Nix has preliminary support for derivations with multiple + outputs. This is useful because it allows parts of a package to + be deployed and garbage-collected separately. For instance, + development parts of a package such as header files or static + libraries would typically not be part of the closure of an + application, resulting in reduced disk usage and installation + time.</para> + </listitem> + + <listitem> + <para>The Nix store garbage collector is faster and holds the + global lock for a shorter amount of time.</para> + </listitem> + + <listitem> + <para>The option <option>--timeout</option> (corresponding to the + configuration setting <literal>build-timeout</literal>) allows you + to set an absolute timeout on builds — if a build runs for more than + the given number of seconds, it is terminated. This is useful for + recovering automatically from builds that are stuck in an infinite + loop but keep producing output, and for which + <literal>--max-silent-time</literal> is ineffective.</para> + </listitem> + + <listitem> + <para>Nix development has moved to GitHub (<link + xlink:href="https://github.com/NixOS/nix" />).</para> + </listitem> + +</itemizedlist> + +</section> + + +<!--==================================================================--> + +<section xml:id="ssec-relnotes-0.16"><title>Release 0.16 (August 17, 2010)</title> + +<para>This release has the following improvements:</para> + +<itemizedlist> + + <listitem> + <para>The Nix expression evaluator is now much faster in most + cases: typically, <link + xlink:href="http://www.mail-archive.com/nix-dev@cs.uu.nl/msg04113.html">3 + to 8 times compared to the old implementation</link>. It also + uses less memory. It no longer depends on the ATerm + library.</para> + </listitem> + + <listitem> + <para> + Support for configurable parallelism inside builders. Build + scripts have always had the ability to perform multiple build + actions in parallel (for instance, by running <command>make -j + 2</command>), but this was not desirable because the number of + actions to be performed in parallel was not configurable. Nix + now has an option <option>--cores + <replaceable>N</replaceable></option> as well as a configuration + setting <varname>build-cores = + <replaceable>N</replaceable></varname> that causes the + environment variable <envar>NIX_BUILD_CORES</envar> to be set to + <replaceable>N</replaceable> when the builder is invoked. The + builder can use this at its discretion to perform a parallel + build, e.g., by calling <command>make -j + <replaceable>N</replaceable></command>. In Nixpkgs, this can be + enabled on a per-package basis by setting the derivation + attribute <varname>enableParallelBuilding</varname> to + <literal>true</literal>. + </para> + </listitem> + + <listitem> + <para><command>nix-store -q</command> now supports XML output + through the <option>--xml</option> flag.</para> + </listitem> + + <listitem> + <para>Several bug fixes.</para> + </listitem> + +</itemizedlist> + +</section> + + +<!--==================================================================--> + +<section xml:id="ssec-relnotes-0.15"><title>Release 0.15 (March 17, 2010)</title> + +<para>This is a bug-fix release. Among other things, it fixes +building on Mac OS X (Snow Leopard), and improves the contents of +<filename>/etc/passwd</filename> and <filename>/etc/group</filename> +in <literal>chroot</literal> builds.</para> + +</section> + + +<!--==================================================================--> + +<section xml:id="ssec-relnotes-0.14"><title>Release 0.14 (February 4, 2010)</title> + +<para>This release has the following improvements:</para> + +<itemizedlist> + + <listitem> + <para>The garbage collector now starts deleting garbage much + faster than before. It no longer determines liveness of all paths + in the store, but does so on demand.</para> + </listitem> + + <listitem> + <para>Added a new operation, <command>nix-store --query + --roots</command>, that shows the garbage collector roots that + directly or indirectly point to the given store paths.</para> + </listitem> + + <listitem> + <para>Removed support for converting Berkeley DB-based Nix + databases to the new schema.</para> + </listitem> + + <listitem> + <para>Removed the <option>--use-atime</option> and + <option>--max-atime</option> garbage collector options. They were + not very useful in practice.</para> + </listitem> + + <listitem> + <para>On Windows, Nix now requires Cygwin 1.7.x.</para> + </listitem> + + <listitem> + <para>A few bug fixes.</para> + </listitem> + +</itemizedlist> + +</section> + + +<!--==================================================================--> + +<section xml:id="ssec-relnotes-0.13"><title>Release 0.13 (November 5, +2009)</title> + +<para>This is primarily a bug fix release. It has some new +features:</para> + +<itemizedlist> + + <listitem> + <para>Syntactic sugar for writing nested attribute sets. Instead of + +<programlisting> +{ + foo = { + bar = 123; + xyzzy = true; + }; + a = { b = { c = "d"; }; }; +} +</programlisting> + + you can write + +<programlisting> +{ + foo.bar = 123; + foo.xyzzy = true; + a.b.c = "d"; +} +</programlisting> + + This is useful, for instance, in NixOS configuration files.</para> + + </listitem> + + <listitem> + <para>Support for Nix channels generated by Hydra, the Nix-based + continuous build system. (Hydra generates NAR archives on the + fly, so the size and hash of these archives isn’t known in + advance.)</para> + </listitem> + + <listitem> + <para>Support <literal>i686-linux</literal> builds directly on + <literal>x86_64-linux</literal> Nix installations. This is + implemented using the <function>personality()</function> syscall, + which causes <command>uname</command> to return + <literal>i686</literal> in child processes.</para> + </listitem> + + <listitem> + <para>Various improvements to the <literal>chroot</literal> + support. Building in a <literal>chroot</literal> works quite well + now.</para> + </listitem> + + <listitem> + <para>Nix no longer blocks if it tries to build a path and another + process is already building the same path. Instead it tries to + build another buildable path first. This improves + parallelism.</para> + </listitem> + + <listitem> + <para>Support for large (> 4 GiB) files in NAR archives.</para> + </listitem> + + <listitem> + <para>Various (performance) improvements to the remote build + mechanism.</para> + </listitem> + + <listitem> + <para>New primops: <varname>builtins.addErrorContext</varname> (to + add a string to stack traces — useful for debugging), + <varname>builtins.isBool</varname>, + <varname>builtins.isString</varname>, + <varname>builtins.isInt</varname>, + <varname>builtins.intersectAttrs</varname>.</para> + </listitem> + + <listitem> + <para>OpenSolaris support (Sander van der Burg).</para> + </listitem> + + <listitem> + <para>Stack traces are no longer displayed unless the + <option>--show-trace</option> option is used.</para> + </listitem> + + <listitem> + <para>The scoping rules for <literal>inherit + (<replaceable>e</replaceable>) ...</literal> in recursive + attribute sets have changed. The expression + <replaceable>e</replaceable> can now refer to the attributes + defined in the containing set.</para> + </listitem> + +</itemizedlist> + +</section> + + +<!--==================================================================--> + +<section xml:id="ssec-relnotes-0.12"><title>Release 0.12 (November 20, +2008)</title> + +<itemizedlist> + + <listitem> + <para>Nix no longer uses Berkeley DB to store Nix store metadata. + The principal advantages of the new storage scheme are: it works + properly over decent implementations of NFS (allowing Nix stores + to be shared between multiple machines); no recovery is needed + when a Nix process crashes; no write access is needed for + read-only operations; no more running out of Berkeley DB locks on + certain operations.</para> + + <para>You still need to compile Nix with Berkeley DB support if + you want Nix to automatically convert your old Nix store to the + new schema. If you don’t need this, you can build Nix with the + <filename>configure</filename> option + <option>--disable-old-db-compat</option>.</para> + + <para>After the automatic conversion to the new schema, you can + delete the old Berkeley DB files: + + <screen> +$ cd /nix/var/nix/db +$ rm __db* log.* derivers references referrers reserved validpaths DB_CONFIG</screen> + + The new metadata is stored in the directories + <filename>/nix/var/nix/db/info</filename> and + <filename>/nix/var/nix/db/referrer</filename>. Though the + metadata is stored in human-readable plain-text files, they are + not intended to be human-editable, as Nix is rather strict about + the format.</para> + + <para>The new storage schema may or may not require less disk + space than the Berkeley DB environment, mostly depending on the + cluster size of your file system. With 1 KiB clusters (which + seems to be the <literal>ext3</literal> default nowadays) it + usually takes up much less space.</para> + </listitem> + + <listitem><para>There is a new substituter that copies paths + directly from other (remote) Nix stores mounted somewhere in the + filesystem. For instance, you can speed up an installation by + mounting some remote Nix store that already has the packages in + question via NFS or <literal>sshfs</literal>. The environment + variable <envar>NIX_OTHER_STORES</envar> specifies the locations of + the remote Nix directories, + e.g. <literal>/mnt/remote-fs/nix</literal>.</para></listitem> + + <listitem><para>New <command>nix-store</command> operations + <option>--dump-db</option> and <option>--load-db</option> to dump + and reload the Nix database.</para></listitem> + + <listitem><para>The garbage collector has a number of new options to + allow only some of the garbage to be deleted. The option + <option>--max-freed <replaceable>N</replaceable></option> tells the + collector to stop after at least <replaceable>N</replaceable> bytes + have been deleted. The option <option>--max-links + <replaceable>N</replaceable></option> tells it to stop after the + link count on <filename>/nix/store</filename> has dropped below + <replaceable>N</replaceable>. This is useful for very large Nix + stores on filesystems with a 32000 subdirectories limit (like + <literal>ext3</literal>). The option <option>--use-atime</option> + causes store paths to be deleted in order of ascending last access + time. This allows non-recently used stuff to be deleted. The + option <option>--max-atime <replaceable>time</replaceable></option> + specifies an upper limit to the last accessed time of paths that may + be deleted. For instance, + + <screen> + $ nix-store --gc -v --max-atime $(date +%s -d "2 months ago")</screen> + + deletes everything that hasn’t been accessed in two months.</para></listitem> + + <listitem><para><command>nix-env</command> now uses optimistic + profile locking when performing an operation like installing or + upgrading, instead of setting an exclusive lock on the profile. + This allows multiple <command>nix-env -i / -u / -e</command> + operations on the same profile in parallel. If a + <command>nix-env</command> operation sees at the end that the profile + was changed in the meantime by another process, it will just + restart. This is generally cheap because the build results are + still in the Nix store.</para></listitem> + + <listitem><para>The option <option>--dry-run</option> is now + supported by <command>nix-store -r</command> and + <command>nix-build</command>.</para></listitem> + + <listitem><para>The information previously shown by + <option>--dry-run</option> (i.e., which derivations will be built + and which paths will be substituted) is now always shown by + <command>nix-env</command>, <command>nix-store -r</command> and + <command>nix-build</command>. The total download size of + substitutable paths is now also shown. For instance, a build will + show something like + + <screen> +the following derivations will be built: + /nix/store/129sbxnk5n466zg6r1qmq1xjv9zymyy7-activate-configuration.sh.drv + /nix/store/7mzy971rdm8l566ch8hgxaf89x7lr7ik-upstart-jobs.drv + ... +the following paths will be downloaded/copied (30.02 MiB): + /nix/store/4m8pvgy2dcjgppf5b4cj5l6wyshjhalj-samba-3.2.4 + /nix/store/7h1kwcj29ip8vk26rhmx6bfjraxp0g4l-libunwind-0.98.6 + ...</screen> + + </para></listitem> + + <listitem><para>Language features: + + <itemizedlist> + + <listitem><para>@-patterns as in Haskell. For instance, in a + function definition + + <programlisting>f = args @ {x, y, z}: <replaceable>...</replaceable>;</programlisting> + + <varname>args</varname> refers to the argument as a whole, which + is further pattern-matched against the attribute set pattern + <literal>{x, y, z}</literal>.</para></listitem> + + <listitem><para>“<literal>...</literal>” (ellipsis) patterns. + An attribute set pattern can now say <literal>...</literal> at + the end of the attribute name list to specify that the function + takes <emphasis>at least</emphasis> the listed attributes, while + ignoring additional attributes. For instance, + + <programlisting>{stdenv, fetchurl, fuse, ...}: <replaceable>...</replaceable></programlisting> + + defines a function that accepts any attribute set that includes + at least the three listed attributes.</para></listitem> + + <listitem><para>New primops: + <varname>builtins.parseDrvName</varname> (split a package name + string like <literal>"nix-0.12pre12876"</literal> into its name + and version components, e.g. <literal>"nix"</literal> and + <literal>"0.12pre12876"</literal>), + <varname>builtins.compareVersions</varname> (compare two version + strings using the same algorithm that <command>nix-env</command> + uses), <varname>builtins.length</varname> (efficiently compute + the length of a list), <varname>builtins.mul</varname> (integer + multiplication), <varname>builtins.div</varname> (integer + division). + <!-- <varname>builtins.genericClosure</varname> --> + </para></listitem> + + </itemizedlist> + + </para></listitem> + + <listitem><para><command>nix-prefetch-url</command> now supports + <literal>mirror://</literal> URLs, provided that the environment + variable <envar>NIXPKGS_ALL</envar> points at a Nixpkgs + tree.</para></listitem> + + <listitem><para>Removed the commands + <command>nix-pack-closure</command> and + <command>nix-unpack-closure</command>. You can do almost the same + thing but much more efficiently by doing <literal>nix-store --export + $(nix-store -qR <replaceable>paths</replaceable>) > closure</literal> and + <literal>nix-store --import < + closure</literal>.</para></listitem> + + <listitem><para>Lots of bug fixes, including a big performance bug in + the handling of <literal>with</literal>-expressions.</para></listitem> + +</itemizedlist> + +</section> + + +<!--==================================================================--> + +<section xml:id="ssec-relnotes-0.11"><title>Release 0.11 (December 31, +2007)</title> + +<para>Nix 0.11 has many improvements over the previous stable release. +The most important improvement is secure multi-user support. It also +features many usability enhancements and language extensions, many of +them prompted by NixOS, the purely functional Linux distribution based +on Nix. Here is an (incomplete) list:</para> + + +<itemizedlist> + + + <listitem><para>Secure multi-user support. A single Nix store can + now be shared between multiple (possible untrusted) users. This is + an important feature for NixOS, where it allows non-root users to + install software. The old setuid method for sharing a store between + multiple users has been removed. Details for setting up a + multi-user store can be found in the manual.</para></listitem> + + + <listitem><para>The new command <command>nix-copy-closure</command> + gives you an easy and efficient way to exchange software between + machines. It copies the missing parts of the closure of a set of + store path to or from a remote machine via + <command>ssh</command>.</para></listitem> + + + <listitem><para>A new kind of string literal: strings between double + single-quotes (<literal>''</literal>) have indentation + “intelligently” removed. This allows large strings (such as shell + scripts or configuration file fragments in NixOS) to cleanly follow + the indentation of the surrounding expression. It also requires + much less escaping, since <literal>''</literal> is less common in + most languages than <literal>"</literal>.</para></listitem> + + + <listitem><para><command>nix-env</command> <option>--set</option> + modifies the current generation of a profile so that it contains + exactly the specified derivation, and nothing else. For example, + <literal>nix-env -p /nix/var/nix/profiles/browser --set + firefox</literal> lets the profile named + <filename>browser</filename> contain just Firefox.</para></listitem> + + + <listitem><para><command>nix-env</command> now maintains + meta-information about installed packages in profiles. The + meta-information is the contents of the <varname>meta</varname> + attribute of derivations, such as <varname>description</varname> or + <varname>homepage</varname>. The command <literal>nix-env -q --xml + --meta</literal> shows all meta-information.</para></listitem> + + + <listitem><para><command>nix-env</command> now uses the + <varname>meta.priority</varname> attribute of derivations to resolve + filename collisions between packages. Lower priority values denote + a higher priority. For instance, the GCC wrapper package and the + Binutils package in Nixpkgs both have a file + <filename>bin/ld</filename>, so previously if you tried to install + both you would get a collision. Now, on the other hand, the GCC + wrapper declares a higher priority than Binutils, so the former’s + <filename>bin/ld</filename> is symlinked in the user + environment.</para></listitem> + + + <listitem><para><command>nix-env -i / -u</command>: instead of + breaking package ties by version, break them by priority and version + number. That is, if there are multiple packages with the same name, + then pick the package with the highest priority, and only use the + version if there are multiple packages with the same + priority.</para> + + <para>This makes it possible to mark specific versions/variant in + Nixpkgs more or less desirable than others. A typical example would + be a beta version of some package (e.g., + <literal>gcc-4.2.0rc1</literal>) which should not be installed even + though it is the highest version, except when it is explicitly + selected (e.g., <literal>nix-env -i + gcc-4.2.0rc1</literal>).</para></listitem> + + + <listitem><para><command>nix-env --set-flag</command> allows meta + attributes of installed packages to be modified. There are several + attributes that can be usefully modified, because they affect the + behaviour of <command>nix-env</command> or the user environment + build script: + + <itemizedlist> + + <listitem><para><varname>meta.priority</varname> can be changed + to resolve filename clashes (see above).</para></listitem> + + <listitem><para><varname>meta.keep</varname> can be set to + <literal>true</literal> to prevent the package from being + upgraded or replaced. Useful if you want to hang on to an older + version of a package.</para></listitem> + + <listitem><para><varname>meta.active</varname> can be set to + <literal>false</literal> to “disable” the package. That is, no + symlinks will be generated to the files of the package, but it + remains part of the profile (so it won’t be garbage-collected). + Set it back to <literal>true</literal> to re-enable the + package.</para></listitem> + + </itemizedlist> + + </para></listitem> + + + <listitem><para><command>nix-env -q</command> now has a flag + <option>--prebuilt-only</option> (<option>-b</option>) that causes + <command>nix-env</command> to show only those derivations whose + output is already in the Nix store or that can be substituted (i.e., + downloaded from somewhere). In other words, it shows the packages + that can be installed “quickly”, i.e., don’t need to be built from + source. The <option>-b</option> flag is also available in + <command>nix-env -i</command> and <command>nix-env -u</command> to + filter out derivations for which no pre-built binary is + available.</para></listitem> + + + <listitem><para>The new option <option>--argstr</option> (in + <command>nix-env</command>, <command>nix-instantiate</command> and + <command>nix-build</command>) is like <option>--arg</option>, except + that the value is a string. For example, <literal>--argstr system + i686-linux</literal> is equivalent to <literal>--arg system + \"i686-linux\"</literal> (note that <option>--argstr</option> + prevents annoying quoting around shell arguments).</para></listitem> + + + <listitem><para><command>nix-store</command> has a new operation + <option>--read-log</option> (<option>-l</option>) + <parameter>paths</parameter> that shows the build log of the given + paths.</para></listitem> + + + <!-- + <listitem><para>TODO: semantic cleanups of string concatenation + etc. (mostly in r6740).</para></listitem> + --> + + + <listitem><para>Nix now uses Berkeley DB 4.5. The database is + upgraded automatically, but you should be careful not to use old + versions of Nix that still use Berkeley DB 4.4.</para></listitem> + + + <!-- foo + <listitem><para>TODO: option <option>- -reregister</option> in + <command>nix-store - -register-validity</command>.</para></listitem> + --> + + + <listitem><para>The option <option>--max-silent-time</option> + (corresponding to the configuration setting + <literal>build-max-silent-time</literal>) allows you to set a + timeout on builds — if a build produces no output on + <literal>stdout</literal> or <literal>stderr</literal> for the given + number of seconds, it is terminated. This is useful for recovering + automatically from builds that are stuck in an infinite + loop.</para></listitem> + + + <listitem><para><command>nix-channel</command>: each subscribed + channel is its own attribute in the top-level expression generated + for the channel. This allows disambiguation (e.g. <literal>nix-env + -i -A nixpkgs_unstable.firefox</literal>).</para></listitem> + + + <listitem><para>The substitutes table has been removed from the + database. This makes operations such as <command>nix-pull</command> + and <command>nix-channel --update</command> much, much + faster.</para></listitem> + + + <listitem><para><command>nix-pull</command> now supports + bzip2-compressed manifests. This speeds up + channels.</para></listitem> + + + <listitem><para><command>nix-prefetch-url</command> now has a + limited form of caching. This is used by + <command>nix-channel</command> to prevent unnecessary downloads when + the channel hasn’t changed.</para></listitem> + + + <listitem><para><command>nix-prefetch-url</command> now by default + computes the SHA-256 hash of the file instead of the MD5 hash. In + calls to <function>fetchurl</function> you should pass the + <literal>sha256</literal> attribute instead of + <literal>md5</literal>. You can pass either a hexadecimal or a + base-32 encoding of the hash.</para></listitem> + + + <listitem><para>Nix can now perform builds in an automatically + generated “chroot”. This prevents a builder from accessing stuff + outside of the Nix store, and thus helps ensure purity. This is an + experimental feature.</para></listitem> + + + <listitem><para>The new command <command>nix-store + --optimise</command> reduces Nix store disk space usage by finding + identical files in the store and hard-linking them to each other. + It typically reduces the size of the store by something like + 25-35%.</para></listitem> + + + <listitem><para><filename>~/.nix-defexpr</filename> can now be a + directory, in which case the Nix expressions in that directory are + combined into an attribute set, with the file names used as the + names of the attributes. The command <command>nix-env + --import</command> (which set the + <filename>~/.nix-defexpr</filename> symlink) is + removed.</para></listitem> + + + <listitem><para>Derivations can specify the new special attribute + <varname>allowedReferences</varname> to enforce that the references + in the output of a derivation are a subset of a declared set of + paths. For example, if <varname>allowedReferences</varname> is an + empty list, then the output must not have any references. This is + used in NixOS to check that generated files such as initial ramdisks + for booting Linux don’t have any dependencies.</para></listitem> + + + <listitem><para>The new attribute + <varname>exportReferencesGraph</varname> allows builders access to + the references graph of their inputs. This is used in NixOS for + tasks such as generating ISO-9660 images that contain a Nix store + populated with the closure of certain paths.</para></listitem> + + + <listitem><para>Fixed-output derivations (like + <function>fetchurl</function>) can define the attribute + <varname>impureEnvVars</varname> to allow external environment + variables to be passed to builders. This is used in Nixpkgs to + support proxy configuration, among other things.</para></listitem> + + + <listitem><para>Several new built-in functions: + <function>builtins.attrNames</function>, + <function>builtins.filterSource</function>, + <function>builtins.isAttrs</function>, + <function>builtins.isFunction</function>, + <function>builtins.listToAttrs</function>, + <function>builtins.stringLength</function>, + <function>builtins.sub</function>, + <function>builtins.substring</function>, + <function>throw</function>, + <function>builtins.trace</function>, + <function>builtins.readFile</function>.</para></listitem> + + +</itemizedlist> + +</section> + + + +<!--==================================================================--> + +<section><title>Release 0.10.1 (October 11, 2006)</title> + +<para>This release fixes two somewhat obscure bugs that occur when +evaluating Nix expressions that are stored inside the Nix store +(<literal>NIX-67</literal>). These do not affect most users.</para> + +</section> + + + +<!--==================================================================--> + +<section><title>Release 0.10 (October 6, 2006)</title> + +<note><para>This version of Nix uses Berkeley DB 4.4 instead of 4.3. +The database is upgraded automatically, but you should be careful not +to use old versions of Nix that still use Berkeley DB 4.3. In +particular, if you use a Nix installed through Nix, you should run + +<screen> +$ nix-store --clear-substitutes</screen> + +first.</para></note> + +<warning><para>Also, the database schema has changed slighted to fix a +performance issue (see below). When you run any Nix 0.10 command for +the first time, the database will be upgraded automatically. This is +irreversible.</para></warning> + +<itemizedlist> + + + <!-- Usability / features --> + + + <listitem><para><command>nix-env</command> usability improvements: + + <itemizedlist> + + <listitem><para>An option <option>--compare-versions</option> + (or <option>-c</option>) has been added to <command>nix-env + --query</command> to allow you to compare installed versions of + packages to available versions, or vice versa. An easy way to + see if you are up to date with what’s in your subscribed + channels is <literal>nix-env -qc \*</literal>.</para></listitem> + + <listitem><para><literal>nix-env --query</literal> now takes as + arguments a list of package names about which to show + information, just like <option>--install</option>, etc.: for + example, <literal>nix-env -q gcc</literal>. Note that to show + all derivations, you need to specify + <literal>\*</literal>.</para></listitem> + + <listitem><para><literal>nix-env -i + <replaceable>pkgname</replaceable></literal> will now install + the highest available version of + <replaceable>pkgname</replaceable>, rather than installing all + available versions (which would probably give collisions) + (<literal>NIX-31</literal>).</para></listitem> + + <listitem><para><literal>nix-env (-i|-u) --dry-run</literal> now + shows exactly which missing paths will be built or + substituted.</para></listitem> + + <listitem><para><literal>nix-env -qa --description</literal> + shows human-readable descriptions of packages, provided that + they have a <literal>meta.description</literal> attribute (which + most packages in Nixpkgs don’t have yet).</para></listitem> + + </itemizedlist> + + </para></listitem> + + + <listitem><para>New language features: + + <itemizedlist> + + <listitem><para>Reference scanning (which happens after each + build) is much faster and takes a constant amount of + memory.</para></listitem> + + <listitem><para>String interpolation. Expressions like + +<programlisting> +"--with-freetype2-library=" + freetype + "/lib"</programlisting> + + can now be written as + +<programlisting> +"--with-freetype2-library=${freetype}/lib"</programlisting> + + You can write arbitrary expressions within + <literal>${<replaceable>...</replaceable>}</literal>, not just + identifiers.</para></listitem> + + <listitem><para>Multi-line string literals.</para></listitem> + + <listitem><para>String concatenations can now involve + derivations, as in the example <code>"--with-freetype2-library=" + + freetype + "/lib"</code>. This was not previously possible + because we need to register that a derivation that uses such a + string is dependent on <literal>freetype</literal>. The + evaluator now properly propagates this information. + Consequently, the subpath operator (<literal>~</literal>) has + been deprecated.</para></listitem> + + <listitem><para>Default values of function arguments can now + refer to other function arguments; that is, all arguments are in + scope in the default values + (<literal>NIX-45</literal>).</para></listitem> + + <!-- + <listitem><para>TODO: domain checks (r5895).</para></listitem> + --> + + <listitem><para>Lots of new built-in primitives, such as + functions for list manipulation and integer arithmetic. See the + manual for a complete list. All primops are now available in + the set <varname>builtins</varname>, allowing one to test for + the availability of primop in a backwards-compatible + way.</para></listitem> + + <listitem><para>Real let-expressions: <literal>let x = ...; + ... z = ...; in ...</literal>.</para></listitem> + + </itemizedlist> + + </para></listitem> + + + <listitem><para>New commands <command>nix-pack-closure</command> and + <command>nix-unpack-closure</command> than can be used to easily + transfer a store path with all its dependencies to another machine. + Very convenient whenever you have some package on your machine and + you want to copy it somewhere else.</para></listitem> + + + <listitem><para>XML support: + + <itemizedlist> + + <listitem><para><literal>nix-env -q --xml</literal> prints the + installed or available packages in an XML representation for + easy processing by other tools.</para></listitem> + + <listitem><para><literal>nix-instantiate --eval-only + --xml</literal> prints an XML representation of the resulting + term. (The new flag <option>--strict</option> forces ‘deep’ + evaluation of the result, i.e., list elements and attributes are + evaluated recursively.)</para></listitem> + + <listitem><para>In Nix expressions, the primop + <function>builtins.toXML</function> converts a term to an XML + representation. This is primarily useful for passing structured + information to builders.</para></listitem> + + </itemizedlist> + + </para></listitem> + + + <listitem><para>You can now unambiguously specify which derivation to + build or install in <command>nix-env</command>, + <command>nix-instantiate</command> and <command>nix-build</command> + using the <option>--attr</option> / <option>-A</option> flags, which + takes an attribute name as argument. (Unlike symbolic package names + such as <literal>subversion-1.4.0</literal>, attribute names in an + attribute set are unique.) For instance, a quick way to perform a + test build of a package in Nixpkgs is <literal>nix-build + pkgs/top-level/all-packages.nix -A + <replaceable>foo</replaceable></literal>. <literal>nix-env -q + --attr</literal> shows the attribute names corresponding to each + derivation.</para></listitem> + + + <listitem><para>If the top-level Nix expression used by + <command>nix-env</command>, <command>nix-instantiate</command> or + <command>nix-build</command> evaluates to a function whose arguments + all have default values, the function will be called automatically. + Also, the new command-line switch <option>--arg + <replaceable>name</replaceable> + <replaceable>value</replaceable></option> can be used to specify + function arguments on the command line.</para></listitem> + + + <listitem><para><literal>nix-install-package --url + <replaceable>URL</replaceable></literal> allows a package to be + installed directly from the given URL.</para></listitem> + + + <listitem><para>Nix now works behind an HTTP proxy server; just set + the standard environment variables <envar>http_proxy</envar>, + <envar>https_proxy</envar>, <envar>ftp_proxy</envar> or + <envar>all_proxy</envar> appropriately. Functions such as + <function>fetchurl</function> in Nixpkgs also respect these + variables.</para></listitem> + + + <listitem><para><literal>nix-build -o + <replaceable>symlink</replaceable></literal> allows the symlink to + the build result to be named something other than + <literal>result</literal>.</para></listitem> + + + <!-- Stability / performance / etc. --> + + + <listitem><para>Platform support: + + <itemizedlist> + + <listitem><para>Support for 64-bit platforms, provided a <link + xlink:href="http://bugzilla.sen.cwi.nl:8080/show_bug.cgi?id=606">suitably + patched ATerm library</link> is used. Also, files larger than 2 + GiB are now supported.</para></listitem> + + <listitem><para>Added support for Cygwin (Windows, + <literal>i686-cygwin</literal>), Mac OS X on Intel + (<literal>i686-darwin</literal>) and Linux on PowerPC + (<literal>powerpc-linux</literal>).</para></listitem> + + <listitem><para>Users of SMP and multicore machines will + appreciate that the number of builds to be performed in parallel + can now be specified in the configuration file in the + <literal>build-max-jobs</literal> setting.</para></listitem> + + </itemizedlist> + + </para></listitem> + + + <listitem><para>Garbage collector improvements: + + <itemizedlist> + + <listitem><para>Open files (such as running programs) are now + used as roots of the garbage collector. This prevents programs + that have been uninstalled from being garbage collected while + they are still running. The script that detects these + additional runtime roots + (<filename>find-runtime-roots.pl</filename>) is inherently + system-specific, but it should work on Linux and on all + platforms that have the <command>lsof</command> + utility.</para></listitem> + + <listitem><para><literal>nix-store --gc</literal> + (a.k.a. <command>nix-collect-garbage</command>) prints out the + number of bytes freed on standard output. <literal>nix-store + --gc --print-dead</literal> shows how many bytes would be freed + by an actual garbage collection.</para></listitem> + + <listitem><para><literal>nix-collect-garbage -d</literal> + removes all old generations of <emphasis>all</emphasis> profiles + before calling the actual garbage collector (<literal>nix-store + --gc</literal>). This is an easy way to get rid of all old + packages in the Nix store.</para></listitem> + + <listitem><para><command>nix-store</command> now has an + operation <option>--delete</option> to delete specific paths + from the Nix store. It won’t delete reachable (non-garbage) + paths unless <option>--ignore-liveness</option> is + specified.</para></listitem> + + </itemizedlist> + + </para></listitem> + + + <listitem><para>Berkeley DB 4.4’s process registry feature is used + to recover from crashed Nix processes.</para></listitem> + + <!-- <listitem><para>TODO: shared stores.</para></listitem> --> + + <listitem><para>A performance issue has been fixed with the + <literal>referer</literal> table, which stores the inverse of the + <literal>references</literal> table (i.e., it tells you what store + paths refer to a given path). Maintaining this table could take a + quadratic amount of time, as well as a quadratic amount of Berkeley + DB log file space (in particular when running the garbage collector) + (<literal>NIX-23</literal>).</para></listitem> + + <listitem><para>Nix now catches the <literal>TERM</literal> and + <literal>HUP</literal> signals in addition to the + <literal>INT</literal> signal. So you can now do a <literal>killall + nix-store</literal> without triggering a database + recovery.</para></listitem> + + <listitem><para><command>bsdiff</command> updated to version + 4.3.</para></listitem> + + <listitem><para>Substantial performance improvements in expression + evaluation and <literal>nix-env -qa</literal>, all thanks to <link + xlink:href="http://valgrind.org/">Valgrind</link>. Memory use has + been reduced by a factor 8 or so. Big speedup by memoisation of + path hashing.</para></listitem> + + <listitem><para>Lots of bug fixes, notably: + + <itemizedlist> + + <listitem><para>Make sure that the garbage collector can run + successfully when the disk is full + (<literal>NIX-18</literal>).</para></listitem> + + <listitem><para><command>nix-env</command> now locks the profile + to prevent races between concurrent <command>nix-env</command> + operations on the same profile + (<literal>NIX-7</literal>).</para></listitem> + + <listitem><para>Removed misleading messages from + <literal>nix-env -i</literal> (e.g., <literal>installing + `foo'</literal> followed by <literal>uninstalling + `foo'</literal>) (<literal>NIX-17</literal>).</para></listitem> + + </itemizedlist> + + </para></listitem> + + <listitem><para>Nix source distributions are a lot smaller now since + we no longer include a full copy of the Berkeley DB source + distribution (but only the bits we need).</para></listitem> + + <listitem><para>Header files are now installed so that external + programs can use the Nix libraries.</para></listitem> + +</itemizedlist> + +</section> + + + +<!--==================================================================--> + +<section><title>Release 0.9.2 (September 21, 2005)</title> + +<para>This bug fix release fixes two problems on Mac OS X: + +<itemizedlist> + + <listitem><para>If Nix was linked against statically linked versions + of the ATerm or Berkeley DB library, there would be dynamic link + errors at runtime.</para></listitem> + + <listitem><para><command>nix-pull</command> and + <command>nix-push</command> intermittently failed due to race + conditions involving pipes and child processes with error messages + such as <literal>open2: open(GLOB(0x180b2e4), >&=9) failed: Bad + file descriptor at /nix/bin/nix-pull line 77</literal> (issue + <literal>NIX-14</literal>).</para></listitem> + +</itemizedlist> + +</para> + +</section> + + + +<!--==================================================================--> + +<section><title>Release 0.9.1 (September 20, 2005)</title> + +<para>This bug fix release addresses a problem with the ATerm library +when the <option>--with-aterm</option> flag in +<command>configure</command> was <emphasis>not</emphasis> used.</para> + +</section> + + + +<!--==================================================================--> + +<section><title>Release 0.9 (September 16, 2005)</title> + +<para>NOTE: this version of Nix uses Berkeley DB 4.3 instead of 4.2. +The database is upgraded automatically, but you should be careful not +to use old versions of Nix that still use Berkeley DB 4.2. In +particular, if you use a Nix installed through Nix, you should run + +<screen> +$ nix-store --clear-substitutes</screen> + +first.</para> + + +<itemizedlist> + + <listitem><para>Unpacking of patch sequences is much faster now + since we no longer do redundant unpacking and repacking of + intermediate paths.</para></listitem> + + <listitem><para>Nix now uses Berkeley DB 4.3.</para></listitem> + + <listitem><para>The <function>derivation</function> primitive is + lazier. Attributes of dependent derivations can mutually refer to + each other (as long as there are no data dependencies on the + <varname>outPath</varname> and <varname>drvPath</varname> attributes + computed by <function>derivation</function>).</para> + + <para>For example, the expression <literal>derivation + attrs</literal> now evaluates to (essentially) + + <programlisting> +attrs // { + type = "derivation"; + outPath = derivation! attrs; + drvPath = derivation! attrs; +}</programlisting> + + where <function>derivation!</function> is a primop that does the + actual derivation instantiation (i.e., it does what + <function>derivation</function> used to do). The advantage is that + it allows commands such as <command>nix-env -qa</command> and + <command>nix-env -i</command> to be much faster since they no longer + need to instantiate all derivations, just the + <varname>name</varname> attribute.</para> + + <para>Also, it allows derivations to cyclically reference each + other, for example, + + <programlisting> +webServer = derivation { + ... + hostName = "svn.cs.uu.nl"; + services = [svnService]; +}; +  +svnService = derivation { + ... + hostName = webServer.hostName; +};</programlisting> + + Previously, this would yield a black hole (infinite recursion).</para> + + </listitem> + + <listitem><para><command>nix-build</command> now defaults to using + <filename>./default.nix</filename> if no Nix expression is + specified.</para></listitem> + + <listitem><para><command>nix-instantiate</command>, when applied to + a Nix expression that evaluates to a function, will call the + function automatically if all its arguments have + defaults.</para></listitem> + + <listitem><para>Nix now uses libtool to build dynamic libraries. + This reduces the size of executables.</para></listitem> + + <listitem><para>A new list concatenation operator + <literal>++</literal>. For example, <literal>[1 2 3] ++ [4 5 + 6]</literal> evaluates to <literal>[1 2 3 4 5 + 6]</literal>.</para></listitem> + + <listitem><para>Some currently undocumented primops to support + low-level build management using Nix (i.e., using Nix as a Make + replacement). See the commit messages for <literal>r3578</literal> + and <literal>r3580</literal>.</para></listitem> + + <listitem><para>Various bug fixes and performance + improvements.</para></listitem> + +</itemizedlist> + +</section> + + + +<!--==================================================================--> + +<section><title>Release 0.8.1 (April 13, 2005)</title> + +<para>This is a bug fix release.</para> + +<itemizedlist> + + <listitem><para>Patch downloading was broken.</para></listitem> + + <listitem><para>The garbage collector would not delete paths that + had references from invalid (but substitutable) + paths.</para></listitem> + +</itemizedlist> + +</section> + + + +<!--==================================================================--> + +<section><title>Release 0.8 (April 11, 2005)</title> + +<para>NOTE: the hashing scheme in Nix 0.8 changed (as detailed below). +As a result, <command>nix-pull</command> manifests and channels built +for Nix 0.7 and below will now work anymore. However, the Nix +expression language has not changed, so you can still build from +source. Also, existing user environments continue to work. Nix 0.8 +will automatically upgrade the database schema of previous +installations when it is first run.</para> + +<para>If you get the error message + +<screen> +you have an old-style manifest `/nix/var/nix/manifests/[...]'; please +delete it</screen> + +you should delete previously downloaded manifests: + +<screen> +$ rm /nix/var/nix/manifests/*</screen> + +If <command>nix-channel</command> gives the error message + +<screen> +manifest `http://catamaran.labs.cs.uu.nl/dist/nix/channels/[channel]/MANIFEST' +is too old (i.e., for Nix <= 0.7)</screen> + +then you should unsubscribe from the offending channel +(<command>nix-channel --remove +<replaceable>URL</replaceable></command>; leave out +<literal>/MANIFEST</literal>), and subscribe to the same URL, with +<literal>channels</literal> replaced by <literal>channels-v3</literal> +(e.g., <link +xlink:href='http://catamaran.labs.cs.uu.nl/dist/nix/channels-v3/nixpkgs-unstable' +/>).</para> + +<para>Nix 0.8 has the following improvements: + +<itemizedlist> + + <listitem><para>The cryptographic hashes used in store paths are now + 160 bits long, but encoded in base-32 so that they are still only 32 + characters long (e.g., + <filename>/nix/store/csw87wag8bqlqk7ipllbwypb14xainap-atk-1.9.0</filename>). + (This is actually a 160 bit truncation of a SHA-256 + hash.)</para></listitem> + + <listitem><para>Big cleanups and simplifications of the basic store + semantics. The notion of “closure store expressions” is gone (and + so is the notion of “successors”); the file system references of a + store path are now just stored in the database.</para> + + <para>For instance, given any store path, you can query its closure: + + <screen> +$ nix-store -qR $(which firefox) +... lots of paths ...</screen> + + Also, Nix now remembers for each store path the derivation that + built it (the “deriver”): + + <screen> +$ nix-store -qR $(which firefox) +/nix/store/4b0jx7vq80l9aqcnkszxhymsf1ffa5jd-firefox-1.0.1.drv</screen> + + So to see the build-time dependencies, you can do + + <screen> +$ nix-store -qR $(nix-store -qd $(which firefox))</screen> + + or, in a nicer format: + + <screen> +$ nix-store -q --tree $(nix-store -qd $(which firefox))</screen> + + </para> + + <para>File system references are also stored in reverse. For + instance, you can query all paths that directly or indirectly use a + certain Glibc: + + <screen> +$ nix-store -q --referrers-closure \ + /nix/store/8lz9yc6zgmc0vlqmn2ipcpkjlmbi51vv-glibc-2.3.4</screen> + + </para> + + </listitem> + + <listitem><para>The concept of fixed-output derivations has been + formalised. Previously, functions such as + <function>fetchurl</function> in Nixpkgs used a hack (namely, + explicitly specifying a store path hash) to prevent changes to, say, + the URL of the file from propagating upwards through the dependency + graph, causing rebuilds of everything. This can now be done cleanly + by specifying the <varname>outputHash</varname> and + <varname>outputHashAlgo</varname> attributes. Nix itself checks + that the content of the output has the specified hash. (This is + important for maintaining certain invariants necessary for future + work on secure shared stores.)</para></listitem> + + <listitem><para>One-click installation :-) It is now possible to + install any top-level component in Nixpkgs directly, through the web + — see, e.g., <link + xlink:href='http://catamaran.labs.cs.uu.nl/dist/nixpkgs-0.8/' />. + All you have to do is associate + <filename>/nix/bin/nix-install-package</filename> with the MIME type + <literal>application/nix-package</literal> (or the extension + <filename>.nixpkg</filename>), and clicking on a package link will + cause it to be installed, with all appropriate dependencies. If you + just want to install some specific application, this is easier than + subscribing to a channel.</para></listitem> + + <listitem><para><command>nix-store -r + <replaceable>PATHS</replaceable></command> now builds all the + derivations PATHS in parallel. Previously it did them sequentially + (though exploiting possible parallelism between subderivations). + This is nice for build farms.</para></listitem> + + <listitem><para><command>nix-channel</command> has new operations + <option>--list</option> and + <option>--remove</option>.</para></listitem> + + <listitem><para>New ways of installing components into user + environments: + + <itemizedlist> + + <listitem><para>Copy from another user environment: + + <screen> +$ nix-env -i --from-profile .../other-profile firefox</screen> + + </para></listitem> + + <listitem><para>Install a store derivation directly (bypassing the + Nix expression language entirely): + + <screen> +$ nix-env -i /nix/store/z58v41v21xd3...-aterm-2.3.1.drv</screen> + + (This is used to implement <command>nix-install-package</command>, + which is therefore immune to evolution in the Nix expression + language.)</para></listitem> + + <listitem><para>Install an already built store path directly: + + <screen> +$ nix-env -i /nix/store/hsyj5pbn0d9i...-aterm-2.3.1</screen> + + </para></listitem> + + <listitem><para>Install the result of a Nix expression specified + as a command-line argument: + + <screen> +$ nix-env -f .../i686-linux.nix -i -E 'x: x.firefoxWrapper'</screen> + + The difference with the normal installation mode is that + <option>-E</option> does not use the <varname>name</varname> + attributes of derivations. Therefore, this can be used to + disambiguate multiple derivations with the same + name.</para></listitem> + + </itemizedlist></para></listitem> + + <listitem><para>A hash of the contents of a store path is now stored + in the database after a successful build. This allows you to check + whether store paths have been tampered with: <command>nix-store + --verify --check-contents</command>.</para></listitem> + + <listitem> + + <para>Implemented a concurrent garbage collector. It is now + always safe to run the garbage collector, even if other Nix + operations are happening simultaneously.</para> + + <para>However, there can still be GC races if you use + <command>nix-instantiate</command> and <command>nix-store + --realise</command> directly to build things. To prevent races, + use the <option>--add-root</option> flag of those commands.</para> + + </listitem> + + <listitem><para>The garbage collector now finally deletes paths in + the right order (i.e., topologically sorted under the “references” + relation), thus making it safe to interrupt the collector without + risking a store that violates the closure + invariant.</para></listitem> + + <listitem><para>Likewise, the substitute mechanism now downloads + files in the right order, thus preserving the closure invariant at + all times.</para></listitem> + + <listitem><para>The result of <command>nix-build</command> is now + registered as a root of the garbage collector. If the + <filename>./result</filename> link is deleted, the GC root + disappears automatically.</para></listitem> + + <listitem> + + <para>The behaviour of the garbage collector can be changed + globally by setting options in + <filename>/nix/etc/nix/nix.conf</filename>. + + <itemizedlist> + + <listitem><para><literal>gc-keep-derivations</literal> specifies + whether deriver links should be followed when searching for live + paths.</para></listitem> + + <listitem><para><literal>gc-keep-outputs</literal> specifies + whether outputs of derivations should be followed when searching + for live paths.</para></listitem> + + <listitem><para><literal>env-keep-derivations</literal> + specifies whether user environments should store the paths of + derivations when they are added (thus keeping the derivations + alive).</para></listitem> + + </itemizedlist> + + </para></listitem> + + <listitem><para>New <command>nix-env</command> query flags + <option>--drv-path</option> and + <option>--out-path</option>.</para></listitem> + + <listitem><para><command>fetchurl</command> allows SHA-1 and SHA-256 + in addition to MD5. Just specify the attribute + <varname>sha1</varname> or <varname>sha256</varname> instead of + <varname>md5</varname>.</para></listitem> + + <listitem><para>Manual updates.</para></listitem> + +</itemizedlist> + +</para> + +</section> + + + +<!--==================================================================--> + +<section><title>Release 0.7 (January 12, 2005)</title> + +<itemizedlist> + + <listitem><para>Binary patching. When upgrading components using + pre-built binaries (through nix-pull / nix-channel), Nix can + automatically download and apply binary patches to already installed + components instead of full downloads. Patching is “smart”: if there + is a <emphasis>sequence</emphasis> of patches to an installed + component, Nix will use it. Patches are currently generated + automatically between Nixpkgs (pre-)releases.</para></listitem> + + <listitem><para>Simplifications to the substitute + mechanism.</para></listitem> + + <listitem><para>Nix-pull now stores downloaded manifests in + <filename>/nix/var/nix/manifests</filename>.</para></listitem> + + <listitem><para>Metadata on files in the Nix store is canonicalised + after builds: the last-modified timestamp is set to 0 (00:00:00 + 1/1/1970), the mode is set to 0444 or 0555 (readable and possibly + executable by all; setuid/setgid bits are dropped), and the group is + set to the default. This ensures that the result of a build and an + installation through a substitute is the same; and that timestamp + dependencies are revealed.</para></listitem> + +</itemizedlist> + +</section> + + + +<!--==================================================================--> + +<section><title>Release 0.6 (November 14, 2004)</title> + +<itemizedlist> + + <listitem> + <para>Rewrite of the normalisation engine. + + <itemizedlist> + + <listitem><para>Multiple builds can now be performed in parallel + (option <option>-j</option>).</para></listitem> + + <listitem><para>Distributed builds. Nix can now call a shell + script to forward builds to Nix installations on remote + machines, which may or may not be of the same platform + type.</para></listitem> + + <listitem><para>Option <option>--fallback</option> allows + recovery from broken substitutes.</para></listitem> + + <listitem><para>Option <option>--keep-going</option> causes + building of other (unaffected) derivations to continue if one + failed.</para></listitem> + + </itemizedlist> + + </para> + + </listitem> + + <listitem><para>Improvements to the garbage collector (i.e., it + should actually work now).</para></listitem> + + <listitem><para>Setuid Nix installations allow a Nix store to be + shared among multiple users.</para></listitem> + + <listitem><para>Substitute registration is much faster + now.</para></listitem> + + <listitem><para>A utility <command>nix-build</command> to build a + Nix expression and create a symlink to the result int the current + directory; useful for testing Nix derivations.</para></listitem> + + <listitem><para>Manual updates.</para></listitem> + + <listitem> + + <para><command>nix-env</command> changes: + + <itemizedlist> + + <listitem><para>Derivations for other platforms are filtered out + (which can be overridden using + <option>--system-filter</option>).</para></listitem> + + <listitem><para><option>--install</option> by default now + uninstall previous derivations with the same + name.</para></listitem> + + <listitem><para><option>--upgrade</option> allows upgrading to a + specific version.</para></listitem> + + <listitem><para>New operation + <option>--delete-generations</option> to remove profile + generations (necessary for effective garbage + collection).</para></listitem> + + <listitem><para>Nicer output (sorted, + columnised).</para></listitem> + + </itemizedlist> + + </para> + + </listitem> + + <listitem><para>More sensible verbosity levels all around (builder + output is now shown always, unless <option>-Q</option> is + given).</para></listitem> + + <listitem> + + <para>Nix expression language changes: + + <itemizedlist> + + <listitem><para>New language construct: <literal>with + <replaceable>E1</replaceable>; + <replaceable>E2</replaceable></literal> brings all attributes + defined in the attribute set <replaceable>E1</replaceable> in + scope in <replaceable>E2</replaceable>.</para></listitem> + + <listitem><para>Added a <function>map</function> + function.</para></listitem> + + <listitem><para>Various new operators (e.g., string + concatenation).</para></listitem> + + </itemizedlist> + + </para> + + </listitem> + + <listitem><para>Expression evaluation is much + faster.</para></listitem> + + <listitem><para>An Emacs mode for editing Nix expressions (with + syntax highlighting and indentation) has been + added.</para></listitem> + + <listitem><para>Many bug fixes.</para></listitem> + +</itemizedlist> + +</section> + + + +<!--==================================================================--> + +<section><title>Release 0.5 and earlier</title> + +<para>Please refer to the Subversion commit log messages.</para> + +</section> + + + +</article> diff --git a/doc/manual/schemas.xml b/doc/manual/schemas.xml new file mode 100644 index 000000000000..691a517b9c26 --- /dev/null +++ b/doc/manual/schemas.xml @@ -0,0 +1,4 @@ +<?xml version="1.0"?> +<locatingRules xmlns="http://thaiopensource.com/ns/locating-rules/1.0"> + <uri pattern="*.xml" typeId="DocBook"/> +</locatingRules> diff --git a/doc/manual/style.css b/doc/manual/style.css new file mode 100644 index 000000000000..ac76a64bbb21 --- /dev/null +++ b/doc/manual/style.css @@ -0,0 +1,255 @@ +/* Copied from http://bakefile.sourceforge.net/, which appears + licensed under the GNU GPL. */ + + +/*************************************************************************** + Basic headers and text: + ***************************************************************************/ + +body +{ + font-family: "Nimbus Sans L", sans-serif; + background: white; + margin: 2em 1em 2em 1em; +} + +h1, h2, h3, h4 +{ + color: #005aa0; +} + +h1 /* title */ +{ + font-size: 200%; +} + +h2 /* chapters, appendices, subtitle */ +{ + font-size: 180%; +} + +/* Extra space between chapters, appendices. */ +div.chapter > div.titlepage h2, div.appendix > div.titlepage h2 +{ + margin-top: 1.5em; +} + +div.section > div.titlepage h2 /* sections */ +{ + font-size: 150%; + margin-top: 1.5em; +} + +h3 /* subsections */ +{ + font-size: 125%; +} + +div.simplesect h2 +{ + font-size: 110%; +} + +div.appendix h3 +{ + font-size: 150%; + margin-top: 1.5em; +} + +div.refnamediv h2, div.refsynopsisdiv h2, div.refsection h2 /* refentry parts */ +{ + margin-top: 1.4em; + font-size: 125%; +} + +div.refsection h3 +{ + font-size: 110%; +} + + +/*************************************************************************** + Examples: + ***************************************************************************/ + +div.example +{ + border: 1px solid #b0b0b0; + padding: 6px 6px; + margin-left: 1.5em; + margin-right: 1.5em; + background: #f4f4f8; + border-radius: 0.4em; + box-shadow: 0.4em 0.4em 0.5em #e0e0e0; +} + +div.example p.title +{ + margin-top: 0em; +} + +div.example pre +{ + box-shadow: none; +} + + +/*************************************************************************** + Screen dumps: + ***************************************************************************/ + +pre.screen, pre.programlisting +{ + border: 1px solid #b0b0b0; + padding: 3px 3px; + margin-left: 1.5em; + margin-right: 1.5em; + color: #600000; + background: #f4f4f8; + font-family: monospace; + border-radius: 0.4em; + box-shadow: 0.4em 0.4em 0.5em #e0e0e0; +} + +div.example pre.programlisting +{ + border: 0px; + padding: 0 0; + margin: 0 0 0 0; +} + + +/*************************************************************************** + Notes, warnings etc: + ***************************************************************************/ + +.note, .warning +{ + border: 1px solid #b0b0b0; + padding: 3px 3px; + margin-left: 1.5em; + margin-right: 1.5em; + margin-bottom: 1em; + padding: 0.3em 0.3em 0.3em 0.3em; + background: #fffff5; + border-radius: 0.4em; + box-shadow: 0.4em 0.4em 0.5em #e0e0e0; +} + +div.note, div.warning +{ + font-style: italic; +} + +div.note h3, div.warning h3 +{ + color: red; + font-size: 100%; + padding-right: 0.5em; + display: inline; +} + +div.note p, div.warning p +{ + margin-bottom: 0em; +} + +div.note h3 + p, div.warning h3 + p +{ + display: inline; +} + +div.note h3 +{ + color: blue; + font-size: 100%; +} + +div.navfooter * +{ + font-size: 90%; +} + + +/*************************************************************************** + Links colors and highlighting: + ***************************************************************************/ + +a { text-decoration: none; } +a:hover { text-decoration: underline; } +a:link { color: #0048b3; } +a:visited { color: #002a6a; } + + +/*************************************************************************** + Table of contents: + ***************************************************************************/ + +div.toc +{ + font-size: 90%; +} + +div.toc dl +{ + margin-top: 0em; + margin-bottom: 0em; +} + + +/*************************************************************************** + Special elements: + ***************************************************************************/ + +tt, code +{ + color: #400000; +} + +.term +{ + font-weight: bold; + +} + +div.variablelist dd p, div.glosslist dd p +{ + margin-top: 0em; +} + +div.variablelist dd, div.glosslist dd +{ + margin-left: 1.5em; +} + +div.glosslist dt +{ + font-style: italic; +} + +.varname +{ + color: #400000; +} + +span.command strong +{ + font-weight: normal; + color: #400000; +} + +div.calloutlist table +{ + box-shadow: none; +} + +table +{ + border-collapse: collapse; + box-shadow: 0.4em 0.4em 0.5em #e0e0e0; +} + +div.affiliation +{ + font-style: italic; +} \ No newline at end of file diff --git a/doc/manual/troubleshooting.xml b/doc/manual/troubleshooting.xml new file mode 100644 index 000000000000..ec8c4c924fc7 --- /dev/null +++ b/doc/manual/troubleshooting.xml @@ -0,0 +1,92 @@ +<appendix xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink"> + +<title>Troubleshooting</title> + + +<para>This section provides solutions for some common problems. See +the <link xlink:href="http://bugs.strategoxt.org/browse/NIX">Nix +bug tracker</link> for a list of currently known issues.</para> + + +<section><title>Collisions in <command>nix-env</command></title> + +<para>Symptom: when installing or upgrading, you get an error message such as + +<screen> +$ nix-env -i docbook-xml +... +adding /nix/store/s5hyxgm62gk2...-docbook-xml-4.2 +collision between `/nix/store/s5hyxgm62gk2...-docbook-xml-4.2/xml/dtd/docbook/calstblx.dtd' + and `/nix/store/06h377hr4b33...-docbook-xml-4.3/xml/dtd/docbook/calstblx.dtd' + at /nix/store/...-builder.pl line 62.</screen> + +</para> + +<para>The cause is that two installed packages in the user environment +have overlapping filenames (e.g., +<filename>xml/dtd/docbook/calstblx.dtd</filename>. This usually +happens when you accidentally try to install two versions of the same +package. For instance, in the example above, the Nix Packages +collection contains two versions of <literal>docbook-xml</literal>, so +<command>nix-env -i</command> will try to install both. The default +user environment builder has no way to way to resolve such conflicts, +so it just gives up.</para> + +<para>Solution: remove one of the offending packages from the user +environment (if already installed) using <command>nix-env +-e</command>, or specify exactly which version should be installed +(e.g., <literal>nix-env -i docbook-xml-4.2</literal>).</para> + +<para>Alternatively, you can modify the user environment builder +script (in +<filename><replaceable>prefix</replaceable>/share/nix/corepkgs/buildenv/builder.pl</filename>) +to implement some conflict resolution policy. E.g., the script could +be modified to rename conflicting file names, or to pick one over the +other.</para> + +</section> + + +<section><title><quote>Too many links</quote> error in the Nix +store</title> + + +<para>Symptom: when building something, you get an error message such as + +<screen> +... +<literal>mkdir: cannot create directory `/nix/store/<replaceable>name</replaceable>': Too many links</literal></screen> + +</para> + +<para>This is usually because you have more than 32,000 subdirectories +in <filename>/nix/store</filename>, as can be seen using <command>ls +-l</command>: + +<screen> +$ ls -l /nix/store +drwxrwxrwt 32000 nix nix 4620288 Sep 8 15:08 store</screen> + +The <literal>ext2</literal> file system is limited to an inode link +count of 32,000 (each subdirectory increasing the count by one). +Furthermore, the <literal>st_nlink</literal> field of the +<function>stat</function> system call is a 16-bit value.</para> + +<para>This only happens on very large Nix installations (such as build +machines).</para> + +<para>Quick solution: run the garbage collector. You may want to use +the <option>--max-links</option> option.</para> + +<para>Real solution: put the Nix store on a file system that supports +more than 32,000 subdirectories per directory, such as ReiserFS. +(This doesn’t solve the <literal>st_nlink</literal> limit, but +ReiserFS lies to the kernel by reporting a link count of 1 if it +exceeds the limit.)</para> + +</section> + + + +</appendix> diff --git a/doc/manual/writing-nix-expressions.xml b/doc/manual/writing-nix-expressions.xml new file mode 100644 index 000000000000..3e6cd7379015 --- /dev/null +++ b/doc/manual/writing-nix-expressions.xml @@ -0,0 +1,1874 @@ +<chapter xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xml:id='chap-writing-nix-expressions' + xmlns:xi="http://www.w3.org/2001/XInclude"> + +<title>Writing Nix Expressions</title> + + +<para>This chapter shows you how to write Nix expressions, which are +the things that tell Nix how to build packages. It starts with a +simple example (a Nix expression for GNU Hello), and then moves +on to a more in-depth look at the Nix expression language.</para> + +<note><para>This chapter is mostly about the Nix expression language. +For more extensive information on adding packages to the Nix Packages +collection (such as functions in the standard environment and coding +conventions), please consult <link +xlink:href="http://nixos.org/nixpkgs/manual/">its +manual</link>.</para></note> + + +<section><title>A simple Nix expression</title> + +<para>This section shows how to add and test the <link +xlink:href='http://www.gnu.org/software/hello/hello.html'>GNU Hello +package</link> to the Nix Packages collection. Hello is a program +that prints out the text <quote>Hello, world!</quote>.</para> + +<para>To add a package to the Nix Packages collection, you generally +need to do three things: + +<orderedlist> + + <listitem><para>Write a Nix expression for the package. This is a + file that describes all the inputs involved in building the package, + such as dependencies, sources, and so on.</para></listitem> + + <listitem><para>Write a <emphasis>builder</emphasis>. This is a + shell script<footnote><para>In fact, it can be written in any + language, but typically it's a <command>bash</command> shell + script.</para></footnote> that actually builds the package from + the inputs.</para></listitem> + + <listitem><para>Add the package to the file + <filename>pkgs/top-level/all-packages.nix</filename>. The Nix + expression written in the first step is a + <emphasis>function</emphasis>; it requires other packages in order + to build it. In this step you put it all together, i.e., you call + the function with the right arguments to build the actual + package.</para></listitem> + +</orderedlist> + +</para> + + +<section><title>The Nix expression</title> + +<example xml:id='ex-hello-nix'><title>Nix expression for GNU Hello +(<filename>default.nix</filename>)</title> +<programlisting> +{ stdenv, fetchurl, perl }: <co xml:id='ex-hello-nix-co-1' /> + +stdenv.mkDerivation { <co xml:id='ex-hello-nix-co-2' /> + name = "hello-2.1.1"; <co xml:id='ex-hello-nix-co-3' /> + builder = ./builder.sh; <co xml:id='ex-hello-nix-co-4' /> + src = fetchurl { <co xml:id='ex-hello-nix-co-5' /> + url = ftp://ftp.nluug.nl/pub/gnu/hello/hello-2.1.1.tar.gz; + md5 = "70c9ccf9fac07f762c24f2df2290784d"; + }; + inherit perl; <co xml:id='ex-hello-nix-co-6' /> +}</programlisting> +</example> + +<para><xref linkend='ex-hello-nix' /> shows a Nix expression for GNU +Hello. It's actually already in the Nix Packages collection in +<filename>pkgs/applications/misc/hello/ex-1/default.nix</filename>. +It is customary to place each package in a separate directory and call +the single Nix expression in that directory +<filename>default.nix</filename>. The file has the following elements +(referenced from the figure by number): + +<calloutlist> + + <callout arearefs='ex-hello-nix-co-1'> + + <para>This states that the expression is a + <emphasis>function</emphasis> that expects to be called with three + arguments: <varname>stdenv</varname>, <varname>fetchurl</varname>, + and <varname>perl</varname>. They are needed to build Hello, but + we don't know how to build them here; that's why they are function + arguments. <varname>stdenv</varname> is a package that is used + by almost all Nix Packages packages; it provides a + <quote>standard</quote> environment consisting of the things you + would expect in a basic Unix environment: a C/C++ compiler (GCC, + to be precise), the Bash shell, fundamental Unix tools such as + <command>cp</command>, <command>grep</command>, + <command>tar</command>, etc. <varname>fetchurl</varname> is a + function that downloads files. <varname>perl</varname> is the + Perl interpreter.</para> + + <para>Nix functions generally have the form <literal>{ x, y, ..., + z }: e</literal> where <varname>x</varname>, <varname>y</varname>, + etc. are the names of the expected arguments, and where + <replaceable>e</replaceable> is the body of the function. So + here, the entire remainder of the file is the body of the + function; when given the required arguments, the body should + describe how to build an instance of the Hello package.</para> + + </callout> + + <callout arearefs='ex-hello-nix-co-2'> + + <para>So we have to build a package. Building something from + other stuff is called a <emphasis>derivation</emphasis> in Nix (as + opposed to sources, which are built by humans instead of + computers). We perform a derivation by calling + <varname>stdenv.mkDerivation</varname>. + <varname>mkDerivation</varname> is a function provided by + <varname>stdenv</varname> that builds a package from a set of + <emphasis>attributes</emphasis>. A set is just a list of + key/value pairs where each key is a string and each value is an + arbitrary Nix expression. They take the general form <literal>{ + <replaceable>name1</replaceable> = + <replaceable>expr1</replaceable>; <replaceable>...</replaceable> + <replaceable>nameN</replaceable> = + <replaceable>exprN</replaceable>; }</literal>.</para> + + </callout> + + <callout arearefs='ex-hello-nix-co-3'> + + <para>The attribute <varname>name</varname> specifies the symbolic + name and version of the package. Nix doesn't really care about + these things, but they are used by for instance <command>nix-env + -q</command> to show a <quote>human-readable</quote> name for + packages. This attribute is required by + <varname>mkDerivation</varname>.</para> + + </callout> + + <callout arearefs='ex-hello-nix-co-4'> + + <para>The attribute <varname>builder</varname> specifies the + builder. This attribute can sometimes be omitted, in which case + <varname>mkDerivation</varname> will fill in a default builder + (which does a <literal>configure; make; make install</literal>, in + essence). Hello is sufficiently simple that the default builder + would suffice, but in this case, we will show an actual builder + for educational purposes. The value + <command>./builder.sh</command> refers to the shell script shown + in <xref linkend='ex-hello-builder' />, discussed below.</para> + + </callout> + + <callout arearefs='ex-hello-nix-co-5'> + + <para>The builder has to know what the sources of the package + are. Here, the attribute <varname>src</varname> is bound to the + result of a call to the <command>fetchurl</command> function. + Given a URL and an MD5 hash of the expected contents of the file + at that URL, this function builds a derivation that downloads the + file and checks its hash. So the sources are a dependency that + like all other dependencies is built before Hello itself is + built.</para> + + <para>Instead of <varname>src</varname> any other name could have + been used, and in fact there can be any number of sources (bound + to different attributes). However, <varname>src</varname> is + customary, and it's also expected by the default builder (which we + don't use in this example).</para> + + </callout> + + <callout arearefs='ex-hello-nix-co-6'> + + <para>Since the derivation requires Perl, we have to pass the + value of the <varname>perl</varname> function argument to the + builder. All attributes in the set are actually passed as + environment variables to the builder, so declaring an attribute + + <programlisting> +perl = perl;</programlisting> + + will do the trick: it binds an attribute <varname>perl</varname> + to the function argument which also happens to be called + <varname>perl</varname>. However, it looks a bit silly, so there + is a shorter syntax. The <literal>inherit</literal> keyword + causes the specified attributes to be bound to whatever variables + with the same name happen to be in scope.</para> + + </callout> + +</calloutlist> + +</para> + +</section> + + +<section><title>The builder</title> + +<example xml:id='ex-hello-builder'><title>Build script for GNU Hello +(<filename>builder.sh</filename>)</title> +<programlisting> +source $stdenv/setup <co xml:id='ex-hello-builder-co-1' /> + +PATH=$perl/bin:$PATH <co xml:id='ex-hello-builder-co-2' /> + +tar xvfz $src <co xml:id='ex-hello-builder-co-3' /> +cd hello-* +./configure --prefix=$out <co xml:id='ex-hello-builder-co-4' /> +make <co xml:id='ex-hello-builder-co-5' /> +make install</programlisting> +</example> + +<para><xref linkend='ex-hello-builder' /> shows the builder referenced +from Hello's Nix expression (stored in +<filename>pkgs/applications/misc/hello/ex-1/builder.sh</filename>). +The builder can actually be made a lot shorter by using the +<emphasis>generic builder</emphasis> functions provided by +<varname>stdenv</varname>, but here we write out the build steps to +elucidate what a builder does. It performs the following +steps:</para> + +<calloutlist> + + <callout arearefs='ex-hello-builder-co-1'> + + <para>When Nix runs a builder, it initially completely clears the + environment (except for the attributes declared in the + derivation). For instance, the <envar>PATH</envar> variable is + empty<footnote><para>Actually, it's initialised to + <filename>/path-not-set</filename> to prevent Bash from setting it + to a default value.</para></footnote>. This is done to prevent + undeclared inputs from being used in the build process. If for + example the <envar>PATH</envar> contained + <filename>/usr/bin</filename>, then you might accidentally use + <filename>/usr/bin/gcc</filename>.</para> + + <para>So the first step is to set up the environment. This is + done by calling the <filename>setup</filename> script of the + standard environment. The environment variable + <envar>stdenv</envar> points to the location of the standard + environment being used. (It wasn't specified explicitly as an + attribute in <xref linkend='ex-hello-nix' />, but + <varname>mkDerivation</varname> adds it automatically.)</para> + + </callout> + + <callout arearefs='ex-hello-builder-co-2'> + + <para>Since Hello needs Perl, we have to make sure that Perl is in + the <envar>PATH</envar>. The <envar>perl</envar> environment + variable points to the location of the Perl package (since it + was passed in as an attribute to the derivation), so + <filename><replaceable>$perl</replaceable>/bin</filename> is the + directory containing the Perl interpreter.</para> + + </callout> + + <callout arearefs='ex-hello-builder-co-3'> + + <para>Now we have to unpack the sources. The + <varname>src</varname> attribute was bound to the result of + fetching the Hello source tarball from the network, so the + <envar>src</envar> environment variable points to the location in + the Nix store to which the tarball was downloaded. After + unpacking, we <command>cd</command> to the resulting source + directory.</para> + + <para>The whole build is performed in a temporary directory + created in <varname>/tmp</varname>, by the way. This directory is + removed after the builder finishes, so there is no need to clean + up the sources afterwards. Also, the temporary directory is + always newly created, so you don't have to worry about files from + previous builds interfering with the current build.</para> + + </callout> + + <callout arearefs='ex-hello-builder-co-4'> + + <para>GNU Hello is a typical Autoconf-based package, so we first + have to run its <filename>configure</filename> script. In Nix + every package is stored in a separate location in the Nix store, + for instance + <filename>/nix/store/9a54ba97fb71b65fda531012d0443ce2-hello-2.1.1</filename>. + Nix computes this path by cryptographically hashing all attributes + of the derivation. The path is passed to the builder through the + <envar>out</envar> environment variable. So here we give + <filename>configure</filename> the parameter + <literal>--prefix=$out</literal> to cause Hello to be installed in + the expected location.</para> + + </callout> + + <callout arearefs='ex-hello-builder-co-5'> + + <para>Finally we build Hello (<literal>make</literal>) and install + it into the location specified by <envar>out</envar> + (<literal>make install</literal>).</para> + + </callout> + +</calloutlist> + +<para>If you are wondering about the absence of error checking on the +result of various commands called in the builder: this is because the +shell script is evaluated with Bash's <option>-e</option> option, +which causes the script to be aborted if any command fails without an +error check.</para> + +</section> + + +<section><title>Composition</title> + +<example xml:id='ex-hello-composition'><title>Composing GNU Hello +(<filename>all-packages.nix</filename>)</title> +<programlisting> +... + +rec { <co xml:id='ex-hello-composition-co-1' /> + + hello = import ../applications/misc/hello/ex-1 <co xml:id='ex-hello-composition-co-2' /> { <co xml:id='ex-hello-composition-co-3' /> + inherit fetchurl stdenv perl; + }; + + perl = import ../development/interpreters/perl { <co xml:id='ex-hello-composition-co-4' /> + inherit fetchurl stdenv; + }; + + fetchurl = import ../build-support/fetchurl { + inherit stdenv; ... + }; + + stdenv = ...; + +} +</programlisting> +</example> + +<para>The Nix expression in <xref linkend='ex-hello-nix' /> is a +function; it is missing some arguments that have to be filled in +somewhere. In the Nix Packages collection this is done in the file +<filename>pkgs/top-level/all-packages.nix</filename>, where all +Nix expressions for packages are imported and called with the +appropriate arguments. <xref linkend='ex-hello-composition' /> shows +some fragments of +<filename>all-packages.nix</filename>.</para> + +<calloutlist> + + <callout arearefs='ex-hello-composition-co-1'> + + <para>This file defines a set of attributes, all of which are + concrete derivations (i.e., not functions). In fact, we define a + <emphasis>mutually recursive</emphasis> set of attributes. That + is, the attributes can refer to each other. This is precisely + what we want since we want to <quote>plug</quote> the + various packages into each other.</para> + + </callout> + + <callout arearefs='ex-hello-composition-co-2'> + + <para>Here we <emphasis>import</emphasis> the Nix expression for + GNU Hello. The import operation just loads and returns the + specified Nix expression. In fact, we could just have put the + contents of <xref linkend='ex-hello-nix' /> in + <filename>all-packages.nix</filename> at this point. That + would be completely equivalent, but it would make the file rather + bulky.</para> + + <para>Note that we refer to + <filename>../applications/misc/hello/ex-1</filename>, not + <filename>../applications/misc/hello/ex-1/default.nix</filename>. + When you try to import a directory, Nix automatically appends + <filename>/default.nix</filename> to the file name.</para> + + </callout> + + <callout arearefs='ex-hello-composition-co-3'> + + <para>This is where the actual composition takes place. Here we + <emphasis>call</emphasis> the function imported from + <filename>../applications/misc/hello/ex-1</filename> with a set + containing the things that the function expects, namely + <varname>fetchurl</varname>, <varname>stdenv</varname>, and + <varname>perl</varname>. We use inherit again to use the + attributes defined in the surrounding scope (we could also have + written <literal>fetchurl = fetchurl;</literal>, etc.).</para> + + <para>The result of this function call is an actual derivation + that can be built by Nix (since when we fill in the arguments of + the function, what we get is its body, which is the call to + <varname>stdenv.mkDerivation</varname> in <xref + linkend='ex-hello-nix' />).</para> + + <note><para>Nixpkgs has a convenience function + <function>callPackage</function> that imports and calls a + function, filling in any missing arguments by passing the + corresponding attribute from the Nixpkgs set, like this: + +<programlisting> +hello = callPackage ../applications/misc/hello/ex-1 { }; +</programlisting> + + If necessary, you can set or override arguments: + +<programlisting> +hello = callPackage ../applications/misc/hello/ex-1 { stdenv = myStdenv; }; +</programlisting> + + </para></note> + + </callout> + + <callout arearefs='ex-hello-composition-co-4'> + + <para>Likewise, we have to instantiate Perl, + <varname>fetchurl</varname>, and the standard environment.</para> + + </callout> + +</calloutlist> + +</section> + + +<section><title>Testing</title> + +<para>You can now try to build Hello. Of course, you could do +<literal>nix-env -f pkgs/top-level/all-packages.nix -i hello</literal>, +but you may not want to install a possibly broken package just yet. +The best way to test the package is by using the command <command +linkend="sec-nix-build">nix-build</command>, which builds a Nix +expression and creates a symlink named <filename>result</filename> in +the current directory: + +<screen> +$ nix-build pkgs/top-level/all-packages.nix -A hello +building path `/nix/store/632d2b22514d...-hello-2.1.1' +hello-2.1.1/ +hello-2.1.1/intl/ +hello-2.1.1/intl/ChangeLog +<replaceable>...</replaceable> + +$ ls -l result +lrwxrwxrwx ... 2006-09-29 10:43 result -> /nix/store/632d2b22514d...-hello-2.1.1 + +$ ./result/bin/hello +Hello, world!</screen> + +The <link linkend='opt-attr'><option>-A</option></link> option selects +the <literal>hello</literal> attribute from +<filename>all-packages.nix</filename>. This is faster than using the +symbolic package name specified by the <literal>name</literal> +attribute (which also happens to be <literal>hello</literal>) and is +unambiguous (there can be multiple packages with the symbolic name +<literal>hello</literal>, but there can be only one attribute in a set +named <literal>hello</literal>).</para> + +<para><command>nix-build</command> registers the +<filename>./result</filename> symlink as a garbage collection root, so +unless and until you delete the <filename>./result</filename> symlink, +the output of the build will be safely kept on your system. You can +use <command>nix-build</command>’s <option +linkend='opt-out-link'>-o</option> switch to give the symlink another +name.</para> + +<para>Nix has a transactional semantics. Once a build finishes +successfully, Nix makes a note of this in its database: it registers +that the path denoted by <envar>out</envar> is now +<quote>valid</quote>. If you try to build the derivation again, Nix +will see that the path is already valid and finish immediately. If a +build fails, either because it returns a non-zero exit code, because +Nix or the builder are killed, or because the machine crashes, then +the output paths will not be registered as valid. If you try to build +the derivation again, Nix will remove the output paths if they exist +(e.g., because the builder died half-way through <literal>make +install</literal>) and try again. Note that there is no +<quote>negative caching</quote>: Nix doesn't remember that a build +failed, and so a failed build can always be repeated. This is because +Nix cannot distinguish between permanent failures (e.g., a compiler +error due to a syntax error in the source) and transient failures +(e.g., a disk full condition).</para> + +<para>Nix also performs locking. If you run multiple Nix builds +simultaneously, and they try to build the same derivation, the first +Nix instance that gets there will perform the build, while the others +block (or perform other derivations if available) until the build +finishes: + +<screen> +$ nix-build pkgs/top-level/all-packages.nix -A hello +waiting for lock on `/nix/store/0h5b7hp8d4hqfrw8igvx97x1xawrjnac-hello-2.1.1x'</screen> + +So it is always safe to run multiple instances of Nix in parallel +(which isn’t the case with, say, <command>make</command>).</para> + +<para>If you have a system with multiple CPUs, you may want to have +Nix build different derivations in parallel (insofar as possible). +Just pass the option <link linkend='opt-max-jobs'><option>-j +<replaceable>N</replaceable></option></link>, where +<replaceable>N</replaceable> is the maximum number of jobs to be run +in parallel, or set. Typically this should be the number of +CPUs.</para> + +</section> + + +<section><title>The generic builder</title> + +<para>Recall from <xref linkend='ex-hello-builder' /> that the builder +looked something like this: + +<programlisting> +PATH=$perl/bin:$PATH +tar xvfz $src +cd hello-* +./configure --prefix=$out +make +make install</programlisting> + +The builders for almost all Unix packages look like this — set up some +environment variables, unpack the sources, configure, build, and +install. For this reason the standard environment provides some Bash +functions that automate the build process. A builder using the +generic build facilities in shown in <xref linkend='ex-hello-builder2' +/>.</para> + +<example xml:id='ex-hello-builder2'><title>Build script using the generic +build functions</title> +<programlisting> +buildInputs="$perl" <co xml:id='ex-hello-builder2-co-1' /> + +source $stdenv/setup <co xml:id='ex-hello-builder2-co-2' /> + +genericBuild <co xml:id='ex-hello-builder2-co-3' /></programlisting> +</example> + +<calloutlist> + + <callout arearefs='ex-hello-builder2-co-1'> + + <para>The <envar>buildInputs</envar> variable tells + <filename>setup</filename> to use the indicated packages as + <quote>inputs</quote>. This means that if a package provides a + <filename>bin</filename> subdirectory, it's added to + <envar>PATH</envar>; if it has a <filename>include</filename> + subdirectory, it's added to GCC's header search path; and so + on.<footnote><para>How does it work? <filename>setup</filename> + tries to source the file + <filename><replaceable>pkg</replaceable>/nix-support/setup-hook</filename> + of all dependencies. These “setup hooks” can then set up whatever + environment variables they want; for instance, the setup hook for + Perl sets the <envar>PERL5LIB</envar> environment variable to + contain the <filename>lib/site_perl</filename> directories of all + inputs.</para></footnote> + </para> + + </callout> + + <callout arearefs='ex-hello-builder2-co-2'> + + <para>The function <function>genericBuild</function> is defined in + the file <literal>$stdenv/setup</literal>.</para> + + </callout> + + <callout arearefs='ex-hello-builder2-co-3'> + + <para>The final step calls the shell function + <function>genericBuild</function>, which performs the steps that + were done explicitly in <xref linkend='ex-hello-builder' />. The + generic builder is smart enough to figure out whether to unpack + the sources using <command>gzip</command>, + <command>bzip2</command>, etc. It can be customised in many ways; + see <xref linkend='sec-standard-environment' />.</para> + + </callout> + +</calloutlist> + +<para>Discerning readers will note that the +<envar>buildInputs</envar> could just as well have been set in the Nix +expression, like this: + +<programlisting> + buildInputs = [ perl ];</programlisting> + +The <varname>perl</varname> attribute can then be removed, and the +builder becomes even shorter: + +<programlisting> +source $stdenv/setup +genericBuild</programlisting> + +In fact, <varname>mkDerivation</varname> provides a default builder +that looks exactly like that, so it is actually possible to omit the +builder for Hello entirely.</para> + +</section> + + +</section> + + + +<section><title>The Nix expression language</title> + +<para>The Nix expression language is a pure, lazy, functional +language. Purity means that operations in the language don't have +side-effects (for instance, there is no variable assignment). +Laziness means that arguments to functions are evaluated only when +they are needed. Functional means that functions are +<quote>normal</quote> values that can be passed around and manipulated +in interesting ways. The language is not a full-featured, general +purpose language. Its main job is to describe packages, +compositions of packages, and the variability within +packages.</para> + +<para>This section presents the various features of the +language.</para> + + +<section xml:id='ssec-values'><title>Values</title> + + +<simplesect><title>Simple values</title> + +<para>Nix has the following basic data types: + +<itemizedlist> + + <listitem> + + <para><emphasis>Strings</emphasis> can be written in three + ways.</para> + + <para>The most common way is to enclose the string between double + quotes, e.g., <literal>"foo bar"</literal>. Strings can span + multiple lines. The special characters <literal>"</literal> and + <literal>\</literal> and the character sequence + <literal>${</literal> must be escaped by prefixing them with a + backslash (<literal>\</literal>). Newlines, carriage returns and + tabs can be written as <literal>\n</literal>, + <literal>\r</literal> and <literal>\t</literal>, + respectively.</para> + + <para>You can include the result of an expression into a string by + enclosing it in + <literal>${<replaceable>...</replaceable>}</literal>, a feature + known as <emphasis>antiquotation</emphasis>. The enclosed + expression must evaluate to something that can be coerced into a + string (meaning that it must be a string, a path, or a + derivation). For instance, rather than writing + +<programlisting> +"--with-freetype2-library=" + freetype + "/lib"</programlisting> + + (where <varname>freetype</varname> is a derivation), you can + instead write the more natural + +<programlisting> +"--with-freetype2-library=${freetype}/lib"</programlisting> + + The latter is automatically translated to the former. A more + complicated example (from the Nix expression for <link + xlink:href='http://www.trolltech.com/products/qt'>Qt</link>): + +<programlisting> +configureFlags = " + -system-zlib -system-libpng -system-libjpeg + ${if openglSupport then "-dlopen-opengl + -L${mesa}/lib -I${mesa}/include + -L${libXmu}/lib -I${libXmu}/include" else ""} + ${if threadSupport then "-thread" else "-no-thread"} +";</programlisting> + + Note that Nix expressions and strings can be arbitrarily nested; + in this case the outer string contains various antiquotations that + themselves contain strings (e.g., <literal>"-thread"</literal>), + some of which in turn contain expressions (e.g., + <literal>${mesa}</literal>).</para> + + <para>The second way to write string literals is as an + <emphasis>indented string</emphasis>, which is enclosed between + pairs of <emphasis>double single-quotes</emphasis>, like so: + +<programlisting> +'' + This is the first line. + This is the second line. + This is the third line. +''</programlisting> + + This kind of string literal intelligently strips indentation from + the start of each line. To be precise, it strips from each line a + number of spaces equal to the minimal indentation of the string as + a whole (disregarding the indentation of empty lines). For + instance, the first and second line are indented two space, while + the third line is indented four spaces. Thus, two spaces are + stripped from each line, so the resulting string is + +<programlisting> +"This is the first line.\nThis is the second line.\n This is the third line.\n"</programlisting> + + </para> + + <para>Note that the whitespace and newline following the opening + <literal>''</literal> is ignored if there is no non-whitespace + text on the initial line.</para> + + <para>Antiquotation + (<literal>${<replaceable>expr</replaceable>}</literal>) is + supported in indented strings.</para> + + <para>Since <literal>${</literal> and <literal>''</literal> have + special meaning in indented strings, you need a way to quote them. + <literal>${</literal> can be escaped by prefixing it with + <literal>''</literal> (that is, two single quotes), i.e., + <literal>''${</literal>. <literal>''</literal> can be escaped by + prefixing it with <literal>'</literal>, i.e., + <literal>'''</literal>. Finally, linefeed, carriage-return and + tab characters can be written as <literal>''\n</literal>, + <literal>''\r</literal>, <literal>''\t</literal>.</para> + + <para>Indented strings are primarily useful in that they allow + multi-line string literals to follow the indentation of the + enclosing Nix expression, and that less escaping is typically + necessary for strings representing languages such as shell scripts + and configuration files because <literal>''</literal> is much less + common than <literal>"</literal>. Example: + +<programlisting> +stdenv.mkDerivation { + <replaceable>...</replaceable> + postInstall = + '' + mkdir $out/bin $out/etc + cp foo $out/bin + echo "Hello World" > $out/etc/foo.conf + ${if enableBar then "cp bar $out/bin" else ""} + ''; + <replaceable>...</replaceable> +} +</programlisting> + + </para> + + <para>Finally, as a convenience, <emphasis>URIs</emphasis> as + defined in appendix B of <link + xlink:href='http://www.ietf.org/rfc/rfc2396.txt'>RFC 2396</link> + can be written <emphasis>as is</emphasis>, without quotes. For + instance, the string + <literal>"http://example.org/foo.tar.bz2"</literal> + can also be written as + <literal>http://example.org/foo.tar.bz2</literal>.</para> + + </listitem> + + <listitem><para><emphasis>Integers</emphasis>, e.g., + <literal>123</literal>.</para></listitem> + + <listitem><para><emphasis>Paths</emphasis>, e.g., + <filename>/bin/sh</filename> or <filename>./builder.sh</filename>. + A path must contain at least one slash to be recognised as such; for + instance, <filename>builder.sh</filename> is not a + path<footnote><para>It's parsed as an expression that selects the + attribute <varname>sh</varname> from the variable + <varname>builder</varname>.</para></footnote>. If the file name is + relative, i.e., if it does not begin with a slash, it is made + absolute at parse time relative to the directory of the Nix + expression that contained it. For instance, if a Nix expression in + <filename>/foo/bar/bla.nix</filename> refers to + <filename>../xyzzy/fnord.nix</filename>, the absolute path is + <filename>/foo/xyzzy/fnord.nix</filename>.</para></listitem> + + <listitem><para><emphasis>Booleans</emphasis> with values + <literal>true</literal> and + <literal>false</literal>.</para></listitem> + + <listitem><para>The null value, denoted as + <literal>null</literal>.</para></listitem> + +</itemizedlist> + +</para> + +</simplesect> + + +<simplesect><title>Lists</title> + +<para>Lists are formed by enclosing a whitespace-separated list of +values between square brackets. For example, + +<programlisting> +[ 123 ./foo.nix "abc" (f { x = y; }) ]</programlisting> + +defines a list of four elements, the last being the result of a call +to the function <varname>f</varname>. Note that function calls have +to be enclosed in parentheses. If they had been omitted, e.g., + +<programlisting> +[ 123 ./foo.nix "abc" f { x = y; } ]</programlisting> + +the result would be a list of five elements, the fourth one being a +function and the fifth being a set.</para> + +</simplesect> + + +<simplesect><title>Sets</title> + +<para>Sets are really the core of the language, since ultimately the +Nix language is all about creating derivations, which are really just +sets of attributes to be passed to build scripts.</para> + +<para>Sets are just a list of name/value pairs (called +<emphasis>attributes</emphasis>) enclosed in curly brackets, where +each value is an arbitrary expression terminated by a semicolon. For +example: + +<programlisting> +{ x = 123; + text = "Hello"; + y = f { bla = 456; }; +}</programlisting> + +This defines a set with attributes named <varname>x</varname>, +<varname>text</varname>, <varname>y</varname>. The order of the +attributes is irrelevant. An attribute name may only occur +once.</para> + +<para>Attributes can be selected from a set using the +<literal>.</literal> operator. For instance, + +<programlisting> +{ a = "Foo"; b = "Bar"; }.a</programlisting> + +evaluates to <literal>"Foo"</literal>. It is possible to provide a +default value in an attribute selection using the +<literal>or</literal> keyword. For example, + +<programlisting> +{ a = "Foo"; b = "Bar"; }.c or "Xyzzy"</programlisting> + +will evaluate to <literal>"Xyzzy"</literal> because there is no +<varname>c</varname> attribute in the set.</para> + +<para>You can use arbitrary string constants as attribute names by +enclosing them in quotes: + +<programlisting> +{ "foo bar" = 123; "nix-1.0" = 456; }."foo bar" </programlisting> + +This will evaluate to <literal>123</literal>.</para> + +</simplesect> + + +</section> + + +<section><title>Language constructs</title> + + +<simplesect><title>Recursive sets</title> + +<para>Recursive sets are just normal sets, but the attributes can +refer to each other. For example, + +<programlisting> +rec { + x = y; + y = 123; +}.x +</programlisting> + +evaluates to <literal>123</literal>. Note that without +<literal>rec</literal> the binding <literal>x = y;</literal> would +refer to the variable <varname>y</varname> in the surrounding scope, +if one exists, and would be invalid if no such variable exists. That +is, in a normal (non-recursive) set, attributes are not added to the +lexical scope; in a recursive set, they are.</para> + +<para>Recursive sets of course introduce the danger of infinite +recursion. For example, + +<programlisting> +rec { + x = y; + y = x; +}.x</programlisting> + +does not terminate<footnote><para>Actually, Nix detects infinite +recursion in this case and aborts (<quote>infinite recursion +encountered</quote>).</para></footnote>.</para> + +</simplesect> + + +<simplesect><title>Let-expressions</title> + +<para>A let-expression allows you define local variables for an +expression. For instance, + +<programlisting> +let + x = "foo"; + y = "bar"; +in x + y</programlisting> + +evaluates to <literal>"foobar"</literal>. + +</para> + +</simplesect> + + +<simplesect><title>Inheriting attributes</title> + +<para>When defining a set it is often convenient to copy variables +from the surrounding lexical scope (e.g., when you want to propagate +attributes). This can be shortened using the +<literal>inherit</literal> keyword. For instance, + +<programlisting> +let x = 123; in +{ inherit x; + y = 456; +}</programlisting> + +evaluates to <literal>{ x = 123; y = 456; }</literal>. (Note that +this works because <varname>x</varname> is added to the lexical scope +by the <literal>let</literal> construct.) It is also possible to +inherit attributes from another set. For instance, in this fragment +from <filename>all-packages.nix</filename>, + +<programlisting> + graphviz = (import ../tools/graphics/graphviz) { + inherit fetchurl stdenv libpng libjpeg expat x11 yacc; + inherit (xlibs) libXaw; + }; + + xlibs = { + libX11 = ...; + libXaw = ...; + ... + } + + libpng = ...; + libjpg = ...; + ...</programlisting> + +the set used in the function call to the function defined in +<filename>../tools/graphics/graphviz</filename> inherits a number of +variables from the surrounding scope (<varname>fetchurl</varname> +... <varname>yacc</varname>), but also inherits +<varname>libXaw</varname> (the X Athena Widgets) from the +<varname>xlibs</varname> (X11 client-side libraries) set.</para> + +</simplesect> + + +<simplesect xml:id="ss-functions"><title>Functions</title> + +<para>Functions have the following form: + +<programlisting> +<replaceable>pattern</replaceable>: <replaceable>body</replaceable></programlisting> + +The pattern specifies what the argument of the function must look +like, and binds variables in the body to (parts of) the +argument. There are three kinds of patterns:</para> + +<itemizedlist> + + + <listitem><para>If a pattern is a single identifier, then the + function matches any argument. Example: + + <programlisting> +let negate = x: !x; + concat = x: y: x + y; +in if negate true then concat "foo" "bar" else ""</programlisting> + + Note that <function>concat</function> is a function that takes one + argument and returns a function that takes another argument. This + allows partial parameterisation (i.e., only filling some of the + arguments of a function); e.g., + + <programlisting> +map (concat "foo") [ "bar" "bla" "abc" ]</programlisting> + + evaluates to <literal>[ "foobar" "foobla" + "fooabc" ]</literal>.</para></listitem> + + + <listitem><para>A <emphasis>set pattern</emphasis> of the form + <literal>{ name1, name2, …, nameN }</literal> matches a set + containing the listed attributes, and binds the values of those + attributes to variables in the function body. For example, the + function + +<programlisting> +{ x, y, z }: z + y + x</programlisting> + + can only be called with a set containing exactly the attributes + <varname>x</varname>, <varname>y</varname> and + <varname>z</varname>. No other attributes are allowed. If you want + to allow additional arguments, you can use an ellipsis + (<literal>...</literal>): + +<programlisting> +{ x, y, z, ... }: z + y + x</programlisting> + + This works on any set that contains at least the three named + attributes.</para> + + <para>It is possible to provide <emphasis>default values</emphasis> + for attributes, in which case they are allowed to be missing. A + default value is specified by writing + <literal><replaceable>name</replaceable> ? + <replaceable>e</replaceable></literal>, where + <replaceable>e</replaceable> is an arbitrary expression. For example, + +<programlisting> +{ x, y ? "foo", z ? "bar" }: z + y + x</programlisting> + + specifies a function that only requires an attribute named + <varname>x</varname>, but optionally accepts <varname>y</varname> + and <varname>z</varname>.</para></listitem> + + + <listitem><para>An <literal>@</literal>-pattern requires that the + argument matches with the patterns on the left- and right-hand side + of the <literal>@</literal>-sign. For example: + +<programlisting> +args@{ x, y, z, ... }: z + y + x + args.a</programlisting> + + Here <varname>args</varname> is bound to the entire argument, which + is further matches against the pattern <literal>{ x, y, z, + ... }</literal>.</para></listitem> + + +</itemizedlist> + +<para>Note that functions do not have names. If you want to give them +a name, you can bind them to an attribute, e.g., + +<programlisting> +let concat = { x, y }: x + y; +in concat { x = "foo"; y = "bar"; }</programlisting> + +</para> + +</simplesect> + + +<simplesect><title>Conditionals</title> + +<para>Conditionals look like this: + +<programlisting> +if <replaceable>e1</replaceable> then <replaceable>e2</replaceable> else <replaceable>e3</replaceable></programlisting> + +where <replaceable>e1</replaceable> is an expression that should +evaluate to a Boolean value (<literal>true</literal> or +<literal>false</literal>).</para> + +</simplesect> + + +<simplesect><title>Assertions</title> + +<para>Assertions are generally used to check that certain requirements +on or between features and dependencies hold. They look like this: + +<programlisting> +assert <replaceable>e1</replaceable>; <replaceable>e2</replaceable></programlisting> + +where <replaceable>e1</replaceable> is an expression that should +evaluate to a Boolean value. If it evaluates to +<literal>true</literal>, <replaceable>e2</replaceable> is returned; +otherwise expression evaluation is aborted and a backtrace is printed.</para> + +<example xml:id='ex-subversion-nix'><title>Nix expression for Subversion</title> +<programlisting> +{ localServer ? false +, httpServer ? false +, sslSupport ? false +, pythonBindings ? false +, javaSwigBindings ? false +, javahlBindings ? false +, stdenv, fetchurl +, openssl ? null, httpd ? null, db4 ? null, expat, swig ? null, j2sdk ? null +}: + +assert localServer -> db4 != null; <co xml:id='ex-subversion-nix-co-1' /> +assert httpServer -> httpd != null && httpd.expat == expat; <co xml:id='ex-subversion-nix-co-2' /> +assert sslSupport -> openssl != null && (httpServer -> httpd.openssl == openssl); <co xml:id='ex-subversion-nix-co-3' /> +assert pythonBindings -> swig != null && swig.pythonSupport; +assert javaSwigBindings -> swig != null && swig.javaSupport; +assert javahlBindings -> j2sdk != null; + +stdenv.mkDerivation { + name = "subversion-1.1.1"; + ... + openssl = if sslSupport then openssl else null; <co xml:id='ex-subversion-nix-co-4' /> + ... +}</programlisting> +</example> + +<para><xref linkend='ex-subversion-nix' /> show how assertions are +used in the Nix expression for Subversion.</para> + +<calloutlist> + + <callout arearefs='ex-subversion-nix-co-1'> + <para>This assertion states that if Subversion is to have support + for local repositories, then Berkeley DB is needed. So if the + Subversion function is called with the + <varname>localServer</varname> argument set to + <literal>true</literal> but the <varname>db4</varname> argument + set to <literal>null</literal>, then the evaluation fails.</para> + </callout> + + <callout arearefs='ex-subversion-nix-co-2'> + <para>This is a more subtle condition: if Subversion is built with + Apache (<literal>httpServer</literal>) support, then the Expat + library (an XML library) used by Subversion should be same as the + one used by Apache. This is because in this configuration + Subversion code ends up being linked with Apache code, and if the + Expat libraries do not match, a build- or runtime link error or + incompatibility might occur.</para> + </callout> + + <callout arearefs='ex-subversion-nix-co-3'> + <para>This assertion says that in order for Subversion to have SSL + support (so that it can access <literal>https</literal> URLs), an + OpenSSL library must be passed. Additionally, it says that + <emphasis>if</emphasis> Apache support is enabled, then Apache's + OpenSSL should match Subversion's. (Note that if Apache support + is not enabled, we don't care about Apache's OpenSSL.)</para> + </callout> + + <callout arearefs='ex-subversion-nix-co-4'> + <para>The conditional here is not really related to assertions, + but is worth pointing out: it ensures that if SSL support is + disabled, then the Subversion derivation is not dependent on + OpenSSL, even if a non-<literal>null</literal> value was passed. + This prevents an unnecessary rebuild of Subversion if OpenSSL + changes.</para> + </callout> + +</calloutlist> + +</simplesect> + + + +<simplesect><title>With-expressions</title> + +<para>A <emphasis>with-expression</emphasis>, + +<programlisting> +with <replaceable>e1</replaceable>; <replaceable>e2</replaceable></programlisting> + +introduces the set <replaceable>e1</replaceable> into the lexical +scope of the expression <replaceable>e2</replaceable>. For instance, + +<programlisting> +let as = { x = "foo"; y = "bar"; }; +in with as; x + y</programlisting> + +evaluates to <literal>"foobar"</literal> since the +<literal>with</literal> adds the <varname>x</varname> and +<varname>y</varname> attributes of <varname>as</varname> to the +lexical scope in the expression <literal>x + y</literal>. The most +common use of <literal>with</literal> is in conjunction with the +<function>import</function> function. E.g., + +<programlisting> +with (import ./definitions.nix); ...</programlisting> + +makes all attributes defined in the file +<filename>definitions.nix</filename> available as if they were defined +locally in a <literal>rec</literal>-expression.</para> + +</simplesect> + + +<simplesect><title>Comments</title> + +<para>Comments can be single-line, started with a <literal>#</literal> +character, or inline/multi-line, enclosed within <literal>/* +... */</literal>.</para> + +</simplesect> + + +</section> + + +<section><title>Operators</title> + +<para><xref linkend='table-operators' /> lists the operators in the +Nix expression language, in order of precedence (from strongest to +weakest binding).</para> + +<table xml:id='table-operators'> + <title>Operators</title> + <tgroup cols='3'> + <thead> + <row> + <entry>Syntax</entry> + <entry>Associativity</entry> + <entry>Description</entry> + </row> + </thead> + <tbody> + <row> + <entry><replaceable>e</replaceable> <literal>.</literal> + <replaceable>attrpath</replaceable> + [ <literal>or</literal> <replaceable>def</replaceable> ] + </entry> + <entry>none</entry> + <entry>Select attribute denoted by the attribute path + <replaceable>attrpath</replaceable> from set + <replaceable>e</replaceable>. (An attribute path is a + dot-separated list of attribute names.) If the attribute + doesn’t exist, return <replaceable>def</replaceable> if + provided, otherwise abort evaluation.</entry> + </row> + <row> + <entry><replaceable>e1</replaceable> <replaceable>e2</replaceable></entry> + <entry>left</entry> + <entry>Call function <replaceable>e1</replaceable> with + argument <replaceable>e2</replaceable>.</entry> + </row> + <row> + <entry><replaceable>e</replaceable> <literal>?</literal> + <replaceable>attrpath</replaceable></entry> + <entry>none</entry> + <entry>Test whether set <replaceable>e</replaceable> contains + the attribute denoted by <replaceable>attrpath</replaceable>; + return <literal>true</literal> or + <literal>false</literal>.</entry> + </row> + <row> + <entry><replaceable>e1</replaceable> <literal>++</literal> <replaceable>e2</replaceable></entry> + <entry>right</entry> + <entry>List concatenation.</entry> + </row> + <row> + <entry><replaceable>e1</replaceable> <literal>+</literal> <replaceable>e2</replaceable></entry> + <entry>left</entry> + <entry>String or path concatenation.</entry> + </row> + <row> + <entry><literal>!</literal> <replaceable>e</replaceable></entry> + <entry>left</entry> + <entry>Boolean negation.</entry> + </row> + <row> + <entry><replaceable>e1</replaceable> <literal>//</literal> + <replaceable>e2</replaceable></entry> + <entry>right</entry> + <entry>Return a set consisting of the attributes in + <replaceable>e1</replaceable> and + <replaceable>e2</replaceable> (with the latter taking + precedence over the former in case of equally named + attributes).</entry> + </row> + <row> + <entry><replaceable>e1</replaceable> <literal>==</literal> + <replaceable>e2</replaceable></entry> + <entry>none</entry> + <entry>Equality.</entry> + </row> + <row> + <entry><replaceable>e1</replaceable> <literal>!=</literal> + <replaceable>e2</replaceable></entry> + <entry>none</entry> + <entry>Inequality.</entry> + </row> + <row> + <entry><replaceable>e1</replaceable> <literal>&&</literal> + <replaceable>e2</replaceable></entry> + <entry>left</entry> + <entry>Logical AND.</entry> + </row> + <row> + <entry><replaceable>e1</replaceable> <literal>||</literal> + <replaceable>e2</replaceable></entry> + <entry>left</entry> + <entry>Logical OR.</entry> + </row> + <row> + <entry><replaceable>e1</replaceable> <literal>-></literal> + <replaceable>e2</replaceable></entry> + <entry>none</entry> + <entry>Logical implication (equivalent to + <literal>!<replaceable>e1</replaceable> || + <replaceable>e2</replaceable></literal>).</entry> + </row> + </tbody> + </tgroup> +</table> + +</section> + + +<section xml:id="ssec-derivation"><title>Derivations</title> + +<para>The most important built-in function is +<function>derivation</function>, which is used to describe a single +derivation (a build action). It takes as input a set, the attributes +of which specify the inputs of the build.</para> + +<itemizedlist> + + <listitem xml:id="attr-system"><para>There must be an attribute named + <varname>system</varname> whose value must be a string specifying a + Nix platform identifier, such as <literal>"i686-linux"</literal> or + <literal>"powerpc-darwin"</literal><footnote><para>To figure out + your platform identifier, look at the line <quote>Checking for the + canonical Nix system name</quote> in the output of Nix's + <filename>configure</filename> script.</para></footnote> The build + can only be performed on a machine and operating system matching the + platform identifier. (Nix can automatically forward builds for + other platforms by forwarding them to other machines; see <xref + linkend='chap-distributed-builds' />.)</para></listitem> + + <listitem><para>There must be an attribute named + <varname>name</varname> whose value must be a string. This is used + as a symbolic name for the package by <command>nix-env</command>, + and it is appended to the output paths of the + derivation.</para></listitem> + + <listitem><para>There must be an attribute named + <varname>builder</varname> that identifies the program that is + executed to perform the build. It can be either a derivation or a + source (a local file reference, e.g., + <filename>./builder.sh</filename>).</para></listitem> + + <listitem><para>Every attribute is passed as an environment variable + to the builder. Attribute values are translated to environment + variables as follows: + + <itemizedlist> + + <listitem><para>Strings and integers are just passed + verbatim.</para></listitem> + + <listitem><para>A <emphasis>path</emphasis> (e.g., + <filename>../foo/sources.tar</filename>) causes the referenced + file to be copied to the store; its location in the store is put + in the environment variable. The idea is that all sources + should reside in the Nix store, since all inputs to a derivation + should reside in the Nix store.</para></listitem> + + <listitem><para>A <emphasis>derivation</emphasis> causes that + derivation to be built prior to the present derivation; its + default output path is put in the environment + variable.</para></listitem> + + <listitem><para>Lists of the previous types are also allowed. + They are simply concatenated, separated by + spaces.</para></listitem> + + <listitem><para><literal>true</literal> is passed as the string + <literal>1</literal>, <literal>false</literal> and + <literal>null</literal> are passed as an empty string. + </para></listitem> + </itemizedlist> + + </para></listitem> + + <listitem><para>The optional attribute <varname>args</varname> + specifies command-line arguments to be passed to the builder. It + should be a list.</para></listitem> + + <listitem><para>The optional attribute <varname>outputs</varname> + specifies a list of symbolic outputs of the derivation. By default, + a derivation produces a single output path, denoted as + <literal>out</literal>. However, derivations can produce multiple + output paths. This is useful because it allows outputs to be + downloaded or garbage-collected separately. For instance, imagine a + library package that provides a dynamic library, header files, and + documentation. A program that links against the library doesn’t + need the header files and documentation at runtime, and it doesn’t + need the documentation at build time. Thus, the library package + could specify: +<programlisting> +outputs = [ "lib" "headers" "doc" ]; +</programlisting> + This will cause Nix to pass environment variables + <literal>lib</literal>, <literal>headers</literal> and + <literal>doc</literal> to the builder containing the intended store + paths of each output. The builder would typically do something like +<programlisting> +./configure --libdir=$lib/lib --includedir=$headers/include --docdir=$doc/share/doc +</programlisting> + for an Autoconf-style package. You can refer to each output of a + derivation by selecting it as an attribute, e.g. +<programlisting> +buildInputs = [ pkg.lib pkg.headers ]; +</programlisting> + The first element of <varname>output</varname> determines the + <emphasis>default output</emphasis>. Thus, you could also write +<programlisting> +buildInputs = [ pkg pkg.headers ]; +</programlisting> + since <literal>pkg</literal> is equivalent to + <literal>pkg.lib</literal>.</para></listitem> + +</itemizedlist> + +<para>The function <function>mkDerivation</function> in the standard +environment is a wrapper around <function>derivation</function> that +adds a default value for <varname>system</varname> and always uses +Bash as the builder, to which the supplied builder is passed as a +command-line argument. See <xref linkend='sec-standard-environment' +/>.</para> + +<para>The builder is executed as follows: + +<itemizedlist> + + <listitem><para>A temporary directory is created under the directory + specified by <envar>TMPDIR</envar> (default + <filename>/tmp</filename>) where the build will take place. The + current directory is changed to this directory.</para></listitem> + + <listitem><para>The environment is cleared and set to the derivation + attributes, as specified above.</para></listitem> + + <listitem><para>In addition, the following variables are set: + + <itemizedlist> + + <listitem><para><envar>NIX_BUILD_TOP</envar> contains the path of + the temporary directory for this build.</para></listitem> + + <listitem><para>Also, <envar>TMPDIR</envar>, + <envar>TEMPDIR</envar>, <envar>TMP</envar>, <envar>TEMP</envar> + are set to point to the temporary directory. This is to prevent + the builder from accidentally writing temporary files anywhere + else. Doing so might cause interference by other + processes.</para></listitem> + + <listitem><para><envar>PATH</envar> is set to + <filename>/path-not-set</filename> to prevent shells from + initialising it to their built-in default value.</para></listitem> + + <listitem><para><envar>HOME</envar> is set to + <filename>/homeless-shelter</filename> to prevent programs from + using <filename>/etc/passwd</filename> or the like to find the + user's home directory, which could cause impurity. Usually, when + <envar>HOME</envar> is set, it is used as the location of the home + directory, even if it points to a non-existent + path.</para></listitem> + + <listitem><para><envar>NIX_STORE</envar> is set to the path of the + top-level Nix store directory (typically, + <filename>/nix/store</filename>).</para></listitem> + + <listitem><para>For each output declared in + <varname>outputs</varname>, the corresponding environment variable + is set to point to the intended path in the Nix store for that + output. Each output path is a concatenation of the cryptographic + hash of all build inputs, the <varname>name</varname> attribute + and the output name. (The output name is omitted if it’s + <literal>out</literal>.)</para></listitem> + + </itemizedlist> + + </para></listitem> + + <listitem><para>If an output path already exists, it is removed. + Also, locks are acquired to prevent multiple Nix instances from + performing the same build at the same time.</para></listitem> + + <listitem><para>A log of the combined standard output and error is + written to <filename>/nix/var/log/nix</filename>.</para></listitem> + + <listitem><para>The builder is executed with the arguments specified + by the attribute <varname>args</varname>. If it exits with exit + code 0, it is considered to have succeeded.</para></listitem> + + <listitem><para>The temporary directory is removed (unless the + <option>-K</option> option was specified).</para></listitem> + + <listitem><para>If the build was successful, Nix scans each output + path for references to input paths by looking for the hash parts of + the input paths. Since these are potential runtime dependencies, + Nix registers them as dependencies of the output + paths.</para></listitem> + + <listitem><para>After the build, Nix sets the last-modified + timestamp on all files in the build result to 1 (00:00:01 1/1/1970 + UTC), sets the group to the default group, and sets the mode of the + file to 0444 or 0555 (i.e., read-only, with execute permission + enabled if the file was originally executable). Note that possible + <literal>setuid</literal> and <literal>setgid</literal> bits are + cleared. Setuid and setgid programs are not currently supported by + Nix. This is because the Nix archives used in deployment have no + concept of ownership information, and because it makes the build + result dependent on the user performing the build.</para></listitem> + +</itemizedlist> + +</para> + + +<section><title>Advanced attributes</title> + +<para>Derivations can declare some infrequently used optional +attributes.</para> + +<variablelist> + + <varlistentry><term><varname>allowedReferences</varname></term> + + <listitem><para>The optional attribute + <varname>allowedReferences</varname> specifies a list of legal + references (dependencies) of the output of the builder. For + example, + +<programlisting> +allowedReferences = []; +</programlisting> + + enforces that the output of a derivation cannot have any runtime + dependencies on its inputs. This is used in NixOS to check that + generated files such as initial ramdisks for booting Linux don’t + have accidental dependencies on other paths in the Nix + store.</para></listitem> + + </varlistentry> + + + <varlistentry><term><varname>exportReferencesGraph</varname></term> + + <listitem><para>This attribute allows builders access to the + references graph of their inputs. The attribute is a list of + inputs in the Nix store whose references graph the builder needs + to know. The value of this attribute should be a list of pairs + <literal>[ <replaceable>name1</replaceable> + <replaceable>path1</replaceable> <replaceable>name2</replaceable> + <replaceable>path2</replaceable> <replaceable>...</replaceable> + ]</literal>. The references graph of each + <replaceable>pathN</replaceable> will be stored in a text file + <replaceable>nameN</replaceable> in the temporary build directory. + The text files have the format used by <command>nix-store + --register-validity</command> (with the deriver fields left + empty). For example, when the following derivation is built: + +<programlisting> +derivation { + ... + exportReferencesGraph = [ "libfoo-graph" libfoo ]; +}; +</programlisting> + + the references graph of <literal>libfoo</literal> is placed in the + file <filename>libfoo-graph</filename> in the temporary build + directory.</para> + + <para><varname>exportReferencesGraph</varname> is useful for + builders that want to do something with the closure of a store + path. Examples include the builders in NixOS that generate the + initial ramdisk for booting Linux (a <command>cpio</command> + archive containing the closure of the boot script) and the + ISO-9660 image for the installation CD (which is populated with a + Nix store containing the closure of a bootable NixOS + configuration).</para></listitem> + + </varlistentry> + + + <varlistentry xml:id="fixed-output-drvs"> + <term><varname>outputHash</varname></term> + <term><varname>outputHashAlgo</varname></term> + <term><varname>outputHashMode</varname></term> + + <listitem><para>These attributes declare that the derivation is a + so-called <emphasis>fixed-output derivation</emphasis>, which + means that a cryptographic hash of the output is already known in + advance. When the build of a fixed-output derivation finishes, + Nix computes the cryptographic hash of the output and compares it + to the hash declared with these attributes. If there is a + mismatch, the build fails.</para> + + <para>The rationale for fixed-output derivations is derivations + such as those produced by the <function>fetchurl</function> + function. This function downloads a file from a given URL. To + ensure that the downloaded file has not been modified, the caller + must also specify a cryptographic hash of the file. For example, + +<programlisting> +fetchurl { + url = http://ftp.gnu.org/pub/gnu/hello/hello-2.1.1.tar.gz; + md5 = "70c9ccf9fac07f762c24f2df2290784d"; +} +</programlisting> + + It sometimes happens that the URL of the file changes, e.g., + because servers are reorganised or no longer available. We then + must update the call to <function>fetchurl</function>, e.g., + +<programlisting> +fetchurl { + url = ftp://ftp.nluug.nl/pub/gnu/hello/hello-2.1.1.tar.gz; + md5 = "70c9ccf9fac07f762c24f2df2290784d"; +} +</programlisting> + + If a <function>fetchurl</function> derivation was treated like a + normal derivation, the output paths of the derivation and + <emphasis>all derivations depending on it</emphasis> would change. + For instance, if we were to change the URL of the Glibc source + distribution in Nixpkgs (a package on which almost all other + packages depend) massive rebuilds would be needed. This is + unfortunate for a change which we know cannot have a real effect + as it propagates upwards through the dependency graph.</para> + + <para>For fixed-output derivations, on the other hand, the name of + the output path only depends on the <varname>outputHash*</varname> + and <varname>name</varname> attributes, while all other attributes + are ignored for the purpose of computing the output path. (The + <varname>name</varname> attribute is included because it is part + of the path.)</para> + + <para>As an example, here is the (simplified) Nix expression for + <varname>fetchurl</varname>: + +<programlisting> +{ stdenv, curl }: # The <command>curl</command> program is used for downloading. + +{ url, md5 }: + +stdenv.mkDerivation { + name = baseNameOf (toString url); + builder = ./builder.sh; + buildInputs = [ curl ]; + + # This is a fixed-output derivation; the output must be a regular + # file with MD5 hash <varname>md5</varname>. + outputHashMode = "flat"; + outputHashAlgo = "md5"; + outputHash = md5; + + inherit url; +} +</programlisting> + + </para> + + <para>The <varname>outputHashAlgo</varname> attribute specifies + the hash algorithm used to compute the hash. It can currently be + <literal>"md5"</literal>, <literal>"sha1"</literal> or + <literal>"sha256"</literal>.</para> + + <para>The <varname>outputHashMode</varname> attribute determines + how the hash is computed. It must be one of the following two + values: + + <variablelist> + + <varlistentry><term><literal>"flat"</literal></term> + + <listitem><para>The output must be a non-executable regular + file. If it isn’t, the build fails. The hash is simply + computed over the contents of that file (so it’s equal to what + Unix commands like <command>md5sum</command> or + <command>sha1sum</command> produce).</para> + + <para>This is the default.</para></listitem> + + </varlistentry> + + <varlistentry><term><literal>"recursive"</literal></term> + + <listitem><para>The hash is computed over the NAR archive dump + of the output (i.e., the result of <link + linkend="refsec-nix-store-dump"><command>nix-store + --dump</command></link>). In this case, the output can be + anything, including a directory tree.</para></listitem> + + </varlistentry> + + </variablelist> + + </para> + + <para>The <varname>outputHash</varname> attribute, finally, must + be a string containing the hash in either hexadecimal or base-32 + notation. (See the <link + linkend="sec-nix-hash"><command>nix-hash</command> command</link> + for information about converting to and from base-32 + notation.)</para></listitem> + + </varlistentry> + + + <varlistentry><term><varname>impureEnvVars</varname></term> + + <listitem><para>This attribute allows you to specify a list of + environment variables that should be passed from the environment + of the calling user to the builder. Usually, the environment is + cleared completely when the builder is executed, but with this + attribute you can allow specific environment variables to be + passed unmodified. For example, <function>fetchurl</function> in + Nixpkgs has the line + +<programlisting> +impureEnvVars = [ "http_proxy" "https_proxy" <replaceable>...</replaceable> ]; +</programlisting> + + to make it use the proxy server configuration specified by the + user in the environment variables <envar>http_proxy</envar> and + friends.</para> + + <para>This attribute is only allowed in <link + linkend="fixed-output-drvs">fixed-output derivations</link>, where + impurities such as these are okay since (the hash of) the output + is known in advance. It is ignored for all other + derivations.</para></listitem> + + </varlistentry> + + + <varlistentry><term><varname>preferLocalBuild</varname></term> + + <listitem><para>If this attribute is set to + <literal>true</literal> and <link + linkend="chap-distributed-builds">distributed building is + enabled</link>, then, if possible, perform this build locally + instead of forwarding it to a remote machine. This is appropriate + for trivial builders where the cost of doing a remote build would + exceed the cost of building locally.</para></listitem> + + </varlistentry> + +</variablelist> + +</section> + + +</section> + + + +<xi:include href="builtins.xml" /> + + +</section> + + + +<section xml:id='sec-standard-environment'><title>The standard environment</title> + + +<para>The standard environment is used by passing it as an input +called <envar>stdenv</envar> to the derivation, and then doing + +<programlisting> +source $stdenv/setup</programlisting> + +at the top of the builder.</para> + +<para>Apart from adding the aforementioned commands to the +<envar>PATH</envar>, <filename>setup</filename> also does the +following: + +<itemizedlist> + + <listitem><para>All input packages specified in the + <envar>buildInputs</envar> environment variable have their + <filename>/bin</filename> subdirectory added to <envar>PATH</envar>, + their <filename>/include</filename> subdirectory added to the C/C++ + header file search path, and their <filename>/lib</filename> + subdirectory added to the linker search path. This can be extended. + For instance, when the <command>pkgconfig</command> package is + used, the subdirectory <filename>/lib/pkgconfig</filename> of each + input is added to the <envar>PKG_CONFIG_PATH</envar> environment + variable.</para></listitem> + + <listitem><para>The environment variable + <envar>NIX_CFLAGS_STRIP</envar> is set so that the compiler strips + debug information from object files. This can be disabled by + setting <envar>NIX_STRIP_DEBUG</envar> to + <literal>0</literal>.</para></listitem> + +</itemizedlist> + +</para> + +<para>The <filename>setup</filename> script also exports a function +called <function>genericBuild</function> that knows how to build +typical Autoconf-style packages. It can be customised to perform +builds for any type of package. It is advisable to use +<function>genericBuild</function> since it provides facilities that +are almost always useful such as unpacking of sources, patching of +sources, nested logging, etc.</para> + +<para>The definitive, up-to-date documentation of the generic builder +is the source itself, which resides in +<filename>pkgs/stdenv/generic/setup.sh</filename>.</para> + + +<section><title>Customising the generic builder</title> + +<para>The operation of the generic builder can be modified in many +places by setting certain variables. These <emphasis>hook +variables</emphasis> are typically set to the name of some shell +function defined by you. For instance, to perform some additional +steps after <command>make install</command> you would set the +<varname>postInstall</varname> variable: + +<programlisting> +postInstall=myPostInstall + +myPostInstall() { + mkdir $out/share/extra + cp extrafiles/* $out/share/extra +}</programlisting> + +</para> + + +</section> + + +<section><title>Debugging failed builds</title> + +<para>At the beginning of each phase, the set of all shell variables +is written to the file <filename>env-vars</filename> at the top-level +build directory. This is useful for debugging: it allows you to +recreate the environment in which a build was performed. For +instance, if a build fails, then assuming you used the +<option>-K</option> flag, you can go to the output directory and +<quote>switch</quote> to the environment of the builder: + +<screen> +$ nix-build -K ./foo.nix +... fails, keeping build directory `/tmp/nix-1234-0' + +$ cd /tmp/nix-1234-0 + +$ source env-vars + +<lineannotation>(edit some files...)</lineannotation> + +$ make + +<lineannotation>(execution continues with the same GCC, make, etc.)</lineannotation></screen> + +</para> + +</section> + + +</section> + + +</chapter> |
