about summary refs log tree commit diff
path: root/third_party/nix/doc/manual/advanced-topics/distributed-builds.xml
diff options
context:
space:
mode:
Diffstat (limited to 'third_party/nix/doc/manual/advanced-topics/distributed-builds.xml')
-rw-r--r--third_party/nix/doc/manual/advanced-topics/distributed-builds.xml190
1 files changed, 190 insertions, 0 deletions
diff --git a/third_party/nix/doc/manual/advanced-topics/distributed-builds.xml b/third_party/nix/doc/manual/advanced-topics/distributed-builds.xml
new file mode 100644
index 000000000000..9ac4a92cd5b1
--- /dev/null
+++ b/third_party/nix/doc/manual/advanced-topics/distributed-builds.xml
@@ -0,0 +1,190 @@
+<chapter xmlns="http://docbook.org/ns/docbook"
+      xmlns:xlink="http://www.w3.org/1999/xlink"
+      xmlns:xi="http://www.w3.org/2001/XInclude"
+      version="5.0"
+      xml:id='chap-distributed-builds'>
+
+<title>Remote Builds</title>
+
+<para>Nix supports remote builds, where a local Nix installation can
+forward Nix builds to other machines.  This allows multiple builds to
+be performed in parallel and allows Nix to perform multi-platform
+builds in a semi-transparent way.  For instance, if you perform a
+build for a <literal>x86_64-darwin</literal> on an
+<literal>i686-linux</literal> machine, Nix can automatically forward
+the build to a <literal>x86_64-darwin</literal> machine, if
+available.</para>
+
+<para>To forward a build to a remote machine, it’s required that the
+remote machine is accessible via SSH and that it has Nix
+installed. You can test whether connecting to the remote Nix instance
+works, e.g.
+
+<screen>
+$ nix ping-store --store ssh://mac
+</screen>
+
+will try to connect to the machine named <literal>mac</literal>. It is
+possible to specify an SSH identity file as part of the remote store
+URI, e.g.
+
+<screen>
+$ nix ping-store --store ssh://mac?ssh-key=/home/alice/my-key
+</screen>
+
+Since builds should be non-interactive, the key should not have a
+passphrase. Alternatively, you can load identities ahead of time into
+<command>ssh-agent</command> or <command>gpg-agent</command>.</para>
+
+<para>If you get the error
+
+<screen>
+bash: nix-store: command not found
+error: cannot connect to 'mac'
+</screen>
+
+then you need to ensure that the <envar>PATH</envar> of
+non-interactive login shells contains Nix.</para>
+
+<warning><para>If you are building via the Nix daemon, it is the Nix
+daemon user account (that is, <literal>root</literal>) that should
+have SSH access to the remote machine. If you can’t or don’t want to
+configure <literal>root</literal> to be able to access to remote
+machine, you can use a private Nix store instead by passing
+e.g. <literal>--store ~/my-nix</literal>.</para></warning>
+
+<para>The list of remote machines can be specified on the command line
+or in the Nix configuration file. The former is convenient for
+testing. For example, the following command allows you to build a
+derivation for <literal>x86_64-darwin</literal> on a Linux machine:
+
+<screen>
+$ uname
+Linux
+
+$ nix build \
+  '(with import &lt;nixpkgs> { system = "x86_64-darwin"; }; runCommand "foo" {} "uname > $out")' \
+  --builders 'ssh://mac x86_64-darwin'
+[1/0/1 built, 0.0 MiB DL] building foo on ssh://mac
+
+$ cat ./result
+Darwin
+</screen>
+
+It is possible to specify multiple builders separated by a semicolon
+or a newline, e.g.
+
+<screen>
+  --builders 'ssh://mac x86_64-darwin ; ssh://beastie x86_64-freebsd'
+</screen>
+</para>
+
+<para>Each machine specification consists of the following elements,
+separated by spaces. Only the first element is required.
+To leave a field at its default, set it to <literal>-</literal>.
+
+<orderedlist>
+
+  <listitem><para>The URI of the remote store in the format
+  <literal>ssh://[<replaceable>username</replaceable>@]<replaceable>hostname</replaceable></literal>,
+  e.g. <literal>ssh://nix@mac</literal> or
+  <literal>ssh://mac</literal>. For backward compatibility,
+  <literal>ssh://</literal> may be omitted. The hostname may be an
+  alias defined in your
+  <filename>~/.ssh/config</filename>.</para></listitem>
+
+  <listitem><para>A comma-separated list of Nix platform type
+  identifiers, such as <literal>x86_64-darwin</literal>.  It is
+  possible for a machine to support multiple platform types, e.g.,
+  <literal>i686-linux,x86_64-linux</literal>. If omitted, this
+  defaults to the local platform type.</para></listitem>
+
+  <listitem><para>The SSH identity file to be used to log in to the
+  remote machine. If omitted, SSH will use its regular
+  identities.</para></listitem>
+
+  <listitem><para>The maximum number of builds that Nix will execute
+  in parallel on the machine.  Typically this should be equal to the
+  number of CPU cores.  For instance, the machine
+  <literal>itchy</literal> in the example will execute up to 8 builds
+  in parallel.</para></listitem>
+
+  <listitem><para>The “speed factor”, indicating the relative speed of
+  the machine.  If there are multiple machines of the right type, Nix
+  will prefer the fastest, taking load into account.</para></listitem>
+
+  <listitem><para>A comma-separated list of <emphasis>supported
+  features</emphasis>.  If a derivation has the
+  <varname>requiredSystemFeatures</varname> attribute, then Nix will
+  only perform the derivation on a machine that has the specified
+  features.  For instance, the attribute
+
+<programlisting>
+requiredSystemFeatures = [ "kvm" ];
+</programlisting>
+
+  will cause the build to be performed on a machine that has the
+  <literal>kvm</literal> feature.</para></listitem>
+
+  <listitem><para>A comma-separated list of <emphasis>mandatory
+  features</emphasis>.  A machine will only be used to build a
+  derivation if all of the machine’s mandatory features appear in the
+  derivation’s <varname>requiredSystemFeatures</varname>
+  attribute..</para></listitem>
+
+</orderedlist>
+
+For example, the machine specification
+
+<programlisting>
+nix@scratchy.labs.cs.uu.nl  i686-linux      /home/nix/.ssh/id_scratchy_auto        8 1 kvm
+nix@itchy.labs.cs.uu.nl     i686-linux      /home/nix/.ssh/id_scratchy_auto        8 2
+nix@poochie.labs.cs.uu.nl   i686-linux      /home/nix/.ssh/id_scratchy_auto        1 2 kvm benchmark
+</programlisting>
+
+specifies several machines that can perform
+<literal>i686-linux</literal> builds. However,
+<literal>poochie</literal> will only do builds that have the attribute
+
+<programlisting>
+requiredSystemFeatures = [ "benchmark" ];
+</programlisting>
+
+or
+
+<programlisting>
+requiredSystemFeatures = [ "benchmark" "kvm" ];
+</programlisting>
+
+<literal>itchy</literal> cannot do builds that require
+<literal>kvm</literal>, but <literal>scratchy</literal> does support
+such builds. For regular builds, <literal>itchy</literal> will be
+preferred over <literal>scratchy</literal> because it has a higher
+speed factor.</para>
+
+<para>Remote builders can also be configured in
+<filename>nix.conf</filename>, e.g.
+
+<programlisting>
+builders = ssh://mac x86_64-darwin ; ssh://beastie x86_64-freebsd
+</programlisting>
+
+Finally, remote builders can be configured in a separate configuration
+file included in <option>builders</option> via the syntax
+<literal>@<replaceable>file</replaceable></literal>. For example,
+
+<programlisting>
+builders = @/etc/nix/machines
+</programlisting>
+
+causes the list of machines in <filename>/etc/nix/machines</filename>
+to be included. (This is the default.)</para>
+
+<para>If you want the builders to use caches, you likely want to set
+the option <link linkend='conf-builders-use-substitutes'><literal>builders-use-substitutes</literal></link>
+in your local <filename>nix.conf</filename>.</para>
+
+<para>To build only on remote builders and disable building on the local machine,
+you can use the option <option>--max-jobs 0</option>.</para>
+
+</chapter>