From 8901acc97664aa8ebf687ee904428aa57a5192be Mon Sep 17 00:00:00 2001 From: Mikey Ariel Date: Wed, 27 Aug 2014 18:41:09 +0200 Subject: Restructuring the Nix manual --- doc/manual/builds/build-farm.xml | 22 +++++++ doc/manual/builds/enabling-builds.xml | 106 ++++++++++++++++++++++++++++++++++ 2 files changed, 128 insertions(+) create mode 100644 doc/manual/builds/build-farm.xml create mode 100644 doc/manual/builds/enabling-builds.xml (limited to 'doc/manual/builds') diff --git a/doc/manual/builds/build-farm.xml b/doc/manual/builds/build-farm.xml new file mode 100644 index 000000000000..e0e9f10f1173 --- /dev/null +++ b/doc/manual/builds/build-farm.xml @@ -0,0 +1,22 @@ + + +Distributed Builds + + +Nix supports distributed builds, where a local Nix installation can +forward Nix builds to other machines over the network. This allows +multiple builds to be performed in parallel (thus improving +performance) and allows Nix to perform multi-platform builds in a +semi-transparent way. For instance, if you perform a build for a +powerpc-darwin on an i686-linux +machine, Nix can automatically forward the build to a +powerpc-darwin machine, if available. + + + + + diff --git a/doc/manual/builds/enabling-builds.xml b/doc/manual/builds/enabling-builds.xml new file mode 100644 index 000000000000..4b45812ee918 --- /dev/null +++ b/doc/manual/builds/enabling-builds.xml @@ -0,0 +1,106 @@ + + +Enabling Distributed Builds + +You can enable distributed builds by setting the environment +variable NIX_BUILD_HOOK to point to a program that Nix +will call whenever it wants to build a derivation. The build hook +(typically a shell or Perl script) can decline the build, in which Nix +will perform it in the usual way if possible, or it can accept it, in +which case it is responsible for somehow getting the inputs of the +build to another machine, doing the build there, and getting the +results back. The details of the build hook protocol are described in +the documentation of the NIX_BUILD_HOOK +variable. + +Remote machine configuration: +<filename>remote-systems.conf</filename> + +nix@mcflurry.labs.cs.uu.nl powerpc-darwin /home/nix/.ssh/id_quarterpounder_auto 2 +nix@scratchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 1 kvm +nix@itchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 2 +nix@poochie.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 2 kvm perf + + + +Nix ships with a build hook that should be suitable for most +purposes. It uses ssh and +nix-copy-closure to copy the build inputs and +outputs and perform the remote build. To use it, you should set +NIX_BUILD_HOOK to +prefix/libexec/nix/build-remote.pl. +You should also define a list of available build machines and point +the environment variable NIX_REMOTE_SYSTEMS to it. An +example configuration is shown in . Each line in the file specifies a machine, with the following +bits of information: + + + + The name of the remote machine, with optionally the + user under which the remote build should be performed. This is + actually passed as an argument to ssh, so it can + be an alias defined in your + ~/.ssh/config. + + A comma-separated list of Nix platform type + identifiers, such as powerpc-darwin. It is + possible for a machine to support multiple platform types, e.g., + i686-linux,x86_64-linux. + + The SSH private key to be used to log in to the + remote machine. Since builds should be non-interactive, this key + should not have a passphrase! + + The maximum number of builds that + build-remote.pl will execute in parallel on the + machine. Typically this should be equal to the number of CPU cores. + For instance, the machine itchy in the example + will execute up to 8 builds in parallel. + + The “speed factor”, indicating the relative speed of + the machine. If there are multiple machines of the right type, Nix + will prefer the fastest, taking load into account. + + A comma-separated list of supported + features. If a derivation has the + requiredSystemFeatures attribute, then + build-remote.pl will only perform the + derivation on a machine that has the specified features. For + instance, the attribute + + +requiredSystemFeatures = [ "kvm" ]; + + + will cause the build to be performed on a machine that has the + kvm feature (i.e., scratchy in + the example above). + + A comma-separated list of mandatory + features. A machine will only be used to build a + derivation if all of the machine’s mandatory features appear in the + derivation’s requiredSystemFeatures attribute. + Thus, in the example, the machine poochie will + only do derivations that have + requiredSystemFeatures set to ["kvm" + "perf"] or ["perf"]. + + + +You should also set up the environment variable +NIX_CURRENT_LOAD to point at a directory (e.g., +/var/run/nix/current-load) that +build-remote.pl uses to remember how many builds +it is currently executing remotely. It doesn't look at the actual +load on the remote machine, so if you have multiple instances of Nix +running, they should use the same NIX_CURRENT_LOAD +file. Maybe in the future build-remote.pl will +look at the actual remote load. + + \ No newline at end of file -- cgit 1.4.1