diff options
Diffstat (limited to 'doc/manual')
| -rw-r--r-- | doc/manual/advanced-topics/advanced-topics.xml | 1 | ||||
| -rw-r--r-- | doc/manual/advanced-topics/post-build-hook.xml | 157 | ||||
| -rw-r--r-- | doc/manual/command-ref/conf-file.xml | 55 |
3 files changed, 213 insertions, 0 deletions
diff --git a/doc/manual/advanced-topics/advanced-topics.xml b/doc/manual/advanced-topics/advanced-topics.xml index c304367aaf8a..1b8841ad21c7 100644 --- a/doc/manual/advanced-topics/advanced-topics.xml +++ b/doc/manual/advanced-topics/advanced-topics.xml @@ -8,5 +8,6 @@ <xi:include href="distributed-builds.xml" /> <xi:include href="diff-hook.xml" /> +<xi:include href="post-build-hook.xml" /> </part> diff --git a/doc/manual/advanced-topics/post-build-hook.xml b/doc/manual/advanced-topics/post-build-hook.xml new file mode 100644 index 000000000000..4335b308b9d1 --- /dev/null +++ b/doc/manual/advanced-topics/post-build-hook.xml @@ -0,0 +1,157 @@ +<chapter xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + xml:id="chap-post-build-hook" + version="5.0" + > + +<title>Using the <xref linkend="conf-post-build-hook" /></title> +<subtitle>Uploading to an S3-compatible binary cache after each build</subtitle> + + +<section xml:id="chap-post-build-hook-caveats"> + <title>Implementation Caveats</title> + <para>Here we use the post-build hook to upload to a binary cache. + This is a simple and working example, but it is not suitable for all + use cases.</para> + + <para>The post build hook program runs after each executed build, + and blocks the build loop. The build loop exits if the hook program + fails.</para> + + <para>Concretely, this implementation will make Nix slow or unusable + when the internet is slow or unreliable.</para> + + <para>A more advanced implementation might pass the store paths to a + user-supplied daemon or queue for processing the store paths outside + of the build loop.</para> +</section> + +<section> + <title>Prerequisites</title> + + <para> + This tutorial assumes you have configured an S3-compatible binary cache + according to the instructions at + <xref linkend="ssec-s3-substituter-authenticated-writes" />, and + that the <literal>root</literal> user's default AWS profile can + upload to the bucket. + </para> +</section> + +<section> + <title>Set up a Signing Key</title> + <para>Use <command>nix-store --generate-binary-cache-key</command> to + create our public and private signing keys. We will sign paths + with the private key, and distribute the public key for verifying + the authenticity of the paths.</para> + + <screen> +# nix-store --generate-binary-cache-key example-nix-cache-1 /etc/nix/key.private /etc/nix/key.public +# cat /etc/nix/key.public +example-nix-cache-1:1/cKDz3QCCOmwcztD2eV6Coggp6rqc9DGjWv7C0G+rM= +</screen> + +<para>Then, add the public key and the cache URL to your +<filename>nix.conf</filename>'s <xref linkend="conf-trusted-public-keys" /> +and <xref linkend="conf-substituters" /> like:</para> + +<programlisting> +substituters = https://cache.nixos.org/ s3://example-nix-cache +trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= example-nix-cache-1:1/cKDz3QCCOmwcztD2eV6Coggp6rqc9DGjWv7C0G+rM= +</programlisting> + +<para>we will restart the Nix daemon a later step.</para> +</section> + +<section> + <title>Implementing the build hook</title> + <para>Write the following script to + <filename>/etc/nix/upload-to-cache.sh</filename>: + </para> + + <programlisting> +#!/bin/sh + +set -eu + +echo "Signing paths" $OUT_PATHS +nix sign-paths --key-file /etc/nix/key.private $OUT_PATHS +echo "Uploading paths" $OUT_PATHS +exec nix copy --to 's3://example-nix-cache' $OUT_PATHS +</programlisting> + + <note> + <title>Should <literal>$OUT_PATHS</literal> be quoted?</title> + <para> + The <literal>$OUT_PATHS</literal> variable is a space-separated + list of Nix store paths. In this case, we expect and want the + shell to perform word splitting to make each output path its + own argument to <command>nix sign-paths</command>. Nix guarantees + the paths will only contain characters which are safe for word + splitting, and free of any globs. + </para> + </note> + <para> + Then make sure the hook program is executable by the <literal>root</literal> user: + <screen> +# chmod +x /etc/nix/upload-to-cache.sh +</screen></para> +</section> + +<section> + <title>Updating Nix Configuration</title> + + <para>Edit <filename>/etc/nix/nix.conf</filename> to run our hook, + by adding the following configuration snippet at the end:</para> + + <programlisting> +post-build-hook = /etc/nix/upload-to-cache.sh +</programlisting> + +<para>Then, restart the <command>nix-daemon</command>.</para> +</section> + +<section> + <title>Testing</title> + + <para>Build any derivation, for example:</para> + + <screen> +$ nix-build -E '(import <nixpkgs> {}).writeText "example" (builtins.toString builtins.currentTime)' +these derivations will be built: + /nix/store/s4pnfbkalzy5qz57qs6yybna8wylkig6-example.drv +building '/nix/store/s4pnfbkalzy5qz57qs6yybna8wylkig6-example.drv'... +running post-build-hook '/home/grahamc/projects/github.com/NixOS/nix/post-hook.sh'... +post-build-hook: Signing paths /nix/store/ibcyipq5gf91838ldx40mjsp0b8w9n18-example +post-build-hook: Uploading paths /nix/store/ibcyipq5gf91838ldx40mjsp0b8w9n18-example +/nix/store/ibcyipq5gf91838ldx40mjsp0b8w9n18-example +</screen> + + <para>Then delete the path from the store, and try substituting it from the binary cache:</para> + <screen> +$ rm ./result +$ nix-store --delete /nix/store/ibcyipq5gf91838ldx40mjsp0b8w9n18-example +</screen> + +<para>Now, copy the path back from the cache:</para> +<screen> +$ nix store --realize /nix/store/ibcyipq5gf91838ldx40mjsp0b8w9n18-example +copying path '/nix/store/m8bmqwrch6l3h8s0k3d673xpmipcdpsa-example from 's3://example-nix-cache'... +warning: you did not specify '--add-root'; the result might be removed by the garbage collector +/nix/store/m8bmqwrch6l3h8s0k3d673xpmipcdpsa-example +</screen> +</section> +<section> + <title>Conclusion</title> + <para> + We now have a Nix installation configured to automatically sign and + upload every local build to a remote binary cache. + </para> + + <para> + Before deploying this to production, be sure to consider the + implementation caveats in <xref linkend="chap-post-build-hook-caveats" />. + </para> +</section> +</chapter> diff --git a/doc/manual/command-ref/conf-file.xml b/doc/manual/command-ref/conf-file.xml index 09aad2e05533..d2c9c7502ebb 100644 --- a/doc/manual/command-ref/conf-file.xml +++ b/doc/manual/command-ref/conf-file.xml @@ -656,6 +656,61 @@ password <replaceable>my-password</replaceable> </varlistentry> + <varlistentry xml:id="conf-post-build-hook"> + <term><literal>post-build-hook</literal></term> + <listitem> + <para>Optional. The path to a program to execute after each build.</para> + + <para>This option is only settable in the global + <filename>nix.conf</filename>, or on the command line by trusted + users.</para> + + <para>When using the nix-daemon, the daemon executes the hook as + <literal>root</literal>. If the nix-daemon is not involved, the + hook runs as the user executing the nix-build.</para> + + <itemizedlist> + <listitem><para>The hook executes after an evaluation-time build.</para></listitem> + <listitem><para>The hook does not execute on substituted paths.</para></listitem> + <listitem><para>The hook's output always goes to the user's terminal.</para></listitem> + <listitem><para>If the hook fails, the build succeeds but no further builds execute.</para></listitem> + </itemizedlist> + + <para>The program executes with no arguments. The program's environment + contains the following environment variables:</para> + + <variablelist> + <varlistentry> + <term><envar>DRV_PATH</envar></term> + <listitem> + <para>The derivation for the built paths.</para> + <para>Example: + <literal>/nix/store/5nihn1a7pa8b25l9zafqaqibznlvvp3f-bash-4.4-p23.drv</literal> + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term><envar>OUT_PATHS</envar></term> + <listitem> + <para>Output paths of the built derivation, separated by a space (<literal> </literal>) character.</para> + <para>Example: + <literal>/nix/store/zf5lbh336mnzf1nlswdn11g4n2m8zh3g-bash-4.4-p23-dev + /nix/store/rjxwxwv1fpn9wa2x5ssk5phzwlcv4mna-bash-4.4-p23-doc + /nix/store/6bqvbzjkcp9695dq0dpl5y43nvy37pq1-bash-4.4-p23-info + /nix/store/r7fng3kk3vlpdlh2idnrbn37vh4imlj2-bash-4.4-p23-man + /nix/store/xfghy8ixrhz3kyy6p724iv3cxji088dx-bash-4.4-p23</literal>. + </para> + </listitem> + </varlistentry> + </variablelist> + + <para>See <xref linkend="chap-post-build-hook" /> for an example + implementation.</para> + + </listitem> + </varlistentry> + <varlistentry xml:id="conf-repeat"><term><literal>repeat</literal></term> <listitem><para>How many times to repeat builds to check whether |