diff options
| -rw-r--r-- | doc/manual/release-notes/rl-2.0.xml | 983 | ||||
| -rw-r--r-- | doc/manual/style.css | 12 | ||||
| -rw-r--r-- | src/nix/ls.cc | 20 | ||||
| -rw-r--r-- | src/nix/run.cc | 4 |
4 files changed, 755 insertions, 264 deletions
diff --git a/doc/manual/release-notes/rl-2.0.xml b/doc/manual/release-notes/rl-2.0.xml index effd2e39d309..0ad17373ef4f 100644 --- a/doc/manual/release-notes/rl-2.0.xml +++ b/doc/manual/release-notes/rl-2.0.xml @@ -6,165 +6,572 @@ <title>Release 2.0 (2018-02-??)</title> -<para>This release has the following new features:</para> +<para>The following incompatible changes have been made:</para> <itemizedlist> <listitem> - <para>Start of new <command>nix</command> command line - interface. This is a work in progress and the interface is subject - to change.</para> + <para>The manifest-based substituter mechanism + (<command>download-using-manifests</command>) has been <link + xlink:href="https://github.com/NixOS/nix/commit/867967265b80946dfe1db72d40324b4f9af988ed">removed</link>. It + has been superseded by the binary cache substituter mechanism + since several years. As a result, the following programs have been + removed: <itemizedlist> + <listitem><para><command>nix-pull</command></para></listitem> + <listitem><para><command>nix-generate-patches</command></para></listitem> + <listitem><para><command>bsdiff</command></para></listitem> + <listitem><para><command>bspatch</command></para></listitem> + </itemizedlist> + </para> + </listitem> - <listitem><para>Self-documenting: <option>--help</option> shows - all available command-line arguments.</para></listitem> - - <listitem><para><option>--help-config</option> shows all - configuration options.</para></listitem> + <listitem> + <para>The “copy from other stores” substituter mechanism + (<command>copy-from-other-stores</command> and the + <envar>NIX_OTHER_STORES</envar> environment variable) has been + removed. It was primarily used by the NixOS installer to copy + available paths from the installation medium. The replacement is + to use a chroot store as a substituter + (e.g. <literal>--substituters /mnt</literal>), or to build into a + chroot store (e.g. <literal>--store /mnt --substituter /</literal>).</para> + </listitem> - <listitem><para><command>nix build</command>: Replacement for - <command>nix-build</command>.</para></listitem> + <listitem> + <para>The command <command>nix-push</command> has been removed as + part of the effort to eliminate Nix's dependency on Perl. You can + use <command>nix copy</command> instead, e.g. <literal>nix copy + --to /tmp/my-binary-cache <replaceable>paths…</replaceable></literal></para> + </listitem> - <listitem><para><command>nix ls-store</command> and <command>nix - ls-nar</command> allow listing the contents of a store path or - NAR file.</para></listitem> + <listitem> + <para>The “nested” log output feature (<option>--log-type + pretty</option>) has been removed. As a result, + <command>nix-log2xml</command> was also removed.</para> + </listitem> - <listitem><para><command>nix cat-store</command> and - <command>nix cat-nar</command> allow extracting a file from a - store path or NAR file.</para></listitem> + <listitem> + <para>OpenSSL-based signing has been <link + xlink:href="https://github.com/NixOS/nix/commit/f435f8247553656774dd1b2c88e9de5d59cab203">removed</link>. This + feature was never well-supported. A better alternative is provided + by the <option>secret-key-files</option> and + <option>trusted-public-keys</option> options.</para> + </listitem> - <listitem><para><command>nix verify</command> checks whether a - store path is unmodified and/or is trusted.</para></listitem> + <listitem> + <para>Failed build caching has been <link + xlink:href="https://github.com/NixOS/nix/commit/8cffec84859cec8b610a2a22ab0c4d462a9351ff">removed</link>. This + feature was introduced to support the Hydra continuous build + system, but Hydra no longer uses it.</para> + </listitem> - <listitem><para><command>nix copy-sigs</command> copies - signatures from one store to another.</para></listitem> + <listitem> + <para><filename>nix-mode.el</filename> has been removed from + Nix. It is now <link + xlink:href="https://github.com/NixOS/nix-mode">a separate + repository</link> and can be installed through the MELPA package + repository.</para> + </listitem> - <listitem><para><command>nix sign-paths</command> signs store - paths.</para></listitem> +</itemizedlist> - <listitem><para><command>nix copy</command> copies paths between - arbitrary Nix stores, generalising - <command>nix-copy-closure</command> and - <command>nix-push</command>.</para></listitem> +<para>This release has the following new features:</para> - <listitem><para><command>nix path-info</command> shows - information about store paths.</para></listitem> +<itemizedlist> - <listitem><para><command>nix run</command> starts a shell in - which the specified packages are available.</para></listitem> + <listitem> + <para>It introduces a new command named <command>nix</command>, + which is intended to eventually replace all + <command>nix-*</command> commands with a more consistent and + better designed user interface. It currently provides replacements + for some (but not all) of the functionality provided by + <command>nix-store</command>, <command>nix-build</command>, + <command>nix-shell -p</command>, <command>nix-env -qa</command>, + <command>nix-instantiate --eval</command>, + <command>nix-push</command> and + <command>nix-copy-closure</command>. It has the following major + features:</para> - <listitem><para><command>nix log</command> shows the build log - of a package or path. If the build log is not available locally, - it will try to obtain it from a binary cache.</para></listitem> + <itemizedlist> - <listitem><para><command>nix eval</command> replaces - <command>nix-instantiate --eval</command>.</para></listitem> + <listitem> + <para>Unlike the legacy commands, it has a consistent way to + refer to packages and package-like argumements (like store + paths). For example, the following commands all copy the GNU + Hello package to a remote machine: + + <screen>nix copy --to ssh://machine nixpkgs.hello</screen> + <screen>nix copy --to ssh://machine /nix/store/0i2jd68mp5g6h2sa5k9c85rb80sn8hi9-hello-2.10</screen> + <screen>nix copy --to ssh://machine '(with import <nixpkgs> {}; hello)'</screen> + + By contrast, <command>nix-copy-closure</command> only accepted + store paths as arguments.</para> + </listitem> + + <listitem> + <para>It is self-documenting: <option>--help</option> shows + all available command-line arguments. If + <option>--help</option> is given after a subcommand, it shows + examples for that subcommand. <command>nix + --help-config</command> shows all configuration + options.</para> + </listitem> + + <listitem> + <para>It is much less verbose. By default, it displays a + single-line progress indicator that shows how many packages + are left to be built or downloaded, and (if there are running + builds) the most recent line of builder output. If a build + fails, it shows the last few lines of builder output. The full + build log can be retrieved using <command>nix + log</command>.</para> + </listitem> + + <listitem> + <para>It <link + xlink:href="https://github.com/NixOS/nix/commit/b8283773bd64d7da6859ed520ee19867742a03ba">provides</link> + all <filename>nix.conf</filename> configuration options as + command line flags. For example, instead of <literal>--option + http-connections 100</literal> you can write + <literal>--http-connections 100</literal>. Boolean options can + be written as + <literal>--<replaceable>foo</replaceable></literal> or + <literal>--no-<replaceable>foo</replaceable></literal> + (e.g. <option>--no-auto-optimise-store</option>).</para> + </listitem> + + <listitem> + <para>Many subcommands have a <option>--json</option> flag to + write results to stdout in JSON format.</para> + </listitem> - <listitem><para><command>nix dump-path</command> to get a NAR - from a store path.</para></listitem> + </itemizedlist> - <listitem><para><command>nix edit</command> opens the source - code of a package in an editor.</para></listitem> + <warning><para>Please note that the <command>nix</command> command + is a work in progress and the interface is subject to + change.</para></warning> - <listitem><para><command>nix search</command> replaces - <command>nix-env -qa</command>. It searches the available - packages for occurences of a search string in the attribute - name, package name or description. It caches available packages - to speed up searches.</para></listitem> + <para>It provides the following high-level (“porcelain”) + subcommands:</para> - <listitem><para><command>nix why-depends</command> (d41c5eb13f4f3a37d80dbc6d3888644170c3b44a).</para></listitem> + <itemizedlist> - <listitem><para><command>nix show-derivation</command> (e8d6ee7c1b90a2fe6d824f1a875acc56799ae6e2).</para></listitem> + <listitem> + <para><command>nix build</command> is a replacement for + <command>nix-build</command>.</para> + </listitem> + + <listitem> + <para><command>nix run</command> executes a command in an + environment in which the specified packages are available. It + is (roughly) a replacement for <command>nix-shell + -p</command>. Unlike that command, it does not execute the + command in a shell, and has a flag (<command>-c</command>) + that specifies the unquoted command line to be + executed.</para> + + <para>It is particularly useful in conjunction with chroot + stores, allowing Linux users who do not have permission to + install Nix in <command>/nix/store</command> to still use + binary substitutes that assume + <command>/nix/store</command>. For example, + + <screen>nix run --store ~/my-nix nixpkgs.hello -c hello --greeting 'Hi everybody!'</screen> + + downloads (or if not substitutes are available, builds) the + GNU Hello package into + <filename>~/my-nix/nix/store</filename>, then runs + <command>hello</command> in a mount namespace where + <filename>~/my-nix/nix/store</filename> is mounted onto + <command>/nix/store</command>.</para> + </listitem> + + <listitem> + <para><command>nix search</command> replaces <command>nix-env + -qa</command>. It searches the available packages for + occurences of a search string in the attribute name, package + name or description. Unlike <command>nix-env -qa</command>, it + has a cache to speed up subsequent searches.</para> + </listitem> + + <listitem> + <para><command>nix copy</command> copies paths between + arbitrary Nix stores, generalising + <command>nix-copy-closure</command> and + <command>nix-push</command>.</para> + </listitem> + + <listitem> + <para><command>nix repl</command> replaces the external + program <command>nix-repl</command>. It provides an + interactive environment for evaluating and building Nix + expressions. Note that it uses <literal>linenoise-ng</literal> + instead of GNU Readline.</para> + </listitem> + + <listitem> + <para><command>nix upgrade-nix</command> upgrades Nix to the + latest stable version. This requires that Nix is installed in + a profile. (Thus it won’t work on NixOS, or if it’s installed + outside of the Nix store.)</para> + </listitem> + + <listitem> + <para><command>nix verify</command> checks whether store paths + are unmodified and/or “trusted” (see below). It replaces + <command>nix-store --verify</command> and <command>nix-store + --verify-path</command>.</para> + </listitem> + + <listitem> + <para><command>nix log</command> shows the build log of a + package or path. If the build log is not available locally, it + will try to obtain it from the configured substituters (such + as <uri>cache.nixos.org</uri>, which now provides build + logs).</para> + </listitem> + + <listitem> + <para><command>nix edit</command> opens the source code of a + package in your editor.</para> + </listitem> + + <listitem> + <para><command>nix eval</command> replaces + <command>nix-instantiate --eval</command>.</para> + </listitem> + + <listitem> + <para><command + xlink:href="https://github.com/NixOS/nix/commit/d41c5eb13f4f3a37d80dbc6d3888644170c3b44a">nix + why-depends</command> shows why one store path has another in + its closure. This is primarily useful to finding the causes of + closure bloat. For example, + + <screen>nix why-depends nixpkgs.vlc nixpkgs.libdrm.dev</screen> + + shows a chain of files and fragments of file contents that + cause the VLC package to have the “dev” output of + <literal>libdrm</literal> in its closure — an undesirable + situation.</para> + </listitem> + + <listitem> + <para><command>nix path-info</command> shows information about + store paths, replacing <command>nix-store -q</command>. A + useful feature is the option <option>--closure-size</option> + (<option>-S</option>). For example, the following command show + the closure sizes of every path in the current NixOS system + closure, sorted by size: + + <screen>nix path-info -rS /run/current-system | sort -nk2</screen> + + </para> + </listitem> + + <listitem> + <para><command>nix optimise-store</command> replaces + <command>nix-store --optimise</command>. The main difference + is that it has a progress indicator.</para> + </listitem> - <listitem><para><command>nix add-to-store</command> (970366266b8df712f5f9cedb45af183ef5a8357f).</para></listitem> + </itemizedlist> - <listitem><para><command>nix upgrade-nix</command> upgrades Nix - to the latest stable version. This requires that Nix is - installed in a profile. (Thus it won’t work on NixOS, or if it’s - installed outside of the Nix store.)</para></listitem> + <para>A number of low-level (“plumbing”) commands are also + available:</para> - <listitem><para>Progress indicator.</para></listitem> + <itemizedlist> - <listitem><para>All options are available as flags now - (b8283773bd64d7da6859ed520ee19867742a03ba).</para></listitem> + <listitem> + <para><command>nix ls-store</command> and <command>nix + ls-nar</command> list the contents of a store path or NAR + file. The former is primarily useful in conjunction with + remote stores, e.g. + + <screen>nix ls-store --store https://cache.nixos.org/ -lR /nix/store/0i2jd68mp5g6h2sa5k9c85rb80sn8hi9-hello-2.10</screen> + + lists the contents of path in a binary cache.</para> + </listitem> + + <listitem> + <para><command>nix cat-store</command> and <command>nix + cat-nar</command> allow extracting a file from a store path or + NAR file.</para> + </listitem> + + <listitem> + <para><command>nix dump-path</command> writes the contents of + a store path to stdout in NAR format. This replaces + <command>nix-store --dump</command>.</para> + </listitem> + + <listitem> + <para><command + xlink:href="https://github.com/NixOS/nix/commit/e8d6ee7c1b90a2fe6d824f1a875acc56799ae6e2">nix + show-derivation</command> displays a store derivation in JSON + format. This is an alternative to + <command>pp-aterm</command>.</para> + </listitem> + + <listitem> + <para><command + xlink:href="https://github.com/NixOS/nix/commit/970366266b8df712f5f9cedb45af183ef5a8357f">nix + add-to-store</command> replaces <command>nix-store + --add</command>.</para> + </listitem> + + <listitem> + <para><command>nix sign-paths</command> signs store + paths. (TODO: add examples)</para> + </listitem> + + <listitem> + <para><command>nix copy-sigs</command> copies signatures from + one store to another. (TODO: add examples and + tests)</para> + </listitem> + + <listitem> + <para><command>nix show-config</command> shows all + configuration options and their current values.</para> + </listitem> </itemizedlist> </listitem> <listitem> - <para>The external program <command>nix-repl</command> has been - integrated into Nix as <command>nix repl</command>.</para> - </listitem> + <para>The store abstraction that Nix has had for a long time to + support store access via the Nix daemon has been extended + significantly. In particular, substituters (which used to be + external programs such as + <command>download-from-binary-cache</command>) are now subclasses + of the abstract <classname>Store</classname> class. This allows + many Nix commands to operate on such store types. For example, + <command>nix path-info</command> shows information about paths in + your local Nix store, while <command>nix path-info --store + https://cache.nixos.org/</command> shows information about paths + in the specified binary cache. Similarly, + <command>nix-copy-closure</command>, <command>nix-push</command> + and substitution are all instances of the general notion of + copying paths between different kinds of Nix stores.</para> - <listitem> - <para>If a fixed-output derivation produces a result with an - incorrect hash, the output path will be moved to the location - corresponding to the actual hash and registered as valid. Thus, a - subsequent build of the fixed-output derivation with the correct - hash is unnecessary.</para> - </listitem> + <para>Stores are specified using an URI-like syntax, + e.g. <uri>https://cache.nixos.org/</uri> or + <uri>ssh://machine</uri>. The following store types are supported: + + <itemizedlist> + + <listitem> + + <para><classname>LocalStore</classname> (stori URI + <literal>local</literal> or an absolute path) and the misnamed + <classname>RemoteStore</classname> (<literal>daemon</literal>) + provide access to a local Nix store, the latter via the Nix + daemon. You can use <literal>auto</literal> or the empty + string to auto-select a local or daemon store depending on + whether you have write permission to the Nix store. It is no + longer necessary to set the <envar>NIX_REMOTE</envar> + environment variable to use the Nix daemon.</para> + + <para>As noted above, <classname>LocalStore</classname> now + supports chroot builds, allowing the “physical” location of + the Nix store + (e.g. <filename>/home/alice/nix/store</filename>) to differ + from its “logical” location (typically + <filename>/nix/store</filename>). This allows non-root users + to use Nix while still getting the benefits from prebuilt + binaries from <uri>cache.nixos.org</uri>.</para> + + </listitem> + + <listitem> + + <para><classname>BinaryCacheStore</classname> is the abstract + superclass of all binary cache stores. It supports writing + build logs and NAR content listings in JSON format.</para> + + </listitem> + + <listitem> + + <para><classname>HttpBinaryCacheStore</classname> + (<literal>http://</literal>, <literal>https://</literal>) + supports binary caches via HTTP or HTTPS. If the server + supports <literal>PUT</literal> requests, it supports + uploading store paths via commands such as <command>nix + copy</command>.</para> + + </listitem> + + <listitem> + + <para><classname>LocalBinaryCacheStore</classname> + (<literal>file://</literal>) supports binary caches in the + local filesystem.</para> + + </listitem> + + <listitem> + + <para><classname>S3BinaryCacheStore</classname> + (<literal>s3://</literal>) supports binary caches stored in + Amazon S3, if enabled at compile time.</para> + + </listitem> + + <listitem> + + <para><classname>LegacySSHStore</classname> (<literal>ssh://</literal>) + is used to implement remote builds and + <command>nix-copy-closure</command>.</para> + + </listitem> + + <listitem> + + <para><classname>SSHStore</classname> + (<literal>ssh-ng://</literal>) supports arbitrary Nix + operations on a remote machine via the same protocol used by + <command>nix-daemon</command>.</para> + + </listitem> + + </itemizedlist> + + </para> - <listitem> - <para>It is no longer necessary to set the - <envar>NIX_REMOTE</envar> environment variable if you need to use - the Nix daemon. Nix will use the daemon automatically if you don’t - have write access to the Nix database.</para> </listitem> <listitem> - <para>The Nix language now supports floating point numbers. They are - based on regular C++ <literal>float</literal> and compatible with - existing integers and number-related operations. Export and import to and - from JSON and XML works, too.</para> + + <para>Security has been improved in various ways: + + <itemizedlist> + + <listitem> + <para>Nix now stores signatures for local store + paths. When paths are copied between stores (e.g., copied from + a binary cache to a local store), signatures are + propagated.</para> + + <para>Locally-built paths are signed automatically using the + secret keys specified by the <option>secret-key-files</option> + store option. Secret/public key pairs can be generated using + <command>nix-store + --generate-binary-cache-key</command>. (TODO: rename)</para> + + <para>In addition, locally-built store paths are marked as + “ultimately trusted”, but this bit is not propagated when + paths are copied between stores.</para> + </listitem> + + <listitem> + <para>Content-addressable store paths no longer require + signatures — they can be imported into a store by unprivileged + users even if they lack signatures.</para> + </listitem> + + <listitem> + <para>The command <command>nix verify</command> checks whether + the specified paths are trusted, i.e., have a certain number + of trusted signatures, are ultimately trusted, or are + content-addressed.</para> + </listitem> + + <listitem> + <para>Substitutions from binary caches <link + xlink:href="https://github.com/NixOS/nix/commit/ecbc3fedd3d5bdc5a0e1a0a51b29062f2874ac8b">now</link> + require signatures by default. This was already the case on + NixOS.</para> + </listitem> + + <listitem> + <para>In Linux sandbox builds, we <link + xlink:href="https://github.com/NixOS/nix/commit/eba840c8a13b465ace90172ff76a0db2899ab11b">now</link> + use <filename>/build</filename> instead of + <filename>/tmp</filename> as the temporary build + directory. This fixes potential security problems when a build + accidentally stores its <envar>TMPDIR</envar> in some + security-sensitive place, such as an RPATH.</para> + </listitem> + + </itemizedlist> + + </para> + </listitem> <listitem> - <para><command>nix-shell</command> now sets the - <varname>IN_NIX_SHELL</varname> environment variable during - evaluation and in the shell itself. This can be used to perform - different actions depending on whether you’re in a Nix shell or in - a regular build. Nixpkgs provides - <varname>lib.inNixShell</varname> to check this variable during - evaluation. (bb36a1a3cf3fbe6bc9d0afcc5fa0f928bed03170)</para> + <para><emphasis>Pure evaluation mode</emphasis>. This is a variant + of the existing restricted evaluation mode. In pure mode, the Nix + evaluator forbids access to anything that could cause different + evaluations of the same command line arguments to produce a + different result. This includes builtin functions such as + <function>builtins.getEnv</function>, but more importantly, + <emphasis>all</emphasis> filesystem or network access unless a + content hash or commit hash is specified. For example, calls to + <function>builtins.fetchGit</function> are only allowed if a + <varname>rev</varname> attribute is specified.</para> + + <para>The goal of this feature is to enable true reproducibility + and traceability of builds (including NixOS system configurations) + at the evaluation level. For example, in the future, + <command>nixos-rebuild</command> might build configurations from a + Nix expression in a Git repository in pure mode. That expression + might fetch other repositories such as Nixpkgs via + <function>builtins.fetchGit</function>. The commit hash of the + top-level repository then uniquely identifies a running system, + and, in conjunction with that repository, allows it to be + reproduced or modified.</para> + </listitem> <listitem> - <para>Internal: all <classname>Store</classname> classes are now - thread-safe. <classname>RemoteStore</classname> supports multiple - concurrent connections to the daemon. This is primarily useful in - multi-threaded programs such as - <command>hydra-queue-runner</command>.</para> + <para>There are several new features to support binary + reproducibility (i.e. to help ensure that multiple builds of the + same derivation produce exactly the same output). When + <option>enforce-determinism</option> is set to + <literal>false</literal>, it’s <link + xlink:href="https://github.com/NixOS/nix/commit/8bdf83f936adae6f2c907a6d2541e80d4120f051">no + longer</link> a fatal error if build rounds produce different + output. Also, a hook named <option>diff-hook</option> is <link + xlink:href="https://github.com/NixOS/nix/commit/9a313469a4bdea2d1e8df24d16289dc2a172a169w">provided</link> + to allow you to run tools such as <command>diffoscope</command> + when build rounds produce different output.</para> </listitem> <listitem> - <para>The dependency on Perl has been removed. As a result, some - (obsolete) programs have been removed: <command>nix-push</command> - (replaced by <command>nix copy</command>), - <command>nix-pull</command> (obsoleted by binary caches), - <command>nix-generate-patches</command>, - <command>bsdiff</command>, <command>bspatch</command>.</para> + <para>Configuring remote builds is a lot easier now. Provided you + are not using the Nix daemon, you can now just specify a remote + build machine on the command line, e.g. <literal>--option builders + 'ssh://my-mac x86_64-darwin'</literal>. The environment variable + <envar>NIX_BUILD_HOOK</envar> has been removed and is no longer + needed. The environment variable <envar>NIX_REMOTE_SYSTEMS</envar> + is still supported for compatibility, but it is also possible to + specify builders in <command>nix.conf</command> by setting the + option <literal>builders = + @<replaceable>path</replaceable></literal>.</para> </listitem> <listitem> - <para>Improved store abstraction. Substituters - eliminated. BinaryCacheStore, LocalBinaryCacheStore, - HttpBinaryCacheStore, S3BinaryCacheStore (compile-time - optional), SSHStore. Add docs + examples? - </para> + <para>If a fixed-output derivation produces a result with an + incorrect hash, the output path is moved to the location + corresponding to the actual hash and registered as valid. Thus, a + subsequent build of the fixed-output derivation with the correct + hash is unnecessary.</para> </listitem> <listitem> - <para>Nix now stores signatures for local store - paths. Locally-built paths are now signed automatically using the - secret keys specified by the <option>secret-key-files</option> - store option.</para> - - <para>In addition, store paths that have been built locally are - marked as “ultimately trusted”, and content-addressable store - paths carry a “content-addressability assertion” that allow them - to be trusted without any signatures.</para> + <para><command>nix-shell</command> <link + xlink:href="https://github.com/NixOS/nix/commit/ea59f39326c8e9dc42dfed4bcbf597fbce58797c">now</link> + sets the <varname>IN_NIX_SHELL</varname> environment variable + during evaluation and in the shell itself. This can be used to + perform different actions depending on whether you’re in a Nix + shell or in a regular build. Nixpkgs provides + <varname>lib.inNixShell</varname> to check this variable during + evaluation.</para> </listitem> <listitem> @@ -179,7 +586,8 @@ <uri>https://nixos.org/channels/<replaceable>channel-name</replaceable>/nixexprs.tar.xz</uri>. For example, <literal>nix-build channel:nixos-15.09 -A hello</literal> will build the GNU Hello package from the - <literal>nixos-15.09</literal> channel.</para> + <literal>nixos-15.09</literal> channel. In the future, this may + use Git to fetch updates more efficiently.</para> </listitem> <listitem> @@ -189,45 +597,119 @@ </listitem> <listitem> - <para><function>builtins.fetchGit</function>. - (38539b943a060d9cdfc24d6e5d997c0885b8aa2f)</para> + <para>Networking has been improved: + + <itemizedlist> + + <listitem> + <para>HTTP/2 is now supported. This makes binary cache lookups + <link + xlink:href="https://github.com/NixOS/nix/commit/90ad02bf626b885a5dd8967894e2eafc953bdf92">much + more efficient</link>.</para> + </listitem> + + <listitem> + <para>We now retry downloads on many HTTP errors, making + binary caches substituters more resilient to temporary + failures.</para> + </listitem> + + <listitem> + <para>HTTP credentials can now be configured via the standard + <filename>netrc</filename> mechanism.</para> + </listitem> + + <listitem> + <para>If S3 support is enabled at compile time, + <uri>s3://</uri> URIs are <link + xlink:href="https://github.com/NixOS/nix/commit/9ff9c3f2f80ba4108e9c945bbfda2c64735f987b">supported</link> + in all places where Nix allows URIs.</para> + </listitem> + + <listitem> + <para>Brotli compression is now supported. In particular, + <uri>cache.nixos.org</uri> build logs are now compressed using + Brotli.</para> + </listitem> + + </itemizedlist> + + </para> + </listitem> <listitem> - <para><literal><nix/fetchurl.nix></literal> now uses the - content-addressable tarball cache at - <uri>http://tarballs.nixos.org/</uri>, just like - <function>fetchurl</function> in - Nixpkgs. (f2682e6e18a76ecbfb8a12c17e3a0ca15c084197)</para> + <para><command>nix-env</command> <link + xlink:href="https://github.com/NixOS/nix/commit/b0cb11722626e906a73f10dd9a0c9eea29faf43a">now</link> + ignores packages with bad derivation names (in particular those + starting with a digit or containing a dot).</para> </listitem> <listitem> - <para>Chroot Nix stores: allow the “physical” location of the Nix - store (e.g. <filename>/home/alice/nix/store</filename>) to differ - from its “logical” location (typically - <filename>/nix/store</filename>). This allows non-root users to - use Nix while still getting the benefits from prebuilt binaries - from - <uri>cache.nixos.org</uri>. (4494000e04122f24558e1436e66d20d89028b4bd, - 3eb621750848e0e6b30e5a79f76afbb096bb6c8a)</para> + <para>Many configuration options have been renamed, either because + they were unnecessarily verbose + (e.g. <option>build-use-sandbox</option> is now just + <option>sandbox</option>) or to reflect generalised behaviour + (e.g. <option>binary-caches</option> is now + <option>substituters</option> because it allows arbitrary store + URIs). The old names are still supported for compatibility.</para> </listitem> <listitem> - <para>On Linux, builds are now executed in a user - namespace with uid 1000 and gid 100.</para> + <para>The <option>max-jobs</option> option can <link + xlink:href="https://github.com/NixOS/nix/commit/7251d048fa812d2551b7003bc9f13a8f5d4c95a5">now</link> + be set to <literal>auto</literal> to use the number of CPUs in the + system.</para> </listitem> <listitem> - <para><function>builtins.fetchurl</function> and - <function>builtins.fetchTarball</function> now support - <varname>sha256</varname> and <varname>name</varname> - attributes.</para> + <para>Hashes can <link + xlink:href="https://github.com/NixOS/nix/commit/c0015e87af70f539f24d2aa2bc224a9d8b84276b">now</link> + be specified in base-64 format, in addition to base-16 and the + non-standard base-32.</para> </listitem> <listitem> - <para><literal>HttpBinaryCacheStore</literal> (the replacement of - <command>download-from-binary-cache</command>) now retries - automatically on certain HTTP error codes.</para> + <para><command>nix-shell</command> now uses + <varname>bashInteractive</varname> from Nixpkgs, rather than the + <command>bash</command> command that happens to be in the caller’s + <envar>PATH</envar>. This is especially important on macOS where + the <command>bash</command> provided by the system is seriously + outdated and cannot execute <literal>stdenv</literal>’s setup + script.</para> + </listitem> + + <listitem> + <para>Nix can now automatically trigger a garbage collection if + free disk space drops below a certain level during a build. This + is configured using the <option>min-free</option> and + <option>max-free</option> options.</para> + </listitem> + + <listitem> + <para><command>nix-store -q --roots</command> and + <command>nix-store --gc --print-roots</command> now show temporary + and in-memory roots.</para> + </listitem> + + <listitem> + <para> + Nix can now be extended with plugins. See the documentation of + the <option>plugin-files</option> option for more details. + </para> + </listitem> + +</itemizedlist> + +<para>The Nix language has the following new features: + +<itemizedlist> + + <listitem> + <para>It supports floating point numbers. They are based on the + C++ <literal>float</literal> type and are supported by the + existing numerical operators. Export and import to and from JSON + and XML works, too.</para> </listitem> <listitem> @@ -245,194 +727,187 @@ configureFlags = "--prefix=${placeholder "out"} --includedir=${placeholder "dev" add docs.</para> </listitem> - <listitem> - <para>Support for HTTP/2. This makes binary cache lookups much - more efficient. (90ad02bf626b885a5dd8967894e2eafc953bdf92)</para> - </listitem> +</itemizedlist> - <listitem> - <para>The <option>build-sandbox-paths</option> configuration - option can now specify optional paths by appending a - <literal>?</literal>, e.g. <literal>/dev/nvidiactl?</literal> will - bind-mount <varname>/dev/nvidiactl</varname> only if it - exists.</para> - </listitem> +</para> - <listitem> - <para>More support for testing build reproducibility: when - <option>enforce-determinism</option> is set to - <literal>false</literal>, it’s no longer a fatal error build - rounds produce different output - (8bdf83f936adae6f2c907a6d2541e80d4120f051); add a hook to run - diffoscope when build rounds produce different output - (9a313469a4bdea2d1e8df24d16289dc2a172a169w).</para> - </listitem> +<para>The following builtin functions are new or extended: - <listitem> - <para>Kill builds as soon as stdout/stderr is closed. This fixes a - bug that allowed builds to hang Nix indefinitely (regardless of - timeouts). (21948deed99a3295e4d5666e027a6ca42dc00b40)</para> - </listitem> +<itemizedlist> <listitem> - <para>Add support for passing structured data to builders. TODO: - document. (6de33a9c675b187437a2e1abbcb290981a89ecb1)</para> - </listitem> + <para><function + xlink:href="https://github.com/NixOS/nix/commit/38539b943a060d9cdfc24d6e5d997c0885b8aa2f">builtins.fetchGit</function> + allows Git repositories to be fetched at evaluation time. Thus it + differs from the <function>fetchgit</function> function in + Nixpkgs, which fetches at build time and cannot be used to fetch + Nix expressions during evaluation. A typical use case is to import + external NixOS modules from your configuration, e.g. - <listitem> - <para><varname>exportReferencesGraph</varname>: Export more - complete info in JSON - format. (c2b0d8749f7e77afc1c4b3e8dd36b7ee9720af4a)</para> - </listitem> + <programlisting>imports = [ (builtins.fetchGit https://github.com/edolstra/dwarffs + "/module.nix") ];</programlisting> - <listitem> - <para>Support for - netrc. (e6e74f987f0fa284d220432d426eb965269a97d6, - 302386f775eea309679654e5ea7c972fb6e7b9af)</para> + </para> </listitem> <listitem> - <para>Support <uri>s3://</uri> URIs in all places where Nix allows - URIs. (9ff9c3f2f80ba4108e9c945bbfda2c64735f987b)</para> + <para>Similarly, <function>builtins.fetchMercurial</function> + allows you to fetch Mercurial repositories.</para> </listitem> <listitem> - <para>The <option>build-max-jobs</option> option can be set to - <literal>auto</literal> to use the number of CPUs in the - system. (7251d048fa812d2551b7003bc9f13a8f5d4c95a5)</para> + <para><function>builtins.path</function> generalises + <function>builtins.filterSource</function> and path literals + (e.g. <literal>./foo</literal>). It allows specifying a store path + name that differs from the source path name + (e.g. <literal>builtins.path { path = ./foo; name = "bar"; + }</literal>) and also supports filtering out unwanted + files.</para> </listitem> <listitem> - <para>Add support for Brotli compression. - <uri>cache.nixos.org</uri> compresses build logs using - Brotli.</para> + <para><function>builtins.fetchurl</function> and + <function>builtins.fetchTarball</function> now support + <varname>sha256</varname> and <varname>name</varname> + attributes.</para> </listitem> <listitem> - <para>Substitutions from binary caches now require signatures by - default. This was already the case on - NixOS. (ecbc3fedd3d5bdc5a0e1a0a51b29062f2874ac8b)</para> + <para><function + xlink:href="https://github.com/NixOS/nix/commit/b8867a0239b1930a16f9ef3f7f3e864b01416dff">builtins.split</function> + splits a string using a POSIX extended regular expression as the + separator.</para> </listitem> <listitem> - <para><command>nix-env</command> now ignores packages with bad - derivation names (in particular those starting with a digit or - containing a - dot). (b0cb11722626e906a73f10dd9a0c9eea29faf43a)</para> + <para><function + xlink:href="https://github.com/NixOS/nix/commit/26d92017d3b36cff940dcb7d1611c42232edb81a">builtins.partition</function> + partitions the elements of a list into two lists, depending on a + Boolean predicate.</para> </listitem> <listitem> - <para>Renamed various configuration options. (TODO: in progress)</para> + <para><literal><nix/fetchurl.nix></literal> now uses the + content-addressable tarball cache at + <uri>http://tarballs.nixos.org/</uri>, just like + <function>fetchurl</function> in + Nixpkgs. (f2682e6e18a76ecbfb8a12c17e3a0ca15c084197)</para> </listitem> <listitem> - <para>Remote machines can now be specified on the command - line. TODO: - document. (1a68710d4dff609bbaf61db3e17a2573f0aadf17)</para> + <para>In restricted and pure evaluation mode, builtin functions + that download from the network (such as + <function>fetchGit</function>) are permitted to fetch underneath a + list of URI prefixes specified in the option + <option>allowed-uris</option>.</para> </listitem> - <listitem> - <para>In Linux sandbox builds, we now use - <filename>/build</filename> instead of <filename>/tmp</filename> - as the temporary build directory. This fixes potential security - problems when a build accidentally stores its - <envar>TMPDIR</envar> in some critical place, such as an - RPATH. (eba840c8a13b465ace90172ff76a0db2899ab11b)</para> - </listitem> +</itemizedlist> - <listitem> - <para>In Linux sandbox builds, we now provide a default - <filename>/bin/sh</filename> (namely <filename>ash</filename> from - BusyBox). (a2d92bb20e82a0957067ede60e91fab256948b41)</para> - </listitem> +</para> - <listitem> - <para>Make all configuration options available as command line - flags (b8283773bd64d7da6859ed520ee19867742a03ba).</para> - </listitem> +<para>The Nix build environment has the following changes: - <listitem> - <para>Support base-64 - hashes. (c0015e87af70f539f24d2aa2bc224a9d8b84276b)</para> - </listitem> +<itemizedlist> <listitem> - <para><command>nix-shell</command> now uses - <varname>bashInteractive</varname> from Nixpkgs, rather than the - <command>bash</command> command that happens to be in the caller’s - <envar>PATH</envar>. This is especially important on macOS where - the <command>bash</command> provided by the system is seriously - outdated and cannot execute <literal>stdenv</literal>’s setup - script.</para> + <para>Values such as Booleans, integers, (nested) lists and + attribute sets can <link + xlink:href="https://github.com/NixOS/nix/commit/6de33a9c675b187437a2e1abbcb290981a89ecb1">now</link> + be passed to builders in a non-lossy way. If the special attribute + <varname>__structuredAttrs</varname> is set to + <literal>true</literal>, the other derivation attributes are + serialised in JSON format and made available to the builder via + the file <envar>.attrs.json</envar> in the builder’s temporary + directory. This obviates the need for + <varname>passAsFile</varname> since JSON files have no size + restrictions, unlike process environments.</para> + + <para><link + xlink:href="https://github.com/NixOS/nix/commit/2d5b1b24bf70a498e4c0b378704cfdb6471cc699">As + a convenience to Bash builders</link>, Nix writes a script named + <envar>.attrs.sh</envar> to the builder’s directory that + initialises shell variables corresponding to all attributes that + are representable in Bash. This includes non-nested (associative) + arrays. For example, the attribute <literal>hardening.format = + true</literal> ends up as the Bash associative array element + <literal>${hardening[format]}</literal>.</para> + </listitem> + + <listitem> + <para>Builders can <link + xlink:href="https://github.com/NixOS/nix/commit/88e6bb76de5564b3217be9688677d1c89101b2a3">now</link> + communicate what build phase they are in by writing messages to + the file descriptor specified in <envar>NIX_LOG_FD</envar>. The + current phase is shown by the <command>nix</command> progress + indicator. + </para> </listitem> <listitem> - <para>New builtin functions: <function>builtins.split</function> - (b8867a0239b1930a16f9ef3f7f3e864b01416dff), - <function>builtins.partition</function>.</para> + <para>In Linux sandbox builds, we <link + xlink:href="https://github.com/NixOS/nix/commit/a2d92bb20e82a0957067ede60e91fab256948b41">now</link> + provide a default <filename>/bin/sh</filename> (namely + <filename>ash</filename> from BusyBox).</para> </listitem> <listitem> - <para>Automatic garbage collection.</para> + <para>In structured attribute mode, + <varname>exportReferencesGraph</varname> <link + xlink:href="https://github.com/NixOS/nix/commit/c2b0d8749f7e77afc1c4b3e8dd36b7ee9720af4a">exports</link> + extended information about closures in JSON format. In particular, + it includes the sizes and hashes of paths. This is primarily + useful for NixOS image builders.</para> </listitem> <listitem> - <para><command>nix-store -q --roots</command> and - <command>nix-store --gc --print-roots</command> now show temporary - and in-memory roots.</para> + <para>Builds are <link + xlink:href="https://github.com/NixOS/nix/commit/21948deed99a3295e4d5666e027a6ca42dc00b40">now</link> + killed as soon as Nix receives EOF on the builder’s stdout or + stderr. This fixes a bug that allowed builds to hang Nix + indefinitely, regardless of + timeouts.</para> </listitem> <listitem> - <para>Builders can now communicate what build phase they are in by - writing messages to the file descriptor specified in - <envar>NIX_LOG_FD</envar>. (88e6bb76de5564b3217be9688677d1c89101b2a3) - </para> + <para>The <option>sandbox-paths</option> configuration + option can now specify optional paths by appending a + <literal>?</literal>, e.g. <literal>/dev/nvidiactl?</literal> will + bind-mount <varname>/dev/nvidiactl</varname> only if it + exists.</para> </listitem> <listitem> - <para> - Nix can now be extended with plugins. See the documentation of - the 'plugin-files' option for more details. - </para> + <para>On Linux, builds are now executed in a user + namespace with uid 1000 and gid 100.</para> </listitem> </itemizedlist> -<para>Some features were removed:</para> - -<itemizedlist> - - <listitem> - <para>“Nested” log output. As a result, - <command>nix-log2xml</command> was also removed.</para> - </listitem> +</para> - <listitem> - <para>OpenSSL-based signing. (f435f8247553656774dd1b2c88e9de5d59cab203)</para> - </listitem> +<para>A number of significant internal changes were made: - <listitem> - <para>Caching of failed - builds. (8cffec84859cec8b610a2a22ab0c4d462a9351ff)</para> - </listitem> +<itemizedlist> <listitem> - <para><filename>nix-mode.el</filename> has been removed from - Nix. It is now a separate repository in - <uri>https://github.com/NixOS/nix-mode</uri> and can be installed - through the MELPA package repository.</para> + <para>Nix no longer depends on Perl and all Perl components have + been rewritten in C++ or removed. The Perl bindings that used to + be part of Nix have been moved to a separate package, + <literal>nix-perl</literal>.</para> </listitem> <listitem> - <para>In restricted evaluation mode - (<option>--restrict-eval</option>), builtin functions that - download from the network (such as <function>fetchGit</function>) - are permitted to fetch underneath the list of URI prefixes - specified in the option <option>allowed-uris</option>.</para> + <para>All <classname>Store</classname> classes are now + thread-safe. <classname>RemoteStore</classname> supports multiple + concurrent connections to the daemon. This is primarily useful in + multi-threaded programs such as + <command>hydra-queue-runner</command>.</para> </listitem> </itemizedlist> +</para> + <para>This release has contributions from TBD.</para> </section> diff --git a/doc/manual/style.css b/doc/manual/style.css index 53fd9d5709c3..592583ab086a 100644 --- a/doc/manual/style.css +++ b/doc/manual/style.css @@ -96,7 +96,6 @@ div.example margin-right: 1.5em; background: #f4f4f8; border-radius: 0.4em; - box-shadow: 0.4em 0.4em 0.5em #e0e0e0; } div.example p.title @@ -106,7 +105,6 @@ div.example p.title div.example pre { - box-shadow: none; } @@ -116,15 +114,12 @@ div.example pre pre.screen, pre.programlisting { - border: 1px solid #b0b0b0; - padding: 3px 3px; + padding: 6px 6px; margin-left: 1.5em; margin-right: 1.5em; color: #600000; background: #f4f4f8; font-family: monospace; - border-radius: 0.4em; - box-shadow: 0.4em 0.4em 0.5em #e0e0e0; } div.example pre.programlisting @@ -149,7 +144,6 @@ div.example pre.programlisting padding: 0.3em 0.3em 0.3em 0.3em; background: #fffff5; border-radius: 0.4em; - box-shadow: 0.4em 0.4em 0.5em #e0e0e0; } div.note, div.warning @@ -256,16 +250,14 @@ span.command strong div.calloutlist table { - box-shadow: none; } table { border-collapse: collapse; - box-shadow: 0.4em 0.4em 0.5em #e0e0e0; } div.affiliation { font-style: italic; -} \ No newline at end of file +} diff --git a/src/nix/ls.cc b/src/nix/ls.cc index 69620595d8ca..e99622faf472 100644 --- a/src/nix/ls.cc +++ b/src/nix/ls.cc @@ -90,6 +90,16 @@ struct CmdLsStore : StoreCommand, MixLs expectArg("path", &path); } + Examples examples() override + { + return { + Example{ + "To list the contents of a store path in a binary cache:", + "nix ls-store --store https://cache.nixos.org/ -lR /nix/store/0i2jd68mp5g6h2sa5k9c85rb80sn8hi9-hello-2.10" + }, + }; + } + std::string name() override { return "ls-store"; @@ -116,6 +126,16 @@ struct CmdLsNar : Command, MixLs expectArg("path", &path); } + Examples examples() override + { + return { + Example{ + "To list a specific file in a NAR:", + "nix ls-nar -l hello.nar /bin/hello" + }, + }; + } + std::string name() override { return "ls-nar"; diff --git a/src/nix/run.cc b/src/nix/run.cc index ade87e63a49c..822654daf488 100644 --- a/src/nix/run.cc +++ b/src/nix/run.cc @@ -85,6 +85,10 @@ struct CmdRun : InstallablesCommand "To run GNU Hello:", "nix run nixpkgs.hello -c hello --greeting 'Hi everybody!'" }, + Example{ + "To run GNU Hello in a chroot store:", + "nix run --store ~/my-nix nixpkgs.hello -c hello" + }, }; } |