about summary refs log tree commit diff
path: root/scripts/nix-push.in
AgeCommit message (Collapse)AuthorFilesLines
2016-02-16 Add C++ functions for .narinfo processing / signingEelco Dolstra1-6/+3
This is currently only used by the Hydra queue runner rework, but like eff5021eaa6dc69f65ea1a8abe8f3ab11ef5eb0a it presumably will be useful for the C++ rewrite of nix-push and download-from-binary-cache. (@shlevy)
2015-06-08 nix-push: Support -jEelco Dolstra1-1/+6
Fixes #548.
2015-02-17 Include NAR size in fingerprint computationEelco Dolstra1-1/+1
This is not strictly needed for integrity (since we already include the NAR hash in the fingerprint) but it helps against endless data attacks [1]. (However, this will also require download-from-binary-cache.pl to bail out if it receives more than the specified number of bytes.) [1] https://isis.poly.edu/~jcappos/papers/cappos_mirror_ccs_08.pdf
2015-02-04 Sign a subset of the .narinfoEelco Dolstra1-2/+3
We only need to sign the store path, NAR hash and references (the "fingerprint"). Everything else is irrelevant to security. For instance, the compression algorithm or the hash of the compressed NAR don't matter as long as the contents of the uncompressed NAR are correct. (Maybe we should include derivers in the fingerprint, but they're broken and nobody cares about them. Also, it might be nice in the future if .narinfos contained signatures from multiple independent signers. But that's impossible if the deriver is included in the fingerprint, since everybody will tend to have a different deriver for the same store path.) Also renamed the "Signature" field to "Sig" since the format changed in an incompatible way.
2015-02-04 Use libsodium instead of OpenSSL for binary cache signingEelco Dolstra1-13/+12
Sodium's Ed25519 signatures are much shorter than OpenSSL's RSA signatures. Public keys are also much shorter, so they're now specified directly in the nix.conf option ‘binary-cache-public-keys’. The new command ‘nix-store --generate-binary-cache-key’ generates and prints a public and secret key.
2014-08-29 Shut up "Wide character" warnings in Perl scriptsEelco Dolstra1-0/+3
2014-08-20 Use proper quotes everywhereEelco Dolstra1-8/+8
2014-08-13 Use $XDG_RUNTIME_DIR for temporary filesEelco Dolstra1-3/+1
2014-01-08 Support cryptographically signed binary cachesEelco Dolstra1-1/+17
NAR info files in binary caches can now have a cryptographic signature that Nix will verify before using the corresponding NAR file. To create a private/public key pair for signing and verifying a binary cache, do: $ openssl genrsa -out ./cache-key.sec 2048 $ openssl rsa -in ./cache-key.sec -pubout > ./cache-key.pub You should also come up with a symbolic name for the key, such as "cache.example.org-1". This will be used by clients to look up the public key. (It's a good idea to number keys, in case you ever need to revoke/replace one.) To create a binary cache signed with the private key: $ nix-push --dest /path/to/binary-cache --key ./cache-key.sec --key-name cache.example.org-1 The public key (cache-key.pub) should be distributed to the clients. They should have a nix.conf should contain something like: signed-binary-caches = * binary-cache-public-key-cache.example.org-1 = /path/to/cache-key.pub If all works well, then if Nix fetches something from the signed binary cache, you will see a message like: *** Downloading ‘http://cache.example.org/nar/7dppcj5sc1nda7l54rjc0g5l1hamj09j-subversion-1.7.11’ (signed by ‘cache.example.org-1’) to ‘/nix/store/7dppcj5sc1nda7l54rjc0g5l1hamj09j-subversion-1.7.11’... On the other hand, if the signature is wrong, you get a message like NAR info file `http://cache.example.org/7dppcj5sc1nda7l54rjc0g5l1hamj09j.narinfo' has an invalid signature; ignoring Signatures are implemented as a single line appended to the NAR info file, which looks like this: Signature: 1;cache.example.org-1;HQ9Xzyanq9iV...muQ== Thus the signature has 3 fields: a version (currently "1"), the ID of key, and the base64-encoded signature of the SHA-256 hash of the contents of the NAR info file up to but not including the Signature line. Issue #75.
2013-07-01 Add support for uncompressed NARs in binary cachesEelco Dolstra1-1/+3
Issue NixOS/hydra#102.
2012-11-06 nix-push: Handle pushing a symlinkEelco Dolstra1-2/+0
2012-10-18 nix-push: Add a flag ‘--manifest-path’ to write the manifest to another ↵Eelco Dolstra1-1/+8
directory
2012-10-17 nix-push: Add ‘--link’ flagEelco Dolstra1-1/+8
If ‘--link’ is given, nix-push will create hard links to the NAR files in the store, rather than copying them. This is faster and requires less disk space. However, it doesn't work if the store is on a different file system.
2012-10-17 nix-push: Only generate and copy a NAR if it doesn't already existEelco Dolstra1-6/+36
This prevents unnecessary and slow rebuilds of NARs that already exist in the binary cache.
2012-10-03 When ‘--help’ is given, just run ‘man’ to show the manual pageEelco Dolstra1-14/+3
I.e. do what git does. I'm too lazy to keep the builtin help text up to date :-) Also add ‘--help’ to various commands that lacked it (e.g. nix-collect-garbage).
2012-09-19 Support xz compression in the download-using-manifests substituterEelco Dolstra1-0/+1
2012-07-27 Allow a binary cache to declare that it doesn't support "nix-env -qas"Eelco Dolstra1-12/+20
Querying all substitutable paths via "nix-env -qas" is potentially hard on a server, since it involves sending thousands of HEAD requests. So a binary cache must now have a meta-info file named "nix-cache-info" that specifies whether the server wants this. It also specifies the store prefix so that we don't send useless queries to a binary cache for a different store prefix.
2012-07-26 nix-push: Support generating a manifest againEelco Dolstra1-10/+34
This makes all the tests succeed. Woohoo!
2012-07-26 nix-push: Remove the upload featureEelco Dolstra1-51/+24
2012-07-02 nix-push: Always generate base-32 hashesEelco Dolstra1-2/+2
2012-07-02 Binary caches: use a better keyEelco Dolstra1-3/+3
Use the hash part of the store path as a key rather than a hash of the store path. This is enough to get the desired privacy property.
2012-07-01 Allow both bzip2 and xz compressionEelco Dolstra1-20/+41
2012-06-29 nix-push: Don't pollute the current directory with result symlinkEelco Dolstra1-1/+1
2012-06-29 DohEelco Dolstra1-1/+1
2012-06-29 Use XZ compression in binary cachesEelco Dolstra1-13/+13
XZ compresses significantly better than bzip2. Here are the compression ratios and execution times (using 4 cores in parallel) on my /var/run/current-system (3.1 GiB): bzip2: total compressed size 849.56 MiB, 30.8% [2m08] xz -6: total compressed size 641.84 MiB, 23.4% [6m53] xz -7: total compressed size 621.82 MiB, 22.6% [7m19] xz -8: total compressed size 599.33 MiB, 21.8% [7m18] xz -9: total compressed size 588.18 MiB, 21.4% [7m40] Note that compression takes much longer. More importantly, however, decompression is much faster: bzip2: 1m47.274s xz -6: 0m55.446s xz -7: 0m54.119s xz -8: 0m52.388s xz -9: 0m51.842s The only downside to using -9 is that decompression takes a fair amount (~65 MB) of memory.
2012-06-28 nix-push: create a manifest-less binary cacheEelco Dolstra1-159/+93
Manifests are a huge pain, since users need to run nix-pull directly or indirectly to obtain them. They tend to be large and lag behind the available binaries; also, the downloaded manifests in /nix/var/nix/manifest need to be in sync with the Nixpkgs sources. So we want to get rid of them. The idea of manifest-free operation works as follows. Nix is configured with a set of URIs of binary caches, e.g. http://nixos.org/binary-cache Whenever Nix needs a store path X, it checks each binary cache for the existence of a file <CACHE-URI>/<SHA-256 hash of X>.narinfo, e.g. http://nixos.org/binary-cache/bi1gh9...ia17.narinfo The .narinfo file contains the necessary information about the store path that was formerly kept in the manifest, i.e., (relative) URI of the compressed NAR, references, size, hash, etc. For example: StorePath: /nix/store/xqp4l88cr9bxv01jinkz861mnc9p7qfi-neon-0.29.6 URL: 1bjxbg52l32wj8ww47sw9f4qz0r8n5vs71l93lcbgk2506v3cpfd.nar.bz2 CompressedHash: sha256:1bjxbg52l32wj8ww47sw9f4qz0r8n5vs71l93lcbgk2506v3cpfd CompressedSize: 202542 NarHash: sha256:1af26536781e6134ab84201b33408759fc59b36cc5530f57c0663f67b588e15f NarSize: 700440 References: 043zrsanirjh8nbc5vqpjn93hhrf107f-bash-4.2-p24 cj7a81wsm1ijwwpkks3725661h3263p5-glibc-2.13 ... Deriver: 4idz1bgi58h3pazxr3akrw4fsr6zrf3r-neon-0.29.6.drv System: x86_64-linux Nix then knows that it needs to download http://nixos.org/binary-cache/1bjxbg52l32wj8ww47sw9f4qz0r8n5vs71l93lcbgk2506v3cpfd.nar.bz2 to substitute the store path. Note that the store directory is omitted from the References and Deriver fields to save space, and that the URL can be relative to the binary cache prefix. This patch just makes nix-push create binary caches in this format. The next step is to make a substituter that supports them.
2012-04-14 Remove unnecessary "system" argumentEelco Dolstra1-1/+1
2012-01-03 * Use Nix::Config.Eelco Dolstra1-10/+8
2012-01-03 * Refactoring: Get rid of a few subdirectories in corepkgs/, and someEelco Dolstra1-4/+1
other simplifications. * Use <nix/...> to locate the corepkgs. This allows them to be overriden through $NIX_PATH. * Use bash's pipefail option in the NAR builder so that we don't need to create a temporary file.
2011-11-29 * For consistency with "nix-store -q --hash", produce hashes inEelco Dolstra1-2/+2
base-32. (This affects Hydra manifests.)
2011-10-10 * Set the executable bit on scripts.Eelco Dolstra1-0/+0
2011-10-10 * Install NixManifest.pm, NixConfig.pm and GeneratePatches.pm underEelco Dolstra1-4/+5
the Nix:: namespace.
2011-04-11 * configure: detect whether DBD::SQLite is present. If necessary theEelco Dolstra1-1/+1
location to DBI and DBD::SQLite can be passed with --with-dbi and --with-dbd-sqlite.
2011-04-11 * Subtle bug: if you import File::stat in one module, it affects otherEelco Dolstra1-1/+2
modules as well. So use symbolic field names everywhere (which is nicer anyway).
2011-02-17 (no commit message)Eelco Dolstra1-1/+1
2011-02-17 * nix-push: handle the case where the hash is not set in the DB.Eelco Dolstra1-0/+10
2010-12-05 * Use CamelCase for the Perl modules.Eelco Dolstra1-1/+1
2010-11-17 * Store the NAR size in the manifest.Eelco Dolstra1-0/+5
2010-11-17 * nix-push: no need to compute the NAR hash, since the Nix databaseEelco Dolstra1-7/+5
already has it (`nix-store -q --hash').
2009-03-18 * Unify exportReferencesGraph and exportBuildReferencesGraph, and makeEelco Dolstra1-5/+1
sure that it works as expected when you pass it a derivation. That is, we have to make sure that all build-time dependencies are built, and that they are all in the input closure (otherwise remote builds might fail, for example). This is ensured at instantiation time by adding all derivations and their sources to inputDrvs and inputSrcs.
2008-12-04 * Dirty hack to make nix-push work properly on derivations: theEelco Dolstra1-2/+1
derivation should be a source rather than a derivation dependency of the call to the NAR derivation. Otherwise the derivation (and all its dependencies) will be built as a side-effect, which may not even succeed.
2008-11-20 * Urgh.Eelco Dolstra1-2/+1
2008-11-19 * Primop builtins.storePath for declaring a store path as aEelco Dolstra1-1/+1
dependency. `storePath /nix/store/bla' gives exactly the same result as `toPath /nix/store/bla', except that the former includes /nix/store/bla in the dependency context of the string. Useful in some generated Nix expressions like nix-push, which now finally does the right thing wrt distributed builds. (Previously the path to be packed wasn't an explicit dependency, so it wouldn't be copied to the remote machine.)
2008-03-20 * Cleanup.Eelco Dolstra1-5/+9
2007-12-30 * Don't use "store expression", it's obsolete.Eelco Dolstra1-2/+2
2007-11-16 * Flag `--no-build-hook' to disable distributed builds.Eelco Dolstra1-1/+1
* queryDeriver in daemon mode: don't barf if the other side returns an empty string (which means there is no deriver).
2007-09-04 * nix-push / generate-patches: bzip the manifest.Eelco Dolstra1-0/+4
2007-08-15 * Show errors in nix-prefetch-url.Eelco Dolstra1-1/+1
2006-10-04 * tmpnam() -> File::Temp::tempdir().Eelco Dolstra1-10/+7
2006-09-25 * Clean up calls to system().Eelco Dolstra1-1/+1
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-12-13T16·40+0000