about summary refs log tree commit diff
path: root/perl/lib
AgeCommit message (Collapse)AuthorFilesLines
2016-04-19 Move path info caching from BinaryCacheStore to StoreEelco Dolstra1-11/+10
Caching path info is generally useful. For instance, it speeds up "nix path-info -rS /run/current-system" (i.e. showing the closure sizes of all paths in the closure of the current system) from 5.6s to 0.15s. This also eliminates some APIs like Store::queryDeriver() and Store::queryReferences().
2016-04-11 Remove manifest supportEelco Dolstra3-508/+1
Manifests have been superseded by binary caches for years. This also gets rid of nix-pull, nix-generate-patches and bsdiff/bspatch.
2016-02-26 importPaths(): Optionally add NARs to binary cache accessorEelco Dolstra1-1/+1
This enables an optimisation in hydra-queue-runner, preventing a download of a NAR it just uploaded to the cache when reading files like hydra-build-products.
2016-02-16 Add C++ functions for .narinfo processing / signingEelco Dolstra1-10/+4
This is currently only used by the Hydra queue runner rework, but like eff5021eaa6dc69f65ea1a8abe8f3ab11ef5eb0a it presumably will be useful for the C++ rewrite of nix-push and download-from-binary-cache. (@shlevy)
2016-02-16 Rename ValidPathInfo::hash -> narHash for consistencyEelco Dolstra1-1/+1
2016-02-15 Fix nix-copy-closureEelco Dolstra1-0/+1
http://hydra.nixos.org/build/32005971
2016-02-04 StoreAPI -> StoreEelco Dolstra1-3/+3
Calling a class an API is a bit redundant...
2016-02-04 Eliminate the "store" global variableEelco Dolstra1-36/+23
Also, move a few free-standing functions into StoreAPI and Derivation. Also, introduce a non-nullable smart pointer, ref<T>, which is just a wrapper around std::shared_ptr ensuring that the pointer is never null. (For reference-counted values, this is better than passing a "T&", because the latter doesn't maintain the refcount. Usually, the caller will have a shared_ptr keeping the value alive, but that's not always the case, e.g., when passing a reference to a std::thread via std::bind.)
2016-01-07 Fix bad use of croak()Eelco Dolstra1-22/+22
Detected by -Werror=format-security. http://hydra.nixos.org/build/29973207
2015-10-09 Provide addTempRoot in the Perl APIEelco Dolstra2-0/+11
Needed by Hydra.
2015-06-03 Handle base-16 NarHash fields in signed .narinfo filesEelco Dolstra3-3/+17
2015-06-03 Be more robust wrt broken .narinfo filesEelco Dolstra1-3/+10
2015-06-01 Include the cache.nixos.org signing key by defaultEelco Dolstra1-1/+3
2015-03-25 addToStore(): Take explicit name argumentEelco Dolstra1-1/+1
2015-03-04 Reduce verbosity in build-remote.plEelco Dolstra2-0/+6
2015-02-17 Include NAR size in fingerprint computationEelco Dolstra1-4/+3
This is not strictly needed for integrity (since we already include the NAR hash in the fingerprint) but it helps against endless data attacks [1]. (However, this will also require download-from-binary-cache.pl to bail out if it receives more than the specified number of bytes.) [1] https://isis.poly.edu/~jcappos/papers/cappos_mirror_ccs_08.pdf
2015-02-10 Make libsodium an optional dependencyEelco Dolstra1-0/+10
2015-02-04 Sign a subset of the .narinfoEelco Dolstra1-11/+26
We only need to sign the store path, NAR hash and references (the "fingerprint"). Everything else is irrelevant to security. For instance, the compression algorithm or the hash of the compressed NAR don't matter as long as the contents of the uncompressed NAR are correct. (Maybe we should include derivers in the fingerprint, but they're broken and nobody cares about them. Also, it might be nice in the future if .narinfos contained signatures from multiple independent signers. But that's impossible if the deriver is included in the fingerprint, since everybody will tend to have a different deriver for the same store path.) Also renamed the "Signature" field to "Sig" since the format changed in an incompatible way.
2015-02-04 Use libsodium instead of OpenSSL for binary cache signingEelco Dolstra5-61/+68
Sodium's Ed25519 signatures are much shorter than OpenSSL's RSA signatures. Public keys are also much shorter, so they're now specified directly in the nix.conf option ‘binary-cache-public-keys’. The new command ‘nix-store --generate-binary-cache-key’ generates and prints a public and secret key.
2014-11-20 Fix bad operatorEelco Dolstra1-1/+1
Spotted by Perl 5.20: Possible precedence issue with control flow operator at /usr/lib/perl5/site_perl/5.20.1/x86_64-linux-gnu-thread-multi/Nix/Utils.pm line 46.
2014-11-10 SSH.pm: Print a friendlier message if connecting failsEelco Dolstra2-4/+12
"got EOF while expecting 8 bytes from remote side" is not very helpful.
2014-10-14 Remove unused @sshOpts flagEelco Dolstra1-6/+6
Closes #300.
2014-09-18 Remove debug statementEelco Dolstra1-1/+0
2014-08-20 Use proper quotes everywhereEelco Dolstra4-15/+18
2014-08-13 Use $XDG_RUNTIME_DIR for temporary filesEelco Dolstra1-1/+9
2014-07-24 nix-copy-closure: Drop --bzip2, --xz, --show-progressEelco Dolstra1-34/+7
These are too difficult to implement via nix-store --serve. ‘--show-progress’ could be re-implemented fairly easily via a sink/source wrapper class.
2014-07-24 Fix NIX_SSHOPTSEelco Dolstra2-4/+8
Closes #302.
2014-07-24 Remove obsolete SSH master connection codeEelco Dolstra1-54/+1
2014-07-24 RefactorEelco Dolstra2-8/+21
2014-07-12 build-remote.pl: Don't keep a shell process aroundEelco Dolstra1-1/+2
2014-07-12 build-remote.pl: Fix build logEelco Dolstra1-2/+4
2014-07-11 build-remote.pl: Use ‘nix-store --serve’ on the remote sideEelco Dolstra4-63/+127
This makes things more efficient (we don't need to use an SSH master connection, and we only start a single remote process) and gets rid of locking issues (the remote nix-store process will keep inputs and outputs locked as long as they're needed). It also makes it more or less secure to connect directly to the root account on the build machine, using a forced command (e.g. ‘command="nix-store --serve --write"’). This bypasses the Nix daemon and is therefore more efficient. Also, don't call nix-store to import the output paths.
2014-07-11 Fix closure size displayEelco Dolstra1-1/+1
2014-07-10 Fix use of sysreadEelco Dolstra1-6/+18
2014-07-10 nix-copy-closure -s: Do substitutions via ‘nix-store --serve’Eelco Dolstra1-8/+11
This means we no longer need an SSH master connection, since we only execute a single command on the remote host.
2014-07-10 nix-copy-closure: Fix --dry-runEelco Dolstra1-8/+8
2014-07-10 nix-copy-closure: Restore compression and the progress viewerEelco Dolstra1-12/+40
2014-07-10 nix-copy-closure: Fix race conditionEelco Dolstra1-3/+74
There is a long-standing race condition when copying a closure to a remote machine, particularly affecting build-remote.pl: the client first asks the remote machine which paths it already has, then copies over the missing paths. If the garbage collector kicks in on the remote machine between the first and second step, the already-present paths may be deleted. The missing paths may then refer to deleted paths, causing nix-copy-closure to fail. The client now performs both steps using a single remote Nix call (using ‘nix-store --serve’), locking all paths in the closure while querying. I changed the --serve protocol a bit (getting rid of QueryCommand), so this breaks the SSH substituter from older versions. But it was marked experimental anyway. Fixes #141.
2014-01-21 Ugly hack to fix building with clangEelco Dolstra1-0/+4
2014-01-21 Merge branch 'master' into makeEelco Dolstra4-4/+83
Conflicts: src/libexpr/eval.cc
2014-01-08 Support cryptographically signed binary cachesEelco Dolstra3-4/+81
NAR info files in binary caches can now have a cryptographic signature that Nix will verify before using the corresponding NAR file. To create a private/public key pair for signing and verifying a binary cache, do: $ openssl genrsa -out ./cache-key.sec 2048 $ openssl rsa -in ./cache-key.sec -pubout > ./cache-key.pub You should also come up with a symbolic name for the key, such as "cache.example.org-1". This will be used by clients to look up the public key. (It's a good idea to number keys, in case you ever need to revoke/replace one.) To create a binary cache signed with the private key: $ nix-push --dest /path/to/binary-cache --key ./cache-key.sec --key-name cache.example.org-1 The public key (cache-key.pub) should be distributed to the clients. They should have a nix.conf should contain something like: signed-binary-caches = * binary-cache-public-key-cache.example.org-1 = /path/to/cache-key.pub If all works well, then if Nix fetches something from the signed binary cache, you will see a message like: *** Downloading ‘http://cache.example.org/nar/7dppcj5sc1nda7l54rjc0g5l1hamj09j-subversion-1.7.11’ (signed by ‘cache.example.org-1’) to ‘/nix/store/7dppcj5sc1nda7l54rjc0g5l1hamj09j-subversion-1.7.11’... On the other hand, if the signature is wrong, you get a message like NAR info file `http://cache.example.org/7dppcj5sc1nda7l54rjc0g5l1hamj09j.narinfo' has an invalid signature; ignoring Signatures are implemented as a single line appended to the NAR info file, which looks like this: Signature: 1;cache.example.org-1;HQ9Xzyanq9iV...muQ== Thus the signature has 3 fields: a version (currently "1"), the ID of key, and the base64-encoded signature of the SHA-256 hash of the contents of the NAR info file up to but not including the Signature line. Issue #75.
2013-12-20 perl: Call loadConfFile() in doInit to avoid screwing sqlite journal mode.Petr Rockai1-0/+2
If the database is opened through perl bindings (and even though nix.conf has use-sqlite-wal set to false), the database is automatically converted into WAL mode. This makes the next nix process to access the database convert it back to "truncate". If the database is still open at the time in wal mode by the perl program, this fails and crashes the nix doing the wal -> truncate conversion.
2013-12-02 Install bsdiff and bspatch in $(libexecdir)/nixEelco Dolstra1-1/+1
2013-11-25 Add a Makefile for the Perl stuffEelco Dolstra1-1/+1
2013-10-24 Fix segfault on DarwinEelco Dolstra1-0/+1
Ever since SQLite in Nixpkgs was updated to 3.8.0.2, Nix has randomly segfaulted on Darwin: http://hydra.nixos.org/build/6175515 http://hydra.nixos.org/build/6611038 It turns out that this is because the binary cache substituter somehow ends up loading two versions of SQLite: the one in Nixpkgs and the other from /usr/lib/libsqlite3.dylib. It's not exactly clear why the latter is loaded, but it appears to be because WWW::Curl indirectly loads /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation, which in turn seems to load /usr/lib/libsqlite3.dylib. This leads to a segfault when Perl exits: #0 0x00000001010375f4 in sqlite3_finalize () #1 0x000000010125806e in sqlite_st_destroy () #2 0x000000010124bc30 in XS_DBD__SQLite__st_DESTROY () #3 0x00000001001c8155 in XS_DBI_dispatch () ... #14 0x0000000100023224 in perl_destruct () #15 0x0000000100000d6a in main () ... The workaround is to explicitly load DBD::SQLite before WWW::Curl.
2013-09-06 Don't apply the CPU affinity hack to nix-shell (and other Perl programs)Eelco Dolstra1-0/+1
As discovered by Todd Veldhuizen, the shell started by nix-shell has its affinity set to a single CPU. This is because nix-shell connects to the Nix daemon, which causes the affinity hack to be applied. So we turn this off for Perl programs.
2013-06-20 Don't keep "disabled" substituters runningEelco Dolstra1-1/+4
For instance, it's pointless to keep copy-from-other-stores running if there are no other stores, or download-using-manifests if there are no manifests. This also speeds things up because we don't send queries to those substituters.
2013-05-10 build-remote.pl: Properly close the SSH connection between attemptsEelco Dolstra1-0/+1
2013-05-03 Fixing the pv position regarding compressionLluís Batlle i Rossell1-1/+1
Problem noticed by niksnut.
2013-05-03 Adding ETA support to the --show-progress in nix-copy-closureLluís Batlle i Rossell1-4/+8
Based on https://github.com/NixOS/nix/pull/6 from shlevy
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-11-01T15·38+0000