Age | Commit message (Collapse) | Author | Files | Lines |
|
Needed by Hydra.
|
|
|
|
The store parameter "write-nar-listing=1" will cause BinaryCacheStore
to write a file ‘<store-hash>.ls.xz’ for each ‘<store-hash>.narinfo’
added to the binary cache. This file contains an XZ-compressed JSON
file describing the contents of the NAR, excluding the contents of
regular files.
E.g.
{
"version": 1,
"root": {
"type": "directory",
"entries": {
"lib": {
"type": "directory",
"entries": {
"Mcrt1.o": {
"type": "regular",
"size": 1288
},
"Scrt1.o": {
"type": "regular",
"size": 3920
},
}
}
}
...
}
}
(The actual file has no indentation.)
This is intended to speed up the NixOS channels programs index
generator [1], since fetching gazillions of large NARs from
cache.nixos.org is currently a bottleneck for updating the regular
(non-small) channel.
[1] https://github.com/NixOS/nixos-channel-scripts/blob/master/generate-programs-index.cc
|
|
This allows the binary cache substituter to pipeline requests.
|
|
|
|
|
|
|
|
The fact that queryPathInfo() is synchronous meant that we needed a
thread for every concurrent binary cache lookup, even though they end
up being handled by the same download thread. Requiring hundreds of
threads is not a good idea. So now there is an asynchronous version of
queryPathInfo() that takes a callback function to process the
result. Similarly, enqueueDownload() now takes a callback rather than
returning a future.
Thus, a command like
nix path-info --store https://cache.nixos.org/ -r /nix/store/slljrzwmpygy1daay14kjszsr9xix063-nixos-16.09beta231.dccf8c5
that returns 4941 paths now takes 1.87s using only 2 threads (the main
thread and the downloader thread). (This is with a prewarmed
CloudFront.)
|
|
|
|
RegisterStoreImplementation
|
|
|
|
This allows such paths to be imported without signatures.
|
|
|
|
|
|
E.g. "local?store=/tmp/store&state=/tmp/var".
|
|
|
|
|
|
As a side effect, this ensures that signatures are propagated when
copying paths between stores.
Also refactored import/export to make use of this.
|
|
|
|
|
|
This is to allow store-specific configuration,
e.g. s3://my-cache?compression=bzip2&secret-key=/path/to/key.
|
|
Substitution is now simply a Store -> Store copy operation, most
typically from BinaryCacheStore to LocalStore.
|
|
|
|
This allows commands like "nix verify --all" or "nix path-info --all"
to work on S3 caches.
Unfortunately, this requires some ugly hackery: when querying the
contents of the bucket, we don't want to have to read every .narinfo
file. But the S3 bucket keys only include the hash part of each store
path, not the name part. So as a special exception
queryAllValidPaths() can now return store paths *without* the name
part, and queryPathInfo() accepts such store paths (returning a
ValidPathInfo object containing the full name).
|
|
This re-implements the binary cache database in C++, allowing it to be
used by other Store backends, in particular the S3 backend.
|
|
Caching path info is generally useful. For instance, it speeds up "nix
path-info -rS /run/current-system" (i.e. showing the closure sizes of
all paths in the closure of the current system) from 5.6s to 0.15s.
This also eliminates some APIs like Store::queryDeriver() and
Store::queryReferences().
|
|
This specifies the number of distinct signatures required to consider
each path "trusted".
Also renamed ‘--no-sigs’ to ‘--no-trust’ for the flag that disables
verifying whether a path is trusted (since a path can also be trusted
if it has no signatures, but was built locally).
|
|
E.g.
$ nix sign-paths -k ./secret -r $(type -p geeqie)
signs geeqie and all its dependencies using the key in ./secret.
|
|
|
|
|
|
This for instance allows hydra-queue-runner to add the S3 backend
at runtime.
|
|
This is primary to allow hydra-queue-runner to extract files like
"nix-support/hydra-build-products" from NARs in binary caches.
|
|
|
|
|
|
So you can now do:
$ NIX_REMOTE=file:///tmp/binary-cache nix-store -qR /nix/store/...
|
|
|
|
|
|
|
|
|
|
Calling a class an API is a bit redundant...
|
|
Also, move a few free-standing functions into StoreAPI and Derivation.
Also, introduce a non-nullable smart pointer, ref<T>, which is just a
wrapper around std::shared_ptr ensuring that the pointer is never
null. (For reference-counted values, this is better than passing a
"T&", because the latter doesn't maintain the refcount. Usually, the
caller will have a shared_ptr keeping the value alive, but that's not
always the case, e.g., when passing a reference to a std::thread via
std::bind.)
|
|
|
|
Fixes #609.
|
|
|
|
|
|
|
|
|
|
Only indirect roots (symlinks to symlinks to the Nix store) are now
supported.
|
|
|
|
That is, delete almost nothing (it will still remove unused links from
/nix/store/.links).
|