depot - monorepo for the virus lounge

Age	Commit message (Collapse)	Author	Files	Lines
2016-09-16	Make computeFSClosure() single-threaded again	Eelco Dolstra	1	-27/+32
	The fact that queryPathInfo() is synchronous meant that we needed a thread for every concurrent binary cache lookup, even though they end up being handled by the same download thread. Requiring hundreds of threads is not a good idea. So now there is an asynchronous version of queryPathInfo() that takes a callback function to process the result. Similarly, enqueueDownload() now takes a callback rather than returning a future. Thus, a command like nix path-info --store https://cache.nixos.org/ -r /nix/store/slljrzwmpygy1daay14kjszsr9xix063-nixos-16.09beta231.dccf8c5 that returns 4941 paths now takes 1.87s using only 2 threads (the main thread and the downloader thread). (This is with a prewarmed CloudFront.)
2016-08-24	Fix queryPathFromHashPart()	Eelco Dolstra	1	-1/+1
	The inner lambda was returning a SQLite-internal char * rather than a std::string, leading to Hydra errors liks Caught exception in Hydra::Controller::Root->narinfo "path âø£â is not in the Nix store at /nix/store/6mvvyb8fgwj23miyal5mdr8ik4ixk15w-hydra-0.1.1234.abcdef/libexec/hydra/lib/Hydra/Controller/Root.pm line 352."
2016-08-10	SQLite:: Add some convenience	Eelco Dolstra	1	-17/+8

2016-08-10	Mark content-addressed paths in the Nix database and in .narinfo	Eelco Dolstra	1	-7/+22
	This allows such paths to be imported without signatures.
2016-08-10	Add a "root" parameter to local stores	Eelco Dolstra	1	-1/+1
	This makes it easier to create a diverted store, i.e. NIX_REMOTE="local?root=/tmp/root" instead of NIX_REMOTE="local?real=/tmp/root/nix/store&state=/tmp/root/nix/var/nix" NIX_LOG_DIR=/tmp/root/nix/var/log
2016-08-10	Remove $NIX_DB_DIR	Eelco Dolstra	1	-1/+1
	This variable has no reason to exist, given $NIX_STATE_DIR.
2016-07-26	makeFixedOutputPath(): Drop superfluous HashType argument	Eelco Dolstra	1	-4/+4

2016-07-11	Modernize AutoCloseFD	Shea Levy	1	-11/+11

2016-06-09	Use O_CLOEXEC in most places	Eelco Dolstra	1	-2/+2

2016-06-03	Fix "creating statement: table ValidPaths has no column named ultimate"	Eelco Dolstra	1	-41/+29

2016-06-03	Fix some more references to storeDir	Eelco Dolstra	1	-2/+2

2016-06-02	LocalStore: Allow the physical and logical store directories to differ	Eelco Dolstra	1	-32/+40
	This is primarily to subsume the functionality of the copy-from-other-stores substituter. For example, in the NixOS installer, we can now do (assuming we're in the target chroot, and the Nix store of the installation CD is bind-mounted on /tmp/nix): $ nix-build ... --option substituters 'local?state=/tmp/nix/var&real=/tmp/nix/store' However, unlike copy-from-other-stores, this also allows write access to such a store. One application might be fetching substitutes for /nix/store in a situation where the user doesn't have sufficient privileges to create /nix, e.g.: $ NIX_REMOTE="local?state=/home/alice/nix/var&real=/home/alice/nix/store" nix-build ...
2016-06-02	Respect build-use-substitutes	Eelco Dolstra	1	-0/+2

2016-06-02	Allow setting the state directory as a store parameter	Eelco Dolstra	1	-16/+11
	E.g. "local?store=/tmp/store&state=/tmp/var".
2016-06-01	Skip substituters with an incompatible store directory	Eelco Dolstra	1	-0/+2

2016-06-01	Make the store directory a member variable of Store	Eelco Dolstra	1	-35/+32

2016-05-31	nix-copy-closure / build-remote.pl: Disable signature checking	Eelco Dolstra	1	-2/+3
	This restores the Nix 1.11 behaviour.
2016-05-30	Re-implement binary cache signature checking	Eelco Dolstra	1	-0/+5
	This is now done in LocalStore::addToStore(), rather than in the binary cache substituter (which no longer exists).
2016-05-30	LocalStore::addToStore: Verify hash of the imported path	Eelco Dolstra	1	-0/+5

2016-05-30	Re-implement the WantMassQuery property of binary caches	Eelco Dolstra	1	-0/+1

2016-05-04	Cleanup: Remove singleton()	Eelco Dolstra	1	-2/+2

2016-05-04	Add a Store::addToStore() variant that accepts a NAR	Eelco Dolstra	1	-160/+34
	As a side effect, this ensures that signatures are propagated when copying paths between stores. Also refactored import/export to make use of this.
2016-05-04	Remove OpenSSL-based signing	Eelco Dolstra	1	-69/+8

2016-04-29	Allow parameters in store URIs	Eelco Dolstra	1	-0/+6
	This is to allow store-specific configuration, e.g. s3://my-cache?compression=bzip2&secret-key=/path/to/key.
2016-04-29	Eliminate the substituter mechanism	Eelco Dolstra	1	-202/+25
	Substitution is now simply a Store -> Store copy operation, most typically from BinaryCacheStore to LocalStore.
2016-04-21	Implement S3BinaryCacheStore::queryAllValidPaths()	Eelco Dolstra	1	-2/+2
	This allows commands like "nix verify --all" or "nix path-info --all" to work on S3 caches. Unfortunately, this requires some ugly hackery: when querying the contents of the bucket, we don't want to have to read every .narinfo file. But the S3 bucket keys only include the hash part of each store path, not the name part. So as a special exception queryAllValidPaths() can now return store paths without the name part, and queryPathInfo() accepts such store paths (returning a ValidPathInfo object containing the full name).
2016-04-20	Cache path info lookups in SQLite	Eelco Dolstra	1	-2/+0
	This re-implements the binary cache database in C++, allowing it to be used by other Store backends, in particular the S3 backend.
2016-04-19	Move path info caching from BinaryCacheStore to Store	Eelco Dolstra	1	-52/+47
	Caching path info is generally useful. For instance, it speeds up "nix path-info -rS /run/current-system" (i.e. showing the closure sizes of all paths in the closure of the current system) from 5.6s to 0.15s. This also eliminates some APIs like Store::queryDeriver() and Store::queryReferences().
2016-04-14	Fix std::atomic_flag build failure	Eelco Dolstra	1	-0/+1
	http://hydra.nixos.org/build/34453794
2016-04-08	Remove failed build caching	Eelco Dolstra	1	-59/+7
	This feature was implemented for Hydra, but Hydra no longer uses it.
2016-04-08	Make LocalStore thread-safe	Eelco Dolstra	1	-129/+136
	Necessary for multi-threaded commands like "nix verify-paths".
2016-04-07	Sign locally-built paths	Eelco Dolstra	1	-1/+15
	Locally-built paths are now signed automatically using the secret keys specified by the ‘secret-key-files’ option.
2016-04-05	Add "nix copy-sigs" command	Eelco Dolstra	1	-3/+19
	This imports signatures from one store into another. E.g. $ nix copy-sigs -r /run/current-system -s https://cache.nixos.org/ imported 595 signatures
2016-03-30	LocalStore: Keep track of ultimately trusted paths	Eelco Dolstra	1	-8/+28
	These are content-addressed paths or outputs of locally performed builds. They are trusted even if they don't have signatures, so "nix verify-paths" won't complain about them.
2016-03-30	Improve the SQLite wrapper API	Eelco Dolstra	1	-171/+64
	In particular, this eliminates a bunch of boilerplate code.
2016-03-30	Factour out SQLite handling	Eelco Dolstra	1	-171/+0

2016-03-30	Turn retrying SQLite transactions into a higher-order function	Eelco Dolstra	1	-33/+42

2016-03-22	Don't overload dumpPath()	Eelco Dolstra	1	-1/+1

2016-03-21	Add Store::dumpPath() method	Eelco Dolstra	1	-1/+1
	This allows applying nix-store --verify-path to binary cache stores: NIX_REMOTE=https://cache.nixos.org nix-store --verify-path /nix/store/s5c7...
2016-03-21	Drop support for upgrading from Nix <= 0.12	Eelco Dolstra	1	-84/+7

2016-03-04	Eliminate some large string copying	Eelco Dolstra	1	-4/+4

2016-02-26	importPaths(): Optionally add NARs to binary cache accessor	Eelco Dolstra	1	-1/+2
	This enables an optimisation in hydra-queue-runner, preventing a download of a NAR it just uploaded to the cache when reading files like hydra-build-products.
2016-02-24	deletePath(): Succeed if path doesn't exist	Eelco Dolstra	1	-3/+3
	Also makes it robust against concurrent deletions.
2016-02-24	Eliminate reserveSpace flag	Eelco Dolstra	1	-17/+13

2016-02-24	LocalStore::addTextToStore(): Don't read the path we just wrote	Eelco Dolstra	1	-3/+5
	This eliminates some unnecessary (presumably cached) I/O.
2016-02-17	Fix build without sodium	Eelco Dolstra	1	-4/+1
	http://hydra.nixos.org/build/32085949
2016-02-16	Rename ValidPathInfo::hash -> narHash for consistency	Eelco Dolstra	1	-16/+16

2016-02-15	Expose the export magic value and move LocalStore::queryReferences to Store	Eelco Dolstra	1	-14/+3

2016-02-15	Add function to extract hash part of a store path	Eelco Dolstra	1	-1/+1

2016-02-04	Eliminate the "store" global variable	Eelco Dolstra	1	-2/+2
	Also, move a few free-standing functions into StoreAPI and Derivation. Also, introduce a non-nullable smart pointer, ref<T>, which is just a wrapper around std::shared_ptr ensuring that the pointer is never null. (For reference-counted values, this is better than passing a "T&", because the latter doesn't maintain the refcount. Usually, the caller will have a shared_ptr keeping the value alive, but that's not always the case, e.g., when passing a reference to a std::thread via std::bind.)