depot - monorepo for the virus lounge

Age	Commit message (Collapse)	Author	Files	Lines
2016-09-16	Make computeFSClosure() single-threaded again	Eelco Dolstra	14	-202/+335
	The fact that queryPathInfo() is synchronous meant that we needed a thread for every concurrent binary cache lookup, even though they end up being handled by the same download thread. Requiring hundreds of threads is not a good idea. So now there is an asynchronous version of queryPathInfo() that takes a callback function to process the result. Similarly, enqueueDownload() now takes a callback rather than returning a future. Thus, a command like nix path-info --store https://cache.nixos.org/ -r /nix/store/slljrzwmpygy1daay14kjszsr9xix063-nixos-16.09beta231.dccf8c5 that returns 4941 paths now takes 1.87s using only 2 threads (the main thread and the downloader thread). (This is with a prewarmed CloudFront.)
2016-09-14	Revive binary-caches-parallel-connections	Eelco Dolstra	1	-2/+8
	It's a slight misnomer now because it actually limits all downloads, not just binary cache lookups. Also add a "enable-http2" option to allow disabling use of HTTP/2 (enabled by default).
2016-09-14	Enable HTTP/2 support	Eelco Dolstra	4	-199/+422
	The binary cache store can now use HTTP/2 to do lookups. This is much more efficient than HTTP/1.1 due to multiplexing: we can issue many requests in parallel over a single TCP connection. Thus it's no longer necessary to use a bunch of concurrent TCP connections (25 by default). For example, downloading 802 .narinfo files from https://cache.nixos.org/, using a single TCP connection, takes 11.8s with HTTP/1.1, but only 0.61s with HTTP/2. This did require a fairly substantial rewrite of the Downloader class to use the curl multi interface, because otherwise curl wouldn't be able to do multiplexing for us. As a bonus, we get connection reuse even with HTTP/1.1. All downloads are now handled by a single worker thread. Clients call Downloader::enqueueDownload() to tell the worker thread to start the download, getting a std::future to the result.
2016-09-14	Move some .drv parsing functions out of util	Eelco Dolstra	1	-0/+46

2016-09-08	Don't run builds as root in the user namespace	Eelco Dolstra	1	-16/+31
	This largely reverts c68e5913c71badc89ff346d1c6948517ba720c93. Running builds as root breaks "cp -p", since when running as root, "cp -p" assumes that it can succesfully chown() files. But that's not actually the case since the user namespace doesn't provide a complete uid mapping. So it barfs with a fatal error message ("cp: failed to preserve ownership for 'foo': Invalid argument").
2016-09-08	Give a better error message if a path in build-sandbox-paths is invalid	Eelco Dolstra	1	-2/+6

2016-09-08	Fix assertion failure after early build setup failure	Eelco Dolstra	1	-1/+1

2016-08-31	download.hh: Fix conflicts from nix-channel-c++ merge	Shea Levy	2	-15/+6

2016-08-31	Merge branch 'nix-channel-c++'	Shea Levy	2	-3/+23

2016-08-31	Merge pull request #1030 from pikajude/rsd-cc	Eelco Dolstra	1	-1/+1
	Implement resolve-system-dependencies in C++
2016-08-30	Increase the sleep time between download retries	Eelco Dolstra	3	-2/+4

2016-08-30	Fix --fallback	Eelco Dolstra	1	-7/+10
	This fixes an assertion failure in "assert(goal);" in Worker::waitForInput() after a substitution goal is cancelled by the termination of another goal. The problem was the line //worker.childTerminated(shared_from_this()); // FIXME in the SubstitutionGoal destructor. This was disabled because shared_from_this() obviously doesn't work from a destructor. So we now use a real pointer for object identity.
2016-08-24	Fix queryPathFromHashPart()	Eelco Dolstra	1	-1/+1
	The inner lambda was returning a SQLite-internal char * rather than a std::string, leading to Hydra errors liks Caught exception in Hydra::Controller::Root->narinfo "path âø£â is not in the Nix store at /nix/store/6mvvyb8fgwj23miyal5mdr8ik4ixk15w-hydra-0.1.1234.abcdef/libexec/hydra/lib/Hydra/Controller/Root.pm line 352."
2016-08-17	Add a mechanism for derivation attributes to reference the derivation's outputs	Eelco Dolstra	3	-14/+24
	For example, you can now say: configureFlags = "--prefix=${placeholder "out"} --includedir=${placeholder "dev"}"; The strings returned by the ‘placeholder’ builtin are replaced at build time by the actual store paths corresponding to the specified outputs. Previously, you had to work around the inability to self-reference by doing stuff like: preConfigure = '' configureFlags+=" --prefix $out --includedir=$dev" ''; or rely on ad-hoc variable interpolation semantics in Autoconf or Make (e.g. --prefix=\$(out)), which doesn't always work.
2016-08-15	builtins.fetch{url,tarball}: Allow name attribute	Shea Levy	2	-5/+6

2016-08-13	remove old traces of resolve-system-dependencies	Jude Taylor	1	-1/+1

2016-08-11	nix-channel: implement in c++	Shea Levy	2	-3/+23

2016-08-10	HttpBinaryCacheStore: Retry on transient HTTP errors	Eelco Dolstra	3	-14/+34
	This makes us more robust against 500 errors from CloudFront or S3 (assuming the 500 error isn't cached by CloudFront...).
2016-08-10	Show HTTP error codes	Eelco Dolstra	1	-2/+6

2016-08-10	SQLite:: Add some convenience	Eelco Dolstra	4	-28/+33

2016-08-10	FSAccessor: Throw InvalidPath	Eelco Dolstra	2	-2/+2

2016-08-10	NarAccessor: Fix handling of non-executable files	Eelco Dolstra	1	-1/+1

2016-08-10	Mark content-addressed paths in the Nix database and in .narinfo	Eelco Dolstra	8	-41/+138
	This allows such paths to be imported without signatures.
2016-08-10	LocalFSStore::narFromPath(): Handle the diverted store case	Eelco Dolstra	1	-1/+1

2016-08-10	Add a "root" parameter to local stores	Eelco Dolstra	4	-3/+7
	This makes it easier to create a diverted store, i.e. NIX_REMOTE="local?root=/tmp/root" instead of NIX_REMOTE="local?real=/tmp/root/nix/store&state=/tmp/root/nix/var/nix" NIX_LOG_DIR=/tmp/root/nix/var/log
2016-08-10	Remove $NIX_DB_DIR	Eelco Dolstra	3	-5/+1
	This variable has no reason to exist, given $NIX_STATE_DIR.
2016-08-10	RemoteStore / nix-daemon: Drop support for Nix < 1.0	Eelco Dolstra	1	-19/+15

2016-07-26	makeFixedOutputPath(): Drop superfluous HashType argument	Eelco Dolstra	8	-22/+21

2016-07-26	builtins.{fetchurl,fetchTarball}: Support a sha256 attribute	Eelco Dolstra	2	-9/+30
	Also, allow builtins.{fetchurl,fetchTarball} in restricted mode if a hash is specified.
2016-07-24	Merge branch 'find-runtime-roots-c++'	Shea Levy	1	-8/+108

2016-07-23	Respect --keep-going when a substituter fails.	Shea Levy	1	-2/+4
	Fixes #977
2016-07-21	Fix random failures caused by CurlDownloader modifying a shared string	Eelco Dolstra	1	-1/+1

2016-07-21	NarInfo::NarInfo(): Ensure that we get a NAR size/hash	Eelco Dolstra	1	-2/+2

2016-07-21	Store::queryMissing(): Use a thread pool	Eelco Dolstra	1	-85/+104
	For one particular NixOS configuration, this cut the runtime of "nix-store -r --dry-run" from 6m51s to 3.4s. It also fixes a bug in the size calculation that was causing certain paths to be counted twice, e.g. before: these paths will be fetched (1249.98 MiB download, 2995.74 MiB unpacked): and after: these paths will be fetched (1219.56 MiB download, 2862.17 MiB unpacked):
2016-07-21	Store::computeFSClosure(): Use thread pool	Eelco Dolstra	1	-27/+48
	This speeds up queries against the binary cache.
2016-07-21	find-runtime-roots: fold into gc.cc	Shea Levy	1	-8/+108

2016-07-11	Modernize AutoCloseFD	Shea Levy	5	-84/+82

2016-06-20	Re-implement negative binary cache lookup caching	Eelco Dolstra	2	-15/+22

2016-06-09	Run builds in a user namespace	Eelco Dolstra	1	-14/+65
	This way, all builds appear to have a uid/gid of 0 inside the chroot. In the future, this may allow using programs like systemd-nspawn inside builds, but that will require assigning a larger UID/GID map to the build. Issue #625.
2016-06-09	Use O_CLOEXEC in most places	Eelco Dolstra	5	-11/+11

2016-06-03	Fix "creating statement: table ValidPaths has no column named ultimate"	Eelco Dolstra	1	-41/+29

2016-06-03	Support sandbox builds by non-root users	Eelco Dolstra	3	-53/+69
	This allows an unprivileged user to perform builds on a diverted store (i.e. where the physical store location differs from the logical location). Example: $ NIX_LOG_DIR=/tmp/log NIX_REMOTE="local?real=/tmp/store&state=/tmp/var" nix-build -E \ 'with import <nixpkgs> {}; runCommand "foo" { buildInputs = [procps nettools]; } "id; ps; ifconfig; echo $out > $out"' will do a build in the Nix store physically in /tmp/store but logically in /nix/store (and thus using substituters for the latter).
2016-06-03	Consistent quotes	Eelco Dolstra	1	-1/+1

2016-06-03	Don't call shared_from_this() from a destructor	Eelco Dolstra	1	-1/+1

2016-06-03	Fix some more references to storeDir	Eelco Dolstra	1	-2/+2

2016-06-03	Merge pull request #925 from abbradar/master	Eelco Dolstra	1	-5/+5
	Show both cycle ends
2016-06-03	Show both cycle ends	Nikolay Amiantov	1	-5/+5

2016-06-03	Fix narrowing conversion from int64_t to bool	Eelco Dolstra	1	-1/+1
	http://hydra.nixos.org/build/36613774
2016-06-02	Make derivationFromPath work on diverted stores	Eelco Dolstra	5	-18/+27

2016-06-02	nix run: Mount the Nix store in a private namespace	Eelco Dolstra	1	-0/+4
	This is a convenience command to allow users who are not privileged to create /nix/store to use Nix with regular binary caches. For example, $ NIX_REMOTE="local?state=$HOME/nix/var&real=/$HOME/nix/store" nix run firefox bashInteractive will download Firefox and bash from cache.nixos.org, then start a shell in which $HOME/nix/store is mounted on /nix/store.