depot - monorepo for the virus lounge

Age	Commit message (Collapse)	Author	Files	Lines
2017-05-16	Improve progress indicator	Eelco Dolstra	1	-32/+6

2017-05-11	Add an option for extending the user agent header	Eelco Dolstra	1	-1/+3
	This is useful e.g. for distinguishing traffic to a binary cache (e.g. certain machines can use a different tag in the user agent).
2017-05-11	Tweak error message	Eelco Dolstra	1	-1/+1

2017-05-11	LocalStore::addToStore(): Check info.narSize	Eelco Dolstra	1	-0/+1
	It allowed the client to specify bogus narSize values. In particular, Downloader::downloadCached wasn't setting narSize at all.
2017-04-19	Reimplement connect-timeout	Eelco Dolstra	1	-0/+2
	Fixes #1339.
2017-04-14	Shut up some warnings	Eelco Dolstra	1	-1/+1

2017-04-13	Convert Settings to the new config system	Eelco Dolstra	1	-1/+1
	This makes all config options self-documenting. Unknown or unparseable config settings and --option flags now cause a warning.
2017-04-13	Merge branch 'rework-options' of https://github.com/copumpkin/nix	Eelco Dolstra	1	-3/+3

2017-04-10	Add CURLE_WRITE_ERROR as a transient error condition	Dan Peebles	1	-0/+5
	We've observed it failing downloads in the wild and retrying the same URL a few moments later seemed to fix it.
2017-04-06	Retry downloads on transient SSL errors too	Dan Peebles	1	-0/+2

2017-03-31	Retry curl error 16	Eelco Dolstra	1	-0/+1

2017-03-15	runProgram(): Distinguish between empty input and no input	Eelco Dolstra	1	-1/+1
	For example, if we call brotli with an empty input, it shouldn't read from the caller's stdin.
2017-03-15	S3BinaryCacheStore: Support compression of narinfo and log files	Eelco Dolstra	1	-5/+11
	You can now set the store parameter "text-compression=br" to compress textual files in the binary cache (i.e. narinfo and logs) using Brotli. This sets the Content-Encoding header; the extension of compressed files is unchanged. You can separately specify the compression of log files using "log-compression=br". This is useful when you don't want to compress narinfo files for backward compatibility.
2017-03-15	Add support for brotli compression	Eelco Dolstra	1	-2/+19
	Build logs on cache.nixos.org are compressed using Brotli (since this allows them to be decompressed automatically by Chrome and Firefox), so it's handy if "nix log" can decompress them.
2017-03-12	Merge branch 'configurable-aws-region'	Shea Levy	1	-1/+4

2017-03-06	Properly set the caFile for aws-sdk-cpp s3	Shea Levy	1	-2/+1

2017-03-03	Add aws-region param to S3 store URLs	Shea Levy	1	-1/+4

2017-02-28	Fix building against older curl versions	Eelco Dolstra	1	-1/+4
	http://hydra.nixos.org/build/49490928
2017-02-27	Retry downloads on HTTP/2 stream errors	Eelco Dolstra	1	-1/+3
	Issue #1254.
2017-02-22	Explicitly model all settings and fail on unrecognized ones	Dan Peebles	1	-3/+3
	Previously, the Settings class allowed other code to query for string properties, which led to a proliferation of code all over the place making up new options without any sort of central registry of valid options. This commit pulls all those options back into the central Settings class and removes the public get() methods, to discourage future abuses like that. Furthermore, because we know the full set of options ahead of time, we now fail loudly if someone enters an unrecognized option, thus preventing subtle typos. With some template fun, we could probably also dump the full set of options (with documentation, defaults, etc.) to the command line, but I'm not doing that yet here.
2017-02-22	RemoteStore::addToStore(): Pass content-addressability assertion	Eelco Dolstra	1	-0/+1
	... and use this in Downloader::downloadCached(). This fixes $ nix-build https://nixos.org/channels/nixos-16.09-small/nixexprs.tar.xz -A hello error: cannot import path ‘/nix/store/csfbp1s60dkgmk9f8g0zk0mwb7hzgabd-nixexprs.tar.xz’ because it lacks a valid signature
2017-02-21	Handle CURLE_RECV_ERROR as a transient error	Eelco Dolstra	1	-1/+1
	This fixes unable to download ‘https://cache.nixos.org/nar/077h8ji74y9b0qx7rjk71xd80vjqp6q5gy137r553jlvdlxdcdlk.nar.xz’: HTTP error 200 (curl error: Failure when receiving data from the peer)
2017-02-21	Fix building without S3 support	Eelco Dolstra	1	-1/+1
	http://hydra.nixos.org/build/49031196/nixlog/2/raw
2017-02-16	Move netrcFile to Settings	Eelco Dolstra	1	-5/+2
	Also get rid of Settings::processEnvironment(), it appears to be useless.
2017-02-16	Include "curl" in the User-Agent header	Eelco Dolstra	1	-1/+1
	Some sites (e.g. BitBucket) give a helpful 401 error when trying to download a private archive if the User-Agent contains "curl", but give a redirect to a login page otherwise (so for instance "nix-prefetch-url" will succeed but produce useless output).
2017-02-16	Merge pull request #1215 from k0001/netrc-1	Eelco Dolstra	1	-0/+8
	Add netrc-file support
2017-02-14	Add support for s3:// URIs	Eelco Dolstra	1	-1/+27
	This adds support for s3:// URIs in all places where Nix allows URIs, e.g. in builtins.fetchurl, builtins.fetchTarball, <nix/fetchurl.nix> and NIX_PATH. It allows fetching resources from private S3 buckets, using credentials obtained from the standard places (i.e. AWS_* environment variables, ~/.aws/credentials and the EC2 metadata server). This may not be super-useful in general, but since we already depend on aws-sdk-cpp, it's a cheap feature to add.
2017-02-09	Documentation.	Renzo Carbonara	1	-0/+3

2017-02-02	Add netrc-file support	Renzo Carbonara	1	-0/+5

2017-01-26	On HTTP errors, also show the curl error	Eelco Dolstra	1	-1/+1
	This is a hopefully temporary measure to diagnose the intermittent "HTTP error 200" failures.
2017-01-24	Enable verbose curl output	Eelco Dolstra	1	-1/+14
	Closes #1182.
2017-01-17	Handle SIGINT etc. via a sigwait() signal handler thread	Eelco Dolstra	1	-5/+15
	This allows other threads to install callbacks that run in a regular, non-signal context. In particular, we can use this to signal the downloader thread to quit. Closes #1183.
2016-11-26	Revert "Get rid of unicode quotes (#1140)"	Eelco Dolstra	1	-10/+10
	This reverts commit f78126bfd6b6c8477fcdbc09b2f98772dbe9a1e7. There really is no need for such a massive change...
2016-11-25	Get rid of unicode quotes (#1140)	Guillaume Maudoux	1	-10/+10

2016-10-21	BinaryCacheStore: Optionally write a NAR listing	Eelco Dolstra	1	-1/+1
	The store parameter "write-nar-listing=1" will cause BinaryCacheStore to write a file ‘<store-hash>.ls.xz’ for each ‘<store-hash>.narinfo’ added to the binary cache. This file contains an XZ-compressed JSON file describing the contents of the NAR, excluding the contents of regular files. E.g. { "version": 1, "root": { "type": "directory", "entries": { "lib": { "type": "directory", "entries": { "Mcrt1.o": { "type": "regular", "size": 1288 }, "Scrt1.o": { "type": "regular", "size": 3920 }, } } } ... } } (The actual file has no indentation.) This is intended to speed up the NixOS channels programs index generator [1], since fetching gazillions of large NARs from cache.nixos.org is currently a bottleneck for updating the regular (non-small) channel. [1] https://github.com/NixOS/nixos-channel-scripts/blob/master/generate-programs-index.cc
2016-10-19	Merge branch 'priorityqueue' of https://github.com/groxxda/nix	Eelco Dolstra	1	-12/+17

2016-10-19	Fix uninitialised variable	Eelco Dolstra	1	-1/+1

2016-10-18	downloader: use priority_queue	Alexander Ried	1	-12/+17

2016-10-14	CURL_AT_LEAST_VERSION -> LIBCURL_VERSION_NUM	Eelco Dolstra	1	-4/+3
	http://hydra.nixos.org/build/42025230
2016-10-13	Fix HTTP/2 support	Eelco Dolstra	1	-3/+3
	Commit 86e8c67efc33cf756500a1dec7fd6313658f2664 broke it, because CURL_* are not actually #defines.
2016-10-13	SSL_CERT_FILE -> NIX_SSL_CERT_FILE	Eelco Dolstra	1	-1/+2
	This prevents collisions with the "native" OpenSSL, in particular on OS X. Fixes #921.
2016-10-05	Use std::random_device	Eelco Dolstra	1	-0/+1

2016-10-04	Fix build with older versions of libcurl	Eelco Dolstra	1	-0/+6

2016-09-21	printMsg(lvlError, ...) -> printError(...) etc.	Eelco Dolstra	1	-5/+5

2016-09-20	Tweak	Eelco Dolstra	1	-5/+1

2016-09-20	Improve robustness	Eelco Dolstra	1	-2/+6

2016-09-16	Make computeFSClosure() single-threaded again	Eelco Dolstra	1	-11/+27
	The fact that queryPathInfo() is synchronous meant that we needed a thread for every concurrent binary cache lookup, even though they end up being handled by the same download thread. Requiring hundreds of threads is not a good idea. So now there is an asynchronous version of queryPathInfo() that takes a callback function to process the result. Similarly, enqueueDownload() now takes a callback rather than returning a future. Thus, a command like nix path-info --store https://cache.nixos.org/ -r /nix/store/slljrzwmpygy1daay14kjszsr9xix063-nixos-16.09beta231.dccf8c5 that returns 4941 paths now takes 1.87s using only 2 threads (the main thread and the downloader thread). (This is with a prewarmed CloudFront.)
2016-09-14	Revive binary-caches-parallel-connections	Eelco Dolstra	1	-2/+8
	It's a slight misnomer now because it actually limits all downloads, not just binary cache lookups. Also add a "enable-http2" option to allow disabling use of HTTP/2 (enabled by default).
2016-09-14	Enable HTTP/2 support	Eelco Dolstra	1	-171/+379
	The binary cache store can now use HTTP/2 to do lookups. This is much more efficient than HTTP/1.1 due to multiplexing: we can issue many requests in parallel over a single TCP connection. Thus it's no longer necessary to use a bunch of concurrent TCP connections (25 by default). For example, downloading 802 .narinfo files from https://cache.nixos.org/, using a single TCP connection, takes 11.8s with HTTP/1.1, but only 0.61s with HTTP/2. This did require a fairly substantial rewrite of the Downloader class to use the curl multi interface, because otherwise curl wouldn't be able to do multiplexing for us. As a bonus, we get connection reuse even with HTTP/1.1. All downloads are now handled by a single worker thread. Clients call Downloader::enqueueDownload() to tell the worker thread to start the download, getting a std::future to the result.
2016-08-31	download.hh: Fix conflicts from nix-channel-c++ merge	Shea Levy	1	-9/+5