about summary refs log tree commit diff
path: root/src/libstore
AgeCommit message (Collapse)AuthorFilesLines
2016-10-05 Use std::random_deviceEelco Dolstra1-0/+1
2016-10-04 Fix build with older versions of libcurlEelco Dolstra1-0/+6
2016-09-22 Handle the case where signed-binary-caches consists of whitespaceEelco Dolstra1-1/+1
2016-09-21 Add sandbox-dev-shm-size optionEelco Dolstra1-1/+2
Fixes #1069.
2016-09-21 printMsg(lvlError, ...) -> printError(...) etc.Eelco Dolstra11-80/+80
2016-09-21 Some notational convenience for formatting stringsEelco Dolstra2-4/+4
We can now write throw Error("file '%s' not found", path); instead of throw Error(format("file '%s' not found") % path); and similarly printError("file '%s' not found", path); instead of printMsg(lvlError, format("file '%s' not found") % path);
2016-09-20 TweakEelco Dolstra1-5/+1
2016-09-20 Improve robustnessEelco Dolstra1-2/+6
2016-09-20 Fix "Promise already satisfied" errorEelco Dolstra1-1/+3
2016-09-16 armv5tel can be built by armv6l and armv7l (#1063)Eric Litak1-0/+1
2016-09-16 Make computeFSClosure() single-threaded againEelco Dolstra14-202/+335
The fact that queryPathInfo() is synchronous meant that we needed a thread for every concurrent binary cache lookup, even though they end up being handled by the same download thread. Requiring hundreds of threads is not a good idea. So now there is an asynchronous version of queryPathInfo() that takes a callback function to process the result. Similarly, enqueueDownload() now takes a callback rather than returning a future. Thus, a command like nix path-info --store https://cache.nixos.org/ -r /nix/store/slljrzwmpygy1daay14kjszsr9xix063-nixos-16.09beta231.dccf8c5 that returns 4941 paths now takes 1.87s using only 2 threads (the main thread and the downloader thread). (This is with a prewarmed CloudFront.)
2016-09-14 Revive binary-caches-parallel-connectionsEelco Dolstra1-2/+8
It's a slight misnomer now because it actually limits *all* downloads, not just binary cache lookups. Also add a "enable-http2" option to allow disabling use of HTTP/2 (enabled by default).
2016-09-14 Enable HTTP/2 supportEelco Dolstra4-199/+422
The binary cache store can now use HTTP/2 to do lookups. This is much more efficient than HTTP/1.1 due to multiplexing: we can issue many requests in parallel over a single TCP connection. Thus it's no longer necessary to use a bunch of concurrent TCP connections (25 by default). For example, downloading 802 .narinfo files from https://cache.nixos.org/, using a single TCP connection, takes 11.8s with HTTP/1.1, but only 0.61s with HTTP/2. This did require a fairly substantial rewrite of the Downloader class to use the curl multi interface, because otherwise curl wouldn't be able to do multiplexing for us. As a bonus, we get connection reuse even with HTTP/1.1. All downloads are now handled by a single worker thread. Clients call Downloader::enqueueDownload() to tell the worker thread to start the download, getting a std::future to the result.
2016-09-14 Move some .drv parsing functions out of utilEelco Dolstra1-0/+46
2016-09-08 Don't run builds as root in the user namespaceEelco Dolstra1-16/+31
This largely reverts c68e5913c71badc89ff346d1c6948517ba720c93. Running builds as root breaks "cp -p", since when running as root, "cp -p" assumes that it can succesfully chown() files. But that's not actually the case since the user namespace doesn't provide a complete uid mapping. So it barfs with a fatal error message ("cp: failed to preserve ownership for 'foo': Invalid argument").
2016-09-08 Give a better error message if a path in build-sandbox-paths is invalidEelco Dolstra1-2/+6
2016-09-08 Fix assertion failure after early build setup failureEelco Dolstra1-1/+1
2016-08-31 download.hh: Fix conflicts from nix-channel-c++ mergeShea Levy2-15/+6
2016-08-31 Merge branch 'nix-channel-c++'Shea Levy2-3/+23
2016-08-31 Merge pull request #1030 from pikajude/rsd-ccEelco Dolstra1-1/+1
Implement resolve-system-dependencies in C++
2016-08-30 Increase the sleep time between download retriesEelco Dolstra3-2/+4
2016-08-30 Fix --fallbackEelco Dolstra1-7/+10
This fixes an assertion failure in "assert(goal);" in Worker::waitForInput() after a substitution goal is cancelled by the termination of another goal. The problem was the line //worker.childTerminated(shared_from_this()); // FIXME in the SubstitutionGoal destructor. This was disabled because shared_from_this() obviously doesn't work from a destructor. So we now use a real pointer for object identity.
2016-08-24 Fix queryPathFromHashPart()Eelco Dolstra1-1/+1
The inner lambda was returning a SQLite-internal char * rather than a std::string, leading to Hydra errors liks Caught exception in Hydra::Controller::Root->narinfo "path ‘ø˜£â€™ is not in the Nix store at /nix/store/6mvvyb8fgwj23miyal5mdr8ik4ixk15w-hydra-0.1.1234.abcdef/libexec/hydra/lib/Hydra/Controller/Root.pm line 352."
2016-08-17 Add a mechanism for derivation attributes to reference the derivation's outputsEelco Dolstra3-14/+24
For example, you can now say: configureFlags = "--prefix=${placeholder "out"} --includedir=${placeholder "dev"}"; The strings returned by the ‘placeholder’ builtin are replaced at build time by the actual store paths corresponding to the specified outputs. Previously, you had to work around the inability to self-reference by doing stuff like: preConfigure = '' configureFlags+=" --prefix $out --includedir=$dev" ''; or rely on ad-hoc variable interpolation semantics in Autoconf or Make (e.g. --prefix=\$(out)), which doesn't always work.
2016-08-15 builtins.fetch{url,tarball}: Allow name attributeShea Levy2-5/+6
2016-08-13 remove old traces of resolve-system-dependenciesJude Taylor1-1/+1
2016-08-11 nix-channel: implement in c++Shea Levy2-3/+23
2016-08-10 HttpBinaryCacheStore: Retry on transient HTTP errorsEelco Dolstra3-14/+34
This makes us more robust against 500 errors from CloudFront or S3 (assuming the 500 error isn't cached by CloudFront...).
2016-08-10 Show HTTP error codesEelco Dolstra1-2/+6
2016-08-10 SQLite:: Add some convenienceEelco Dolstra4-28/+33
2016-08-10 FSAccessor: Throw InvalidPathEelco Dolstra2-2/+2
2016-08-10 NarAccessor: Fix handling of non-executable filesEelco Dolstra1-1/+1
2016-08-10 Mark content-addressed paths in the Nix database and in .narinfoEelco Dolstra8-41/+138
This allows such paths to be imported without signatures.
2016-08-10 LocalFSStore::narFromPath(): Handle the diverted store caseEelco Dolstra1-1/+1
2016-08-10 Add a "root" parameter to local storesEelco Dolstra4-3/+7
This makes it easier to create a diverted store, i.e. NIX_REMOTE="local?root=/tmp/root" instead of NIX_REMOTE="local?real=/tmp/root/nix/store&state=/tmp/root/nix/var/nix" NIX_LOG_DIR=/tmp/root/nix/var/log
2016-08-10 Remove $NIX_DB_DIREelco Dolstra3-5/+1
This variable has no reason to exist, given $NIX_STATE_DIR.
2016-08-10 RemoteStore / nix-daemon: Drop support for Nix < 1.0Eelco Dolstra1-19/+15
2016-07-26 makeFixedOutputPath(): Drop superfluous HashType argumentEelco Dolstra8-22/+21
2016-07-26 builtins.{fetchurl,fetchTarball}: Support a sha256 attributeEelco Dolstra2-9/+30
Also, allow builtins.{fetchurl,fetchTarball} in restricted mode if a hash is specified.
2016-07-24 Merge branch 'find-runtime-roots-c++'Shea Levy1-8/+108
2016-07-23 Respect --keep-going when a substituter fails.Shea Levy1-2/+4
Fixes #977
2016-07-21 Fix random failures caused by CurlDownloader modifying a shared stringEelco Dolstra1-1/+1
2016-07-21 NarInfo::NarInfo(): Ensure that we get a NAR size/hashEelco Dolstra1-2/+2
2016-07-21 Store::queryMissing(): Use a thread poolEelco Dolstra1-85/+104
For one particular NixOS configuration, this cut the runtime of "nix-store -r --dry-run" from 6m51s to 3.4s. It also fixes a bug in the size calculation that was causing certain paths to be counted twice, e.g. before: these paths will be fetched (1249.98 MiB download, 2995.74 MiB unpacked): and after: these paths will be fetched (1219.56 MiB download, 2862.17 MiB unpacked):
2016-07-21 Store::computeFSClosure(): Use thread poolEelco Dolstra1-27/+48
This speeds up queries against the binary cache.
2016-07-21 find-runtime-roots: fold into gc.ccShea Levy1-8/+108
2016-07-11 Modernize AutoCloseFDShea Levy5-84/+82
2016-06-20 Re-implement negative binary cache lookup cachingEelco Dolstra2-15/+22
2016-06-09 Run builds in a user namespaceEelco Dolstra1-14/+65
This way, all builds appear to have a uid/gid of 0 inside the chroot. In the future, this may allow using programs like systemd-nspawn inside builds, but that will require assigning a larger UID/GID map to the build. Issue #625.
2016-06-09 Use O_CLOEXEC in most placesEelco Dolstra5-11/+11