about summary refs log tree commit diff
path: root/src/libutil
AgeCommit message (Collapse)AuthorFilesLines
2016-09-16 Make computeFSClosure() single-threaded againEelco Dolstra2-0/+50
The fact that queryPathInfo() is synchronous meant that we needed a thread for every concurrent binary cache lookup, even though they end up being handled by the same download thread. Requiring hundreds of threads is not a good idea. So now there is an asynchronous version of queryPathInfo() that takes a callback function to process the result. Similarly, enqueueDownload() now takes a callback rather than returning a future. Thus, a command like nix path-info --store https://cache.nixos.org/ -r /nix/store/slljrzwmpygy1daay14kjszsr9xix063-nixos-16.09beta231.dccf8c5 that returns 4941 paths now takes 1.87s using only 2 threads (the main thread and the downloader thread). (This is with a prewarmed CloudFront.)
2016-09-16 printMsg(): Don't check for interruptsEelco Dolstra3-17/+15
Having the logger function potentially throw exceptions is Heisenbuggy.
2016-09-14 Add a toLower utility functionEelco Dolstra2-0/+14
2016-09-14 Move some .drv parsing functions out of utilEelco Dolstra2-55/+0
2016-08-30 Fix 32-bit buildEelco Dolstra2-0/+6
2016-08-29 nix path-info: Add --json flagEelco Dolstra2-0/+354
Also, factor out JSON generation from value-to-json.{cc,hh}, and support producing indented JSON.
2016-07-21 Fix assertion failureEelco Dolstra1-1/+1
2016-07-21 NarInfo::NarInfo(): Ensure that we get a NAR size/hashEelco Dolstra1-0/+1
2016-07-21 Fix assertion failure in ThreadPool::enqueue()Eelco Dolstra2-2/+6
2016-07-13 FdSink: Restore move constructor/assignmentShea Levy1-0/+2
2016-07-13 Make Buffered{Source,Sink} move-safeShea Levy2-27/+12
2016-07-11 Modernize AutoCloseFDShea Levy4-54/+42
2016-06-09 Use O_CLOEXEC in most placesEelco Dolstra3-7/+11
2016-06-02 createTempDir(): Don't do a chown on LinuxEelco Dolstra1-0/+2
It's not needed and can cause problems in a user namespace.
2016-06-02 Allow setting the state directory as a store parameterEelco Dolstra1-3/+14
E.g. "local?store=/tmp/store&state=/tmp/var".
2016-05-31 use $(LIBLZMA_LIBS) instead of -llzmaDmitry Kalinkin1-1/+1
This is needed in case of non-standard lzma installation path that will be specified in pkgconfig manifest as extra -L option for LDFLAGS.
2016-05-04 Cleanup: Remove singleton()Eelco Dolstra1-9/+0
2016-05-04 Do compression in a sinkEelco Dolstra3-92/+186
2016-05-04 Add a Store::addToStore() variant that accepts a NAREelco Dolstra2-10/+10
As a side effect, this ensures that signatures are propagated when copying paths between stores. Also refactored import/export to make use of this.
2016-05-04 Fix segfault in xz compression/decompressionEelco Dolstra1-2/+2
2016-05-04 Add a copyStorePath() utility functionEelco Dolstra1-0/+7
2016-04-29 Support Git repos in the Nix pathEelco Dolstra2-0/+10
E.g. $ nix-build -I nixpkgs=git://github.com/NixOS/nixpkgs '<nixpkgs>' -A hello This is not extremely useful yet because you can't specify a branch/revision.
2016-04-29 BinaryCacheStore: Support bzip2 compressionEelco Dolstra2-35/+117
2016-04-29 BinaryCacheStore: Support "none" compression methodEelco Dolstra2-6/+29
2016-04-29 Eliminate the substituter mechanismEelco Dolstra1-0/+12
Substitution is now simply a Store -> Store copy operation, most typically from BinaryCacheStore to LocalStore.
2016-04-25 Improved logging abstractionEelco Dolstra5-168/+165
This also gets rid of --log-type, since the nested log type isn't useful in a multi-threaded situation, and nobody cares about the "pretty" log type.
2016-04-22 Factor out parallel processing of work items that have dependenciesEelco Dolstra1-0/+60
2016-04-22 ThreadPool: Start doing work as soon as work items are enqueuedEelco Dolstra2-59/+84
2016-04-22 Make compression interruptibleEelco Dolstra1-1/+3
2016-04-21 nix --help: Show short flagsEelco Dolstra2-4/+6
2016-04-20 Cache path info lookups in SQLiteEelco Dolstra4-7/+52
This re-implements the binary cache database in C++, allowing it to be used by other Store backends, in particular the S3 backend.
2016-04-19 Move path info caching from BinaryCacheStore to StoreEelco Dolstra2-0/+12
Caching path info is generally useful. For instance, it speeds up "nix path-info -rS /run/current-system" (i.e. showing the closure sizes of all paths in the closure of the current system) from 5.6s to 0.15s. This also eliminates some APIs like Store::queryDeriver() and Store::queryReferences().
2016-04-15 BinaryCacheStore::readFile(): Return a shared_ptr to a stringEelco Dolstra2-4/+6
This allows readFile() to indicate that a file doesn't exist, and might eliminate some large string copying.
2016-04-08 Make LocalStore thread-safeEelco Dolstra1-3/+3
Necessary for multi-threaded commands like "nix verify-paths".
2016-03-31 Add missing -pthreadEelco Dolstra1-1/+1
https://hydra.nixos.org/build/33908385
2016-03-29 Improve SIGINT handling in multi-threaded programsEelco Dolstra3-5/+10
The flag remembering whether an Interrupted exception was thrown is now thread-local. Thus, all threads will (eventually) throw Interrupted. Previously, one thread would throw Interrupted, and then the other threads wouldn't see that they were supposed to quit.
2016-03-29 Add "nix verify-paths" commandEelco Dolstra4-2/+135
Unlike "nix-store --verify-path", this command verifies signatures in addition to store path contents, is multi-threaded (especially useful when verifying binary caches), and has a progress indicator. Example use: $ nix verify-paths --store https://cache.nixos.org -r $(type -p thunderbird) ... [17/132 checked] checking ‘/nix/store/rawakphadqrqxr6zri2rmnxh03gqkrl3-autogen-5.18.6’
2016-03-15 Fix Darwin buildEelco Dolstra1-0/+1
http://hydra.nixos.org/build/33279996
2016-03-04 Eliminate some large string copyingEelco Dolstra2-3/+5
2016-03-02 Fix reading symlinksEelco Dolstra1-1/+1
The st_size field of a symlink doesn't have to be correct, e.g. for /dev/fd symlinks.
2016-03-02 FdSource: track number of bytes readEelco Dolstra2-5/+8
2016-02-25 Merge branch 'master' into new-cliEelco Dolstra1-0/+1
2016-02-25 Add NAR / Store accessor abstractionEelco Dolstra1-0/+1
This is primary to allow hydra-queue-runner to extract files like "nix-support/hydra-build-products" from NARs in binary caches.
2016-02-25 Fix short boolean flagsEelco Dolstra1-1/+1
2016-02-25 Merge branch 'master' into new-cliEelco Dolstra13-97/+515
2016-02-24 deletePath(): Succeed if path doesn't existEelco Dolstra2-6/+10
Also makes it robust against concurrent deletions.
2016-02-24 BinaryCacheStore: Implement addToStore()Eelco Dolstra2-4/+16
So now you can do $ NIX_REMOTE=file:///tmp/binary-cache nix-instantiate '<nixpkgs>' -A hello and lots of other operations.
2016-02-24 TypoEelco Dolstra1-2/+2
2016-02-24 Move BinaryCacheStore / LocalBinaryCacheStore from HydraEelco Dolstra1-0/+84
So you can now do: $ NIX_REMOTE=file:///tmp/binary-cache nix-store -qR /nix/store/...
2016-02-24 std::condition_variable_any -> std::condition_variableEelco Dolstra2-11/+11
The latter is supposed to be more efficient.