about summary refs log tree commit diff
path: root/src/libstore/local-store.hh
AgeCommit message (Collapse)AuthorFilesLines
2018-05-30 Simplify the callback mechanismEelco Dolstra1-2/+1
2018-04-13 Fix #1921Eelco Dolstra1-1/+3
2018-03-16 Reduce substitution memory consumptionEelco Dolstra1-1/+1
copyStorePath() now pipes the output of srcStore->narFromPath() directly into dstStore->addToStore(). The sink used by the former is converted into a source usable by the latter using boost::coroutine2. This is based on [1]. This reduces the maximum resident size of $ nix build --store ~/my-nix/ /nix/store/b0zlxla7dmy1iwc3g459rjznx59797xy-binutils-2.28.1 --substituters file:///tmp/binary-cache-xz/ --no-require-sigs from 418592 KiB to 53416 KiB. (The previous commit also reduced the runtime from ~4.2s to ~3.4s, not sure why.) A further improvement will be to download files into a Sink. [1] https://github.com/NixOS/nix/compare/master...Mathnerd314:dump-fix-coroutine#diff-dcbcac55a634031f9cc73707da6e4b18 Issue #1969.
2018-02-12 Fix hang in build-remoteEelco Dolstra1-0/+3
2018-01-31 Rename 1.12 -> 2.0Eelco Dolstra1-1/+1
Following discussion with Shea and Graham. It's a big enough change from the last release. Also, from a semver perspective, 2.0 makes more sense because we did remove some interfaces (like nix-pull/nix-push).
2017-11-20 signed-binary-caches -> require-sigsEelco Dolstra1-1/+1
Unlike signed-binary-caches (which could only be '*' or ''), require-sigs is a proper Boolean option. The default is true.
2017-09-14 Show the pid of temporary rootsEelco Dolstra1-1/+1
2017-09-14 nix-store -q --roots / --gc --print-roots: Print temporary / in-memory rootsEelco Dolstra1-2/+4
For example, $ nix-store -q --roots /nix/store/7phd2sav7068nivgvmj2vpm3v47fd27l-patchelf-0.8pre845_0315148 {temp:1} denotes that the path is only being kept alive by a temporary root (i.e. /nix/var/nix/temproots/). Similarly, $ nix-store --gc --print-roots ... {memory:9} -> /nix/store/094gpjn9f15ip17wzxhma4r51nvsj17p-curl-7.53.1 shows that curl is being used by some process.
2017-09-05 Add automatic garbage collectionEelco Dolstra1-0/+21
Nix can now automatically run the garbage collector during builds or while adding paths to the store. The option "min-free = <bytes>" specifies that Nix should run the garbage collector whenever free space in the Nix store drops below <bytes>. It will then delete garbage until "max-free" bytes are available. Garbage collection during builds is asynchronous; running builds are not paused and new builds are not blocked. However, there also is a synchronous GC run prior to the first build/substitution. Currently, no old GC roots are deleted (as in "nix-collect-garbage -d").
2017-09-05 GC: Don't delete own temproots fileEelco Dolstra1-1/+2
Since file locks are per-process rather than per-file-descriptor, the garbage collector would always acquire a lock on its own temproots file and conclude that it's stale.
2017-08-16 nix optimise-store: Show how much space has been freedEelco Dolstra1-1/+1
2017-07-03 Replace a few bool flags with enumsEelco Dolstra1-7/+8
Functions like copyClosure() had 3 bool arguments, which creates a severe risk of mixing up arguments. Also, implement copyClosure() using copyPaths().
2017-04-13 Merge branch 'rework-options' of https://github.com/copumpkin/nixEelco Dolstra1-1/+1
2017-04-13 Add a Config class to simplify adding configuration settingsEelco Dolstra1-1/+5
The typical use is to inherit Config and add Setting<T> members: class MyClass : private Config { Setting<int> foo{this, 123, "foo", "the number of foos to use"}; Setting<std::string> bar{this, "blabla", "bar", "the name of the bar"}; MyClass() : Config(readConfigFile("/etc/my-app.conf")) { std::cout << foo << "\n"; // will print 123 unless overriden } }; Currently, this is used by Store and its subclasses for store parameters. You now get a warning if you specify a non-existant store parameter in a store URI.
2017-04-10 Minor cleanupEelco Dolstra1-8/+3
Also, possible fix for #1310 on 32-bit systems.
2017-03-16 copyPaths(): Use queryValidPaths() to reduce SSH latencyEelco Dolstra1-1/+1
2017-03-15 Store: Add a method for getting build logsEelco Dolstra1-3/+0
This allows various Store implementations to provide different ways to get build logs. For example, BinaryCacheStore can get the build logs from the binary cache. Also, remove the log-servers option since we can use substituters for this.
2016-11-26 Revert "Get rid of unicode quotes (#1140)"Eelco Dolstra1-1/+1
This reverts commit f78126bfd6b6c8477fcdbc09b2f98772dbe9a1e7. There really is no need for such a massive change...
2016-11-25 Get rid of unicode quotes (#1140)Guillaume Maudoux1-1/+1
2016-10-21 Remove addPathToAccessorEelco Dolstra1-4/+5
2016-10-21 BinaryCacheStore: Optionally write a NAR listingEelco Dolstra1-1/+1
The store parameter "write-nar-listing=1" will cause BinaryCacheStore to write a file ‘<store-hash>.ls.xz’ for each ‘<store-hash>.narinfo’ added to the binary cache. This file contains an XZ-compressed JSON file describing the contents of the NAR, excluding the contents of regular files. E.g. { "version": 1, "root": { "type": "directory", "entries": { "lib": { "type": "directory", "entries": { "Mcrt1.o": { "type": "regular", "size": 1288 }, "Scrt1.o": { "type": "regular", "size": 3920 }, } } } ... } } (The actual file has no indentation.) This is intended to speed up the NixOS channels programs index generator [1], since fetching gazillions of large NARs from cache.nixos.org is currently a bottleneck for updating the regular (non-small) channel. [1] https://github.com/NixOS/nixos-channel-scripts/blob/master/generate-programs-index.cc
2016-09-16 Make computeFSClosure() single-threaded againEelco Dolstra1-1/+3
The fact that queryPathInfo() is synchronous meant that we needed a thread for every concurrent binary cache lookup, even though they end up being handled by the same download thread. Requiring hundreds of threads is not a good idea. So now there is an asynchronous version of queryPathInfo() that takes a callback function to process the result. Similarly, enqueueDownload() now takes a callback rather than returning a future. Thus, a command like nix path-info --store https://cache.nixos.org/ -r /nix/store/slljrzwmpygy1daay14kjszsr9xix063-nixos-16.09beta231.dccf8c5 that returns 4941 paths now takes 1.87s using only 2 threads (the main thread and the downloader thread). (This is with a prewarmed CloudFront.)
2016-08-10 Mark content-addressed paths in the Nix database and in .narinfoEelco Dolstra1-2/+2
This allows such paths to be imported without signatures.
2016-06-02 Make derivationFromPath work on diverted storesEelco Dolstra1-0/+2
2016-06-02 nix run: Mount the Nix store in a private namespaceEelco Dolstra1-0/+4
This is a convenience command to allow users who are not privileged to create /nix/store to use Nix with regular binary caches. For example, $ NIX_REMOTE="local?state=$HOME/nix/var&real=/$HOME/nix/store" nix run firefox bashInteractive will download Firefox and bash from cache.nixos.org, then start a shell in which $HOME/nix/store is mounted on /nix/store.
2016-06-02 LocalStore: Allow the physical and logical store directories to differEelco Dolstra1-0/+2
This is primarily to subsume the functionality of the copy-from-other-stores substituter. For example, in the NixOS installer, we can now do (assuming we're in the target chroot, and the Nix store of the installation CD is bind-mounted on /tmp/nix): $ nix-build ... --option substituters 'local?state=/tmp/nix/var&real=/tmp/nix/store' However, unlike copy-from-other-stores, this also allows write access to such a store. One application might be fetching substitutes for /nix/store in a situation where the user doesn't have sufficient privileges to create /nix, e.g.: $ NIX_REMOTE="local?state=/home/alice/nix/var&real=/home/alice/nix/store" nix-build ...
2016-06-02 Allow setting the state directory as a store parameterEelco Dolstra1-2/+10
E.g. "local?store=/tmp/store&state=/tmp/var".
2016-06-01 Make the store directory a member variable of StoreEelco Dolstra1-1/+1
2016-05-31 nix-copy-closure / build-remote.pl: Disable signature checkingEelco Dolstra1-1/+1
This restores the Nix 1.11 behaviour.
2016-05-30 Check signatures before downloading the substituteEelco Dolstra1-0/+1
2016-05-30 Re-implement binary cache signature checkingEelco Dolstra1-0/+4
This is now done in LocalStore::addToStore(), rather than in the binary cache substituter (which no longer exists).
2016-05-04 Add a Store::addToStore() variant that accepts a NAREelco Dolstra1-7/+3
As a side effect, this ensures that signatures are propagated when copying paths between stores. Also refactored import/export to make use of this.
2016-05-04 Remove OpenSSL-based signingEelco Dolstra1-4/+3
2016-04-29 Allow parameters in store URIsEelco Dolstra1-0/+2
This is to allow store-specific configuration, e.g. s3://my-cache?compression=bzip2&secret-key=/path/to/key.
2016-04-29 Eliminate the substituter mechanismEelco Dolstra1-27/+0
Substitution is now simply a Store -> Store copy operation, most typically from BinaryCacheStore to LocalStore.
2016-04-19 Move path info caching from BinaryCacheStore to StoreEelco Dolstra1-7/+3
Caching path info is generally useful. For instance, it speeds up "nix path-info -rS /run/current-system" (i.e. showing the closure sizes of all paths in the closure of the current system) from 5.6s to 0.15s. This also eliminates some APIs like Store::queryDeriver() and Store::queryReferences().
2016-04-08 Remove failed build cachingEelco Dolstra1-17/+2
This feature was implemented for Hydra, but Hydra no longer uses it.
2016-04-08 Make LocalStore thread-safeEelco Dolstra1-66/+52
Necessary for multi-threaded commands like "nix verify-paths".
2016-04-07 Sign locally-built pathsEelco Dolstra1-0/+6
Locally-built paths are now signed automatically using the secret keys specified by the ‘secret-key-files’ option.
2016-04-05 Add "nix copy-sigs" commandEelco Dolstra1-0/+2
This imports signatures from one store into another. E.g. $ nix copy-sigs -r /run/current-system -s https://cache.nixos.org/ imported 595 signatures
2016-03-30 LocalStore: Keep track of ultimately trusted pathsEelco Dolstra1-2/+2
These are content-addressed paths or outputs of locally performed builds. They are trusted even if they don't have signatures, so "nix verify-paths" won't complain about them.
2016-03-30 Improve the SQLite wrapper APIEelco Dolstra1-3/+4
In particular, this eliminates a bunch of boilerplate code.
2016-03-30 Factour out SQLite handlingEelco Dolstra1-33/+2
2016-02-26 importPaths(): Optionally add NARs to binary cache accessorEelco Dolstra1-1/+2
This enables an optimisation in hydra-queue-runner, preventing a download of a NAR it just uploaded to the cache when reading files like hydra-build-products.
2016-02-25 Add NAR / Store accessor abstractionEelco Dolstra1-4/+1
This is primary to allow hydra-queue-runner to extract files like "nix-support/hydra-build-products" from NARs in binary caches.
2016-02-24 Eliminate reserveSpace flagEelco Dolstra1-1/+3
2016-02-15 Expose the export magic value and move LocalStore::queryReferences to StoreEelco Dolstra1-2/+0
2016-02-04 StoreAPI -> StoreEelco Dolstra1-1/+1
Calling a class an API is a bit redundant...
2016-02-04 Eliminate the "store" global variableEelco Dolstra1-0/+4
Also, move a few free-standing functions into StoreAPI and Derivation. Also, introduce a non-nullable smart pointer, ref<T>, which is just a wrapper around std::shared_ptr ensuring that the pointer is never null. (For reference-counted values, this is better than passing a "T&", because the latter doesn't maintain the refcount. Usually, the caller will have a shared_ptr keeping the value alive, but that's not always the case, e.g., when passing a reference to a std::thread via std::bind.)
2016-01-31 Use the daemon when we don't have write access to the Nix databaseEelco Dolstra1-0/+6
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-10-14T14·24+0000