about summary refs log tree commit diff
path: root/src/libstore/remote-store.hh
AgeCommit message (Collapse)AuthorFilesLines
2019-03-14 experimental/optional -> optionalEelco Dolstra1-1/+1
2019-03-14 findRoots(): Add 'censor' parameterEelco Dolstra1-1/+1
This is less brittle than filtering paths after the fact in nix-daemon.
2018-10-27 Merge branch 'nix-doctor' of https://github.com/LnL7/nixEelco Dolstra1-0/+2
2018-10-26 remote-store.hh: ConnectionHandle is struct, minor fix warningWill Dietz1-1/+1
2018-10-16 RemoteStore: Close connection if an exception occursEelco Dolstra1-1/+6
Fixes #2075.
2018-10-16 Get rid of UDSRemoteStore::ConnectionEelco Dolstra1-5/+1
Since its superclass RemoteStore::Connection contains 'to' and 'from' fields that refer to the file descriptor maintained in the subclass, it was possible for the flush() call in Connection::~Connection() to write to a closed file descriptor (or worse, a file descriptor now referencing another file). So make sure that the file descriptor survives 'to' and 'from'.
2018-09-02 Store: expose the protocol version used by a storeDaiderd Jordan1-0/+2
2018-05-30 Simplify the callback mechanismEelco Dolstra1-2/+1
2018-03-21 Make 'nix copy --to daemon' run in constant memoryEelco Dolstra1-1/+1
2018-03-05 ssh-ng: Don't forward options to the daemon.Shea Levy1-1/+2
This can be iterated on and currently leaves out settings we know we want to forward, but it fixes #1713 and fixes #1935 and isn't fundamentally broken like the status quo. Future changes are suggested in a comment.
2018-01-26 remote_store: register for NIX_REMOTE=unix://pathSpencer Baugh1-0/+2
This allows overriding the socket path so the daemon may be listening at an arbitrary Unix domain socket location. Fixes #1800
2017-09-14 RemoteStore: Add option to drop old connections from the poolEelco Dolstra1-0/+6
This is a hack to make hydra-queue-runner free its temproots periodically, thereby ensuring that garbage collection of the corresponding paths is not blocked until the queue runner is restarted. It would be better if temproots could be released earlier than at process exit. I started working on a RAII object returned by functions like addToStore() that releases temproots. However, this would be a pretty massive change so I gave up on it for now.
2017-07-03 Replace a few bool flags with enumsEelco Dolstra1-5/+6
Functions like copyClosure() had 3 bool arguments, which creates a severe risk of mixing up arguments. Also, implement copyClosure() using copyPaths().
2017-05-02 build-remote: Fix fallback to other machines when connecting failsEelco Dolstra1-0/+2
Opening an SSHStore or LegacySSHStore does not actually establish a connection, so the try/catch block here did nothing. Added a Store::connect() method to test whether a connection can be established.
2017-04-13 Add a Config class to simplify adding configuration settingsEelco Dolstra1-0/+3
The typical use is to inherit Config and add Setting<T> members: class MyClass : private Config { Setting<int> foo{this, 123, "foo", "the number of foos to use"}; Setting<std::string> bar{this, "blabla", "bar", "the name of the bar"}; MyClass() : Config(readConfigFile("/etc/my-app.conf")) { std::cout << foo << "\n"; // will print 123 unless overriden } }; Currently, this is used by Store and its subclasses for store parameters. You now get a warning if you specify a non-existant store parameter in a store URI.
2017-04-06 Implement RemoteStore::queryMissing()Eelco Dolstra1-0/+4
This provides a significant speedup, e.g. 64 s -> 12 s for nix-build --dry-run -I nixpkgs=channel:nixos-16.03 '<nixpkgs/nixos/tests/misc.nix>' -A test on a cold local and CloudFront cache. The alternative is to use lots of concurrent daemon connections but that seems wasteful.
2017-03-16 copyPaths(): Use queryValidPaths() to reduce SSH latencyEelco Dolstra1-1/+1
2017-03-03 Cache connection failuresEelco Dolstra1-0/+4
2017-03-03 Improve SSH handlingEelco Dolstra1-2/+2
* Unify SSH code in SSHStore and LegacySSHStore. * Fix a race starting the SSH master. We now wait synchronously for the SSH master to finish starting. This prevents the SSH clients from starting their own connections. * Don't use a master if max-connections == 1. * Add a "max-connections" store parameter. * Add a "compress" store parameter.
2016-11-09 Merge branch 'ssh-store' of https://github.com/shlevy/nixEelco Dolstra1-8/+27
2016-10-21 Remove addPathToAccessorEelco Dolstra1-1/+2
2016-10-21 BinaryCacheStore: Optionally write a NAR listingEelco Dolstra1-1/+1
The store parameter "write-nar-listing=1" will cause BinaryCacheStore to write a file ‘<store-hash>.ls.xz’ for each ‘<store-hash>.narinfo’ added to the binary cache. This file contains an XZ-compressed JSON file describing the contents of the NAR, excluding the contents of regular files. E.g. { "version": 1, "root": { "type": "directory", "entries": { "lib": { "type": "directory", "entries": { "Mcrt1.o": { "type": "regular", "size": 1288 }, "Scrt1.o": { "type": "regular", "size": 3920 }, } } } ... } } (The actual file has no indentation.) This is intended to speed up the NixOS channels programs index generator [1], since fetching gazillions of large NARs from cache.nixos.org is currently a bottleneck for updating the regular (non-small) channel. [1] https://github.com/NixOS/nixos-channel-scripts/blob/master/generate-programs-index.cc
2016-09-16 Make computeFSClosure() single-threaded againEelco Dolstra1-1/+3
The fact that queryPathInfo() is synchronous meant that we needed a thread for every concurrent binary cache lookup, even though they end up being handled by the same download thread. Requiring hundreds of threads is not a good idea. So now there is an asynchronous version of queryPathInfo() that takes a callback function to process the result. Similarly, enqueueDownload() now takes a callback rather than returning a future. Thus, a command like nix path-info --store https://cache.nixos.org/ -r /nix/store/slljrzwmpygy1daay14kjszsr9xix063-nixos-16.09beta231.dccf8c5 that returns 4941 paths now takes 1.87s using only 2 threads (the main thread and the downloader thread). (This is with a prewarmed CloudFront.)
2016-09-02 Add ssh store implementationShea Levy1-4/+4
2016-09-02 Factor out the unix domain socket-specific code from RemoteStoreShea Levy1-7/+26
2016-06-01 Make the store directory a member variable of StoreEelco Dolstra1-1/+1
2016-05-31 nix-copy-closure / build-remote.pl: Disable signature checkingEelco Dolstra1-1/+1
This restores the Nix 1.11 behaviour.
2016-05-04 Add a Store::addToStore() variant that accepts a NAREelco Dolstra1-5/+3
As a side effect, this ensures that signatures are propagated when copying paths between stores. Also refactored import/export to make use of this.
2016-05-04 Remove OpenSSL-based signingEelco Dolstra1-3/+2
2016-04-29 Allow parameters in store URIsEelco Dolstra1-0/+2
This is to allow store-specific configuration, e.g. s3://my-cache?compression=bzip2&secret-key=/path/to/key.
2016-04-19 Move path info caching from BinaryCacheStore to StoreEelco Dolstra1-8/+2
Caching path info is generally useful. For instance, it speeds up "nix path-info -rS /run/current-system" (i.e. showing the closure sizes of all paths in the closure of the current system) from 5.6s to 0.15s. This also eliminates some APIs like Store::queryDeriver() and Store::queryReferences().
2016-04-08 Remove failed build cachingEelco Dolstra1-4/+0
This feature was implemented for Hydra, but Hydra no longer uses it.
2016-04-05 Add "nix copy-sigs" commandEelco Dolstra1-0/+2
This imports signatures from one store into another. E.g. $ nix copy-sigs -r /run/current-system -s https://cache.nixos.org/ imported 595 signatures
2016-02-26 importPaths(): Optionally add NARs to binary cache accessorEelco Dolstra1-1/+2
This enables an optimisation in hydra-queue-runner, preventing a download of a NAR it just uploaded to the cache when reading files like hydra-build-products.
2016-02-25 Add NAR / Store accessor abstractionEelco Dolstra1-1/+3
This is primary to allow hydra-queue-runner to extract files like "nix-support/hydra-build-products" from NARs in binary caches.
2016-02-24 Eliminate reserveSpace flagEelco Dolstra1-1/+1
2016-02-23 Pool<T>: Allow a maximum pool sizeEelco Dolstra1-3/+4
2016-02-23 RemoteStore: Make thread-safeEelco Dolstra1-9/+13
This allows a RemoteStore object to be used safely from multiple threads concurrently. It will make multiple daemon connections if necessary. Note: pool.hh and sync.hh have been copied from the Hydra source tree.
2016-02-04 StoreAPI -> StoreEelco Dolstra1-1/+1
Calling a class an API is a bit redundant...
2015-09-18 Shut up clang warningsEelco Dolstra1-29/+30
2015-07-17 Allow remote builds without sending the derivation closureEelco Dolstra1-0/+3
Previously, to build a derivation remotely, we had to copy the entire closure of the .drv file to the remote machine, even though we only need the top-level derivation. This is very wasteful: the closure can contain thousands of store paths, and in some Hydra use cases, include source paths that are very large (e.g. Git/Mercurial checkouts). So now there is a new operation, StoreAPI::buildDerivation(), that performs a build from an in-memory representation of a derivation (BasicDerivation) rather than from a on-disk .drv file. The only files that need to be in the Nix store are the sources of the derivation (drv.inputSrcs), and the needed output paths of the dependencies (as described by drv.inputDrvs). "nix-store --serve" exposes this interface. Note that this is a privileged operation, because you can construct a derivation that builds any store path whatsoever. Fixing this will require changing the hashing scheme (i.e., the output paths should be computed from the other fields in BasicDerivation, allowing them to be verified without access to other derivations). However, this would be quite nice because it would allow .drv-free building (e.g. "nix-env -i" wouldn't have to write any .drv files to disk). Fixes #173.
2015-06-02 Add a ‘verifyStore’ RPCLudovic Courtès1-0/+1
Hello! The patch below adds a ‘verifyStore’ RPC with the same signature as the current LocalStore::verifyStore method. Thanks, Ludo’. >From aef46c03ca77eb6344f4892672eb6d9d06432041 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@gnu.org> Date: Mon, 1 Jun 2015 23:17:10 +0200 Subject: [PATCH] Add a 'verifyStore' remote procedure call.
2015-03-25 addToStore(): Take explicit name argumentEelco Dolstra1-14/+14
2014-09-01 Add an 'optimiseStore' remote procedure call.Ludovic Courtès1-1/+3
2014-07-23 Remove dead codeEelco Dolstra1-1/+0
2014-02-18 Add a flag ‘--check’ to verify build determinismEelco Dolstra1-1/+1
The flag ‘--check’ to ‘nix-store -r’ or ‘nix-build’ will cause Nix to redo the build of a derivation whose output paths are already valid. If the new output differs from the original output, an error is printed. This makes it easier to test if a build is deterministic. (Obviously this cannot catch all sources of non-determinism, but it catches the most common one, namely the current time.) For example: $ nix-build '<nixpkgs>' -A patchelf ... $ nix-build '<nixpkgs>' -A patchelf --check error: derivation `/nix/store/1ipvxsdnbhl1rw6siz6x92s7sc8nwkkb-patchelf-0.6' may not be deterministic: hash mismatch in output `/nix/store/4pc1dmw5xkwmc6q3gdc9i5nbjl4dkjpp-patchelf-0.6.drv' The --check build fails if not all outputs are valid. Thus the first call to nix-build is necessary to ensure that all outputs are valid. The current outputs are left untouched: the new outputs are either put in a chroot or diverted to a different location in the store using hash rewriting.
2012-12-20 nix-store -q --roots: Respect the gc-keep-outputs/gc-keep-derivations settingsEelco Dolstra1-0/+2
So if a path is not garbage solely because it's reachable from a root due to the gc-keep-outputs or gc-keep-derivations settings, ‘nix-store -q --roots’ now shows that root.
2012-11-09 Remove definition of non-existant functionEelco Dolstra1-2/+0
2012-10-03 Add a ‘--repair’ flag to nix-instantiateEelco Dolstra1-2/+2
This allows repairing corrupted derivations and other source files.
2012-10-02 Add a --repair flag to ‘nix-store -r’ to repair derivation outputsEelco Dolstra1-1/+1
With this flag, if any valid derivation output is missing or corrupt, it will be recreated by using a substitute if available, or by rebuilding the derivation. The latter may use hash rewriting if chroots are not available.
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-11-13T01·32+0000