about summary refs log tree commit diff
path: root/src/libstore
AgeCommit message (Collapse)AuthorFilesLines
2017-03-16 copyPaths(): Use queryValidPaths() to reduce SSH latencyEelco Dolstra7-30/+34
2017-03-16 copyPaths(): Don't query path info for a path the target already hasEelco Dolstra1-0/+1
For example, this cuts "nix-copy-closure --from" on a NixOS system closure from 15.9s to 0.5s.
2017-03-16 LegacySSHStore: Provide a faster implementation of computeFSClosure()Eelco Dolstra2-1/+23
This avoids the latency of the standard implementation, which can make a huge difference (e.g. 16.5s -> 0.5s on a NixOS system closure).
2017-03-16 Fix nix-copy-closure --toEelco Dolstra1-1/+1
2017-03-16 Remove "killing process <pid>" messagesEelco Dolstra1-2/+2
They convey no useful information.
2017-03-15 More precise compression settingsEelco Dolstra1-5/+8
2017-03-15 runProgram(): Distinguish between empty input and no inputEelco Dolstra1-1/+1
For example, if we call brotli with an empty input, it shouldn't read from the caller's stdin.
2017-03-15 Compress NAR listings using the "text-compression" methodEelco Dolstra2-3/+3
So if "text-compression=br", the .ls file in S3 will get a Content-Encoding of "br". Brotli appears to compress better than xz for this kind of file and is natively supported by browsers.
2017-03-15 S3BinaryCacheStore: Set Content-TypeEelco Dolstra5-13/+25
This is necessary for serving log files to browsers.
2017-03-15 S3BinaryCacheStore: Support compression of narinfo and log filesEelco Dolstra4-8/+40
You can now set the store parameter "text-compression=br" to compress textual files in the binary cache (i.e. narinfo and logs) using Brotli. This sets the Content-Encoding header; the extension of compressed files is unchanged. You can separately specify the compression of log files using "log-compression=br". This is useful when you don't want to compress narinfo files for backward compatibility.
2017-03-15 Add support for brotli compressionEelco Dolstra1-2/+19
Build logs on cache.nixos.org are compressed using Brotli (since this allows them to be decompressed automatically by Chrome and Firefox), so it's handy if "nix log" can decompress them.
2017-03-15 BinaryCacheStore: Implement getBuildLog()Eelco Dolstra3-1/+31
We assume that build logs are stored under log/<drv>, e.g. /nix/store/q7ab198v13p0f8x8wgnd75dva7d5mip6-friday-devil-0.1.1.1.drv maps to https://cache.nixos.org/log/q7ab198v13p0f8x8wgnd75dva7d5mip6-friday-devil-0.1.1.1.drv
2017-03-15 Store: Add a method for getting build logsEelco Dolstra6-11/+44
This allows various Store implementations to provide different ways to get build logs. For example, BinaryCacheStore can get the build logs from the binary cache. Also, remove the log-servers option since we can use substituters for this.
2017-03-12 Merge branch 'configurable-aws-region'Shea Levy3-7/+11
2017-03-08 Add option to disable import-from-derivation completely, even if the drv is ↵Shea Levy2-0/+5
already realized
2017-03-06 Properly set the caFile for aws-sdk-cpp s3Shea Levy4-2/+6
2017-03-03 Add aws-region param to S3 store URLsShea Levy3-7/+11
2017-03-03 Cache connection failuresEelco Dolstra2-1/+18
2017-03-03 SSHMaster: Make thread-safeEelco Dolstra2-17/+29
2017-03-03 build-remote: Don't use a SSH masterEelco Dolstra2-0/+6
This is unnecessary because we make only one connection.
2017-03-03 Improve SSH handlingEelco Dolstra6-112/+179
* Unify SSH code in SSHStore and LegacySSHStore. * Fix a race starting the SSH master. We now wait synchronously for the SSH master to finish starting. This prevents the SSH clients from starting their own connections. * Don't use a master if max-connections == 1. * Add a "max-connections" store parameter. * Add a "compress" store parameter.
2017-03-03 Fix fatal "broken pipe" error when $NIX_BUILD_HOOK is missingEelco Dolstra1-27/+39
2017-03-01 TeeSink: Pre-reserve string spaceEelco Dolstra1-6/+5
When receiving a very large file, this can prevent the string from having tobe copied, which temporarily doubles memory consumption.
2017-03-01 RemoteStore::addToStore(): Send NAR rather than string containing NAREelco Dolstra3-26/+6
This allows the NAR to be streamed in the future (though we're not doing that yet).
2017-03-01 Tweak messageEelco Dolstra1-1/+1
2017-03-01 Handle importing NARs containing files greater than 4 GiBEelco Dolstra3-18/+16
Also templatize readInt() to work for various integer types.
2017-02-28 Fix building against older curl versionsEelco Dolstra1-1/+4
http://hydra.nixos.org/build/49490928
2017-02-28 Improve SQLite busy handlingEelco Dolstra2-30/+42
2017-02-28 NarInfoDiskCache: Handle SQLite busy errorsEelco Dolstra1-104/+115
2017-02-28 In SQLite errors, include the database pathEelco Dolstra1-4/+8
This is necessary because we have multiple SQLite databases (e.g. the binary cache cache).
2017-02-28 Support auto-configuration of build-max-jobsEelco Dolstra1-1/+8
"build-max-jobs" and the "-j" option can now be set to "auto" to use the number of CPUs in the system. (Unlike build-cores, it doesn't use 0 to imply auto-configuration, because a) magic values are a bad idea in general; b) 0 is a legitimate value used to disable local building.) Fixes #1198.
2017-02-27 _SC_NPROCESSORS_ONLN -> std::thread::hardware_concurrency()Eelco Dolstra1-7/+3
2017-02-27 Retry downloads on HTTP/2 stream errorsEelco Dolstra1-1/+3
Issue #1254.
2017-02-24 Register content-addressability assertion for fixed outputsEelco Dolstra1-1/+4
2017-02-24 Verify content-addressability assertions at registration timeEelco Dolstra1-0/+2
2017-02-22 RemoteStore::addToStore(): Pass content-addressability assertionEelco Dolstra5-3/+15
... and use this in Downloader::downloadCached(). This fixes $ nix-build https://nixos.org/channels/nixos-16.09-small/nixexprs.tar.xz -A hello error: cannot import path ‘/nix/store/csfbp1s60dkgmk9f8g0zk0mwb7hzgabd-nixexprs.tar.xz’ because it lacks a valid signature
2017-02-21 Handle CURLE_RECV_ERROR as a transient errorEelco Dolstra1-1/+1
This fixes unable to download ‘https://cache.nixos.org/nar/077h8ji74y9b0qx7rjk71xd80vjqp6q5gy137r553jlvdlxdcdlk.nar.xz’: HTTP error 200 (curl error: Failure when receiving data from the peer)
2017-02-21 Fix building without S3 supportEelco Dolstra2-2/+2
http://hydra.nixos.org/build/49031196/nixlog/2/raw
2017-02-21 Log AWS retriesEelco Dolstra1-0/+14
2017-02-16 Support netrc in <nix/fetchurl.nix>Eelco Dolstra3-4/+22
This allows <nix/fetchurl.nix> to fetch private Git/Mercurial repositories, e.g. import <nix/fetchurl.nix> { url = https://edolstra@bitbucket.org/edolstra/my-private-repo/get/80a14018daed.tar.bz2; sha256 = "1mgqzn7biqkq3hf2697b0jc4wabkqhmzq2srdymjfa6sb9zb6qs7"; } where /etc/nix/netrc contains: machine bitbucket.org login edolstra password blabla... This works even when sandboxing is enabled. To do: add unpacking support (i.e. fetchzip functionality).
2017-02-16 Move netrcFile to SettingsEelco Dolstra3-26/+23
Also get rid of Settings::processEnvironment(), it appears to be useless.
2017-02-16 Include "curl" in the User-Agent headerEelco Dolstra1-1/+1
Some sites (e.g. BitBucket) give a helpful 401 error when trying to download a private archive if the User-Agent contains "curl", but give a redirect to a login page otherwise (so for instance "nix-prefetch-url" will succeed but produce useless output).
2017-02-16 Merge pull request #1215 from k0001/netrc-1Eelco Dolstra1-0/+8
Add netrc-file support
2017-02-14 Add support for s3:// URIsEelco Dolstra4-63/+141
This adds support for s3:// URIs in all places where Nix allows URIs, e.g. in builtins.fetchurl, builtins.fetchTarball, <nix/fetchurl.nix> and NIX_PATH. It allows fetching resources from private S3 buckets, using credentials obtained from the standard places (i.e. AWS_* environment variables, ~/.aws/credentials and the EC2 metadata server). This may not be super-useful in general, but since we already depend on aws-sdk-cpp, it's a cheap feature to add.
2017-02-09 Documentation.Renzo Carbonara1-0/+3
2017-02-08 Include config.h implicitly with '-include config.h' in CFLAGSTuomas Tynkkynen5-9/+0
Because config.h can #define things like _FILE_OFFSET_BITS=64 and not every compilation unit includes config.h, we currently compile half of Nix with _FILE_OFFSET_BITS=64 and other half with _FILE_OFFSET_BITS unset. This causes major havoc with the Settings class on e.g. 32-bit ARM, where different compilation units disagree with the struct layout. E.g.: diff --git a/src/libstore/globals.cc b/src/libstore/globals.cc @@ -166,6 +166,8 @@ void Settings::update() _get(useSubstitutes, "build-use-substitutes"); + fprintf(stderr, "at Settings::update(): &useSubstitutes = %p\n", &nix::settings.useSubstitutes); _get(buildUsersGroup, "build-users-group"); diff --git a/src/libstore/remote-store.cc b/src/libstore/remote-store.cc +++ b/src/libstore/remote-store.cc @@ -138,6 +138,8 @@ void RemoteStore::initConnection(Connection & conn) void RemoteStore::setOptions(Connection & conn) { + fprintf(stderr, "at RemoteStore::setOptions(): &useSubstitutes = %p\n", &nix::settings.useSubstitutes); conn.to << wopSetOptions Gave me: at Settings::update(): &useSubstitutes = 0xb6e5c5cb at RemoteStore::setOptions(): &useSubstitutes = 0xb6e5c5c7 That was not a fun one to debug!
2017-02-07 Merge branch 'nix-copy-closure-c++' of https://github.com/shlevy/nixEelco Dolstra3-3/+26
2017-02-07 Add a LegacySSHStore that uses nix-store --serveEelco Dolstra2-0/+248
This is useful for nix-copy-closure.
2017-02-07 copyStorePath(): Don't require signatures for "trusted" storesEelco Dolstra2-0/+13
For example, SSH stores could be trusted.
2017-02-07 Provide default implementations for a couple of Store methodsEelco Dolstra4-26/+43
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-12-30T19·49+0000