about summary refs log tree commit diff
path: root/src/libstore
AgeCommit message (Collapse)AuthorFilesLines
2019-08-27 Merge branch 'test-sandboxing' of https://github.com/matthewbauer/nixEelco Dolstra2-3/+27
2019-08-23 Reset tmpDirInSandbox for unsandboxedMatthew Bauer1-0/+1
2019-08-08 Merge pull request #3031 from grahamc/low-speed-limitEelco Dolstra2-3/+4
conf: stalled-download-timeout: make tunable
2019-08-08 conf: stalled-download-timeout: make tunableGraham Christensen2-3/+4
Make curl's low speed limit configurable via stalled-download-timeout. Before, this limit was five minutes without receiving a single byte. This is much too long as if the remote end may not have even acknowledged the HTTP request.
2019-08-07 Merge pull request #3030 from dtzWill/fix/missing-include-ocloexecEelco Dolstra1-0/+1
pathlocks: add include to fcntl.h for O_CLOEXEC
2019-08-07 Merge pull request #2995 from tweag/post-build-hookEelco Dolstra2-0/+58
Add a post build hook
2019-08-07 pathlocks: add include to fcntl.h for O_CLOEXECWill Dietz1-0/+1
2019-08-02 nix-store --verify: Don't repair while holding the GC lockEelco Dolstra1-5/+4
2019-08-02 SimplifyEelco Dolstra1-18/+14
With BSD locks we don't have to guard against reading our own temproots.
2019-08-02 Use BSD instead of POSIX file locksEelco Dolstra5-108/+46
POSIX file locks are essentially incompatible with multithreading. BSD locks have much saner semantics. We need this now that there can be multiple concurrent LocalStore::buildPaths() invocations.
2019-08-02 Add a test for auto-GCEelco Dolstra2-2/+10
This currently fails because we're using POSIX file locks. So when the garbage collector opens and closes its own temproots file, it causes the lock to be released and then deleted by another GC instance.
2019-08-02 Add a post-build-hookregnat2-0/+58
Passing `--post-build-hook /foo/bar` to a nix-* command will cause `/foo/bar` to be executed after each build with the following environment variables set: DRV_PATH=/nix/store/drv-that-has-been-built.drv OUT_PATHS=/nix/store/...build /nix/store/...build-bin /nix/store/...build-dev This can be useful in particular to upload all the builded artifacts to the cache (including the ones that don't appear in the runtime closure of the final derivation or are built because of IFD). This new feature prints the stderr/stdout output to the `nix-build` and `nix build` client, and the output is printed in a Nix 2 compatible format: [nix]$ ./inst/bin/nix-build ./test.nix these derivations will be built: /nix/store/ishzj9ni17xq4hgrjvlyjkfvm00b0ch9-my-example-derivation.drv building '/nix/store/ishzj9ni17xq4hgrjvlyjkfvm00b0ch9-my-example-derivation.drv'... hello! bye! running post-build-hook '/home/grahamc/projects/github.com/NixOS/nix/post-hook.sh'... post-build-hook: + sleep 1 post-build-hook: + echo 'Signing paths' /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation post-build-hook: Signing paths /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation post-build-hook: + sleep 1 post-build-hook: + echo 'Uploading paths' /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation post-build-hook: Uploading paths /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation post-build-hook: + sleep 1 post-build-hook: + printf 'very important stuff' /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation [nix-shell:~/projects/github.com/NixOS/nix]$ ./inst/bin/nix build -L -f ./test.nix my-example-derivation> hello! my-example-derivation> bye! my-example-derivation (post)> + sleep 1 my-example-derivation (post)> + echo 'Signing paths' /nix/store/c263gzj2kb2609mz8wrbmh53l14wzmfs-my-example-derivation my-example-derivation (post)> Signing paths /nix/store/c263gzj2kb2609mz8wrbmh53l14wzmfs-my-example-derivation my-example-derivation (post)> + sleep 1 my-example-derivation (post)> + echo 'Uploading paths' /nix/store/c263gzj2kb2609mz8wrbmh53l14wzmfs-my-example-derivation my-example-derivation (post)> Uploading paths /nix/store/c263gzj2kb2609mz8wrbmh53l14wzmfs-my-example-derivation my-example-derivation (post)> + sleep 1 my-example-derivation (post)> + printf 'very important stuff' [1 built, 0.0 MiB DL] Co-authored-by: Graham Christensen <graham@grahamc.com> Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2019-07-30 Don’t rely on EPERMMatthew Bauer1-2/+2
startProcess does not appear to send the exit code to the helper correctly. Not sure why this is, but it is probably safe to just fallback on all sandbox errors.
2019-07-30 Disable findRuntimeRoots on darwin when running tests because lsof is slowBas van Dijk1-10/+15
See: https://github.com/NixOS/nix/issues/3011
2019-07-25 Use sandbox fallback when cloning fails in builderMatthew Bauer2-1/+14
When sandbox-fallback = true (the default), the Nix builder will fall back to disabled sandbox mode when the kernel doesn’t allow users to set it up. This prevents hard errors from occuring in tricky places, especially the initial installer. To restore the previous behavior, users can set: sandbox-fallback = false in their /etc/nix/nix.conf configuration.
2019-07-25 Disable CLONE_NEWUSER when it’s unavailableMatthew Bauer1-2/+12
Some kernels disable "unpriveleged user namespaces". This is unfortunate, but we can still use mount namespaces. Anyway, since each builder has its own nixbld user, we already have most of the benefits of user namespaces.
2019-07-13 Merge pull request #2975 from matthewbauer/fix-nsswitch-issueEelco Dolstra1-1/+7
Don’t use entire /etc/nsswitch.conf file
2019-07-10 Resume NAR downloadsEelco Dolstra1-7/+28
This is a much simpler fix to the 'error 9 while decompressing xz file' problem than 78fa47a7f08a4cb6ee7061bf0bd86a40e1d6dc91. We just do a ranged HTTP request starting after the data that we previously wrote into the sink. Fixes #2952, #379.
2019-07-10 HttpBinaryCacheStore: Use default number of retries for NARsEelco Dolstra1-1/+0
2019-07-10 Downloader: Use warn()Eelco Dolstra1-3/+3
2019-07-10 Revert "Fix 'error 9 while decompressing xz file'"Eelco Dolstra5-122/+125
This reverts commit 78fa47a7f08a4cb6ee7061bf0bd86a40e1d6dc91.
2019-07-02 Merge pull request #2779 from LnL7/build-exit-codesEelco Dolstra2-4/+37
build: add exit code for hash and check mismatches
2019-07-02 Merge pull request #2974 from grahamc/invalid-nameEelco Dolstra1-3/+10
checkStoreName: give more precise/verbose error information
2019-07-02 checkStoreName: give more precise/verbose error informationGraham Christensen1-3/+10
$ sudo ./inst/bin/nix-instantiate -E '"${./.git}"' error: The path name '.git' is invalid: it is illegal to start the name with a period. Path names are alphanumeric and can include the symbols +-._?= and must not begin with a period. Note: If '.git' is a source file and you cannot rename it on disk, builtins.path { name = ... } can be used to give it an alternative name.
2019-07-02 build: replace 100 offset for build exit codesDaiderd Jordan1-5/+19
2019-07-02 build: use binary mask for build status flagsDaiderd Jordan1-1/+9
If multiple builds with fail with different errors it will be reflected in the status code. eg. 103 => timeout + hash mismatch 105 => timeout + check mismatch 106 => hash mismatch + check mismatch 107 => timeout + hash mismatch + check mismatch
2019-07-02 build: add exit code for hash and check mismatchesDaiderd Jordan2-4/+15
Makes it easier to identify the failure reason in other tooling, eg. differentiate between a non-deterministic --check vs a failed build. $ nix-build '<nix/fetchurl.nix>' --argstr url http://example.org --argstr sha256 0000000000000000000000000000000000000000000000000000 hash mismatch in fixed-output derivation '/nix/store/nzi9ck45rwlxzcwr25is7qlf3hs5xl83-example.org': wanted: sha256:0000000000000000000000000000000000000000000000000000 got: sha256:08y4734bm2zahw75b16bcmcg587vvyvh0n11gwiyir70divwp1rm $ echo $? 102 $ nix-build -E 'with import <nixpkgs> {}; runCommand "foo" {} "date +%s > $out"' --check warning: rewriting hashes in '/nix/store/g3k47g0399fvjmbm0p0mnad74k4w8vkz-foo'; cross fingers error: derivation '/nix/store/mggc8dz13ackb49qca6m23zq4fpq132q-foo.drv' may not be deterministic: output '/nix/store/g3k47g0399fvjmbm0p0mnad74k4w8vkz-foo' differs $ echo $? 104
2019-06-28 Fix `http2 = false` having no effect. Fixes #2971.Aniket Deshpande1-0/+2
Setting `http2 = false` in nix config (e.g. /etc/nix/nix.conf) had no effect, and `nix-env -vvvvv -i hello` still downloaded .nar packages using HTTP/2. In `src/libstore/download.cc`, the `CURL_HTTP_VERSION_2TLS` option was being explicitly set when `downloadSettings.enableHttp2` was `true`, but, `CURL_HTTP_VERSION_1_1` option was not being explicitly set when `downloadSettings.enableHttp2` was `false`. This may be because `https://curl.haxx.se/libcurl/c/libcurl-env.html` states: "You have to set this option if you want to use libcurl's HTTP/2 support." but, also, in the changelog, states: "DEFAULT Since curl 7.62.0: CURL_HTTP_VERSION_2TLS Before that: CURL_HTTP_VERSION_1_1" So, the default setting for `libcurl` is HTTP/2 for version >= 7.62.0. In this commit, option `CURLOPT_HTTP_VERSION` is explicitly set to `CURL_HTTP_VERSION_1_1` when `downloadSettings.enableHttp2` nix config setting is `false`. This can be tested by running `nix-env -vvvvv -i hello | grep HTTP`
2019-06-27 Don’t use entire /etc/nsswitch.conf fileMatthew Bauer1-1/+7
The default nsswitch.conf(5) file in most distros can handle many different things including host name, user names, groups, etc. In Nix, we want to limit the amount of impurities that come from these things. As a result, we should only allow nss to be used for gethostbyname(3) and getservent(3). /cc @Ericson2314
2019-06-25 nix-channel: Don't fetch binary-cache-urlEelco Dolstra1-2/+1
This has been ignored since the Perl->C++ rewrite.
2019-06-24 Fix 32-bit overflow with --no-netEelco Dolstra1-1/+1
--no-net causes tarballTtl to be set to the largest 32-bit integer, which causes comparison like 'time + tarballTtl < other_time' to fail on 32-bit systems. So cast them to 64-bit first. https://hydra.nixos.org/build/95076624 (cherry picked from commit 29ccb2e9697ee2184012dd13854e487928ae4441)
2019-06-24 Refactor downloadCached() interfaceEelco Dolstra2-27/+37
(cherry picked from commit df3f5a78d5ab0a1f2dc9d288b271b38a9b8b33b5)
2019-06-24 Add '--no-net' convenience flagEelco Dolstra4-20/+24
This flag * Disables substituters. * Sets the tarball-ttl to infinity (ensuring e.g. that the flake registry and any downloaded flakes are considered current). * Disables retrying downloads and sets the connection timeout to the minimum. (So it doesn't completely disable downloads at the moment.) (cherry picked from commit 8ea842260b4fd93315d35c5ba94b1ff99ab391d8)
2019-06-24 Fix 'error 9 while decompressing xz file'Eelco Dolstra5-125/+122
Once we've started writing data to a Sink, we can't restart a download request, because then we end up writing duplicate data to the Sink. Therefore we shouldn't handle retries in Downloader but at a higher level (in particular, in copyStorePath()). Fixes #2952. (cherry picked from commit a67cf5a3585c41dd9f219a2c7aa9cf67fa69520b)
2019-06-24 Downloader: Propagate exceptions from decompressionSink->finish()Eelco Dolstra1-2/+7
(cherry picked from commit 15fa70cd1b853f5e62662b99ccb9ef3da6cfadff)
2019-06-24 CachedDownloadResult: Include store pathEelco Dolstra2-0/+5
Also, make fetchGit and fetchMercurial update allowedPaths properly. (Maybe the evaluator, rather than the caller of the evaluator, should apply toRealPath(), but that's a bigger change.) (cherry picked from commit 5c34d665386f4053d666b0899ecca0639e500fbd)
2019-06-24 downloadCached: Return ETagEelco Dolstra2-6/+23
(cherry picked from commit 529add316c5356a8060c35f987643b7bf5c796dc)
2019-06-17 Another attempt at getting pseudoterminals to work on macOSEelco Dolstra1-0/+3
2019-06-16 Hopefully fix macOS testsEelco Dolstra1-4/+6
2019-06-16 Set $TERMEelco Dolstra1-0/+3
2019-06-16 Run builds in a pseudo-terminalEelco Dolstra1-8/+46
This allows many programs (e.g. gcc, clang, cmake) to print colorized log output (assuming $TERM is set to a value like "xterm"). There are other ways to get colors, in particular setting CLICOLOR_FORCE, but they're less widely supported and can break programs that parse tool output.
2019-06-01 Fix segfault in builtin fetchurl with hashed mirrors + SRI hashesEelco Dolstra1-1/+2
2019-05-13 Merge pull request #2802 from LnL7/fix-needs-hashrewriteEelco Dolstra1-12/+16
build: move needsHashRewrite initialization to startBuilder
2019-05-12 build: make needsHashRewrite a methodDaiderd Jordan1-12/+16
2019-05-12 handleDiffHook: stop passing allowVforkGraham Christensen1-3/+1
2019-05-12 runProgram: support gid, uid, chdirGraham Christensen1-20/+16
2019-05-12 chdir, setgroupsGraham Christensen1-0/+4
2019-05-12 diff hook: execute as the build user, and pass the temp dirGraham Christensen1-12/+29
2019-05-12 build: run diff-hook under --check and document diff-hookGraham Christensen1-12/+18
2019-05-12 build: move needsHashRewrite initialization to startBuilderDaiderd Jordan1-7/+7
The value of useChroot is not set yet in the constructor, resulting in hash rewriting being enabled in certain cases where it should not be. Fixes #2801
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-12-18T12·41+0000