about summary refs log tree commit diff
path: root/src
AgeCommit message (Collapse)AuthorFilesLines
2017-10-17 BinaryCacheStore: Support local caching of NARsEelco Dolstra4-5/+25
This speeds up commands like "nix cat-store". For example: $ time nix cat-store --store https://cache.nixos.org?local-nar-cache=/tmp/nar-cache /nix/store/i60yncmq6w9dyv37zd2k454g0fkl3arl-systemd-234/etc/udev/udev.conf real 0m4.336s $ time nix cat-store --store https://cache.nixos.org?local-nar-cache=/tmp/nar-cache /nix/store/i60yncmq6w9dyv37zd2k454g0fkl3arl-systemd-234/etc/udev/udev.conf real 0m0.045s The primary motivation is to allow hydra-server to serve files from S3 binary caches. Previously Hydra had a hack to do "nix-store -r <path>", but that fetches the entire closure so is prohibitively expensive. There is no garbage collection of the NAR cache yet. Also, the entire NAR is read when accessing a single member file. We could generate the NAR listing to provide random access. Note: the NAR cache is indexed by the store path hash, not the content hash, so NAR caches should not be shared between binary caches, unless you're sure that all your builds are binary-reproducible.
2017-10-17 Remove BinaryCacheStoreAccessorEelco Dolstra2-61/+4
Probably as a result of a bad merge in 4b8f1b0ec066a5b994747b1afd050f5f62d857f6, we had both a BinaryCacheStoreAccessor and a RemoteFSAccessor. BinaryCacheStore::getFSAccessor() returned the latter, but BinaryCacheStore::addToStore() checked for the former. This probably caused hydra-queue-runner to download paths that it just uploaded.
2017-10-17 fetchgit: Remove incomplete/unneeded ref check.Shea Levy1-5/+0
This check fails for tags and branches, and is made redundant by the checks git itself will do when fetching the repo.
2017-10-16 fetchgit: Remove incomplete/unneeded isURI check.Shea Levy1-4/+1
This check spuriously fails for e.g. git@github.com:NixOS/nixpkgs.git, and even for ssh://git@github.com/NixOS/nixpkgs.git, and is made redundant by the checks git itself will do when fetching the repo. We instead pass a -- before passing the URI to git to avoid injection.
2017-10-12 Add option to disable the seccomp filterEelco Dolstra2-0/+8
I needed this to test ACL/xattr removal in canonicalisePathMetaData(). Might also be useful if you need to build old Nixpkgs that doesn't have the required patches to remove setuid/setgid creation.
2017-10-12 Merge pull request #1571 from teto/improve_priority_msgEelco Dolstra1-3/+4
Improve error message for conflicting priorities
2017-10-12 Merge pull request #1580 from copumpkin/retry-after-timeoutEelco Dolstra1-20/+37
Retry in all error cases but a few
2017-10-09 Fix a hang in ThreadPoolEelco Dolstra2-33/+57
The worker threads could exit prematurely if they finished processing all items while the main thread was still adding items. In particular, this caused hanging nix-store --serve processes in the build farm. Also, process items from the main thread.
2017-10-02 Reverse retry logic to retry in all but a few casesDan Peebles1-20/+37
It was getting too much like whac-a-mole listing all the retriable error conditions, so we now retry by default and list the cases where retrying is almost certainly hopeless.
2017-09-28 nix run: Restore signalsEelco Dolstra1-0/+2
Otherwise Ctrl-C doesn't work.
2017-09-27 nix copy: make recursive by defaultEelco Dolstra3-3/+16
2017-09-27 Tweak messageEelco Dolstra1-1/+1
2017-09-27 nix build: Fix realising a store pathEelco Dolstra1-1/+3
2017-09-25 nix why-depends: Write to stdoutEelco Dolstra1-3/+3
2017-09-25 Add "nix show-derivation"Eelco Dolstra3-0/+147
This debug command prints a store derivation in JSON format. For example: $ nix show-derivation nixpkgs.hello { "/nix/store/ayjwpwwiyy04nh9z71rsdgd3q7bra7ch-hello-2.10.drv": { "outputs": { "out": { "path": "/nix/store/w5w4v29ql0qwqhczkdxs94ix2lh7ibgs-hello-2.10" } }, "inputSrcs": [ "/nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh" ], "inputDrvs": { "/nix/store/13839aqdf6x4k3b785rw5f2l7857l6y3-bash-4.4-p12.drv": [ "out" ], "/nix/store/vgdx7fdc7d4iirmnwj2py1nrvr5qwzj7-hello-2.10.tar.gz.drv": [ "out" ], "/nix/store/x3kkd0vsqfflbvwf1055l9mr39bg0ms0-stdenv.drv": [ "out" ] }, "platform": "x86_64-linux", "builder": "/nix/store/qp5fw57d38bd1n07ss4zxh88zg67c3vg-bash-4.4-p12/bin/bash", "args": [ "-e", "/nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh" ], "env": { "buildInputs": "", "builder": "/nix/store/qp5fw57d38bd1n07ss4zxh88zg67c3vg-bash-4.4-p12/bin/bash", "configureFlags": "", "doCheck": "1", "name": "hello-2.10", "nativeBuildInputs": "", "out": "/nix/store/w5w4v29ql0qwqhczkdxs94ix2lh7ibgs-hello-2.10", "propagatedBuildInputs": "", "propagatedNativeBuildInputs": "", "src": "/nix/store/3x7dwzq014bblazs7kq20p9hyzz0qh8g-hello-2.10.tar.gz", "stdenv": "/nix/store/6zngq1rdh0ans9qyckqimqibgnlvlfrm-stdenv", "system": "x86_64-linux" } } } This removes the need for pp-aterm.
2017-09-25 DohEelco Dolstra1-1/+1
2017-09-21 Improve error message for conflicting prioritiesMatthieu Coudron1-3/+4
I find the error message 'nix-env --set-flag priority NUMBER PKGNAME' not as helpful as it could be : - doesn't share the current priorities - doesn't say that the command must be run on the already installed PKGNAME (which is confusing the first time) - the doc needs careful reading: "If there are multiple derivations matching a name in args that have the same name (e.g., gcc-3.3.6 and gcc-4.1.1), then the derivation with the highest priority is used." if one stops reading there, he is screwed. Salvation comes with reading "A derivation can define a priority by declaring the meta.priority attribute. This attribute should be a number, with a higher value denoting a lower priority. The default priority is 0." To sum it up, lower number wins. I tried to convey this idea in the message too.
2017-09-18 nix-shell: Ignore readFile() errorsEelco Dolstra1-5/+5
Fixes #1563.
2017-09-18 Don't remove Content-Encoding in fetchurl / nix-prefetch-urlEelco Dolstra4-2/+7
Fixes #1568.
2017-09-14 RemoteStore: Add option to drop old connections from the poolEelco Dolstra3-1/+31
This is a hack to make hydra-queue-runner free its temproots periodically, thereby ensuring that garbage collection of the corresponding paths is not blocked until the queue runner is restarted. It would be better if temproots could be released earlier than at process exit. I started working on a RAII object returned by functions like addToStore() that releases temproots. However, this would be a pretty massive change so I gave up on it for now.
2017-09-14 Show the pid of temporary rootsEelco Dolstra2-11/+16
2017-09-14 nix-store -q --roots / --gc --print-roots: Print temporary / in-memory rootsEelco Dolstra2-22/+41
For example, $ nix-store -q --roots /nix/store/7phd2sav7068nivgvmj2vpm3v47fd27l-patchelf-0.8pre845_0315148 {temp:1} denotes that the path is only being kept alive by a temporary root (i.e. /nix/var/nix/temproots/). Similarly, $ nix-store --gc --print-roots ... {memory:9} -> /nix/store/094gpjn9f15ip17wzxhma4r51nvsj17p-curl-7.53.1 shows that curl is being used by some process.
2017-09-14 Add "nix add-to-store" commandEelco Dolstra4-4/+67
2017-09-14 nix why-depends: Fix showing self-referencesEelco Dolstra1-8/+16
2017-09-11 nix why-depends: Add option to show all edges causing a dependencyEelco Dolstra1-34/+156
For example, without --all: $ nix why-depends nixpkgs.nixUnstable nixpkgs.libssh2 /nix/store/s9n5gvj2l49b4n19nz6xl832654nf7n7-nix-1.12pre5511_c94f3d55 └───lib/libnixstore.so: …/lib:/nix/store/w9ykqpl5v0r3vfwsgn408jqhs72cx96x-curl-7.55.0/lib… => /nix/store/w9ykqpl5v0r3vfwsgn408jqhs72cx96x-curl-7.55.0 └───lib/libcurl.la: …ib -L/nix/store/4mbayl1y5hpjbjzkx8ndyhkv98kqw1wi-libssh2-1.8.0/l… => /nix/store/4mbayl1y5hpjbjzkx8ndyhkv98kqw1wi-libssh2-1.8.0 but with --all: $ nix why-depends -a nixpkgs.nixUnstable nixpkgs.libssh2 /nix/store/s9n5gvj2l49b4n19nz6xl832654nf7n7-nix-1.12pre5511_c94f3d55 ├───lib/libnixstore.so: …/lib:/nix/store/w9ykqpl5v0r3vfwsgn408jqhs72cx96x-curl-7.55.0/lib… │ => /nix/store/w9ykqpl5v0r3vfwsgn408jqhs72cx96x-curl-7.55.0 │ └───lib/libcurl.la: …ib -L/nix/store/4mbayl1y5hpjbjzkx8ndyhkv98kqw1wi-libssh2-1.8.0/l… │ lib/libcurl.so.4.4.0: …/lib:/nix/store/4mbayl1y5hpjbjzkx8ndyhkv98kqw1wi-libssh2-1.8.0/l… │ => /nix/store/4mbayl1y5hpjbjzkx8ndyhkv98kqw1wi-libssh2-1.8.0 └───lib/libnixstore.so: …/lib:/nix/store/bx2i9vi76lps6w9rr73fxf6my31s4dg5-aws-sdk-cpp-1.0… => /nix/store/bx2i9vi76lps6w9rr73fxf6my31s4dg5-aws-sdk-cpp-1.0.153 └───lib/libaws-cpp-sdk-core.so: …e.so./nix/store/w9ykqpl5v0r3vfwsgn408jqhs72cx96x-curl-7.55.0/lib… lib/libaws-cpp-sdk-s3.so: …/lib:/nix/store/w9ykqpl5v0r3vfwsgn408jqhs72cx96x-curl-7.55.0/lib… => /nix/store/w9ykqpl5v0r3vfwsgn408jqhs72cx96x-curl-7.55.0
2017-09-10 Add command "nix why-depends"Eelco Dolstra1-0/+133
This command shows why a package has another package in its runtime closure. For example, to see why VLC has libdrm.dev in its closure: $ nix why-depends nixpkgs.vlc nixpkgs.libdrm.dev /nix/store/g901z9pcj0n5yy5n6ykxk3qm4ina1d6z-vlc-2.2.5.1: lib/libvlccore.so.8.0.0: …nfig:/nix/store/405lmx6jl8lp0ad1vrr6j498chrqhz8g-libdrm-2.4.75-d… /nix/store/s3nm7kd8hlcg0facn2q1ff2n7wrwdi2l-mesa-noglu-17.0.7-dev: nix-support/propagated-native-build-inputs: …-dev /nix/store/405lmx6jl8lp0ad1vrr6j498chrqhz8g-libdrm-2.4.75-d… Thus, VLC's lib/libvlccore.so.8.0.0 as well as mesa-noglu's nix-support/propagated-native-build-inputs cause the dependency.
2017-09-10 TypoEelco Dolstra1-1/+1
2017-09-10 nix build: Only download the requested derivation outputsEelco Dolstra5-18/+52
Also some refactoring.
2017-09-08 LegacySSHStore: Include signatures etc.Eelco Dolstra3-1/+10
2017-09-08 nix copy: Add --substitute flagEelco Dolstra1-1/+9
2017-09-08 ThreadPool: On exception, interrupt the other worker threadsEelco Dolstra4-7/+14
2017-09-08 copyStorePath(): Fill in missing narHash regardless of checkSigsEelco Dolstra1-5/+1
I don't remember what the reasoning was here, but security is provided by the signatures, not by whether the hash is provided by the other store.
2017-09-08 nix copy: Don't open the --from store twiceEelco Dolstra1-1/+1
2017-09-08 ThreadPool: Improve exception handlingEelco Dolstra2-24/+53
In particular, process() won't return as long as there are active items. This prevents work item lambdas from referring to stack frames that no longer exist.
2017-09-08 nix copy: Add examplesEelco Dolstra1-1/+9
2017-09-08 Communicate the structured log FD to buildersEelco Dolstra1-1/+6
Since we may use a dedicated file descriptor in the future, this allows us to change it. So builders can do if [[ -n $NIX_LOG_FD ]]; then echo "@nix { message... }" >&$NIX_LOG_FD fi
2017-09-07 nix search: Add examplesEelco Dolstra1-0/+18
2017-09-07 nix eval: Add examplesEelco Dolstra1-0/+18
2017-09-07 nix log: Add examplesEelco Dolstra1-1/+19
2017-09-07 nix build: Add examplesEelco Dolstra1-0/+14
2017-09-07 nix run: Add some examplesEelco Dolstra1-0/+18
2017-09-06 nix build: Add --out-link and --no-link optionsEelco Dolstra4-10/+37
2017-09-06 nix build: Create result symlinksEelco Dolstra4-42/+105
2017-09-06 Fix abort when the GC thread gets an exceptionEelco Dolstra1-13/+21
2017-09-05 Add automatic garbage collectionEelco Dolstra5-1/+123
Nix can now automatically run the garbage collector during builds or while adding paths to the store. The option "min-free = <bytes>" specifies that Nix should run the garbage collector whenever free space in the Nix store drops below <bytes>. It will then delete garbage until "max-free" bytes are available. Garbage collection during builds is asynchronous; running builds are not paused and new builds are not blocked. However, there also is a synchronous GC run prior to the first build/substitution. Currently, no old GC roots are deleted (as in "nix-collect-garbage -d").
2017-09-05 GC: Don't delete own temproots fileEelco Dolstra3-34/+34
Since file locks are per-process rather than per-file-descriptor, the garbage collector would always acquire a lock on its own temproots file and conclude that it's stale.
2017-09-01 Abort curl downloads if there is no progress for 5 minutesEelco Dolstra1-1/+6
Maybe this will fix the curl hangs on macOS. (We could also use CURLOPT_TIMEOUT but that seems more of a sledgehammer.)
2017-09-01 Fix verbosity level for nix build --dry-runEelco Dolstra4-16/+16
2017-08-31 src/libmain/stack.cc: fix 'ucontext' usage on glibc-2.26Sergei Trofimovich1-2/+2
Build fails as: $ make CXX src/libmain/stack.o src/libmain/stack.cc: In function 'void nix::sigsegvHandler(int, siginfo_t*, void*)': src/libmain/stack.cc:21:21: error: 'ucontext' was not declared in this scope sp = (char *) ((ucontext *) ctx)->uc_mcontext.gregs[REG_RSP]; ^~~~~~~~ src/libmain/stack.cc:21:21: note: suggested alternative: 'ucontext_t' sp = (char *) ((ucontext *) ctx)->uc_mcontext.gregs[REG_RSP]; ^~~~~~~~ ucontext_t It's caused by upstream rename: https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=251287734e89a52da3db682a8241eb6bccc050c9 which basically changes typedef struct ucontext {} ucontext_t; to typedef struct ucontext_t {} ucontext_t; The change uses ucontext_t. Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org>
2017-08-31 Call queryMissing() prior to buildingEelco Dolstra2-0/+14
Without this, substitute info is fetched sequentially, which is superslow. In the old UI (e.g. nix-build), we call printMissing(), which calls queryMissing(), thereby preheating the binary cache cache. But the new UI doesn't do that.
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-11-17T23·49+0000