about summary refs log tree commit diff
path: root/src/libstore/build.cc
AgeCommit message (Collapse)AuthorFilesLines
2017-04-14 Shut up some warningsEelco Dolstra1-7/+7
2017-04-13 Convert Settings to the new config systemEelco Dolstra1-7/+4
This makes all config options self-documenting. Unknown or unparseable config settings and --option flags now cause a warning.
2017-04-13 Merge branch 'rework-options' of https://github.com/copumpkin/nixEelco Dolstra1-32/+11
2017-04-13 Add a Config class to simplify adding configuration settingsEelco Dolstra1-5/+3
The typical use is to inherit Config and add Setting<T> members: class MyClass : private Config { Setting<int> foo{this, 123, "foo", "the number of foos to use"}; Setting<std::string> bar{this, "blabla", "bar", "the name of the bar"}; MyClass() : Config(readConfigFile("/etc/my-app.conf")) { std::cout << foo << "\n"; // will print 123 unless overriden } }; Currently, this is used by Store and its subclasses for store parameters. You now get a warning if you specify a non-existant store parameter in a store URI.
2017-04-04 Allow default sandbox paths to be overridenEelco Dolstra1-1/+1
E.g. you can now redirect /etc/resolv.conf to a different file.
2017-04-04 Make /var/run/nscd/socket optionalEelco Dolstra1-1/+2
Not every distribution uses nscd.
2017-03-31 Sandbox: Fix /dev/ptmx on recent kernelsEelco Dolstra1-8/+1
This fixes "No such file or directory" when opening /dev/ptmx (e.g. http://hydra.nixos.org/build/51094249). The reason appears to be some changes to /dev/ptmx / /dev/pts handling between Linux 4.4 and 4.9. See https://patchwork.kernel.org/patch/7832531/. The fix is to go back to mounting a proper /dev/pts instance inside the sandbox. Happily, this now works inside user namespaces, even for unprivileged users. So NIX_REMOTE=local?root=/tmp/nix nix-build \ '<nixpkgs/nixos/tests/misc.nix>' -A test works for non-root users. The downside is that the fix breaks sandbox builds on older kernels (probably pre-4.6), since mounting a devpts fails inside user namespaces for some reason I've never been able to figure out. Builds on those systems will fail with error: while setting up the build environment: mounting /dev/pts: Invalid argument Ah well.
2017-03-16 Remove "killing process <pid>" messagesEelco Dolstra1-2/+2
They convey no useful information.
2017-03-15 Store: Add a method for getting build logsEelco Dolstra1-4/+1
This allows various Store implementations to provide different ways to get build logs. For example, BinaryCacheStore can get the build logs from the binary cache. Also, remove the log-servers option since we can use substituters for this.
2017-03-03 Fix fatal "broken pipe" error when $NIX_BUILD_HOOK is missingEelco Dolstra1-27/+39
2017-02-24 Register content-addressability assertion for fixed outputsEelco Dolstra1-1/+4
2017-02-22 Explicitly model all settings and fail on unrecognized onesDan Peebles1-32/+11
Previously, the Settings class allowed other code to query for string properties, which led to a proliferation of code all over the place making up new options without any sort of central registry of valid options. This commit pulls all those options back into the central Settings class and removes the public get() methods, to discourage future abuses like that. Furthermore, because we know the full set of options ahead of time, we now fail loudly if someone enters an unrecognized option, thus preventing subtle typos. With some template fun, we could probably also dump the full set of options (with documentation, defaults, etc.) to the command line, but I'm not doing that yet here.
2017-02-16 Support netrc in <nix/fetchurl.nix>Eelco Dolstra1-2/+12
This allows <nix/fetchurl.nix> to fetch private Git/Mercurial repositories, e.g. import <nix/fetchurl.nix> { url = https://edolstra@bitbucket.org/edolstra/my-private-repo/get/80a14018daed.tar.bz2; sha256 = "1mgqzn7biqkq3hf2697b0jc4wabkqhmzq2srdymjfa6sb9zb6qs7"; } where /etc/nix/netrc contains: machine bitbucket.org login edolstra password blabla... This works even when sandboxing is enabled. To do: add unpacking support (i.e. fetchzip functionality).
2017-02-08 Include config.h implicitly with '-include config.h' in CFLAGSTuomas Tynkkynen1-2/+0
Because config.h can #define things like _FILE_OFFSET_BITS=64 and not every compilation unit includes config.h, we currently compile half of Nix with _FILE_OFFSET_BITS=64 and other half with _FILE_OFFSET_BITS unset. This causes major havoc with the Settings class on e.g. 32-bit ARM, where different compilation units disagree with the struct layout. E.g.: diff --git a/src/libstore/globals.cc b/src/libstore/globals.cc @@ -166,6 +166,8 @@ void Settings::update() _get(useSubstitutes, "build-use-substitutes"); + fprintf(stderr, "at Settings::update(): &useSubstitutes = %p\n", &nix::settings.useSubstitutes); _get(buildUsersGroup, "build-users-group"); diff --git a/src/libstore/remote-store.cc b/src/libstore/remote-store.cc +++ b/src/libstore/remote-store.cc @@ -138,6 +138,8 @@ void RemoteStore::initConnection(Connection & conn) void RemoteStore::setOptions(Connection & conn) { + fprintf(stderr, "at RemoteStore::setOptions(): &useSubstitutes = %p\n", &nix::settings.useSubstitutes); conn.to << wopSetOptions Gave me: at Settings::update(): &useSubstitutes = 0xb6e5c5cb at RemoteStore::setOptions(): &useSubstitutes = 0xb6e5c5c7 That was not a fun one to debug!
2017-02-02 exportReferencesGraph: Only export in JSON format when in structured modeEelco Dolstra1-10/+16
This prevents breaking compatibility with builders that read "closure.*", since they would accidentally pick up the new JSON files.
2017-02-01 Restore default signal handling in child processesEelco Dolstra1-2/+2
In particular, this fixes Ctrl-C in nix-shell sessions.
2017-01-26 exportReferencesGraph: Export more complete info in JSON formatEelco Dolstra1-1/+11
This writes info about every path in the closure in the same format as ‘nix path-info --json’. Thus it also includes NAR hashes and sizes. Example: [ { "path": "/nix/store/10h6li26i7g6z3mdpvra09yyf10mmzdr-hello-2.10", "narHash": "sha256:0ckdc4z20kkmpqdilx0wl6cricxv90lh85xpv2qljppcmz6vzcxl", "narSize": 197648, "references": [ "/nix/store/10h6li26i7g6z3mdpvra09yyf10mmzdr-hello-2.10", "/nix/store/27binbdy296qvjycdgr1535v8872vz3z-glibc-2.24" ], "closureSize": 20939776 }, { "path": "/nix/store/27binbdy296qvjycdgr1535v8872vz3z-glibc-2.24", "narHash": "sha256:1nfn3m3p98y1c0kd0brp80dn9n5mycwgrk183j17rajya0h7gax3", "narSize": 20742128, "references": [ "/nix/store/27binbdy296qvjycdgr1535v8872vz3z-glibc-2.24" ], "closureSize": 20742128 } ] Fixes #1134.
2017-01-26 Add support for passing structured data to buildersEelco Dolstra1-18/+39
Previously, all derivation attributes had to be coerced into strings so that they could be passed via the environment. This is lossy (e.g. lists get flattened, necessitating configureFlags vs. configureFlagsArray, of which the latter cannot be specified as an attribute), doesn't support attribute sets at all, and has size limitations (necessitating hacks like passAsFile). This patch adds a new mode for passing attributes to builders, namely encoded as a JSON file ".attrs.json" in the current directory of the builder. This mode is activated via the special attribute __structuredAttrs = true; (The idea is that one day we can set this in stdenv.mkDerivation.) For example, stdenv.mkDerivation { __structuredAttrs = true; name = "foo"; buildInputs = [ pkgs.hello pkgs.cowsay ]; doCheck = true; hardening.format = false; } results in a ".attrs.json" file containing (sans the indentation): { "buildInputs": [], "builder": "/nix/store/ygl61ycpr2vjqrx775l1r2mw1g2rb754-bash-4.3-p48/bin/bash", "configureFlags": [ "--with-foo", "--with-bar=1 2" ], "doCheck": true, "hardening": { "format": false }, "name": "foo", "nativeBuildInputs": [ "/nix/store/10h6li26i7g6z3mdpvra09yyf10mmzdr-hello-2.10", "/nix/store/4jnvjin0r6wp6cv1hdm5jbkx3vinlcvk-cowsay-3.03" ], "propagatedBuildInputs": [], "propagatedNativeBuildInputs": [], "stdenv": "/nix/store/f3hw3p8armnzy6xhd4h8s7anfjrs15n2-stdenv", "system": "x86_64-linux" } "passAsFile" is ignored in this mode because it's not needed - large strings are included directly in the JSON representation. It is up to the builder to do something with the JSON representation. For example, in bash-based builders, lists/attrsets of string values could be mapped to bash (associative) arrays.
2017-01-26 UserLock: Fix multi-threaded access to a global variableEelco Dolstra1-30/+40
2017-01-26 UserLock: Make more RAII-ishEelco Dolstra1-53/+38
2017-01-26 Moving more code out of DerivationGoal::startBuilder()Eelco Dolstra1-100/+111
2017-01-26 Move exportReferencesGraph into a separate methodEelco Dolstra1-44/+55
startBuilder() is getting rather obese.
2017-01-19 Kill builds when we get EOF on the log FDEelco Dolstra1-11/+10
This closes a long-time bug that allowed builds to hang Nix indefinitely (regardless of timeouts) simply by doing exec > /dev/null 2>&1; while true; do true; done Now, on EOF, we just send SIGKILL to the child to make sure it's really gone.
2017-01-19 Use std::unique_ptr for HookInstanceEelco Dolstra1-5/+4
2017-01-09 Merge pull request #1139 from Mic92/masterEelco Dolstra1-6/+2
Simplify remouting with MS_PRIVATE in sandbox build
2017-01-02 Provide /var/run/nscd/socket in the sandboxEelco Dolstra1-0/+1
Otherwise sandbox builds can fail, e.g. $ NIX_REMOTE=local?root=/tmp/nix nix-build '<nixpkgs>' -A hello --option build-use-substitutes false ... downloading ‘http://ftpmirror.gnu.org/bash/bash-4.3-patches/bash43-047’... error: unable to download ‘http://ftpmirror.gnu.org/bash/bash-4.3-patches/bash43-047’: Couldn't resolve host name (6)
2016-12-22 Add commentEelco Dolstra1-1/+2
2016-12-19 Revert "Merge branch 'seccomp' of https://github.com/aszlig/nix"Eelco Dolstra1-65/+29
This reverts commit 9f3f2e21edb17dbcd674539dff96efb6cceca10c, reversing changes made to 47f587700d646f5b03a42f2fa57c28875a31efbe.
2016-12-19 Revert "Give root a valid home directory"Eelco Dolstra1-3/+3
This reverts commit ec7d498b72b0e283814c27cbc6c064496f95445d.
2016-12-15 Give root a valid home directoryEelco Dolstra1-3/+3
Some programs barf if the current user has a non-writable home directory, e.g. http://hydra.nixos.org/build/44818144.
2016-12-15 Merge branch 'seccomp' of https://github.com/aszlig/nixEelco Dolstra1-29/+65
2016-12-08 Don't delete .check directories of running buildsEelco Dolstra1-2/+2
We need to keep them around for diffoscope.
2016-12-08 Fix buildEelco Dolstra1-2/+2
http://hydra.nixos.org/build/44628517
2016-12-07 Add a hook to run diffoscope when non-determinism is detectedEelco Dolstra1-7/+32
2016-12-07 Keep track of the exact build start/stop timesEelco Dolstra1-0/+4
2016-12-07 Bail out early when non-determinism is detectedEelco Dolstra1-0/+1
2016-12-07 Add an option to make non-determinism non-fatalEelco Dolstra1-11/+13
That is, when build-repeat > 0, and the output of two rounds differ, then print a warning rather than fail the build. This is primarily to let Hydra check reproducibility of all packages.
2016-12-06 Use a steady clock for timeoutsEelco Dolstra1-22/+28
Fixes #1146.
2016-12-06 nix-store --serve: Suppress log output on stderr when repeating a buildEelco Dolstra1-1/+2
2016-11-26 Revert "Get rid of unicode quotes (#1140)"Eelco Dolstra1-101/+101
This reverts commit f78126bfd6b6c8477fcdbc09b2f98772dbe9a1e7. There really is no need for such a massive change...
2016-11-25 Get rid of unicode quotes (#1140)Guillaume Maudoux1-101/+101
2016-11-25 Simplify remouting with MS_PRIVATE in sandbox buildJörg Thalheim1-6/+2
also fix race condition if mounts are added after mountinfo is read.
2016-11-17 Update darwin build for optional sandbox pathsShea Levy1-3/+6
Fixes #1132
2016-11-16 seccomp: Forge return values for *chown32aszlig1-0/+4
These syscalls are only available in 32bit architectures, but libseccomp should handle them correctly even if we're on native architectures that do not have these syscalls. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-11-16 seccomp: Forge return codes for POSIX ACL syscallsaszlig1-0/+4
Commands such as "cp -p" also use fsetxattr() in addition to fchown(), so we need to make sure these syscalls always return successful as well in order to avoid nasty "Invalid value" errors. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-11-16 libstore/build: Forge chown() to return successaszlig1-0/+41
What we basically want is a seccomp mode 2 BPF program like this but for every architecture: BPF_STMT(BPF_LD+BPF_W+BPF_ABS, offsetof(struct seccomp_data, nr)), BPF_JUMP(BPF_JMP+BPF_JEQ+BPF_K, __NR_chown, 4, 0), BPF_JUMP(BPF_JMP+BPF_JEQ+BPF_K, __NR_fchown, 3, 0), BPF_JUMP(BPF_JMP+BPF_JEQ+BPF_K, __NR_fchownat, 2, 0), BPF_JUMP(BPF_JMP+BPF_JEQ+BPF_K, __NR_lchown, 1, 0), BPF_STMT(BPF_RET+BPF_K, SECCOMP_RET_ALLOW), BPF_STMT(BPF_RET+BPF_K, SECCOMP_RET_ERRNO) However, on 32 bit architectures we do have chown32, lchown32 and fchown32, so we'd need to add all the architecture blurb which libseccomp handles for us. So we only need to make sure that we add the 32bit seccomp arch while we're on x86_64 and otherwise we just stay at the native architecture which was set during seccomp_init(), which more or less replicates setting 32bit personality during runChild(). The FORCE_SUCCESS() macro here could be a bit less ugly but I think repeating the seccomp_rule_add() all over the place is way uglier. Another way would have been to create a vector of syscalls to iterate over, but that would make error messages uglier because we can either only print the (libseccomp-internal) syscall number or use seccomp_syscall_resolve_num_arch() to get the name or even make the vector a pair number/name, essentially duplicating everything again. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-11-16 Run builds as root in user namespace againaszlig1-29/+16
This reverts commit ff0c0b645cc1448959126185bb2fafe41cf0bddf. We're going to use seccomp to allow "cp -p" and force chown-related syscalls to always return 0. Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-11-14 buildPaths(): Handle ecIncompleteClosureEelco Dolstra1-1/+1
buildPaths() on a non-derivation would incorrectly not throw an error if the path didn't have a substitute.
2016-11-10 Store::computeFSClosure(): Support a set of pathsEelco Dolstra1-2/+1
This way, callers can exploits the parallelism of computeFSClosure() when they have multiple paths that they need the (combined) closure of.
2016-11-08 TypoEelco Dolstra1-1/+1
generated by cgit-pink 1.4.1 (git 2.44.2) at 2025-01-07T22·29+0000