about summary refs log tree commit diff
path: root/src/libutil
AgeCommit message (Collapse)AuthorFilesLines
2013-08-07 Run the daemon worker on the same CPU as the clientEelco Dolstra3-2/+65
On a system with multiple CPUs, running Nix operations through the daemon is significantly slower than "direct" mode: $ NIX_REMOTE= nix-instantiate '<nixos>' -A system real 0m0.974s user 0m0.875s sys 0m0.088s $ NIX_REMOTE=daemon nix-instantiate '<nixos>' -A system real 0m2.118s user 0m1.463s sys 0m0.218s The main reason seems to be that the client and the worker get moved to a different CPU after every call to the worker. This patch adds a hack to lock them to the same CPU. With this, the overhead of going through the daemon is very small: $ NIX_REMOTE=daemon nix-instantiate '<nixos>' -A system real 0m1.074s user 0m0.809s sys 0m0.098s
2013-07-30 killUser: Don't let the child kill itself on AppleShea Levy1-0/+13
The kill(2) in Apple's libc follows POSIX semantics, which means that kill(-1, SIGKILL) will kill the calling process too. Since nix has no way to distinguish between the process successfully killing everything and the process being killed by a rogue builder in that case, it can't safely conclude that killUser was successful. Luckily, the actual kill syscall takes a parameter that determines whether POSIX semantics are followed, so we can call that syscall directly and avoid the issue on Apple. Signed-off-by: Shea Levy <shea@shealevy.com>
2013-07-12 Garbage collector: Don't follow symlinks arbitrarilyEelco Dolstra2-0/+13
Only indirect roots (symlinks to symlinks to the Nix store) are now supported.
2013-06-20 Don't keep "disabled" substituters runningEelco Dolstra1-0/+1
For instance, it's pointless to keep copy-from-other-stores running if there are no other stores, or download-using-manifests if there are no manifests. This also speeds things up because we don't send queries to those substituters.
2013-06-07 Process stderr from substituters while doing have/info queriesEelco Dolstra2-0/+8
2013-01-03 Remove tabsEelco Dolstra2-46/+46
2013-01-03 Clear any immutable bits in the Nix storeEelco Dolstra4-64/+2
Doing this once makes subsequent operations like garbage collecting more efficient since we don't have to call makeMutable() first.
2012-11-26 Make "nix-build -A <derivation>.<output>" do the right thingEelco Dolstra2-1/+14
For example, given a derivation with outputs "out", "man" and "bin": $ nix-build -A pkg produces ./result pointing to the "out" output; $ nix-build -A pkg.man produces ./result-man pointing to the "man" output; $ nix-build -A pkg.all produces ./result, ./result-man and ./result-bin; $ nix-build -A pkg.all -A pkg2 produces ./result, ./result-man, ./result-bin and ./result-2.
2012-11-15 Don't use std::cerr in a few placesEelco Dolstra2-16/+23
Slightly scared of using std::cerr in a vforked process...
2012-11-15 Add some debug codeEelco Dolstra1-1/+1
2012-11-09 Use vfork() instead of fork() if availableEelco Dolstra2-3/+14
Hopefully this reduces the chance of hitting ‘unable to fork: Cannot allocate memory’ errors. vfork() is used for everything except starting builders.
2012-11-09 Remove some redundant close() callsEelco Dolstra1-10/+8
They are unnecessary because we set the close-on-exec flag.
2012-11-09 Remove the quickExit functionEelco Dolstra2-13/+3
2012-10-23 If hashes do not match, print them in base-32 for SHA-1/SHA-256Eelco Dolstra2-0/+9
Fixes #57.
2012-10-04 XML writer: flush after newlinesEelco Dolstra1-4/+4
This is useful for hydra-eval-jobs.
2012-09-28 Handle octal escapes in /proc/self/mountinfoEelco Dolstra2-0/+21
2012-09-19 Remove setting of the immutable bitEelco Dolstra2-26/+2
Using the immutable bit is problematic, especially in conjunction with store optimisation. For instance, if the garbage collector deletes a file, it has to clear its immutable bit, but if the file has additional hard links, we can't set the bit afterwards because we don't know the remaining paths. So now that we support having the entire Nix store as a read-only mount, we may as well drop the immutable bit. Unfortunately, we have to keep the code to clear the immutable bit for backwards compatibility.
2012-09-19 Templatise tokenizeString()Eelco Dolstra2-3/+6
2012-09-11 Include config.h before any other headerEelco Dolstra1-0/+2
"config.h" must be included first, because otherwise the compiler might not see the right value of _FILE_OFFSET_BITS. We've had this before; see 705868a8a96a10f70e629433cfffc2d5cd2703eb. In this case, GCC would compute a different address for ‘settings.useSubstitutes’ in misc.cc because of the off_t in ‘settings’. Reverts 3854fc9b42d16b810f62b64194b699033b03aaf1. http://hydra.nixos.org/build/3016700
2012-08-27 Merge branch 'master' into no-manifestsEelco Dolstra2-18/+15
2012-08-20 In the chroot, make all mounted filesystems privateEelco Dolstra2-3/+3
This is required on systemd, which mounts filesystems as "shared" subtrees. Changes to shared trees in a private mount namespace are propagated to the outside world, which is bad.
2012-08-12 Add some basic profiling support to the evaluatorEelco Dolstra1-0/+3
Setting the environment variable NIX_COUNT_CALLS to 1 enables some basic profiling in the evaluator. It will count calls to functions and primops as well as evaluations of attributes. For example, to see where evaluation of a NixOS configuration spends its time: $ NIX_SHOW_STATS=1 NIX_COUNT_CALLS=1 ./src/nix-instantiate/nix-instantiate '<nixos>' -A system --readonly-mode ... calls to 39 primops: 239532 head 233962 tail 191252 hasAttr ... calls to 1595 functions: 224157 `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/pkgs/lib/lists.nix:17:19' 221767 `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/pkgs/lib/lists.nix:17:14' 221767 `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/pkgs/lib/lists.nix:17:10' ... evaluations of 7088 attributes: 167377 undefined position 132459 `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/pkgs/lib/attrsets.nix:119:41' 47322 `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/pkgs/lib/attrsets.nix:13:21' ...
2012-08-01 Drop the block count in the garbage collectorEelco Dolstra2-15/+9
2012-08-01 DohEelco Dolstra1-1/+1
2012-08-01 Report substituter errors to clients of the Nix daemonEelco Dolstra2-1/+12
2012-07-26 Set permissions on temporary build directories to 0700Eelco Dolstra2-3/+3
Fixes #39.
2012-07-18 Use "#pragma once" to prevent repeated header file inclusionEelco Dolstra7-34/+7
2012-06-25 When using chroots, use a private PID namespaceEelco Dolstra1-2/+5
In a private PID namespace, processes have PIDs that are separate from the rest of the system. The initial child gets PID 1. Processes in the chroot cannot see processes outside of the chroot. This improves isolation between builds. However, processes on the outside can see processes in the chroot and send signals to them (if they have appropriate rights). Since the builder gets PID 1, it serves as the reaper for zombies in the chroot. This might turn out to be a problem. In that case we'll need to have a small PID 1 process that sits in a loop calling wait().
2012-03-18 Drop the externals directoryEelco Dolstra1-1/+3
Nix now requires SQLite and bzip2 to be pre-installed. SQLite is detected using pkg-config. We required DBD::SQLite anyway, so depending on SQLite is not a big problem. The --with-bzip2, --with-openssl and --with-sqlite flags are gone.
2012-03-05 Set the close-on-exec flag on file descriptorsEelco Dolstra2-0/+15
2012-02-15 On Linux, make the Nix store really read-only by using the immutable bitEelco Dolstra4-2/+91
I was bitten one time too many by Python modifying the Nix store by creating *.pyc files when run as root. On Linux, we can prevent this by setting the immutable bit on files and directories (as in ‘chattr +i’). This isn't supported by all filesystems, so it's not an error if setting the bit fails. The immutable bit is cleared by the garbage collector before deleting a path. The only tricky aspect is in optimiseStore(), since it's forbidden to create hard links to an immutable file. Thus optimiseStore() temporarily clears the immutable bit before creating the link.
2012-02-09 Use data() instead of c_str() where appropriateEelco Dolstra3-7/+7
2011-12-22 * In the garbage collector, delete invalid paths before deletingEelco Dolstra2-1/+10
unreachable paths. This matters when using --max-freed etc.: unreachable paths could become reachable again, so it's nicer to keep them if there is "real" garbage to be deleted. Also, don't use readDirectory() but read the Nix store and delete invalid paths in parallel. This reduces GC latency on very large Nix stores.
2011-12-16 * importPath() -> importPaths(). Because of buffering of the inputEelco Dolstra2-7/+13
stream it's now necessary for the daemon to process the entire sequence of exported paths, rather than letting the client do it.
2011-12-16 * Avoid expensive conversions from char arrays to STL strings.Eelco Dolstra2-3/+20
2011-12-16 * Make the import operation through the daemon much more efficientEelco Dolstra2-29/+44
(way fewer roundtrips) by allowing the client to send data in bigger chunks. * Some refactoring.
2011-12-16 * Clean up exception handling.Eelco Dolstra2-3/+10
2011-12-15 * Refactoring: move sink/source buffering into separate classes.Eelco Dolstra4-76/+116
* Buffer the HashSink. This speeds up hashing a bit because it prevents lots of calls to the hash update functions (e.g. nix-hash went from 9.3s to 8.7s of user time on the closure of my /var/run/current-system).
2011-12-15 * Buffer reads in FdSource. Together with write buffering, thisEelco Dolstra2-10/+32
significantly cuts down the number of syscalls (e.g., for "nix-store -qR /var/run/current-system" via the daemon, it reduced the number of syscalls in the client from 29134 to 4766 and in the daemon from 44266 to 20666).
2011-12-14 * Buffer writes in FdSink. This significantly reduces the number ofEelco Dolstra2-7/+37
system calls / context switches when dumping a NAR and in the worker protocol.
2011-12-02 * Move parseHash16or32 into libutil, and use in nix-hash.Eelco Dolstra2-0/+19
2011-02-09 * Sync with the trunk.Eelco Dolstra1-2/+0
2011-01-14 Remove useless <config.h> inclusion from public header.Ludovic Courtès1-2/+0
2010-12-13 * nix-instantiate: return exit status 100 to denote a permanent buildEelco Dolstra2-3/+5
failure. The build hook can use this to distinguish between transient and permanent failures on the remote side.
2010-12-13 * createDirs(path): if path already exists, make sure it's aEelco Dolstra2-20/+26
directory. * Provide a C++ wrapper around lstat().
2010-11-16 * Store the size of a store path in the database (to be precise, theEelco Dolstra2-6/+11
size of the NAR serialisation of the path, i.e., `nix-store --dump PATH'). This is useful for Hydra.
2010-11-16 * Sync with the trunk.Eelco Dolstra1-0/+2
2010-10-04 * Make sure that config.h is included before the system headers,Eelco Dolstra1-0/+2
because it defines _FILE_OFFSET_BITS. Without this, on OpenSolaris the system headers define it to be 32, and then the 32-bit stat() ends up being called with a 64-bit "struct stat", or vice versa. This also ensures that we get 64-bit file sizes everywhere. * Remove the redundant call to stat() in parseExprFromFile(). The file cannot be a symlink because that's the exit condition of the loop before.
2010-10-04 * Sync with the trunk.Eelco Dolstra1-1/+10
2010-10-04 * In printMsg(), ignore failing writes to stderr if we're in anEelco Dolstra1-1/+10
exception handler, otherwise throw an exception. We need to ignore write errors in exception handlers to ensure that cleanup code runs to completion if the other side of stderr has been closed unexpectedly.
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-12-23T10·40+0000