depot - monorepo for the virus lounge

Age	Commit message (Collapse)	Author	Files	Lines
2014-07-24	nix-copy-closure: Drop --bzip2, --xz, --show-progress	Eelco Dolstra	1	-1/+1
	These are too difficult to implement via nix-store --serve. ‘--show-progress’ could be re-implemented fairly easily via a sink/source wrapper class.
2014-07-24	Remove obsolete SSH master connection code	Eelco Dolstra	1	-1/+1

2014-07-24	Implement nix-copy-closure --from via nix-store --serve	Eelco Dolstra	1	-1/+2

2014-07-24	build-remote.pl: Be less verbose on failing builds	Eelco Dolstra	1	-1/+2

2014-07-12	build-remote.pl: Fix build log	Eelco Dolstra	1	-1/+1

2014-07-11	build-remote.pl: Use ‘nix-store --serve’ on the remote side	Eelco Dolstra	1	-46/+22
	This makes things more efficient (we don't need to use an SSH master connection, and we only start a single remote process) and gets rid of locking issues (the remote nix-store process will keep inputs and outputs locked as long as they're needed). It also makes it more or less secure to connect directly to the root account on the build machine, using a forced command (e.g. ‘command="nix-store --serve --write"’). This bypasses the Nix daemon and is therefore more efficient. Also, don't call nix-store to import the output paths.
2013-09-18	build-remote.pl: Don't use substituters on the remote	Eelco Dolstra	1	-1/+1
	It's kinda pointless to check substituters on the remote side, since we just checked them locally.
2013-07-18	Revert "build-remote.pl: Enforce timeouts locally"	Eelco Dolstra	1	-2/+3
	This reverts commit 69b8f9980f39c14a59365a188b300a34d625a2cd. The timeout should be enforced remotely. Otherwise, if the garbage collector is running either locally or remotely, if will block the build or closure copying for some time. If the garbage collector takes too long, the build may time out, which is not what we want. Also, on heavily loaded systems, copying large paths to and from the remote machine can take a long time, also potentially resulting in a timeout.
2013-07-11	build-remote.pl: Move "building ..." message to a better place	Eelco Dolstra	1	-1/+1

2013-05-29	build-remote.pl: Remove meaningless signing when importing the output paths	Eelco Dolstra	1	-3/+1
	The "$UID != 0" makes no sense: if the local side has write access to the Nix store (which is always the case) then it doesn't matter if we're root - we can import unsigned paths either way.
2013-05-23	build-remote.pl: Copy all outputs in one operation	Eelco Dolstra	1	-5/+5

2013-05-23	build-remote.pl: Indicate if remote machine is refusing builds	Eelco Dolstra	1	-2/+3
	Fixes #120.
2013-05-10	build-remote.pl: Properly close the SSH connection between attempts	Eelco Dolstra	1	-3/+4

2013-05-10	build-remote.pl: Pass /dev/null as SSH's stdin	Eelco Dolstra	1	-1/+1
	Otherwise it will set the parent's stdin to non-blocking mode, causing the subsequent read of the set of inputs/outputs to fail randomly. That's insane.
2013-05-10	build-remote.pl: Allow a machine to refuse a build	Eelco Dolstra	1	-1/+3
	Before selecting a machine, build-remote.pl will try to run the command "nix-builds-inhibited" on the machine. If this command exists and returns a 0 exit code, then the machine won't be used. It's up to the user to provide this command, but it would typically be a script that checks whether there is enough disk space and whether the load is not too high.
2013-05-09	build-remote.pl: Create one process fewer on the remote side	Eelco Dolstra	1	-1/+1

2013-05-09	build-remote.pl: Enforce timeouts locally	Eelco Dolstra	1	-3/+2
	Don't pass --timeout / --max-silent-time to the remote builder. Instead, let the local Nix process terminate the build if it exceeds a timeout. The remote builder will be killed as a side-effect. This gives better error reporting (since the timeout message from the remote side wasn't properly propagated) and handles non-Nix problems like SSH hangs.
2013-03-08	Revert "Prevent config.h from being clobbered"	Eelco Dolstra	1	-2/+3
	This reverts commit 28bba8c44f484eae38e8a15dcec73cfa999156f6.
2013-03-07	Prevent config.h from being clobbered	Eelco Dolstra	1	-3/+2

2013-02-19	build-remote: Use the --quiet flag	Eelco Dolstra	1	-1/+1
	‘--option verbosity 0’ doesn't actually do anything.
2013-01-21	build-remote.pl: Don't keep build logs on the build slave	Eelco Dolstra	1	-2/+3

2012-09-12	Build hook: Pass the timeout to the remote builder	Eelco Dolstra	1	-3/+4
	Note that this will only work if the client has a very recent Nix version (post 15e1b2c223494ecb5efefc3ea0e3b926a6b1d7dc), otherwise the --option flag will just be ignored. Fixes #50.
2012-09-11	Support building a derivation if some outputs are already valid	Eelco Dolstra	1	-11/+12
	This handles the chroot and build hook cases, which are easy. Supporting the non-chroot-build case will require more work (hash rewriting!). Issue #21.
2012-04-30	Support mandatory system features in the build hook	Eelco Dolstra	1	-4/+8
	Mandatory features are features that MUST be present in a derivation's requiredSystemFeatures attribute. One application is performance testing, where we have a dedicated machine to run performance tests (and nothing else). Then we would add the label "perf" to the machine's mandatory features and to the performance testing derivations.
2012-04-24	Fix a warning in the build hook about $progressViewer	Eelco Dolstra	1	-1/+1

2012-03-05	build-remote.pl: don't wait forever for the upload lock	Eelco Dolstra	1	-3/+17
	In the build hook, don't wait forever to get the upload lock. This ensures progress if another process gets stuck while holding the upload lock.
2011-12-21	* Security: make sure the lock files used by build-remote.pl are not	Eelco Dolstra	1	-5/+5
	readable to other users. Otherwise, any user can open the lock file for reading and lock it, thus DoSing the remote build mechanism.
2011-12-21	* Use a lock to ensure that only one build-remote instance can copy a	Eelco Dolstra	1	-1/+10
	closure to a given machine at the same time. This prevents the case where multiple instances try to copy the same missing store path to the target machine, which is very wasteful.
2011-11-29	* Get rid of the shell in ssh calls.	Eelco Dolstra	1	-2/+2

2011-11-23	* Add an API function exportPaths() that provides the functionality of	Eelco Dolstra	1	-2/+2
	‘nix-store --export’. * Add a Perl module that provides the functionality of ‘nix-copy-closure --to’. This is used by build-remote.pl so it no longer needs to start a separate nix-copy-closure process. Also, it uses the Perl API to do the export, so it doesn't need to start a separate nix-store process either. As a result, nix-copy-closure and build-remote.pl should no longer fail on very large closures due to an "Argument list too long" error. (Note that having very many dependencies in a single derivation can still fail because the environment can become too large. Can't be helped though.)
2011-11-23	* build-remote.pl: drop a hard-coded reference to /nix/etc/nix.	Eelco Dolstra	1	-1/+2

2011-10-11	* Move SSH.pm.	Eelco Dolstra	1	-2/+2

2010-12-13	* nix-instantiate: return exit status 100 to denote a permanent build	Eelco Dolstra	1	-7/+7
	failure. The build hook can use this to distinguish between transient and permanent failures on the remote side.
2010-12-05	* Use CamelCase for the Perl modules.	Eelco Dolstra	1	-1/+1

2010-10-04	* Don't use SSH's `-tt' flag because it doesn't seem to work	Eelco Dolstra	1	-6/+10
	on OpenSolaris when using connection sharing. Instead have the remote side check for disconnection and kill the process group when that happens.
2010-08-31	* Always print hook output on stderr, even if --no-build-output is	Eelco Dolstra	1	-2/+5
	set. * In the build hook, print a trace message to allow Hydra to pick up the name of the remote machine used for the build.
2010-08-30	* When using the build hook, distinguish between the stderr of the	Eelco Dolstra	1	-2/+2
	hook script proper, and the stdout/stderr of the builder. Only the latter should be saved in /nix/var/log/nix/drvs. * Allow the verbosity to be set through an option. * Added a flag --quiet to lower the verbosity level.
2010-08-27	* Experimental feature: allow a derivation to tell the build hook that	Eelco Dolstra	1	-10/+17
	it requires a certain feature on the build machine, e.g. requiredSystemFeatures = [ "kvm" ]; We need this in Hydra to make sure that builds that require KVM support are forwarded to machines that have KVM support. Probably this should also be enforced for local builds.
2010-08-25	* Made the build hook mechanism more efficient. Rather than starting	Eelco Dolstra	1	-125/+125
	the hook every time we want to ask whether we can run a remote build (which can be very often), we now reuse a hook process for answering those queries until it accepts a build. So if there are N derivations to be built, at most N hooks will be started.
2010-08-25	* Don't call "cat".	Eelco Dolstra	1	-7/+4

2010-08-25	* In the build hook, temporarily register the derivation and its	Eelco Dolstra	1	-5/+23
	output as GC roots. This prevents a race if the garbage collector is running during the build.
2010-06-22	* Pass `--fallback' to the remote build to ignore failing	Eelco Dolstra	1	-2/+2
	substituters.
2010-02-04	* Doh! The scope of $slotLock should extend to the end of the script,	Eelco Dolstra	1	-1/+2
	because otherwise the lock will be released at the end of the while loop.
2010-02-04	* Typo.	Eelco Dolstra	1	-1/+1

2010-02-04	* Revert r19796 for now.	Eelco Dolstra	1	-1/+1

2010-02-03	* Respect @sysconfdir@.	Eelco Dolstra	1	-1/+1

2010-02-03	* In the build hook, if connecting to a machine fails, try the other	Eelco Dolstra	1	-70/+73
	machines of the right type (if available). This makes the build farm more robust to failures.
2010-02-03	* Use SSH connection sharing in the remote build script.	Eelco Dolstra	1	-36/+16
	* Removed the Cygwin password hack since the problem is apparently fixed in Visual Studio.
2010-01-25	* Disable gzip compression in build-remote.pl because it puts too much	Eelco Dolstra	1	-2/+2
	load on the Hydra build farm (where it's unnecessary anyway because it has a fast connection to the build machines). In any case, compression can be enabled by using the `-C' option to ssh.
2009-11-17	* Don't pass -K. It should really inherit the setting of the calling	Eelco Dolstra	1	-1/+1
	Nix though.