about summary refs log tree commit diff
path: root/scripts/build-remote.pl.in
AgeCommit message (Collapse)AuthorFilesLines
2012-04-30 Support mandatory system features in the build hookEelco Dolstra1-4/+8
Mandatory features are features that MUST be present in a derivation's requiredSystemFeatures attribute. One application is performance testing, where we have a dedicated machine to run performance tests (and nothing else). Then we would add the label "perf" to the machine's mandatory features and to the performance testing derivations.
2012-04-24 Fix a warning in the build hook about $progressViewerEelco Dolstra1-1/+1
2012-03-05 build-remote.pl: don't wait forever for the upload lockEelco Dolstra1-3/+17
In the build hook, don't wait forever to get the upload lock. This ensures progress if another process gets stuck while holding the upload lock.
2011-12-21 * Security: make sure the lock files used by build-remote.pl are notEelco Dolstra1-5/+5
readable to other users. Otherwise, any user can open the lock file for reading and lock it, thus DoSing the remote build mechanism.
2011-12-21 * Use a lock to ensure that only one build-remote instance can copy aEelco Dolstra1-1/+10
closure to a given machine at the same time. This prevents the case where multiple instances try to copy the same missing store path to the target machine, which is very wasteful.
2011-11-29 * Get rid of the shell in ssh calls.Eelco Dolstra1-2/+2
2011-11-23 * Add an API function exportPaths() that provides the functionality ofEelco Dolstra1-2/+2
‘nix-store --export’. * Add a Perl module that provides the functionality of ‘nix-copy-closure --to’. This is used by build-remote.pl so it no longer needs to start a separate nix-copy-closure process. Also, it uses the Perl API to do the export, so it doesn't need to start a separate nix-store process either. As a result, nix-copy-closure and build-remote.pl should no longer fail on very large closures due to an "Argument list too long" error. (Note that having very many dependencies in a single derivation can still fail because the environment can become too large. Can't be helped though.)
2011-11-23 * build-remote.pl: drop a hard-coded reference to /nix/etc/nix.Eelco Dolstra1-1/+2
2011-10-11 * Move SSH.pm.Eelco Dolstra1-2/+2
2010-12-13 * nix-instantiate: return exit status 100 to denote a permanent buildEelco Dolstra1-7/+7
failure. The build hook can use this to distinguish between transient and permanent failures on the remote side.
2010-12-05 * Use CamelCase for the Perl modules.Eelco Dolstra1-1/+1
2010-10-04 * Don't use SSH's `-tt' flag because it doesn't seem to workEelco Dolstra1-6/+10
on OpenSolaris when using connection sharing. Instead have the remote side check for disconnection and kill the process group when that happens.
2010-08-31 * Always print hook output on stderr, even if --no-build-output isEelco Dolstra1-2/+5
set. * In the build hook, print a trace message to allow Hydra to pick up the name of the remote machine used for the build.
2010-08-30 * When using the build hook, distinguish between the stderr of theEelco Dolstra1-2/+2
hook script proper, and the stdout/stderr of the builder. Only the latter should be saved in /nix/var/log/nix/drvs. * Allow the verbosity to be set through an option. * Added a flag --quiet to lower the verbosity level.
2010-08-27 * Experimental feature: allow a derivation to tell the build hook thatEelco Dolstra1-10/+17
it requires a certain feature on the build machine, e.g. requiredSystemFeatures = [ "kvm" ]; We need this in Hydra to make sure that builds that require KVM support are forwarded to machines that have KVM support. Probably this should also be enforced for local builds.
2010-08-25 * Made the build hook mechanism more efficient. Rather than startingEelco Dolstra1-125/+125
the hook every time we want to ask whether we can run a remote build (which can be very often), we now reuse a hook process for answering those queries until it accepts a build. So if there are N derivations to be built, at most N hooks will be started.
2010-08-25 * Don't call "cat".Eelco Dolstra1-7/+4
2010-08-25 * In the build hook, temporarily register the derivation and itsEelco Dolstra1-5/+23
output as GC roots. This prevents a race if the garbage collector is running during the build.
2010-06-22 * Pass `--fallback' to the remote build to ignore failingEelco Dolstra1-2/+2
substituters.
2010-02-04 * Doh! The scope of $slotLock should extend to the end of the script,Eelco Dolstra1-1/+2
because otherwise the lock will be released at the end of the while loop.
2010-02-04 * Typo.Eelco Dolstra1-1/+1
2010-02-04 * Revert r19796 for now.Eelco Dolstra1-1/+1
2010-02-03 * Respect @sysconfdir@.Eelco Dolstra1-1/+1
2010-02-03 * In the build hook, if connecting to a machine fails, try the otherEelco Dolstra1-70/+73
machines of the right type (if available). This makes the build farm more robust to failures.
2010-02-03 * Use SSH connection sharing in the remote build script.Eelco Dolstra1-36/+16
* Removed the Cygwin password hack since the problem is apparently fixed in Visual Studio.
2010-01-25 * Disable gzip compression in build-remote.pl because it puts too muchEelco Dolstra1-2/+2
load on the Hydra build farm (where it's unnecessary anyway because it has a fast connection to the build machines). In any case, compression can be enabled by using the `-C' option to ssh.
2009-11-17 * Don't pass -K. It should really inherit the setting of the callingEelco Dolstra1-1/+1
Nix though.
2009-09-17 (no commit message)Eelco Dolstra1-0/+1
2009-09-17 * build-remote.pl: Pick machines in a round-robin order, rather thanEelco Dolstra1-23/+67
giving jobs to the first machine until it hits its job limit, then the second machine and so on. This should improve utilisation of the Hydra build farm a lot. Also take an optional speed factor into account to cause fast machines to be preferred over slower machines with a similar load.
2009-09-17 * build-remote.pl: allow the system type to be a comma-separated listEelco Dolstra1-5/+4
of system types. Don't treat the x86_64-linux system type specially.
2009-04-21 * Machines of type x86_64-linux can do i686-linux builds.Eelco Dolstra1-1/+3
2009-03-29 * Update the mtime on the lock file to make it easy to see when a slotEelco Dolstra1-4/+8
was last used.
2009-03-29 * Use polling to wait for a remote build slot when using a build hookEelco Dolstra1-5/+2
(that is, call the build hook with a certain interval until it accepts the build). * build-remote.pl was totally broken: for all system types other than the local system type, it would send all builds to the *first* machine of the appropriate type.
2009-03-28 * nix-copy-closure: compute the closure in one call to nix-store,Eelco Dolstra1-1/+1
which is much faster.
2009-03-28 * Simplify communication with the hook a bit (don't use fileEelco Dolstra1-7/+2
descriptors 3/4, just use stdin/stderr).
2009-03-28 * Clean up the output a bit.Eelco Dolstra1-5/+5
2009-02-03 * Build hook: compress the transferred data.Eelco Dolstra1-2/+2
2009-02-02 * Build hooks: use nix-store --import. This prevents a redundantEelco Dolstra1-13/+2
scan for runtime dependencies (i.e. the local machine shouldn't do a scan that the remote machine has already done). Also pipe directly into `nix-store --import': don't use a temporary file.
2009-01-13 * When using a build hook, distinguish between transient failuresEelco Dolstra1-2/+10
(e.g. an SSH connection problem) and permanent failures (i.e. the builder failed). This matters to Hydra (it wants to know whether it makes sense to retry a build).
2008-12-04 (no commit message)Eelco Dolstra1-2/+1
2008-12-04 * Propagate --max-silent-time to remote machines.Eelco Dolstra1-2/+6
2008-12-04 * Prefer building on a remote machine over a local machine. ThisEelco Dolstra1-5/+6
makes more sense for the build farm, otherwise every nix-store invocation will lead to at least one local build. Will come up with a better solution later...
2008-12-04 * When using build hooks, for any nix-store -r build operation, it isEelco Dolstra1-6/+6
necessary that at least one build hook doesn't return "postpone", otherwise nix-store will barf ("waiting for a build slot, yet there are no running children"). So inform the build hook when this is the case, so that it can start a build even when that would exceed the maximum load on a machine.
2008-12-04 * Force allocation of a pseudo-terminal to clean up the remoteEelco Dolstra1-1/+6
nix-store process when the connection is interrupted.
2008-12-04 * Support multiple system types per remote machine, e.g. a machineEelco Dolstra1-24/+24
list like root@example.org x86_64-linux /root/.ssh/id_buildfarm 1 root@example.org i686-darwin /root/.ssh/id_buildfarm 1 This is possible when the Nix installation on example.org itself has remote builds enabled.
2007-11-15 * Add build-remote.pl to the Nix distribution.Eelco Dolstra1-0/+208