depot - monorepo for the virus lounge

Age	Commit message (Collapse)	Author	Files	Lines
2004-06-25	* A flag `--keep-going / -k' to keep building goals if one fails, as	Eelco Dolstra	5	-139/+167
	much as possible. (This is similar to GNU Make's `-k' flag.) * Refactoring to implement this: previously we just bombed out when a build failed, but now we have to clean up. In particular this means that goals must be freed quickly --- they shouldn't hang around until the worker exits. So the worker now maintains weak pointers in order not to prevent garbage collection. * Documented the `-k' and `-j' flags.
2004-06-25	* Don't throw an exception when a build fails. Just terminate the	Eelco Dolstra	1	-58/+214
	goal and allow the problem to be handled elsewhere (e.g., at top-level).
2004-06-24	* Obsolete.	Eelco Dolstra	1	-162/+0

2004-06-24	* Multiple and/or failing substitutes now work.	Eelco Dolstra	1	-12/+42

2004-06-22	* Arghhhhhh	Eelco Dolstra	1	-1/+1

2004-06-22	* Some more diagnostics changes.	Eelco Dolstra	2	-25/+41

2004-06-22	* Well, it's better than printf.	Eelco Dolstra	1	-1/+1

2004-06-22	* Started making Nix's diagnostic messages a bit more useful.	Eelco Dolstra	1	-10/+26

2004-06-22	* Put WEXITSTATUS stuff somewhere else.	Eelco Dolstra	4	-3/+11

2004-06-22	* Reduce gratuitous cut & pasting.	Eelco Dolstra	1	-68/+55

2004-06-22	* Wrapper class around pids.	Eelco Dolstra	4	-87/+162

2004-06-22	* Substitutes should occupy a build slot.	Eelco Dolstra	1	-1/+15

2004-06-22	* Refactoring.	Eelco Dolstra	4	-20/+25

2004-06-21	* Remove debug output.	Eelco Dolstra	1	-1/+0

2004-06-21	* Acquire a lock on the output path when running a substitute. Also	Eelco Dolstra	1	-0/+22
	delete obstructing invalid paths.
2004-06-21	* Ugh, nasty Heisenbug due to an uninitialiased variable. The bug	Eelco Dolstra	1	-0/+1
	only caused a crash if the program was not invoked with a high verbosity level.
2004-06-21	* Remove obstructing invalid store paths add[Text]ToStore().	Eelco Dolstra	1	-0/+6

2004-06-21	* Wrap calls to registerSubstitute() in a single transaction to	Eelco Dolstra	3	-11/+13
	improve throughput. * Don't build the `substitute-rev' table for now, since it caused Theta(N^2) time and log file consumption when adding N substitutes. Maybe we can do without it.
2004-06-21	* Disable calls to fsync() since Berkeley DB's DB_TXN_WRITE_NOSYNC	Eelco Dolstra	1	-0/+8
	flag doesn't seem to work as advertised.
2004-06-20	* Re-enable support for substitutes in the normaliser.	Eelco Dolstra	5	-107/+383
	* A better substitute mechanism. Instead of generating a store expression for each store path for which we have a substitute, we can have a single store expression that builds a generic program that is invoked to build the desired store path, which is passed as an argument. This means that operations like `nix-pull' only produce O(1) files instead of O(N) files in the store when registering N substitutes. (It consumes O(N) database storage, of course, but that's not a performance problem). * Added a test for the substitute mechanism. * `nix-store --substitute' reads the substitutes from standard input, instead of from the command line. This prevents us from running into the kernel's limit on command line length.
2004-06-20	* Refactoring.	Eelco Dolstra	3	-41/+52

2004-06-19	* Re-enable build hooks.	Eelco Dolstra	1	-72/+325

2004-06-18	* Big refactoring. Move to a much more explicitly state machine based	Eelco Dolstra	3	-764/+869
	approach. This makes it much easier to add extra complexity in the normaliser / realiser (e.g., build hooks, substitutes).
2004-06-18	* This is also useful.	Eelco Dolstra	3	-1/+263

2004-06-18	* Shared (garbage collecting) pointers. Copied from Boost.	Eelco Dolstra	12	-17/+1268

2004-06-15	* Refactoring.	Eelco Dolstra	3	-83/+57

2004-06-08	* Cleanup.	Eelco Dolstra	1	-10/+17

2004-05-18	* Drain the output of the build hook to show error messages. Ugly	Eelco Dolstra	1	-4/+26
	hack.
2004-05-18	* setpgrp() is not POSIX (and on Mac OS X it's different than on	Eelco Dolstra	1	-1/+1
	Linux), so use setpgid().
2004-05-14	* execl() requires a terminating 0.	Eelco Dolstra	1	-10/+21
	* When a fast build wakes up a goal, try to start that goal in the same iteration of the startBuild() loop of run(). Otherwise no job might be started until the next job terminates.
2004-05-13	* Distributed builds and load balancing now seem to work pretty well.	Eelco Dolstra	1	-24/+74
	(Though the `build-remote.pl' script has a gigantic race condition).
2004-05-13	* Load balancing. `build-remote.pl' will only execute up to a	Eelco Dolstra	1	-1/+1
	per-machine maximum number of parallel jobs on a remote machine.
2004-05-13	* The build hooks used to implement distributed builds can now be run	Eelco Dolstra	2	-134/+352
	in parallel. Hooks are more efficient: locks on output paths are only acquired when the hook says that it is willing to accept a build job. Hooks now work in two phases. First, they should first tell Nix whether they are willing to accept a job. Nix guarantuees that no two hooks will ever be in the first phase at the same time (this simplifies the implementation of hooks, since they don't have to perform locking (?)). Second, if they accept a job, they are then responsible for building it (on the remote system), and copying the result back. These can be run in parallel with other hooks and locally executed jobs. The implementation is a bit messy right now, though. * The directory `distributed' shows a (hacky) example of a hook that distributes build jobs over a set of machines listed in a configuration file.
2004-05-12	* A switch `-j NUMBER' to set the maximum number of parallel jobs (0 =	Eelco Dolstra	4	-8/+29
	no limit). * Add missing file to distribution.
2004-05-12	* Pass to the build hook all sorts of information useful for	Eelco Dolstra	1	-6/+41
	distributing a build action to another machine. In particular, the paths in the input closures, the output paths, and successor mapping for sub-derivations.
2004-05-12	* An quick and dirty hack to support distributed builds.	Eelco Dolstra	5	-11/+33

2004-05-11	* True parallel builds. Nix can now run as many build jobs in	Eelco Dolstra	8	-288/+676
	parallel as possible (similar to GNU Make's `-j' switch). This is useful on SMP systems, but it is especially useful for doing builds on multiple machines. The idea is that a large derivation is initiated on one master machine, which then distributes sub-derivations to any number of slave machines. This should not happen synchronously or in lock-step, so the master must be capable of dealing with multiple parallel build jobs. We now have the infrastructure to support this. TODO: substitutes are currently broken.
2004-05-11	* Ignore interrupt signals while handling an exception.	Eelco Dolstra	1	-4/+15
	* Ignore EINTR in reads and writes.
2004-05-04	* Allow the location of the store etc. to be specified using	Eelco Dolstra	1	-12/+19
	environment variables. * Started adding some automatic tests. * Do a `make check' when building RPMs.
2004-04-23	* Pass SYSTEM through config.h, and allow spaces.	Eelco Dolstra	2	-1/+3

2004-04-22	* Don't create $(localstatedir)/nix/profiles if --disable-init-state	Eelco Dolstra	1	-3/+0
	is specified.
2004-04-21	* Do initialise state (the DB etc.) when doing a `make install',	Eelco Dolstra	1	-7/+0
	unless `--disable-init-state' is passed to configure.
2004-04-21	* Recurse into attribute sets and lists when getting derivations from	Eelco Dolstra	1	-0/+4
	an expression.
2004-04-14	* Be stricter in verifying store paths.	Eelco Dolstra	3	-4/+9

2004-04-05	* When something goes wrong in the evaluation of a Nix expression,	Eelco Dolstra	8	-89/+200
	print a nice backtrace of the stack, rather than vomiting a gigantic (and useless) aterm on the screen. Example: error: while evaluating file `.../pkgs/system/test.nix': while evaluating attribute `subversion' at `.../pkgs/system/all-packages-generic.nix', line 533: while evaluating function at `.../pkgs/applications/version-management/subversion/default.nix', line 1: assertion failed at `.../pkgs/applications/version-management/subversion/default.nix', line 13 Since the Nix expression language is lazy, the trace may be misleading. The purpose is to provide a hint as to the location of the problem.
2004-04-02	* Print a more useful error message in case of an invalid derivation	Eelco Dolstra	1	-1/+6
	binding.
2004-03-30	* The recent change in nixpkgs of calling `stdenv.mkDerivation'	Eelco Dolstra	2	-2/+11
	instead of `derivation' triggered a huge slowdown in the Nix expression evaluator. Total execution time of `nix-env -qa' went up by a factor of 60 or so. This scalability problem was caused by expressions such as (x: y: ... x ...) a b where `a' is a large term (say, the one in `all-packages-generic.nix'). Then the first beta-reduction would produce (y: ... a ...) b by substituting `a' for `x'. The second beta-reduction would then substitute `b' for `y' into the body `... a ...', which is a large term due to `a', and thus causes a large traversal to be performed by substitute() in the second reduction. This is however entirely redundant, since `a' cannot contain free variables (since we never substitute below a weak head normal form). The solution is to wrap substituted terms into a `Closed' constructor, i.e., subst(subs, Var(x)) = Closed(e) iff subs[x] = e have substitution not descent into closed terms, subst(subs, Closed(x)) = Closed(x) and otherwise ignore them for evaluation, eval(Closed(x)) = eval(x). * Fix a typo that caused incorrect substitutions to be performed in simple lambdas, e.g., `(x: x: x) a' would reduce to `(x: a)'.
2004-03-28	* Added an operator `?' to test for attribute existence, e.g.,	Eelco Dolstra	2	-1/+10
	`attrs ? x' yields true iff `attrs' has an attribute named `x'.
2004-03-28	* Added an operator `~' to select paths within a derivation. E.g.,	Eelco Dolstra	3	-0/+16
	{stdenv, bash}: derivation { builder = bash ~ /bin/sh; args = ["-e" "-x" ./builder.sh]; ... } Here the attribute `builder' will evaluate to, e.g., `/nix/store/1234abcd...-bash-2.0.1/bin/sh'.
2004-03-28	* Added plain lambdas, e.g., `let { id = x: x; const = x: y: x; }'.	Eelco Dolstra	5	-37/+63
	`bla:' is now no longer parsed as a URL. * Re-enabled support for the `args' attribute in derivations to specify command line arguments to the builder, e.g., ... builder = /usr/bin/python; args = ["-c" ./builder.py]; ...