depot - monorepo for the virus lounge

Age	Commit message (Collapse)	Author	Files	Lines
2014-02-26	Only start download-via-ssh if it's enabled	Eelco Dolstra	1	-12/+14

2014-02-26	Add use-ssh-substituter setting.	Shea Levy	2	-0/+5
	It defaults to false and can be overridden by RemoteStore. Untested currently, just quickly put this together
2014-02-18	Slight simplification	Eelco Dolstra	1	-17/+10

2014-02-18	Add a flag ‘--check’ to verify build determinism	Eelco Dolstra	5	-49/+86
	The flag ‘--check’ to ‘nix-store -r’ or ‘nix-build’ will cause Nix to redo the build of a derivation whose output paths are already valid. If the new output differs from the original output, an error is printed. This makes it easier to test if a build is deterministic. (Obviously this cannot catch all sources of non-determinism, but it catches the most common one, namely the current time.) For example: $ nix-build '<nixpkgs>' -A patchelf ... $ nix-build '<nixpkgs>' -A patchelf --check error: derivation `/nix/store/1ipvxsdnbhl1rw6siz6x92s7sc8nwkkb-patchelf-0.6' may not be deterministic: hash mismatch in output `/nix/store/4pc1dmw5xkwmc6q3gdc9i5nbjl4dkjpp-patchelf-0.6.drv' The --check build fails if not all outputs are valid. Thus the first call to nix-build is necessary to ensure that all outputs are valid. The current outputs are left untouched: the new outputs are either put in a chroot or diverted to a different location in the store using hash rewriting.
2014-02-17	Make --repair work on Darwin	Eelco Dolstra	1	-3/+3
	Mac OS X doesn't allow renaming a read-only directory. http://hydra.nixos.org/build/9113895
2014-02-17	Refactoring	Eelco Dolstra	1	-29/+36

2014-02-17	When using a build hook, only copy missing paths	Eelco Dolstra	1	-9/+6

2014-02-17	Move some code around	Eelco Dolstra	1	-92/+82
	In particular, do replacing of valid paths during repair later. This prevents us from replacing a valid path after the build fails.
2014-02-17	Heuristically detect if a build may have failed due to a full disk	Eelco Dolstra	2	-5/+28
	This will allow Hydra to detect that a build should not be marked as "permanently failed", allowing it to be retried later.
2014-02-14	Minor style fixes	Eelco Dolstra	2	-2/+2

2014-02-10	Move StoreApi::serve into opServe	Shea Levy	2	-38/+0
	Signed-off-by: Shea Levy <shea@shealevy.com>
2014-02-10	Pass in params by const ref	Shea Levy	2	-2/+2
	Signed-off-by: Shea Levy <shea@shealevy.com>
2014-02-08	Add download-via-ssh substituter	Shea Levy	2	-0/+26
	This substituter connects to a remote host, runs nix-store --serve there, and then forwards substituter commands on to the remote host and sends their results to the calling program. The ssh-substituter-hosts option can be specified as a list of hosts to try. This is an initial implementation and, while it works, it has some limitations: * Only the first host is used * There is no caching of query results (all queries are sent to the remote machine) * There is no informative output (such as progress bars) * Some failure modes may cause unhelpful error messages * There is no concept of trusted-ssh-substituter-hosts Signed-off-by: Shea Levy <shea@shealevy.com>
2014-02-08	nix-store --serve: Flush out after every loop	Shea Levy	2	-2/+3
	Signed-off-by: Shea Levy <shea@shealevy.com>
2014-02-08	nix-store --serve: Use dump instead of export	Shea Levy	2	-3/+4
	Also remove signing support Signed-off-by: Shea Levy <shea@shealevy.com>
2014-02-08	nix-store --serve: Don't fail if asked for info about non-valid path	Shea Levy	1	-0/+2
	Signed-off-by: Shea Levy <shea@shealevy.com>
2014-02-08	nix-store --serve: Don't loop forever	Shea Levy	1	-26/+25
	nix-store --export takes a tmproot, which can only release by exiting. Substituters don't currently work in a way that could take advantage of the looping, anyway. Signed-off-by: Shea Levy <shea@shealevy.com>
2014-02-08	Add the nix-store --serve command	Shea Levy	2	-0/+35
	This is essentially the substituter API operating on the local store, which will be used by the ssh substituter. It runs in a loop rather than just taking one command so that in the future nix will be able to keep one connection open for multiple instances of the substituter. Signed-off-by: Shea Levy <shea@shealevy.com>
2014-02-03	Fix assertion failure in ‘nix-store --load-db’	Eelco Dolstra	1	-1/+12
	Namely: nix-store: derivations.cc:242: nix::Hash nix::hashDerivationModulo(nix::StoreAPI&, nix::Derivation): Assertion `store.isValidPath(i->first)' failed. This happened because of the derivation output correctness check being applied before the references of a derivation are valid.
2014-02-01	Give a friendly error message if the DB directory is not writable	Eelco Dolstra	1	-2/+6
	Previously we would say "error: setting synchronous mode: unable to open database file" which isn't very helpful.
2014-02-01	Fix "make dist"	Eelco Dolstra	1	-1/+3

2014-02-01	Remove Automakefiles	Eelco Dolstra	1	-33/+0

2014-02-01	Update Makefile variable names	Eelco Dolstra	1	-1/+1

2014-01-30	Rename Makefile -> local.mk	Eelco Dolstra	1	-0/+0

2014-01-21	Fix some clang warnings	Eelco Dolstra	2	-6/+1

2014-01-21	Merge branch 'master' into make	Eelco Dolstra	2	-2/+7
	Conflicts: src/libexpr/eval.cc
2014-01-09	Generate schema.sql.hh	Eelco Dolstra	1	-0/+5

2014-01-09	Update Makefiles	Eelco Dolstra	1	-3/+1

2014-01-08	Fix checking of NAR hashes	Eelco Dolstra	1	-0/+1
	headdesk headdesk headdesk So since commit 22144afa8d9f8968da351618a1347072a93bd8aa, Nix hasn't actually checked whether the content of a downloaded NAR matches the hash specified in the manifest / NAR info file. Urghhh...
2014-01-06	wording	Domen Kožar	1	-1/+1

2013-12-10	Garbage collector: Release locks on temporary root files	Eelco Dolstra	1	-0/+1
	This allows processes waiting for such locks to proceed during the trash deletion phase of the garbage collector.
2013-12-05	Print a trace message if a build fails due to the platform being unknown	Eelco Dolstra	1	-1/+4

2013-11-25	Rename Makefile.new -> Makefile	Eelco Dolstra	1	-0/+0

2013-11-23	Use libnix as a prefix for all Nix libraries	Eelco Dolstra	1	-0/+2
	In particular "libutil" was always a problem because it collides with Glibc's libutil. Even if we install into $(libdir)/nix, the linker sometimes got confused (e.g. if a program links against libstore but not libutil, then ld would report undefined symbols in libstore because it was looking at Glibc's libutil).
2013-11-23	Allow (dynamic) libraries to depend on other libraries	Eelco Dolstra	1	-0/+2

2013-11-22	Drop the dependency on Automake	Eelco Dolstra	2	-2/+2

2013-11-22	Rename $(here) to $(d) for brevity, and remove trailing slash	Eelco Dolstra	1	-1/+1

2013-11-22	Add ‘make dist’ support	Eelco Dolstra	1	-1/+1

2013-11-22	New non-recursive, plain Make-based build system	Eelco Dolstra	1	-0/+19

2013-11-22	Include <cstring> for memset	Eelco Dolstra	1	-0/+1
	This should fix building on Illumos.
2013-11-14	Remove nix-setuid-helper	Eelco Dolstra	4	-158/+31
	AFAIK, nobody uses it, it's not maintained, and it has no tests.
2013-11-12	Make function calls show up in stack traces again	Eelco Dolstra	2	-0/+4
	Note that adding --show-trace prevents functions calls from being tail-recursive, so an expression that evaluates without --show-trace may fail with a stack overflow if --show-trace is given.
2013-10-22	For auto roots, show the intermediate link	Eelco Dolstra	1	-1/+1
	I.e. "nix-store -q --roots" will now show (for example) /home/eelco/Dev/nixpkgs/result rather than /nix/var/nix/gcroots/auto/53222qsppi12s2hkap8dm2lg8xhhyk6v
2013-10-16	Retry all SQLite operations	Eelco Dolstra	2	-128/+169
	To deal with SQLITE_PROTOCOL, we also need to retry read-only operations.
2013-10-16	Fix a race in registerFailedPath()	Eelco Dolstra	1	-2/+1
	Registering the path as failed can fail if another process does the same thing after the call to hasPathFailed(). This is extremely unlikely though.
2013-10-16	Convenience macros for retrying a SQLite transaction	Eelco Dolstra	1	-46/+42

2013-10-16	Don't wrap read-only queries in a transaction	Eelco Dolstra	1	-6/+0
	There is no risk of getting an inconsistent result here: if the ID returned by queryValidPathId() is deleted from the database concurrently, subsequent queries involving that ID will simply fail (since IDs are never reused).
2013-10-16	Print a distinct warning for SQLITE_PROTOCOL	Eelco Dolstra	1	-4/+8

2013-10-16	Treat SQLITE_PROTOCOL as SQLITE_BUSY	Eelco Dolstra	1	-1/+1
	In the Hydra build farm we fairly regularly get SQLITE_PROTOCOL errors (e.g., "querying path in database: locking protocol"). The docs for this error code say that it "is returned if some other process is messing with file locks and has violated the file locking protocol that SQLite uses on its rollback journal files." However, the SQLite source code reveals that this error can also occur under high load: if( cnt>5 ){ int nDelay = 1; /* Pause time in microseconds / if( cnt>100 ){ VVA_ONLY( pWal->lockError = 1; ) return SQLITE_PROTOCOL; } if( cnt>=10 ) nDelay = (cnt-9)238; /* Max delay 21ms. Total delay 996ms */ sqlite3OsSleep(pWal->pVfs, nDelay); } i.e. if certain locks cannot be not acquired, SQLite will retry a number of times before giving up and returing SQLITE_PROTOCOL. The comments say: Circumstances that cause a RETRY should only last for the briefest instances of time. No I/O or other system calls are done while the locks are held, so the locks should not be held for very long. But if we are unlucky, another process that is holding a lock might get paged out or take a page-fault that is time-consuming to resolve, during the few nanoseconds that it is holding the lock. In that case, it might take longer than normal for the lock to free. ... The total delay time before giving up is less than 1 second. On a heavily loaded machine like lucifer (the main Hydra server), which often has dozens of processes waiting for I/O, it seems to me that a page fault could easily take more than a second to resolve. So, let's treat SQLITE_PROTOCOL as SQLITE_BUSY and retry the transaction. Issue NixOS/hydra#14.
2013-09-06	Don't apply the CPU affinity hack to nix-shell (and other Perl programs)	Eelco Dolstra	3	-1/+5
	As discovered by Todd Veldhuizen, the shell started by nix-shell has its affinity set to a single CPU. This is because nix-shell connects to the Nix daemon, which causes the affinity hack to be applied. So we turn this off for Perl programs.