depot - monorepo for the virus lounge

Age	Commit message (Collapse)	Author	Files	Lines
2014-08-13	Use regular file GC roots if possible	Eelco Dolstra	1	-1/+4
	This makes hydra-eval-jobs create roots as regular files. See 1c208f2b7ef8ffb5e6d435d703dad83223a67bd6.
2014-08-01	findRoots(): Prevent a call to lstat()	Eelco Dolstra	1	-9/+14
	This means that getting the roots from /nix/var/nix/.../hydra-roots doesn't need any I/O other than reading the directory.
2014-08-01	Make readDirectory() return inode / file type	Eelco Dolstra	1	-8/+6

2014-08-01	Allow regular files as GC roots	Eelco Dolstra	1	-0/+6
	If a root is a regular file, then its name must denote a store path. For instance, the existence of the file /nix/var/nix/gcroots/per-user/eelco/hydra-roots/wzc3cy1wwwd6d0dgxpa77ijr1yp50s6v-libxml2-2.7.7 would cause /nix/store/wzc3cy1wwwd6d0dgxpa77ijr1yp50s6v-libxml2-2.7.7 to be a root. This is useful because it involves less I/O (no need for a readlink() call) and takes up less disk space (the symlink target typically takes up a full disk block, while directory entries are packed more efficiently). This is particularly important for hydra.nixos.org, which has hundreds of thousands of roots, and where reading the roots can take 25 minutes.
2014-03-30	boost::shared_ptr -> std::shared_ptr	Eelco Dolstra	1	-3/+1

2014-02-27	Set up a minimal /dev in chroots	Eelco Dolstra	1	-7/+5
	Not bind-mounting the /dev from the host also solves the problem with /dev/shm being a symlink to something not in the chroot.
2014-01-21	Fix some clang warnings	Eelco Dolstra	1	-2/+0

2013-12-10	Garbage collector: Release locks on temporary root files	Eelco Dolstra	1	-0/+1
	This allows processes waiting for such locks to proceed during the trash deletion phase of the garbage collector.
2013-11-14	Remove nix-setuid-helper	Eelco Dolstra	1	-1/+1
	AFAIK, nobody uses it, it's not maintained, and it has no tests.
2013-10-22	For auto roots, show the intermediate link	Eelco Dolstra	1	-1/+1
	I.e. "nix-store -q --roots" will now show (for example) /home/eelco/Dev/nixpkgs/result rather than /nix/var/nix/gcroots/auto/53222qsppi12s2hkap8dm2lg8xhhyk6v
2013-07-12	Garbage collector: Don't follow symlinks arbitrarily	Eelco Dolstra	1	-41/+39
	Only indirect roots (symlinks to symlinks to the Nix store) are now supported.
2013-04-26	addAdditionalRoots(): Check each path only once	Eelco Dolstra	1	-2/+2

2013-03-08	Revert "Prevent config.h from being clobbered"	Eelco Dolstra	1	-1/+4
	This reverts commit 28bba8c44f484eae38e8a15dcec73cfa999156f6.
2013-03-07	Prevent config.h from being clobbered	Eelco Dolstra	1	-4/+1

2013-01-05	Delete a left-over trash directory before doing a GC	Eelco Dolstra	1	-1/+4

2013-01-04	Fix "0 store paths deleted" message	Eelco Dolstra	1	-0/+2

2013-01-03	Clear any immutable bits in the Nix store	Eelco Dolstra	1	-2/+0
	Doing this once makes subsequent operations like garbage collecting more efficient since we don't have to call makeMutable() first.
2012-12-20	Yet another rewrite of the garbage collector	Eelco Dolstra	1	-130/+133
	But this time it's obviously correct! No more segfaults due to infinite recursions for sure, etc. Also, move directories to /nix/store/trash instead of renaming them to /nix/store/bla-gc-<pid>. Then we can just delete /nix/store/trash at the end.
2012-12-19	If gc-keep-derivations is set, only keep the actual deriver	Eelco Dolstra	1	-1/+1
	This prevents zillions of derivations from being kept, and fixes an infinite recursion in the garbage collector (due to an obscure cycle that can occur with fixed-output derivations).
2012-09-19	Templatise tokenizeString()	Eelco Dolstra	1	-1/+1

2012-09-13	Vacuum the SQLite DB after running the garbage collector	Eelco Dolstra	1	-0/+3

2012-09-13	Delete paths in a component in topologically sorted order	Eelco Dolstra	1	-1/+3
	The outputs of a derivation can refer to each other (even though they cannot have cycles), so they have to be deleted in the right order. http://hydra.nixos.org/build/3026118
2012-09-12	Handle gc-keep-outputs and gc-keep-derivations both enabled	Eelco Dolstra	1	-94/+97
	If the options gc-keep-outputs and gc-keep-derivations are both enabled, you can get a cycle in the liveness graph. There was a hack to handle this, but it didn't work with multiple-output derivations, causing the garbage collector to fail with errors like ‘error: cannot delete path `...' because it is in use by `...'’. The garbage collector now handles strongly connected components in the liveness graph as a unit and decides whether to delete all or none of the paths in an SCC.
2012-08-27	Merge branch 'master' into no-manifests	Eelco Dolstra	1	-8/+24

2012-08-05	Handle amount of disk space saved by hard linking being negative	Eelco Dolstra	1	-2/+2
	Fixes bogus messages like "currently hard linking saves 17592186044416.00 MiB".
2012-08-01	Count bytes freed deleting unused links	Eelco Dolstra	1	-2/+4

2012-08-01	Drop the block count in the garbage collector	Eelco Dolstra	1	-3/+2

2012-08-01	nix-store --gc: Make ‘--max-freed 0’ do the right thing	Eelco Dolstra	1	-2/+2
	That is, delete almost nothing (it will still remove unused links from /nix/store/.links).
2012-08-01	removeUnusedLinks(): Print stats on disk space saved by hard linking	Eelco Dolstra	1	-1/+16

2012-07-30	Refactor settings processing	Eelco Dolstra	1	-39/+39
	Put all Nix configuration flags in a Settings object.
2012-07-23	Garbage collect unused links in /nix/store/.links	Eelco Dolstra	1	-0/+37
	Incremental optimisation requires creating links in /nix/store/.links to all files in the store. However, this means that if we delete a store path, no files are actually deleted because links in /nix/store/.links still exists. So we need to check /nix/store/.links for files with a link count of 1 and delete them.
2012-04-16	Fix obscure race condition in GC root creation	Eelco Dolstra	1	-16/+18
	This should fix rare Hydra errors of the form: error: symlinking `/nix/var/nix/gcroots/per-user/hydra/hydra-roots/7sfhs5fdmjxm8sqgcpd0pgcsmz1kq0l0-nixos-iso-0.1pre33785-33795' to `/nix/store/7sfhs5fdmjxm8sqgcpd0pgcsmz1kq0l0-nixos-iso-0.1pre33785-33795': File exists
2012-03-26	Mac OS X fix	Eelco Dolstra	1	-0/+3

2012-03-26	Delete non-directory valid paths right away	Eelco Dolstra	1	-16/+27
	It's unlikely that rename() is faster than unlink() on a regular file or symlink, so don't bother.
2012-03-26	Make the garbage collector more concurrent	Eelco Dolstra	1	-11/+42
	Make the garbage collector more concurrent by deleting valid paths outside the region where we're holding the global GC lock. This should greatly reduce the time during which new builds are blocked, since the deletion accounts for the vast majority of the time spent in the GC. To ensure that this is safe, the valid paths are invalidated and renamed to some arbitrary path while we're holding the lock. This ensures that we when we finally delete the path, it's not a (newly) valid or locked path.
2012-03-26	Remove the --max-links GC option	Eelco Dolstra	1	-11/+1
	We don't need this anymore now that current filesystems support more than 32,000 files in a directory.
2012-03-13	Ensure that Perl processes delete their entry in the temproots directory	Eelco Dolstra	1	-0/+12
	By moving the destructor object to libstore.so, it's also run when download-using-manifests and nix-prefetch-url exit. This prevents them from cluttering /nix/var/nix/temproots with stale files.
2012-03-05	Set the close-on-exec flag on file descriptors	Eelco Dolstra	1	-0/+1

2012-02-15	Don't use deletePath() to delete a single file	Eelco Dolstra	1	-1/+1

2012-02-09	Use data() instead of c_str() where appropriate	Eelco Dolstra	1	-1/+1

2011-12-30	* Move topoSortPaths() out of gc.cc.	Eelco Dolstra	1	-36/+0

2011-12-30	* Reject a build if there is a cycle among the outputs. This is	Eelco Dolstra	1	-4/+10
	necessary because existing code assumes that the references graph is acyclic.
2011-12-22	* Oops.	Eelco Dolstra	1	-1/+1

2011-12-22	* In the garbage collector, delete invalid paths before deleting	Eelco Dolstra	1	-11/+35
	unreachable paths. This matters when using --max-freed etc.: unreachable paths could become reachable again, so it's nicer to keep them if there is "real" garbage to be deleted. Also, don't use readDirectory() but read the Nix store and delete invalid paths in parallel. This reduces GC latency on very large Nix stores.
2011-12-22	* Drop unnecessary call to canonPath() (nixStore is already canonical).	Eelco Dolstra	1	-1/+1

2011-08-31	* Eliminate all uses of the global variable ‘store’ from libstore.	Eelco Dolstra	1	-24/+24
	This should also fix: nix-instantiate: ./../boost/shared_ptr.hpp:254: T* boost::shared_ptr<T>::operator->() const [with T = nix::StoreAPI]: Assertion `px != 0' failed. which was caused by hashDerivationModulo() calling the ‘store’ object (during store upgrades) before openStore() assigned it.
2010-10-14	* Wrap deleteFromStore() in a transaction. Otherwise there might be a	Eelco Dolstra	1	-3/+5
	race with other processes that add new referrers to a path, resulting in the garbage collector crashing with "foreign key constraint failed". (Nix/4) * Make --gc --print-dead etc. interruptible.
2010-08-31	`nix-store --verify' improvements:	Eelco Dolstra	1	-2/+1
	* If a path has disappeared, check its referrers first, and don't try to invalidate paths that have valid referrers. Otherwise we get a foreign key constraint violation. * Read the whole Nix store directory instead of statting each valid path, which is slower. * Acquire the global GC lock.
2010-08-30	* When using the build hook, distinguish between the stderr of the	Eelco Dolstra	1	-1/+1
	hook script proper, and the stdout/stderr of the builder. Only the latter should be saved in /nix/var/log/nix/drvs. * Allow the verbosity to be set through an option. * Added a flag --quiet to lower the verbosity level.
2010-08-30	* Disable the GC reachability check for now (when creating new roots),	Eelco Dolstra	1	-1/+1
	as it's hopelessly inefficient.