Age | Commit message (Collapse) | Author | Files | Lines |
|
Also makes it robust against concurrent deletions.
|
|
|
|
|
|
Calling a class an API is a bit redundant...
|
|
Also, move a few free-standing functions into StoreAPI and Derivation.
Also, introduce a non-nullable smart pointer, ref<T>, which is just a
wrapper around std::shared_ptr ensuring that the pointer is never
null. (For reference-counted values, this is better than passing a
"T&", because the latter doesn't maintain the refcount. Usually, the
caller will have a shared_ptr keeping the value alive, but that's not
always the case, e.g., when passing a reference to a std::thread via
std::bind.)
|
|
|
|
Issue #564.
|
|
|
|
http://hydra.nixos.org/build/17862041
|
|
Especially in WAL mode on a highly loaded machine, this is not a good
idea because it results in a WAL file of approximately the same size
ad the database, which apparently cannot be deleted while anybody is
accessing it.
|
|
|
|
|
|
|
|
|
|
This makes hydra-eval-jobs create roots as regular files. See
1c208f2b7ef8ffb5e6d435d703dad83223a67bd6.
|
|
This means that getting the roots from /nix/var/nix/.../hydra-roots
doesn't need any I/O other than reading the directory.
|
|
|
|
If a root is a regular file, then its name must denote a store
path. For instance, the existence of the file
/nix/var/nix/gcroots/per-user/eelco/hydra-roots/wzc3cy1wwwd6d0dgxpa77ijr1yp50s6v-libxml2-2.7.7
would cause
/nix/store/wzc3cy1wwwd6d0dgxpa77ijr1yp50s6v-libxml2-2.7.7
to be a root.
This is useful because it involves less I/O (no need for a readlink()
call) and takes up less disk space (the symlink target typically takes
up a full disk block, while directory entries are packed more
efficiently). This is particularly important for hydra.nixos.org,
which has hundreds of thousands of roots, and where reading the roots
can take 25 minutes.
|
|
|
|
Not bind-mounting the /dev from the host also solves the problem with
/dev/shm being a symlink to something not in the chroot.
|
|
|
|
This allows processes waiting for such locks to proceed during the
trash deletion phase of the garbage collector.
|
|
AFAIK, nobody uses it, it's not maintained, and it has no tests.
|
|
I.e. "nix-store -q --roots" will now show (for example)
/home/eelco/Dev/nixpkgs/result
rather than
/nix/var/nix/gcroots/auto/53222qsppi12s2hkap8dm2lg8xhhyk6v
|
|
Only indirect roots (symlinks to symlinks to the Nix store) are now
supported.
|
|
|
|
This reverts commit 28bba8c44f484eae38e8a15dcec73cfa999156f6.
|
|
|
|
|
|
|
|
Doing this once makes subsequent operations like garbage collecting
more efficient since we don't have to call makeMutable() first.
|
|
But this time it's *obviously* correct! No more segfaults due to
infinite recursions for sure, etc.
Also, move directories to /nix/store/trash instead of renaming them to
/nix/store/bla-gc-<pid>. Then we can just delete /nix/store/trash at
the end.
|
|
This prevents zillions of derivations from being kept, and fixes an
infinite recursion in the garbage collector (due to an obscure cycle
that can occur with fixed-output derivations).
|
|
|
|
|
|
The outputs of a derivation can refer to each other (even though they
cannot have cycles), so they have to be deleted in the right order.
http://hydra.nixos.org/build/3026118
|
|
If the options gc-keep-outputs and gc-keep-derivations are both
enabled, you can get a cycle in the liveness graph. There was a hack
to handle this, but it didn't work with multiple-output derivations,
causing the garbage collector to fail with errors like ‘error: cannot
delete path `...' because it is in use by `...'’. The garbage
collector now handles strongly connected components in the liveness
graph as a unit and decides whether to delete all or none of the paths
in an SCC.
|
|
|
|
Fixes bogus messages like "currently hard linking saves
17592186044416.00 MiB".
|
|
|
|
|
|
That is, delete almost nothing (it will still remove unused links from
/nix/store/.links).
|
|
|
|
Put all Nix configuration flags in a Settings object.
|
|
Incremental optimisation requires creating links in /nix/store/.links
to all files in the store. However, this means that if we delete a
store path, no files are actually deleted because links in
/nix/store/.links still exists. So we need to check /nix/store/.links
for files with a link count of 1 and delete them.
|
|
This should fix rare Hydra errors of the form:
error: symlinking `/nix/var/nix/gcroots/per-user/hydra/hydra-roots/7sfhs5fdmjxm8sqgcpd0pgcsmz1kq0l0-nixos-iso-0.1pre33785-33795' to `/nix/store/7sfhs5fdmjxm8sqgcpd0pgcsmz1kq0l0-nixos-iso-0.1pre33785-33795': File exists
|
|
|
|
It's unlikely that rename() is faster than unlink() on a regular file
or symlink, so don't bother.
|
|
Make the garbage collector more concurrent by deleting valid paths
outside the region where we're holding the global GC lock. This
should greatly reduce the time during which new builds are blocked,
since the deletion accounts for the vast majority of the time spent in
the GC.
To ensure that this is safe, the valid paths are invalidated and
renamed to some arbitrary path while we're holding the lock. This
ensures that we when we finally delete the path, it's not a (newly)
valid or locked path.
|
|
We don't need this anymore now that current filesystems support more
than 32,000 files in a directory.
|