about summary refs log tree commit diff
path: root/src/libstore/optimise-store.cc
AgeCommit message (Collapse)AuthorFilesLines
2017-02-08 Include config.h implicitly with '-include config.h' in CFLAGSTuomas Tynkkynen1-2/+0
Because config.h can #define things like _FILE_OFFSET_BITS=64 and not every compilation unit includes config.h, we currently compile half of Nix with _FILE_OFFSET_BITS=64 and other half with _FILE_OFFSET_BITS unset. This causes major havoc with the Settings class on e.g. 32-bit ARM, where different compilation units disagree with the struct layout. E.g.: diff --git a/src/libstore/globals.cc b/src/libstore/globals.cc @@ -166,6 +166,8 @@ void Settings::update() _get(useSubstitutes, "build-use-substitutes"); + fprintf(stderr, "at Settings::update(): &useSubstitutes = %p\n", &nix::settings.useSubstitutes); _get(buildUsersGroup, "build-users-group"); diff --git a/src/libstore/remote-store.cc b/src/libstore/remote-store.cc +++ b/src/libstore/remote-store.cc @@ -138,6 +138,8 @@ void RemoteStore::initConnection(Connection & conn) void RemoteStore::setOptions(Connection & conn) { + fprintf(stderr, "at RemoteStore::setOptions(): &useSubstitutes = %p\n", &nix::settings.useSubstitutes); conn.to << wopSetOptions Gave me: at Settings::update(): &useSubstitutes = 0xb6e5c5cb at RemoteStore::setOptions(): &useSubstitutes = 0xb6e5c5c7 That was not a fun one to debug!
2017-01-16 AutoCloseDir: Use std::unique_ptrEelco Dolstra1-4/+4
2016-11-26 Revert "Get rid of unicode quotes (#1140)"Eelco Dolstra1-24/+24
This reverts commit f78126bfd6b6c8477fcdbc09b2f98772dbe9a1e7. There really is no need for such a massive change...
2016-11-25 Get rid of unicode quotes (#1140)Guillaume Maudoux1-24/+24
2016-11-14 Don't rely on %mEelco Dolstra1-1/+2
2016-11-14 daemon: Do not error out when deduplication fails due to ENOSPC.Ludovic Courtès1-5/+19
This solves a problem whereby if /gnu/store/.links had enough entries, ext4's directory index would be full, leading to link(2) returning ENOSPC. * nix/libstore/optimise-store.cc (LocalStore::optimisePath_): Upon ENOSPC from link(2), print a message and return instead of throwing a 'SysError'.
2016-09-21 printMsg(lvlError, ...) -> printError(...) etc.Eelco Dolstra1-11/+11
2016-06-02 LocalStore: Allow the physical and logical store directories to differEelco Dolstra1-3/+3
This is primarily to subsume the functionality of the copy-from-other-stores substituter. For example, in the NixOS installer, we can now do (assuming we're in the target chroot, and the Nix store of the installation CD is bind-mounted on /tmp/nix): $ nix-build ... --option substituters 'local?state=/tmp/nix/var&real=/tmp/nix/store' However, unlike copy-from-other-stores, this also allows write access to such a store. One application might be fetching substitutes for /nix/store in a situation where the user doesn't have sufficient privileges to create /nix, e.g.: $ NIX_REMOTE="local?state=/home/alice/nix/var&real=/home/alice/nix/store" nix-build ...
2016-06-01 Make the store directory a member variable of StoreEelco Dolstra1-1/+1
2016-04-25 Improved logging abstractionEelco Dolstra1-1/+1
This also gets rid of --log-type, since the nested log type isn't useful in a multi-threaded situation, and nobody cares about the "pretty" log type.
2015-11-09 optimizePath(): Detect some .links corruptionEelco Dolstra1-2/+9
If automatic store optimisation is enabled, and a hard-linked file in the store gets corrupted, then the corresponding .links entry will also be corrupted. In that case, trying to repair with --repair or --repair-path won't work, because the new "good" file will be replaced by a hard link to the corrupted file. We can catch most of these cases by doing a sanity-check on the file sizes.
2015-07-17 OCD: foreach -> C++11 ranged forEelco Dolstra1-7/+7
2014-12-14 PedantryEelco Dolstra1-1/+1
2014-12-09 Explicitly include required C headersMarko Durkovic1-0/+1
2014-09-01 Add an 'optimiseStore' remote procedure call.Ludovic Courtès1-0/+16
2014-08-20 Use proper quotes everywhereEelco Dolstra1-21/+21
2014-05-15 lvlInfo -> lvlTalkativeEelco Dolstra1-1/+1
2014-05-15 nix-store --optimise: Remove bogus statisticsEelco Dolstra1-6/+5
2014-05-15 Remove tabEelco Dolstra1-2/+2
2014-05-15 Shortcut store files before lstatWout Mertens1-9/+36
readdir() already returns the inode numbers, so we don't need to call lstat to know if a file was already linked or not.
2014-05-14 Use the inodes given by readdir directlyWout Mertens1-17/+21
2014-05-13 Preload linked hashes to speed up lookupsWout Mertens1-9/+28
By preloading all inodes in the /nix/store/.links directory, we can quickly determine of a hardlinked file was already linked to the hashed links. This is tolerant of removing the .links directory, it will simply recalculate all hashes in the store.
2014-05-10 Shortcut already-hardlinked fileswmertens1-1/+10
If an inode in the Nix store has more than 1 link, it probably means that it was linked into .links/ by us. If so, skip. There's a possibility that something else hardlinked the file, so it would be nice to be able to override this. Also, by looking at the number of hardlinks for each of the files in .links/, you can get deduplication numbers and space savings.
2013-03-08 Revert "Prevent config.h from being clobbered"Eelco Dolstra1-1/+1
This reverts commit 28bba8c44f484eae38e8a15dcec73cfa999156f6.
2013-03-07 Prevent config.h from being clobberedEelco Dolstra1-1/+1
2013-02-27 Refactoring: Split off the non-recursive canonicalisePathMetaData()Eelco Dolstra1-1/+1
Also, change the file mode before changing the owner. This prevents a slight time window in which a setuid binary would be setuid root.
2013-02-26 Security: Don't allow builders to change permissions on files they don't ownEelco Dolstra1-1/+1
It turns out that in multi-user Nix, a builder may be able to do ln /etc/shadow $out/foo Afterwards, canonicalisePathMetaData() will be applied to $out/foo, causing /etc/shadow's mode to be set to 444 (readable by everybody but writable by nobody). That's obviously Very Bad. Fortunately, this fails in NixOS's default configuration because /nix/store is a bind mount, so "ln" will fail with "Invalid cross-device link". It also fails if hard-link restrictions are enabled, so a workaround is: echo 1 > /proc/sys/fs/protected_hardlinks The solution is to check that all files in $out are owned by the build user. This means that innocuous operations like "ln ${pkgs.foo}/some-file $out/" are now rejected, but that already failed in chroot builds anyway.
2013-01-03 Clear any immutable bits in the Nix storeEelco Dolstra1-9/+0
Doing this once makes subsequent operations like garbage collecting more efficient since we don't have to call makeMutable() first.
2012-09-19 Remove setting of the immutable bitEelco Dolstra1-57/+31
Using the immutable bit is problematic, especially in conjunction with store optimisation. For instance, if the garbage collector deletes a file, it has to clear its immutable bit, but if the file has additional hard links, we can't set the bit afterwards because we don't know the remaining paths. So now that we support having the entire Nix store as a read-only mount, we may as well drop the immutable bit. Unfortunately, we have to keep the code to clear the immutable bit for backwards compatibility.
2012-08-27 Merge branch 'master' into no-manifestsEelco Dolstra1-31/+43
2012-08-07 Don't warn about maximum link count exceeded on 0-byte filesEelco Dolstra1-2/+4
2012-08-05 Fix race condition when two processes create a hard link to a file in .linksEelco Dolstra1-27/+37
This is a problem because one process may set the immutable bit before the second process has created its link. Addressed random Hydra failures such as: error: cannot rename `/nix/store/.tmp-link-17397-1804289383' to `/nix/store/rsvzm574rlfip3830ac7kmaa028bzl6h-nixos-0.1pre-git/upstart-interface-version': Operation not permitted
2012-08-05 Fix race condition when two processes create the same link in /nix/store/.linksEelco Dolstra1-4/+4
2012-08-01 Merge branch 'master' into no-manifestsEelco Dolstra1-1/+4
2012-08-01 DohEelco Dolstra1-1/+2
2012-08-01 Make ‘nix-store --optimise’ interruptibleEelco Dolstra1-0/+2
2012-07-30 Refactor settings processingEelco Dolstra1-15/+13
Put all Nix configuration flags in a Settings object.
2012-07-26 Merge branch 'master' into no-manifestsEelco Dolstra1-104/+114
2012-07-23 Handle platforms that don't support linking to a symlinkEelco Dolstra1-2/+7
E.g. Darwin doesn't allow this.
2012-07-23 Unlink the right fileEelco Dolstra1-2/+2
2012-07-23 Automatically optimise the Nix store when a new path is addedEelco Dolstra1-9/+13
Auto-optimisation is enabled by default. It can be turned off by setting auto-optimise-store to false in nix.conf.
2012-07-23 optimiseStore(): Use a content-addressed file store in /nix/store/.linksEelco Dolstra1-106/+107
optimiseStore() now creates persistent, content-addressed hard links in /nix/store/.links. For instance, if it encounters a file P with hash H, it will create a hard link P' = /nix/store/.link/<H> to P if P' doesn't already exist; if P' exist, then P is replaced by a hard link to P'. This is better than the previous in-memory map, because it had the tendency to unnecessarily replace hard links with a hard link to whatever happened to be the first file with a given hash it encountered. It also allows on-the-fly, incremental optimisation.
2012-07-11 Rename queryValidPaths() to queryAllValidPaths()Eelco Dolstra1-1/+1
2012-02-15 On Linux, make the Nix store really read-only by using the immutable bitEelco Dolstra1-2/+24
I was bitten one time too many by Python modifying the Nix store by creating *.pyc files when run as root. On Linux, we can prevent this by setting the immutable bit on files and directories (as in ‘chattr +i’). This isn't supported by all filesystems, so it's not an error if setting the bit fails. The immutable bit is cleared by the garbage collector before deleting a path. The only tricky aspect is in optimiseStore(), since it's forbidden to create hard links to an immutable file. Thus optimiseStore() temporarily clears the immutable bit before creating the link.
2010-11-16 * Store the size of a store path in the database (to be precise, theEelco Dolstra1-1/+1
size of the NAR serialisation of the path, i.e., `nix-store --dump PATH'). This is useful for Hydra.
2010-06-04 * Applied a patch from David Brown to prevent `nix-store --optimise'Eelco Dolstra1-1/+15
from failing on rename() on BtrFS.
2009-09-24 * And some more.Eelco Dolstra1-0/+1
2009-04-21 * Use foreach in a lot of places.Eelco Dolstra1-2/+2
2008-06-18 * `nix-store --optimise': handle files with >= 32000 hard links.Eelco Dolstra1-6/+31
(There can easily be more than 32000 occurrences of the empty file.)
2008-06-18 * Some refactoring: put the GC options / results in separate structs.Eelco Dolstra1-0/+1
* The garbage collector now also prints the number of blocks freed.
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-11-26T02·12+0000