about summary refs log tree commit diff
path: root/scripts/download-using-manifests.pl.in
AgeCommit message (Collapse)AuthorFilesLines
2012-07-27 Let build.cc verify the expected hash of a substituter's outputEelco Dolstra1-11/+2
Since SubstitutionGoal::finished() in build.cc computes the hash anyway, we can prevent the inefficiency of computing the hash twice by letting the substituter tell Nix about the expected hash, which can then verify it.
2012-07-27 download-using-manifests: Don't use nix-prefetch-urlEelco Dolstra1-41/+29
Instead call curl directly and pipe it into ‘nix-store --restore’. This saves I/O and prevents creating garbage in the Nix store.
2012-07-11 Update the other substitutersEelco Dolstra1-40/+38
2011-11-29 * download-using-manifests: use the Perl bindings.Eelco Dolstra1-37/+29
2011-10-10 * Set the executable bit on scripts.Eelco Dolstra1-0/+0
2011-10-10 * Install NixManifest.pm, NixConfig.pm and GeneratePatches.pm underEelco Dolstra1-16/+14
the Nix:: namespace.
2011-07-13 * Fix concurrency issues in download-using-manifests' handling of theEelco Dolstra1-0/+6
SQLite manifest cache. The DBI AutoCommit feature caused every process to have an active transaction at all times, which could indefinitely block processes wanting to update the manifest cache. * Disable fsync() in the manifest cache because we don't need integrity (the cache can always be recreated if it gets corrupted).
2011-04-11 * configure: detect whether DBD::SQLite is present. If necessary theEelco Dolstra1-1/+1
location to DBI and DBD::SQLite can be passed with --with-dbi and --with-dbd-sqlite.
2011-04-10 * Cache the manifests in /nix/var/nix/manifests in a SQLite database.Eelco Dolstra1-26/+26
This significantly speeds up the download-using-manifests substituter, especially if manifests are very large. For instance, one "nix-build -A geeqie" operation that updated four packages using binary patches went from 18.5s to 1.6s. It also significantly reduces memory use. The cache is kept in /nix/var/nix/manifests/cache.sqlite. It's updated automatically when manifests are added to or removed from /nix/var/nix/manifests. It might be interesting to have nix-pull store manifests directly in the DB, rather than storing them as separate flat files, but then we would need a command line interface to delete manifests from the DB.
2011-04-06 * Remove the localPaths feature in manifests since it's no longer usedEelco Dolstra1-20/+2
and redundant anyway.
2011-03-16 * Print a better error message.Eelco Dolstra1-1/+1
2010-12-22 * In Hydra manifests the Size field is missing, so don't rely on it.Eelco Dolstra1-2/+2
This caused a lot of "Use of uninitialized value" warnings from Perl.
2010-12-13 * When doing a query (e.g. `nix-store -r --dry-run'), don't make a lotEelco Dolstra1-4/+12
of expensive calls to `nix-store --check-validity'.
2010-12-05 * Use CamelCase for the Perl modules.Eelco Dolstra1-1/+1
2010-11-17 * Oops.Eelco Dolstra1-1/+1
2010-11-17 * When checking whether a patch is applicable, for efficiency, useEelco Dolstra1-3/+7
`nix-store -q --hash' to get the hash of the base path rather than `nix-hash'. However, only do this for estimating the size of a download, not for the actual substitution, because sometimes the contents of store paths are modified (which they shouldn't, of course).
2010-11-17 * In the download size indication, take binary patches into account.Eelco Dolstra1-144/+161
Hopefully this doesn't slow things down too much.
2010-11-17 * Before a build, show the disk space that the downloaded store pathsEelco Dolstra1-0/+2
will approximately require.
2010-02-04 * "Fix" incorrect help message.Eelco Dolstra1-1/+1
2010-02-02 * Don't do a chdir to $tmpDir. It's not necessary, and Windows doesn'tEelco Dolstra1-2/+0
support deleting the current directory.
2009-03-19 * Future proofing: assume we can read manifests up to version 10Eelco Dolstra1-1/+6
(which should therefore be backwards compatible).
2009-02-27 * nix-install-package: don't pollute /nix/var/nix/manifests.Eelco Dolstra1-1/+1
2009-02-26 * Handle base-16 hashes in manifests.Eelco Dolstra1-1/+12
2009-02-19 * download-using-manifests: don't check the cryptographic hash ofEelco Dolstra1-18/+30
downloaded files; rather, check the hash of the unpacked store path. When the server produces bzipped NAR archives on demand (like Hydra does), the hash of the file is not known in advance; it's streamed from the server. Thus the manifest doesn't contain a hash for the bzipped NAR archive. However, the server does know the hash of the *uncompressed* NAR archive (the "NarHash" field), since it's stored in the Nix database (nix-store -q --hash /nix/store/bla). So we use that instead for checking the integrity of the download.
2008-11-20 * Urgh.Eelco Dolstra1-7/+9
2008-09-08 * Doh.Eelco Dolstra1-1/+1
2008-08-04 * Doh.Eelco Dolstra1-0/+2
2008-08-02 * Make nix-env --dry-run print the paths to be substituted correctlyEelco Dolstra1-35/+42
again. (After the previous substituter mechanism refactoring I didn't update the code that obtains the references of substitutable paths.) This required some refactoring: the substituter programs are now kept running and receive/respond to info requests via stdin/stdout.
2008-07-18 * Allow read-only access to the store (e.g., non-root users on NixOSEelco Dolstra1-1/+2
can do operations like "nix-store -qR <path>" even without the Nix daemon).
2007-10-22 (no commit message)Eelco Dolstra1-2/+0
2007-08-12 * Get rid of the substitutes database table (NIX-47). Instead, if weEelco Dolstra1-10/+48
need any info on substitutable paths, we just call the substituters (such as download-using-manifests.pl) directly. This means that it's no longer necessary for nix-pull to register substitutes or for nix-channel to clear them, which makes those operations much faster (NIX-95). Also, we don't have to worry about keeping nix-pull manifests (in /nix/var/nix/manifests) and the database in sync with each other. The downside is that there is some overhead in calling an external program to get the substitutes info. For instance, "nix-env -qas" takes a bit longer. Abolishing the substitutes table also makes the logic in local-store.cc simpler, as we don't need to store info for invalid paths. On the downside, you cannot do things like "nix-store -qR" on a substitutable but invalid path (but nobody did that anyway). * Never catch interrupts (the Interrupted exception).
2007-01-23 * New kind of manifest object: "localPath", which denotes that a storeEelco Dolstra1-1/+15
path can be created by copying it from another location in the file system. This is useful in the NixOS installation.
2007-01-23 * Successors have been gone for ages.Eelco Dolstra1-2/+1
2007-01-13 * Removed chroot support.Eelco Dolstra1-2/+0
2006-10-04 * tmpnam() -> File::Temp::tempdir().Eelco Dolstra1-4/+4
2006-09-25 * Clean up calls to system().Eelco Dolstra1-11/+11
2006-08-05 prevent doing recursive chroots, by unsetting NIX_ROOT in the scripts.Armijn Hemel1-0/+1
2006-08-04 * Remove the dependency on `date', use strftime instead.Eelco Dolstra1-3/+2
2005-09-16 * Set the current directory to something well-defined. Might help inEelco Dolstra1-0/+2
setuid installations.
2005-09-15 * Use a proper temporary directory.Eelco Dolstra1-7/+19
2005-05-10 * Make unpacking of patch sequences much faster by not doing redundantEelco Dolstra1-16/+32
unpacking and repacking of intermediate paths.
2005-04-12 * Argh! The patch downloader was broken due to the renaming of theEelco Dolstra1-1/+1
`--isvalid' flag in nix-store.
2005-03-25 * Better error checking.Eelco Dolstra1-0/+1
2005-03-14 * Pass `--base32' unless using MD5.Eelco Dolstra1-1/+3
2005-03-14 * Parse new hash format properly.Eelco Dolstra1-6/+15
2005-02-25 * Add a version number to manifests.Eelco Dolstra1-1/+4
2005-02-24 * Properly specify the hash algorithm in the manifests, and read itEelco Dolstra1-3/+5
too. * Change the default hash for nix-prefetch-url back to md5, since that's what we use in Nixpkgs (for now; a birthday attack is rather unlikely there).
2005-01-25 * Really fix the substitute mechanism, i.e., ensure the closureEelco Dolstra1-3/+3
invariant by registering references through the manifest. * Added a test for nix-pull.
2005-01-14 * Use absolute paths.Eelco Dolstra1-6/+6
2005-01-12 * Print out less garbage.Eelco Dolstra1-8/+10