about summary refs log tree commit diff
path: root/scripts/download-using-manifests.pl.in
AgeCommit message (Collapse)AuthorFilesLines
2016-01-05 Verify TLS certificate before downloading binariesPhilip Potter1-2/+1
The --insecure flag to curl tells curl not to bother checking if the TLS certificate presented by the server actually matches the hostname requested, and actually is issued by a trusted CA chain. This almost entirely negates any benefit from using TLS in the first place. This removes the --insecure flag to ensure we actually have a secure connection to the intended hostname before downloading binaries. Manually tested locally within a dev-shell; was able to download binaries from https://cache.nixos.org without issue. [Note: --insecure was only used for fetching NARs, whose integrity is verified by Nix anyway using the hash from the .narinfo. But if we can fetch the .narinfo without --insecure, we can also fetch the .nar, so there is not much point to using --insecure. --Eelco]
2014-08-29 Shut up "Wide character" warnings in Perl scriptsEelco Dolstra1-0/+1
2014-08-20 Use proper quotes everywhereEelco Dolstra1-12/+13
2014-08-13 Use $XDG_RUNTIME_DIR for temporary filesEelco Dolstra1-3/+1
2013-12-02 Install bsdiff and bspatch in $(libexecdir)/nixEelco Dolstra1-1/+1
2013-07-01 Add support for uncompressed NARs in binary cachesEelco Dolstra1-4/+5
Issue NixOS/hydra#102.
2013-06-20 Don't keep "disabled" substituters runningEelco Dolstra1-0/+2
For instance, it's pointless to keep copy-from-other-stores running if there are no other stores, or download-using-manifests if there are no manifests. This also speeds things up because we don't send queries to those substituters.
2012-10-17 nix-push: Only generate and copy a NAR if it doesn't already existEelco Dolstra1-2/+2
This prevents unnecessary and slow rebuilds of NARs that already exist in the binary cache.
2012-10-02 Add operation ‘nix-store --repair-path’Eelco Dolstra1-3/+4
This operation allows fixing corrupted or accidentally deleted store paths by redownloading them using substituters, if available. Since the corrupted path cannot be replaced atomically, there is a very small time window (one system call) during which neither the old (corrupted) nor the new (repaired) contents are available. So repairing should be used with some care on critical packages like Glibc.
2012-09-21 Style fixEelco Dolstra1-5/+5
2012-09-19 Support xz compression in the download-using-manifests substituterEelco Dolstra1-2/+8
2012-07-30 Do some validation of URLsEelco Dolstra1-0/+3
2012-07-27 Let build.cc verify the expected hash of a substituter's outputEelco Dolstra1-11/+2
Since SubstitutionGoal::finished() in build.cc computes the hash anyway, we can prevent the inefficiency of computing the hash twice by letting the substituter tell Nix about the expected hash, which can then verify it.
2012-07-27 download-using-manifests: Don't use nix-prefetch-urlEelco Dolstra1-41/+29
Instead call curl directly and pipe it into ‘nix-store --restore’. This saves I/O and prevents creating garbage in the Nix store.
2012-07-11 Update the other substitutersEelco Dolstra1-40/+38
2011-11-29 * download-using-manifests: use the Perl bindings.Eelco Dolstra1-37/+29
2011-10-10 * Set the executable bit on scripts.Eelco Dolstra1-0/+0
2011-10-10 * Install NixManifest.pm, NixConfig.pm and GeneratePatches.pm underEelco Dolstra1-16/+14
the Nix:: namespace.
2011-07-13 * Fix concurrency issues in download-using-manifests' handling of theEelco Dolstra1-0/+6
SQLite manifest cache. The DBI AutoCommit feature caused every process to have an active transaction at all times, which could indefinitely block processes wanting to update the manifest cache. * Disable fsync() in the manifest cache because we don't need integrity (the cache can always be recreated if it gets corrupted).
2011-04-11 * configure: detect whether DBD::SQLite is present. If necessary theEelco Dolstra1-1/+1
location to DBI and DBD::SQLite can be passed with --with-dbi and --with-dbd-sqlite.
2011-04-10 * Cache the manifests in /nix/var/nix/manifests in a SQLite database.Eelco Dolstra1-26/+26
This significantly speeds up the download-using-manifests substituter, especially if manifests are very large. For instance, one "nix-build -A geeqie" operation that updated four packages using binary patches went from 18.5s to 1.6s. It also significantly reduces memory use. The cache is kept in /nix/var/nix/manifests/cache.sqlite. It's updated automatically when manifests are added to or removed from /nix/var/nix/manifests. It might be interesting to have nix-pull store manifests directly in the DB, rather than storing them as separate flat files, but then we would need a command line interface to delete manifests from the DB.
2011-04-06 * Remove the localPaths feature in manifests since it's no longer usedEelco Dolstra1-20/+2
and redundant anyway.
2011-03-16 * Print a better error message.Eelco Dolstra1-1/+1
2010-12-22 * In Hydra manifests the Size field is missing, so don't rely on it.Eelco Dolstra1-2/+2
This caused a lot of "Use of uninitialized value" warnings from Perl.
2010-12-13 * When doing a query (e.g. `nix-store -r --dry-run'), don't make a lotEelco Dolstra1-4/+12
of expensive calls to `nix-store --check-validity'.
2010-12-05 * Use CamelCase for the Perl modules.Eelco Dolstra1-1/+1
2010-11-17 * Oops.Eelco Dolstra1-1/+1
2010-11-17 * When checking whether a patch is applicable, for efficiency, useEelco Dolstra1-3/+7
`nix-store -q --hash' to get the hash of the base path rather than `nix-hash'. However, only do this for estimating the size of a download, not for the actual substitution, because sometimes the contents of store paths are modified (which they shouldn't, of course).
2010-11-17 * In the download size indication, take binary patches into account.Eelco Dolstra1-144/+161
Hopefully this doesn't slow things down too much.
2010-11-17 * Before a build, show the disk space that the downloaded store pathsEelco Dolstra1-0/+2
will approximately require.
2010-02-04 * "Fix" incorrect help message.Eelco Dolstra1-1/+1
2010-02-02 * Don't do a chdir to $tmpDir. It's not necessary, and Windows doesn'tEelco Dolstra1-2/+0
support deleting the current directory.
2009-03-19 * Future proofing: assume we can read manifests up to version 10Eelco Dolstra1-1/+6
(which should therefore be backwards compatible).
2009-02-27 * nix-install-package: don't pollute /nix/var/nix/manifests.Eelco Dolstra1-1/+1
2009-02-26 * Handle base-16 hashes in manifests.Eelco Dolstra1-1/+12
2009-02-19 * download-using-manifests: don't check the cryptographic hash ofEelco Dolstra1-18/+30
downloaded files; rather, check the hash of the unpacked store path. When the server produces bzipped NAR archives on demand (like Hydra does), the hash of the file is not known in advance; it's streamed from the server. Thus the manifest doesn't contain a hash for the bzipped NAR archive. However, the server does know the hash of the *uncompressed* NAR archive (the "NarHash" field), since it's stored in the Nix database (nix-store -q --hash /nix/store/bla). So we use that instead for checking the integrity of the download.
2008-11-20 * Urgh.Eelco Dolstra1-7/+9
2008-09-08 * Doh.Eelco Dolstra1-1/+1
2008-08-04 * Doh.Eelco Dolstra1-0/+2
2008-08-02 * Make nix-env --dry-run print the paths to be substituted correctlyEelco Dolstra1-35/+42
again. (After the previous substituter mechanism refactoring I didn't update the code that obtains the references of substitutable paths.) This required some refactoring: the substituter programs are now kept running and receive/respond to info requests via stdin/stdout.
2008-07-18 * Allow read-only access to the store (e.g., non-root users on NixOSEelco Dolstra1-1/+2
can do operations like "nix-store -qR <path>" even without the Nix daemon).
2007-10-22 (no commit message)Eelco Dolstra1-2/+0
2007-08-12 * Get rid of the substitutes database table (NIX-47). Instead, if weEelco Dolstra1-10/+48
need any info on substitutable paths, we just call the substituters (such as download-using-manifests.pl) directly. This means that it's no longer necessary for nix-pull to register substitutes or for nix-channel to clear them, which makes those operations much faster (NIX-95). Also, we don't have to worry about keeping nix-pull manifests (in /nix/var/nix/manifests) and the database in sync with each other. The downside is that there is some overhead in calling an external program to get the substitutes info. For instance, "nix-env -qas" takes a bit longer. Abolishing the substitutes table also makes the logic in local-store.cc simpler, as we don't need to store info for invalid paths. On the downside, you cannot do things like "nix-store -qR" on a substitutable but invalid path (but nobody did that anyway). * Never catch interrupts (the Interrupted exception).
2007-01-23 * New kind of manifest object: "localPath", which denotes that a storeEelco Dolstra1-1/+15
path can be created by copying it from another location in the file system. This is useful in the NixOS installation.
2007-01-23 * Successors have been gone for ages.Eelco Dolstra1-2/+1
2007-01-13 * Removed chroot support.Eelco Dolstra1-2/+0
2006-10-04 * tmpnam() -> File::Temp::tempdir().Eelco Dolstra1-4/+4
2006-09-25 * Clean up calls to system().Eelco Dolstra1-11/+11
2006-08-05 prevent doing recursive chroots, by unsetting NIX_ROOT in the scripts.Armijn Hemel1-0/+1
2006-08-04 * Remove the dependency on `date', use strftime instead.Eelco Dolstra1-3/+2