about summary refs log tree commit diff
path: root/src/libstore/download.cc
AgeCommit message (Collapse)AuthorFilesLines
2019-06-24 Fix 'error 9 while decompressing xz file'Eelco Dolstra1-55/+12
Once we've started writing data to a Sink, we can't restart a download request, because then we end up writing duplicate data to the Sink. Therefore we shouldn't handle retries in Downloader but at a higher level (in particular, in copyStorePath()). Fixes #2952. (cherry picked from commit a67cf5a3585c41dd9f219a2c7aa9cf67fa69520b)
2019-06-24 Downloader: Propagate exceptions from decompressionSink->finish()Eelco Dolstra1-2/+7
(cherry picked from commit 15fa70cd1b853f5e62662b99ccb9ef3da6cfadff)
2019-06-24 CachedDownloadResult: Include store pathEelco Dolstra1-0/+2
Also, make fetchGit and fetchMercurial update allowedPaths properly. (Maybe the evaluator, rather than the caller of the evaluator, should apply toRealPath(), but that's a bigger change.) (cherry picked from commit 5c34d665386f4053d666b0899ecca0639e500fbd)
2019-06-24 downloadCached: Return ETagEelco Dolstra1-4/+13
(cherry picked from commit 529add316c5356a8060c35f987643b7bf5c796dc)
2019-02-26 fix indentationDzmitry Zaitsau1-2/+2
2019-02-25 Apply param values on S3Helper initializationDzmitry Zaitsau1-1/+8
2019-02-25 Extend S3 URL parsing with parameters extractionDzmitry Zaitsau1-5/+17
2018-12-12 libstore: improve hash mismatch error messageszimbatm1-2/+2
Use the same output ordering and format everywhere. This is such a common issue that we trade the single-line error message for more readability. Old message: ``` fixed-output derivation produced path '/nix/store/d4nw9x2sy9q3r32f3g5l5h1k833c01vq-example.com' with sha256 hash '08y4734bm2zahw75b16bcmcg587vvyvh0n11gwiyir70divwp1rm' instead of the expected hash '1xzwnipjd54wl8g93vpw6hxnpmdabq0wqywriiwmh7x8k0lvpq5m' ``` New message: ``` hash mismatch in fixed-output derivation '/nix/store/d4nw9x2sy9q3r32f3g5l5h1k833c01vq-example.com': wanted: sha256:1xzwnipjd54wl8g93vpw6hxnpmdabq0wqywriiwmh7x8k0lvpq5m got: sha256:08y4734bm2zahw75b16bcmcg587vvyvh0n11gwiyir70divwp1rm ```
2018-12-07 s3: make scheme configurableDaiderd Jordan1-1/+1
This enables using for http for S3 request for debugging or implementations that don't have https configured. This is not a problem for binary caches since they should not contain sensitive information. Both package signatures and AWS auth already protect against tampering.
2018-11-20 Merge pull request #2216 from dtzWill/fix/curl-760-timeoutEelco Dolstra1-1/+2
download: if there are active requests, never sleep for 10s
2018-09-26 Ensure download thread livenessEelco Dolstra1-19/+26
* Don't wait forever for the client to remove data from the buffer. This does mean that the buffer can grow without bounds (e.g. when downloading is faster than writing to disk), but meh. * Don't hold the state lock while calling the sink. The sink could take any amount of time to process the data (in particular when it's actually a coroutine), so we don't want to block the download thread.
2018-09-03 download: fix size reported to progress barWill Dietz1-1/+1
2018-08-06 Fix Brotli decompression in 'nix log'Eelco Dolstra1-24/+44
This didn't work anymore since decompression was only done in the non-coroutine case. Decompressors are now sinks, just like compressors. Also fixed a bug in bzip2 API handling (we have to handle BZ_RUN_OK rather than BZ_OK), which we didn't notice because there was a missing 'throw': if (ret != BZ_OK) CompressionError("error while compressing bzip2 file");
2018-08-06 Fix reporting of HTTP body size when a result callback is usedEelco Dolstra1-1/+4
2018-08-02 s3 binary cache: support specifying an endpointGraham Christensen1-1/+1
Works for uploading and not downloading.
2018-07-31 Fix race downloading into a sinkEelco Dolstra1-6/+8
This fixes 'error 10 while decompressing xz file'. https://hydra.nixos.org/build/78308551
2018-06-18 Detect redirect loopsEelco Dolstra1-0/+2
Fixes #2225.
2018-06-08 download: if there are active requests, never sleep for 10sWill Dietz1-1/+2
In some versions/configurations libcurl doesn't handle timeouts (especially DNS timeouts) in a way that wakes curl_multi_wait. This doesn't appear to be a problem if using c-ares, FWIW.
2018-06-05 Handle HTTP 410 Gone errorsEelco Dolstra1-1/+1
2018-06-05 Improve binary cache fallback testEelco Dolstra1-0/+5
2018-06-05 Further improve upload messagesEelco Dolstra1-8/+6
2018-06-01 download.cc: fix readCallback return type (int -> size_t)Will Dietz1-2/+2
I'm not sure if curl ever asks for enough data at once for truncation to occur but better safe than sorry.
2018-06-01 Improve binary cache upload messagesEelco Dolstra1-4/+14
Don't say "download" when we mean "upload".
2018-05-30 getDownloader(): SimplifyEelco Dolstra1-4/+2
2018-05-30 Make HttpBinaryCacheStore::narFromPath() run in constant memoryEelco Dolstra1-1/+91
This reduces memory consumption of nix copy --from https://cache.nixos.org --to ~/my-nix /nix/store/95cwv4q54dc6giaqv6q6p4r02ia2km35-blender-2.79 from 176 MiB to 82 MiB. (The remaining memory is probably due to xz decompression overhead.) Issue https://github.com/NixOS/nix/issues/1681. Issue https://github.com/NixOS/nix/issues/1969.
2018-05-30 Simplify the callback mechanismEelco Dolstra1-17/+20
2018-05-30 Move some Download-specific settings to download.ccEelco Dolstra1-8/+24
2018-05-25 libstore/download: remove unused function "getTime()"Will Dietz1-7/+0
2018-05-09 make sure not to use cached channels for nix-channel --updateYorick van Pelt1-2/+1
fixes #1964
2018-04-23 Merge branch 'hash-mismatch' of git://github.com/LnL7/nixShea Levy1-2/+2
2018-04-21 Merge pull request #1707 from dtzWill/fix/issue-1692Peter Simons1-1/+5
ignore "interrupted" exception in progress callback
2018-04-20 download: make hash mismatch error consistent with fetchurlDaiderd Jordan1-2/+2
2018-03-22 download: improve error for hash mismatch ("store mismatch")Will Dietz1-2/+7
Fixes #1905
2018-03-22 download.cc: fix path for cached filesWill Dietz1-1/+1
2018-03-16 Don't retry on CURLE_SSL_CACERT_BADFILEEelco Dolstra1-0/+1
The certificates won't get any better if we retry.
2018-03-14 Catch more possible instances of passing NULL to memcpy.Shea Levy1-0/+1
Actually fixes #1976.
2018-02-23 Fix downloadCached() with a chroot storeEelco Dolstra1-2/+2
E.g. nix run --store ~/my-nix -f channel:nixos-17.03 hello -c hello This problem was mentioned in #1897.
2018-01-31 Fix building without aws-sdk-cppEelco Dolstra1-4/+5
2018-01-31 Merge branch 'http-binary-cache-put-upsert' of https://github.com/adelbertc/nixEelco Dolstra1-1/+27
2018-01-31 Indent properlyEelco Dolstra1-3/+3
2018-01-26 HttpBinaryCacheStore: Support upsertFile with PUT.Shea Levy1-1/+27
Some servers, such as Artifactory, allow uploading with PUT and BASIC auth. This allows nix copy to work to upload binaries to those servers. Worked on together with @adelbertc
2018-01-19 Don't retry CURLE_URL_MALFORMATEelco Dolstra1-0/+1
2017-12-09 Fix for builds with system libcurl < 7.30Benjamin Hipple1-1/+3
CentOS 7.4 and RHEL 7.4 ship with libcurl-devel-7.29.0-42.el7.x86_64; this flag was added in 7.30.0 https://curl.haxx.se/libcurl/c/CURLMOPT_MAX_TOTAL_CONNECTIONS.html
2017-11-30 ignore "interrupted" exception in progress callbackWill Dietz1-1/+5
Context/discusson: https://github.com/NixOS/nix/issues/1692#issuecomment-348282301
2017-11-15 Add a "profile" option to S3BinaryCacheStoreEelco Dolstra1-1/+1
This allows specifying the AWS configuration profile to use. E.g. nix copy --from s3://my-cache?profile=aws-dev-account /nix/store/cf3isrlqavvd5w7rpky1fa8j9lcnlggm-...
2017-11-01 Add fetchMercurial primopEelco Dolstra1-1/+1
E.g. $ nix eval '(fetchMercurial https://www.mercurial-scm.org/repo/hello)' { branch = "default"; outPath = "/nix/store/alvb9y1kfz42bjishqmyy3pphnrh1pfa-source"; rev = "82e55d328c8ca4ee16520036c0aaace03a5beb65"; revCount = 1; shortRev = "82e55d328c8c"; } $ nix eval '(fetchMercurial { url = https://www.mercurial-scm.org/repo/hello; rev = "0a04b987be5ae354b710cefeba0e2d9de7ad41a9"; })' { branch = "default"; outPath = "/nix/store/alvb9y1kfz42bjishqmyy3pphnrh1pfa-source"; rev = "0a04b987be5ae354b710cefeba0e2d9de7ad41a9"; revCount = 0; shortRev = "0a04b987be5a"; } $ nix eval '(fetchMercurial /tmp/unclean-hg-tree)' { branch = "default"; outPath = "/nix/store/cm750cdw1x8wfpm3jq7mz09r30l9r024-source"; rev = "0000000000000000000000000000000000000000"; revCount = 0; shortRev = "000000000000"; }
2017-10-30 fetchurl/fetchTarball: Respect name changesEelco Dolstra1-1/+3
The computation of urlHash didn't take the name into account, so subsequent fetchurl calls with the same URL but a different name would resolve to the same cached store path.
2017-10-02 Reverse retry logic to retry in all but a few casesDan Peebles1-20/+37
It was getting too much like whac-a-mole listing all the retriable error conditions, so we now retry by default and list the cases where retrying is almost certainly hopeless.
2017-09-18 Don't remove Content-Encoding in fetchurl / nix-prefetch-urlEelco Dolstra1-1/+2
Fixes #1568.
2017-09-01 Abort curl downloads if there is no progress for 5 minutesEelco Dolstra1-1/+6
Maybe this will fix the curl hangs on macOS. (We could also use CURLOPT_TIMEOUT but that seems more of a sledgehammer.)
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-10-04T22·58+0000