about summary refs log tree commit diff
path: root/src/libutil/compression.cc
AgeCommit message (Collapse)AuthorFilesLines
2018-05-02 Fix bzip2 compression of files > 4 GiBEelco Dolstra1-3/+14
Bzip2's 'avail_in' parameter is declared as an unsigned int, so assigning a size_t length to it led to silent truncation. Fixes #2111.
2018-03-22 Fix buildEelco Dolstra1-2/+2
2018-03-16 decompress(): Use a Source and SinkEelco Dolstra1-57/+109
This allows decompression to happen in O(1) memory.
2018-02-11 compression: make parallel sink separate classWill Dietz1-29/+34
2018-02-11 compression: print warning if parallel requested but not supportedWill Dietz1-1/+8
2018-02-09 configure.ac: check if lzma has MT support, fix deb build/etc.Will Dietz1-0/+2
2018-02-07 make multi threaded compression configurable and use single threadedAmineChikhaoui1-17/+25
by default.
2018-02-07 set block size to 0 to let the lzma lib choose the right one, addAmineChikhaoui1-1/+6
some comments about possible improvements wrt memory usage/threading.
2018-02-06 support multi threaded xz encoder, this might be particularly useful inAmineChikhaoui1-2/+7
the case of hydra where the overhead of single threaded encoding is more noticeable e.g most of the time spent in "Sending inputs"/"Receiving outputs" is due to compression while the actual upload to the binary cache seems to be negligible.
2017-12-30 use libbrotli directly when availableWill Dietz1-8/+163
* Look for both 'brotli' and 'bro' as external command, since upstream has renamed it in newer versions. If neither are found, current runtime behavior is preserved: try to find 'bro' on PATH. * Limit amount handed to BrotliEncoderCompressStream to ensure interrupts are processed in a timely manner. Testing shows negligible performance impact. (Other compression sinks don't seem to require this)
2017-07-30 Replace Unicode quotes in user-facing strings by ASCIIJörg Thalheim1-2/+2
Relevant RFC: NixOS/rfcs#4 $ ag -l | xargs sed -i -e "/\"/s/’/'/g;/\"/s/‘/'/g"
2017-03-22 Fix xz decompressionEelco Dolstra1-4/+1
Fixes #1285.
2017-03-21 LocalFSStore::getBuildLog(): Handle corrupted logsEelco Dolstra1-12/+12
2017-03-21 Don't hang in decompression if bzip2 data ends prematurelyEelco Dolstra1-0/+6
2017-03-15 runProgram(): Distinguish between empty input and no inputEelco Dolstra1-1/+1
For example, if we call brotli with an empty input, it shouldn't read from the caller's stdin.
2017-03-15 S3BinaryCacheStore: Support compression of narinfo and log filesEelco Dolstra1-0/+31
You can now set the store parameter "text-compression=br" to compress textual files in the binary cache (i.e. narinfo and logs) using Brotli. This sets the Content-Encoding header; the extension of compressed files is unchanged. You can separately specify the compression of log files using "log-compression=br". This is useful when you don't want to compress narinfo files for backward compatibility.
2017-03-15 Fix assertion failure on SIGINTEelco Dolstra1-2/+0
nix: src/libutil/compression.cc:142: virtual nix::XzSink::~XzSink(): Assertion `finished' failed.
2017-03-15 Add support for brotli compressionEelco Dolstra1-0/+7
Build logs on cache.nixos.org are compressed using Brotli (since this allows them to be decompressed automatically by Chrome and Firefox), so it's handy if "nix log" can decompress them.
2016-11-26 Revert "Get rid of unicode quotes (#1140)"Eelco Dolstra1-2/+2
This reverts commit f78126bfd6b6c8477fcdbc09b2f98772dbe9a1e7. There really is no need for such a massive change...
2016-11-25 Get rid of unicode quotes (#1140)Guillaume Maudoux1-2/+2
2016-05-04 Do compression in a sinkEelco Dolstra1-92/+168
2016-05-04 Add a Store::addToStore() variant that accepts a NAREelco Dolstra1-8/+8
As a side effect, this ensures that signatures are propagated when copying paths between stores. Also refactored import/export to make use of this.
2016-05-04 Fix segfault in xz compression/decompressionEelco Dolstra1-2/+2
2016-04-29 BinaryCacheStore: Support bzip2 compressionEelco Dolstra1-34/+116
2016-04-29 BinaryCacheStore: Support "none" compression methodEelco Dolstra1-4/+24
2016-04-22 Make compression interruptibleEelco Dolstra1-1/+3
2016-04-15 BinaryCacheStore::readFile(): Return a shared_ptr to a stringEelco Dolstra1-3/+3
This allows readFile() to indicate that a file doesn't exist, and might eliminate some large string copying.
2016-02-15 Add xz compression functionEelco Dolstra1-0/+40
This is used by the Hydra queue runner, but since it may also be useful for the C++ rewrite of nix-push, I'm putting it here.
2016-02-15 decompressXZ: Ensure that lzma_end() is calledEelco Dolstra1-12/+21
Otherwise we might leak memory.
2016-01-05 Fixed unresolved 'BUFSIZ'FrankHB1-0/+1
`BUFSIZ` is defined in header `<cstdio>`.
2015-10-30 <nix/fetchurl.nix>: Support xz-compressed NARsEelco Dolstra1-0/+46