about summary refs log tree commit diff
path: root/src/libutil
AgeCommit message (Collapse)AuthorFilesLines
2018-08-21 Improve 'coroutine has finished' error messageEelco Dolstra2-7/+14
2018-08-21 Fix warnings in compression.ccEelco Dolstra1-7/+5
2018-08-06 Fix Brotli decompression in 'nix log'Eelco Dolstra2-325/+244
This didn't work anymore since decompression was only done in the non-coroutine case. Decompressors are now sinks, just like compressors. Also fixed a bug in bzip2 API handling (we have to handle BZ_RUN_OK rather than BZ_OK), which we didn't notice because there was a missing 'throw': if (ret != BZ_OK) CompressionError("error while compressing bzip2 file");
2018-08-06 Require libbrotliEelco Dolstra1-50/+0
2018-07-31 Add a check for broken Boost versionsEelco Dolstra1-0/+4
In some Boost versions, coroutines don't propagate exceptions properly, causing Nix to fail with the exception 'coroutine has finished'.
2018-07-03 Store floating point numbers in double precisionEelco Dolstra1-0/+1
Even on 32-bit systems, Value has enough space to hold a double.
2018-06-18 sync.hh: return cv_status instead of dropping it, in case useful.Will Dietz1-2/+2
2018-05-30 Make <nix/fetchurl.nix> run in constant memoryEelco Dolstra2-0/+19
E.g. nix-build --store ~/my-nix/ -E 'import <nix/fetchurl.nix> { url = https://cache.nixos.org/nar/0nwi996rgq4b914qyx0mv2wq4k80hjac7xilikavagw7kxmn2iiv.nar.xz; sha256 = "0nwi996rgq4b914qyx0mv2wq4k80hjac7xilikavagw7kxmn2iiv"; }' now runs in 17 MiB (was 70 MiB), while nix-build --store ~/my-nix/ -E 'import <nix/fetchurl.nix> { url = https://cache.nixos.org/nar/0nwi996rgq4b914qyx0mv2wq4k80hjac7xilikavagw7kxmn2iiv.nar.xz; sha256 = "0d2fxljdih3nc5dqx41hjzic3141ajil94m8kdbpryq569dpsbvb"; unpack = true; }' runs in 17 MiB (was 346 MiB).
2018-05-30 Make LocalBinaryCacheStore::narFromPath() run in constant memoryEelco Dolstra2-1/+10
This reduces memory consumption of nix copy --from file://... --to ~/my-nix /nix/store/95cwv4q54dc6giaqv6q6p4r02ia2km35-blender-2.79 from 514 MiB to 18 MiB for an uncompressed binary cache, and from 192 MiB to 53 MiB for a bzipped binary cache. It may also be faster because fetching can happen concurrently with decompression/writing. Continuation of 48662d151bdf4a38670897beacea9d1bd750376a. Issue https://github.com/NixOS/nix/issues/1681.
2018-05-30 Simplify the callback mechanismEelco Dolstra1-33/+20
2018-05-30 Modularize config settingsEelco Dolstra4-49/+150
Allow global config settings to be defined in multiple Config classes. For example, this means that libutil can have settings and evaluator settings can be moved out of libstore. The Config classes are registered in a new GlobalConfig class to which config files etc. are applied. Relevant to https://github.com/NixOS/nix/issues/2009 in that it removes the need for ad hoc handling of useCaseHack, which was the underlying cause of that issue.
2018-05-21 serialise: fix buffer size used, hide method for internal use onlyWill Dietz2-4/+7
Fixes #2169.
2018-05-02 Fix some random -Wconversion warningsEelco Dolstra4-6/+6
2018-05-02 Fix bzip2 compression of files > 4 GiBEelco Dolstra1-3/+14
Bzip2's 'avail_in' parameter is declared as an unsigned int, so assigning a size_t length to it led to silent truncation. Fixes #2111.
2018-03-22 Fix buildEelco Dolstra1-2/+2
2018-03-22 TweakEelco Dolstra1-1/+1
2018-03-22 Merge branch 'fix/avoid-large-stack-buffers' of https://github.com/dtzWill/nixEelco Dolstra3-20/+21
2018-03-21 Make 'nix copy --from ssh://...' run in constant memoryEelco Dolstra2-0/+20
For instance, this reduced the memory consumption of $ nix copy --from ssh://localhost --to ~/my-nix /nix/store/1n7x0yv8vq6zi90hfmian84vdhd04bgp-blender-2.79a from 632 MiB to 16 MiB.
2018-03-20 Hack to get SSH error messages from build-remoteEelco Dolstra2-5/+22
E.g. cannot build on 'ssh://mac1': cannot connect to 'mac1': bash: nix-store: command not found cannot build on 'ssh://mac2': cannot connect to 'mac2': Host key verification failed. cannot build on 'ssh://mac3': cannot connect to 'mac3': Received disconnect from 213... port 6001:2: Too many authentication failures Authentication failed.
2018-03-19 serialise.cc: remove pessimising moveWill Dietz1-1/+1
from clang6: src/libutil/serialise.cc:189:23: warning: moving a temporary object prevents copy elision [-Wpessimizing-move]
2018-03-19 util: rename stdout/stdin members to avoid conflicts w/standard macroWill Dietz2-9/+9
(cherry picked from commit c389a7fb617ed7bcd617efa68c6a48c00405310d)
2018-03-16 Reduce substitution memory consumptionEelco Dolstra3-1/+87
copyStorePath() now pipes the output of srcStore->narFromPath() directly into dstStore->addToStore(). The sink used by the former is converted into a source usable by the latter using boost::coroutine2. This is based on [1]. This reduces the maximum resident size of $ nix build --store ~/my-nix/ /nix/store/b0zlxla7dmy1iwc3g459rjznx59797xy-binutils-2.28.1 --substituters file:///tmp/binary-cache-xz/ --no-require-sigs from 418592 KiB to 53416 KiB. (The previous commit also reduced the runtime from ~4.2s to ~3.4s, not sure why.) A further improvement will be to download files into a Sink. [1] https://github.com/NixOS/nix/compare/master...Mathnerd314:dump-fix-coroutine#diff-dcbcac55a634031f9cc73707da6e4b18 Issue #1969.
2018-03-16 decompress(): Use a Source and SinkEelco Dolstra5-75/+198
This allows decompression to happen in O(1) memory.
2018-03-15 Filter ANSI colors when not writing to a terminalEelco Dolstra3-7/+9
Fixes https://github.com/NixOS/nixpkgs/issues/37114.
2018-03-15 Merge pull request #1965 from masaeedu/masterEelco Dolstra2-6/+16
Wrap thread local in function for Cygwin
2018-03-14 Catch more possible instances of passing NULL to memcpy.Shea Levy1-0/+1
Actually fixes #1976.
2018-03-12 Wrap thread local in function for CygwinAsad Saeeduddin2-6/+16
Fixes #1826. See #1352 for a previous instance of a similar change.
2018-03-09 Fix double free in Store::queryPathInfo()Eelco Dolstra1-4/+4
It was holding on to a Value* (i.e. a std::shared_ptr<ValidPathInfo>*) outside of the pathInfoCache lock, so the std::shared_ptr could be destroyed between the release of the lock and the decrement of the std::shared_ptr refcount. This can happen if more than 'path-info-cache-size' paths are added in the meantime, *or* if clearPathInfoCache() is called. The hydra-queue-runner queue monitor thread periodically calls the later, so is likely to trigger a crash. Fixes https://github.com/NixOS/hydra/issues/542.
2018-03-06 logging: Don't throw from Activity destructorWill Dietz2-2/+9
Move definition of destructor to logging.cc for access to util.hh's ignoreException.
2018-03-02 don't allocate large buffers on the stackWill Dietz3-23/+24
2018-03-02 Merge branch 'write-failure-fixes' of git://github.com/lheckemann/nixShea Levy1-1/+2
2018-02-26 nix run: Fix segfault on macOSEelco Dolstra2-0/+10
Note that clearenv() is not available on macOS. Fixes #1907.
2018-02-19 libutil: Fix invalid assert on decoding base64 hashesTuomas Tynkkynen1-1/+2
The assertion is broken because there is no one-to-one mapping from length of a base64 string to the length of the output. E.g. "1q69lz7Empb06nzfkj651413n9icx0njmyr3xzq1j9q=" results in a 32-byte output. "1q69lz7Empb06nzfkj651413n9icx0njmyr3xzq1j9qy" results in a 33-byte output. To reproduce, evaluate: builtins.derivationStrict { name = "0"; builder = "0"; system = "0"; outputHashAlgo = "sha256"; outputHash = "1q69lz7Empb06nzfkj651413n9icx0njmyr3xzq1j9qy"; } Found by afl-fuzz.
2018-02-19 libutil: Fix infinite loop in filterANSIEscapes on '\r'Tuomas Tynkkynen1-1/+1
E.g. nix-instantiate --eval -E 'abort "\r"' hangs. Found by afl-fuzz.
2018-02-19 Config::handleUnknownSettings(): Remove unused 'fatal' argumentEelco Dolstra2-6/+3
2018-02-19 Merge branch 'register-settings' of https://github.com/shlevy/nixEelco Dolstra2-25/+25
2018-02-19 Merge pull request #1857 from dtzWill/fix/check-for-lzma-mtEelco Dolstra1-23/+37
configure.ac: check if lzma has MT support, fix deb build/etc.
2018-02-14 monitor-fds: Fix on macOS.Shea Levy1-7/+23
Fixes #1871.
2018-02-13 Allow plugins to define new settings.Shea Levy2-25/+25
2018-02-13 Fix #1762Linus Heckemann1-1/+2
nix-store --export, nix-store --dump, and nix dump-path would previously fail silently if writing the data out failed, because a) FdSink::write ignored exceptions, and b) the commands relied on FdSink's destructor, which ignores exceptions, to flush the data out. This could cause rather opaque issues with installing nixos, because nix-store --export would happily proceed even if it couldn't write its data out (e.g. if nix-store --import on the other side of the pipe failed). This commit adds tests that expose these issues in the nix-store commands, and fixes them for all three.
2018-02-13 Allow includes from nix.confShea Levy1-1/+25
2018-02-11 compression: make parallel sink separate classWill Dietz1-29/+34
2018-02-11 compression: print warning if parallel requested but not supportedWill Dietz1-1/+8
2018-02-09 configure.ac: check if lzma has MT support, fix deb build/etc.Will Dietz1-0/+2
2018-02-07 make multi threaded compression configurable and use single threadedAmineChikhaoui2-19/+27
by default.
2018-02-07 Merge branch 'master' of https://github.com/NixOS/nix into parallel-xzAmineChikhaoui3-31/+48
2018-02-07 Improve filtering of ANSI escape sequences in build logsEelco Dolstra3-31/+48
All ANSI sequences except color setting are now filtered out. In particular, terminal resets (such as from NixOS VM tests) are filtered out. Also, fix the completely broken tab character handling.
2018-02-07 set block size to 0 to let the lzma lib choose the right one, addAmineChikhaoui1-1/+6
some comments about possible improvements wrt memory usage/threading.
2018-02-06 support multi threaded xz encoder, this might be particularly useful inAmineChikhaoui1-2/+7
the case of hydra where the overhead of single threaded encoding is more noticeable e.g most of the time spent in "Sending inputs"/"Receiving outputs" is due to compression while the actual upload to the binary cache seems to be negligible.
2018-01-16 Add pure evaluation modeEelco Dolstra2-2/+10
In this mode, the following restrictions apply: * The builtins currentTime, currentSystem and storePath throw an error. * $NIX_PATH and -I are ignored. * fetchGit and fetchMercurial require a revision hash. * fetchurl and fetchTarball require a sha256 attribute. * No file system access is allowed outside of the paths returned by fetch{Git,Mercurial,url,Tarball}. Thus 'nix build -f ./foo.nix' is not allowed. Thus, the evaluation result is completely reproducible from the command line arguments. E.g. nix build --pure-eval '( let nix = fetchGit { url = https://github.com/NixOS/nixpkgs.git; rev = "9c927de4b179a6dd210dd88d34bda8af4b575680"; }; nixpkgs = fetchGit { url = https://github.com/NixOS/nixpkgs.git; ref = "release-17.09"; rev = "66b4de79e3841530e6d9c6baf98702aa1f7124e4"; }; in (import (nix + "/release.nix") { inherit nix nixpkgs; }).build.x86_64-linux )' The goal is to enable completely reproducible and traceable evaluation. For example, a NixOS configuration could be fully described by a single Git commit hash. 'nixos-rebuild' would do something like nix build --pure-eval '( (import (fetchGit { url = file:///my-nixos-config; rev = "..."; })).system ') where the Git repository /my-nixos-config would use further fetchGit calls or Git externals to fetch Nixpkgs and whatever other dependencies it has. Either way, the commit hash would uniquely identify the NixOS configuration and allow it to reproduced.
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-12-21T02·48+0000