about summary refs log tree commit diff
AgeCommit message (Collapse)AuthorFilesLines
2012-08-13 Optimise concatenating a list to an empty listEelco Dolstra1-2/+10
More precisely, in concatLists, if all lists except one are empty, then just return the non-empty list. This reduces the number of list element allocations by 32% when evaluating a NixOS system configuration.
2012-08-13 Add a primop ‘elemAt’ to get an element from a listEelco Dolstra1-5/+19
2012-08-13 Add a primop ‘concatLists’Eelco Dolstra3-8/+33
This can serve as a generic efficient list builder. For instance, the function ‘catAttrs’ in Nixpkgs can be rewritten from attr: l: fold (s: l: if hasAttr attr s then [(getAttr attr s)] ++ l else l) [] l to attr: l: builtins.concatLists (map (s: if hasAttr attr s then [(getAttr attr s)] else []) l) Statistics before: time elapsed: 1.08683 size of a value: 24 environments allocated: 1384376 (35809568 bytes) list elements: 6946783 (55574264 bytes) list concatenations: 37434 values allocated: 1760440 (42250560 bytes) attribute sets allocated: 392040 right-biased unions: 186334 values copied in right-biased unions: 591137 symbols in symbol table: 18273 number of thunks: 1297673 number of thunks avoided: 1380759 number of attr lookups: 430802 number of primop calls: 628912 number of function calls: 1333544 Statistics after (including new catAttrs): time elapsed: 0.959854 size of a value: 24 environments allocated: 1010198 (26829296 bytes) list elements: 1984878 (15879024 bytes) list concatenations: 30488 values allocated: 1589760 (38154240 bytes) attribute sets allocated: 392040 right-biased unions: 186334 values copied in right-biased unions: 591137 symbols in symbol table: 18274 number of thunks: 1040925 number of thunks avoided: 1038428 number of attr lookups: 438419 number of primop calls: 474844 number of function calls: 959366
2012-08-13 Provide an efficient implementation of ‘elem’Eelco Dolstra1-2/+17
The one in Nixpkgs is O(n^2), this one is O(n). Big reduction in the number of list allocations. Statistics before (on a NixOS system config): time elapsed: 1.17982 size of a value: 24 environments allocated: 1543334 (39624560 bytes) list elements: 9612638 (76901104 bytes) list concatenations: 37434 values allocated: 1854933 (44518392 bytes) attribute sets allocated: 392040 right-biased unions: 186334 values copied in right-biased unions: 591137 symbols in symbol table: 18272 number of thunks: 1392467 number of thunks avoided: 1507311 number of attr lookups: 430801 number of primop calls: 691600 number of function calls: 1492502 Statistics after: time elapsed: 1.08683 size of a value: 24 environments allocated: 1384376 (35809568 bytes) list elements: 6946783 (55574264 bytes) list concatenations: 37434 values allocated: 1760440 (42250560 bytes) attribute sets allocated: 392040 right-biased unions: 186334 values copied in right-biased unions: 591137 symbols in symbol table: 18273 number of thunks: 1297673 number of thunks avoided: 1380759 number of attr lookups: 430802 number of primop calls: 628912 number of function calls: 1333544
2012-08-13 Add a "filter" primopEelco Dolstra2-1/+26
Evaluation of a NixOS configuration spends quite a lot of time in the "filter" function in Nixpkgs. As implemented in Nixpkgs, this is a O(n^2) operation, so it's a good candidate for providing a more efficient (i.e. primop) implementation. Using it gives a ~10% speed increase and a significant reduction in the number of evaluations. Statistics before (on a NixOS system config): time elapsed: 1.3258 size of a value: 24 environments allocated: 1980939 (50127080 bytes) list elements: 14679308 (117434464 bytes) list concatenations: 50828 values allocated: 2098938 (50374512 bytes) attribute sets allocated: 392040 right-biased unions: 186334 values copied in right-biased unions: 591137 symbols in symbol table: 18271 number of thunks: 1645752 number of thunks avoided: 1921196 number of attr lookups: 430798 number of primop calls: 838807 number of function calls: 1930107 Statistics after: time elapsed: 1.17982 size of a value: 24 environments allocated: 1543334 (39624560 bytes) list elements: 9612638 (76901104 bytes) list concatenations: 37434 values allocated: 1854933 (44518392 bytes) attribute sets allocated: 392040 right-biased unions: 186334 values copied in right-biased unions: 591137 symbols in symbol table: 18272 number of thunks: 1392467 number of thunks avoided: 1507311 number of attr lookups: 430801 number of primop calls: 691600 number of function calls: 1492502
2012-08-12 Add some more evaluations statsEelco Dolstra2-1/+12
2012-08-12 Add some basic profiling support to the evaluatorEelco Dolstra5-4/+65
Setting the environment variable NIX_COUNT_CALLS to 1 enables some basic profiling in the evaluator. It will count calls to functions and primops as well as evaluations of attributes. For example, to see where evaluation of a NixOS configuration spends its time: $ NIX_SHOW_STATS=1 NIX_COUNT_CALLS=1 ./src/nix-instantiate/nix-instantiate '<nixos>' -A system --readonly-mode ... calls to 39 primops: 239532 head 233962 tail 191252 hasAttr ... calls to 1595 functions: 224157 `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/pkgs/lib/lists.nix:17:19' 221767 `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/pkgs/lib/lists.nix:17:14' 221767 `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/pkgs/lib/lists.nix:17:10' ... evaluations of 7088 attributes: 167377 undefined position 132459 `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/pkgs/lib/attrsets.nix:119:41' 47322 `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/pkgs/lib/attrsets.nix:13:21' ...
2012-08-07 Don't warn about maximum link count exceeded on 0-byte filesEelco Dolstra1-2/+4
2012-08-05 Handle amount of disk space saved by hard linking being negativeEelco Dolstra1-2/+2
Fixes bogus messages like "currently hard linking saves 17592186044416.00 MiB".
2012-08-05 Fix race condition when two processes create a hard link to a file in .linksEelco Dolstra1-27/+37
This is a problem because one process may set the immutable bit before the second process has created its link. Addressed random Hydra failures such as: error: cannot rename `/nix/store/.tmp-link-17397-1804289383' to `/nix/store/rsvzm574rlfip3830ac7kmaa028bzl6h-nixos-0.1pre-git/upstart-interface-version': Operation not permitted
2012-08-05 Fix race condition when two processes create the same link in /nix/store/.linksEelco Dolstra1-4/+4
2012-08-01 Count bytes freed deleting unused linksEelco Dolstra2-3/+5
2012-08-01 Drop the block count in the garbage collectorEelco Dolstra9-37/+23
2012-08-01 nix-store --gc: Make ‘--max-freed 0’ do the right thingEelco Dolstra4-7/+6
That is, delete almost nothing (it will still remove unused links from /nix/store/.links).
2012-08-01 removeUnusedLinks(): Print stats on disk space saved by hard linkingEelco Dolstra1-1/+16
2012-08-01 DohEelco Dolstra1-1/+2
2012-08-01 Make ‘nix-store --optimise’ interruptibleEelco Dolstra1-0/+2
2012-07-30 Manual: Remove reference to non-existent -I optionEelco Dolstra1-15/+0
2012-07-29 Document the --option flagEelco Dolstra2-1/+16
Pointed out by Daniel Santa Cruz on IRC.
2012-07-26 Set permissions on temporary build directories to 0700Eelco Dolstra3-4/+5
Fixes #39.
2012-07-25 nix-profile.sh: Don't set NIX_REMOTE on single user installationsEelco Dolstra1-1/+1
Commit 6a214f3e06fa1c5f0a4d40e555f14d87691af297 reused the NixOS environment initialisation for nix-profile.sh, but this is inappropriate on systems that don't have multi-user support enabled.
2012-07-25 Subscribe to the Nixpkgs rather than NixOS channelEelco Dolstra1-2/+2
2012-07-25 prim_import: When importing .drvs, allocate the intermediate attrset on the ↵Shea Levy1-1/+1
heap just in case it escapes the stack frame.
2012-07-25 Turn tests back onShea Levy1-1/+1
2012-07-25 import: If the path is a valid .drv file, parse it and generate a derivation ↵Shea Levy3-2/+48
attrset. The generated attrset has drvPath and outPath with the right string context, type 'derivation', outputName with the right name, all with a list of outputs, and an attribute for each output. I see three uses for this (though certainly there may be more): * Using derivations generated by something besides nix-instantiate (e.g. guix) * Allowing packages provided by channels to be used in nix expressions. If a channel installed a valid deriver for each package it provides into the store, then those could be imported and used as dependencies or installed in environment.systemPackages, for example. * Enable hydra to be consistent in how it treats inputs that are outputs of another build. Right now, if an input is passed as an argument to the job, it is passed as a derivation, but if it is accessed via NIX_PATH (i.e. through the <> syntax), then it is a path that can be imported. This is problematic because the build being depended upon may have been built with non-obvious arguments passed to its jobset file. With this feature, hydra can just set the name of that input to the path to its drv file in NIX_PATH
2012-07-25 Disable tests temporarilyShea Levy1-1/+1
2012-07-23 Handle platforms that don't support linking to a symlinkEelco Dolstra2-2/+19
E.g. Darwin doesn't allow this.
2012-07-23 Unlink the right fileEelco Dolstra1-2/+2
2012-07-23 Add a test for Nix store optimisationEelco Dolstra2-1/+27
2012-07-23 Garbage collect unused links in /nix/store/.linksEelco Dolstra2-0/+39
Incremental optimisation requires creating links in /nix/store/.links to all files in the store. However, this means that if we delete a store path, no files are actually deleted because links in /nix/store/.links still exists. So we need to check /nix/store/.links for files with a link count of 1 and delete them.
2012-07-23 Automatically optimise the Nix store when a new path is addedEelco Dolstra5-10/+45
Auto-optimisation is enabled by default. It can be turned off by setting auto-optimise-store to false in nix.conf.
2012-07-23 optimiseStore(): Use a content-addressed file store in /nix/store/.linksEelco Dolstra3-115/+110
optimiseStore() now creates persistent, content-addressed hard links in /nix/store/.links. For instance, if it encounters a file P with hash H, it will create a hard link P' = /nix/store/.link/<H> to P if P' doesn't already exist; if P' exist, then P is replaced by a hard link to P'. This is better than the previous in-memory map, because it had the tendency to unnecessarily replace hard links with a hard link to whatever happened to be the first file with a given hash it encountered. It also allows on-the-fly, incremental optimisation.
2012-07-23 nix-build: Support the ‘-’ argument to build an expression from stdinEelco Dolstra1-0/+4
2012-07-23 Use lutimes() if available to canonicalise the timestamp of symlinksEelco Dolstra2-9/+21
Also use utimes() instead of utime() if lutimes() is not available.
2012-07-18 Bump versionEelco Dolstra1-1/+1
2012-07-18 Remove RPM builds that don't evaluateEelco Dolstra1-4/+0
2012-07-18 Use "#pragma once" to prevent repeated header file inclusionEelco Dolstra32-166/+31
2012-07-18 Set release dateEelco Dolstra1-1/+1
2012-07-17 Add function queryPathFromHashPart()Eelco Dolstra9-0/+67
To implement binary caches efficiently, Hydra needs to be able to map the hash part of a store path (e.g. "gbg...zr7") to the full store path (e.g. "/nix/store/gbg...kzr7-subversion-1.7.5"). (The binary cache mechanism uses hash parts as a key for looking up store paths to ensure privacy.) However, doing a search in the Nix store for /nix/store/<hash>* is expensive since it requires reading the entire directory. queryPathFromHashPart() prevents this by doing a cheap database lookup.
2012-07-17 queryPathInfo(): return hash in base-32 if desiredEelco Dolstra1-2/+2
Cherry-picked from the no-manifests branch.
2012-07-17 Return an exit code of 100 for cached failed buildsEelco Dolstra1-0/+1
Exit code 100 should be returned for all permanent failures. This includes cached failures. Fixes #34.
2012-07-17 Remove dead codeEelco Dolstra2-13/+0
2012-07-17 Manual: Don't claim we support CygwinEelco Dolstra1-1/+1
2012-07-17 Update Nix 1.1 release notesEelco Dolstra2-8/+70
2012-07-17 Allow disabling log compressionEelco Dolstra2-12/+35
2012-07-12 builtins.storePath: resolve symlinksEelco Dolstra1-1/+5
Needed for Charon/Hydra interaction.
2012-07-09 Pass --insecure to curl so that https worksEelco Dolstra1-1/+1
2012-07-09 Inline fetchurl.shEelco Dolstra3-9/+13
2012-07-09 Add a test for the fetchurl functionEelco Dolstra3-1/+17
2012-07-09 Remove obsolete commentShea Levy1-3/+0
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-12-29T07·33+0000