about summary refs log tree commit diff
path: root/third_party/nix/src/libstore/build.cc (follow)
AgeCommit message (Collapse)AuthorFilesLines
2020-11-28 r/1946 feat(tvix): Thread a log sink through calls to buildDerivationGriffin Smith1-3/+3
Similarly to how we did for buildPaths, add a std::ostream& log_sink parameter to the build_derivation method on Store, and pass it std::cerr when called at the top level by nix commands - most notably, the build-remote hook binary, so that we get build logs when using tvix as a remote builder. Change-Id: I0f8f729ba8429d4838a0a135a5c2ac1e1a95d575 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2176 Tested-by: BuildkiteCI Reviewed-by: andi <andi@notmuch.email> Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-11-27 r/1943 refactor(tvix): Prefer absl::StrFormat/StrAppendGriffin Smith1-6/+13
This fmt call was particularly egregious Change-Id: I2a3b1006c285170ab3374d1c8d81fc53e82a7b05 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2174 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-11-27 r/1940 fix(tvix): Remove more superfluous log outputGriffin Smith1-1/+1
- A couple of log messages had double-newlines still - Remove the [ drvPath ] bit from the "building path" output, since the derivation path is already present in the message itself. Change-Id: Ib4b514a18fcb63c7191f4d7cf0a9b857aaa3f3fa Reviewed-on: https://cl.tvl.fyi/c/depot/+/2171 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-11-27 r/1933 fix(tvix): Don't double-up on endlines from log messagesGriffin Smith1-1/+1
Change-Id: I676dd4e9d8b1a7ff5389323e0462ba6681d62cf0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2168 Reviewed-by: andi <andi@notmuch.email> Reviewed-by: kanepyork <rikingcoding@gmail.com> Tested-by: BuildkiteCI
2020-11-22 r/1909 fix(tvix): Add endlines to build log messagesGriffin Smith1-16/+19
Change-Id: Id6e871a2b7fbb24fb5c8e84e1638bfe94e363b57 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2135 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-11-21 r/1894 chore(3p): Bump nixpkgs to nixos-unstable from 2020-11-21Vincent Ambo1-10/+7
Included fixes for random breakage: * 3p/awscli: pick from the stable channel; it is broken on unstable * 3p/googletest: bumped version & removed patches that nixpkgs applies * 3p/lisp/cffi: bumped library version for SBCL compat * 3p/nix: fix libsystemd attribute * 3p/nix: reformatted (clang-format handling of ternaries changed) * glittershark/home: Use home-manager from nixkpgs * glittershark/kernel: bumped linux-ck patch hash * glittershark/kernel: removed "patch patch" * multi/whitby: Use home-manager from nixpkgs * tazjin/frog: drop Sourcetrail (it doesn't build currently) Note that in addition to these changes, some previous CLs updated the versions of git and cgit which was necessary for this channel bump, but which could not be done in the same commit due to the nature of the subtree merges. Change-Id: If2563e8a68e2750c4b913a976ff7b93b42e8b7f3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2110 Tested-by: BuildkiteCI Reviewed-by: multi <depot@in-addr.xyz> Reviewed-by: glittershark <grfn@gws.fyi>
2020-09-14 r/1791 refactor(tvix): Make static strings constexpr string_viewsGriffin Smith1-3/+3
Make all static std::strings constexpr std::string_views, and replace concatenation with absl::StrCat where necessary. Technically all of these are constant, so they really don't need to be top-level statics - and since I'm trying to get rid of as much global state as possible in preparation for making the nix daemon properly multithreaded I figured I'd knock these out while I was at it. Change-Id: Ibd3ad9ef68f0a0eacb135541b39fdb13dae042e1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1939 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-09-10 r/1782 fix(tvix): Make working flag thread_localGriffin Smith1-5/+4
Since the daemon is running in threads now rather than forking a process per connection (thanks to grpc) this static flag to prevent accidentally initializing a new worker during a build is getting stepped on by multiple threads. This converts it to thread-local, and also adds an actual message to the check so that if it hits in the future we know what's going on. Fixes: b/58 Change-Id: I07a2f1582e56709c104f79935e5405fa24888f59 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1940 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com> Reviewed-by: V <v@anomalous.eu>
2020-09-06 r/1767 feat(tvix): Write even more build logs to clientGriffin Smith1-89/+84
Write *all* build logs (essentially everything that wasn't a DLOG), not just the stdout of the builder, to the client-bound log_sink_ that's threaded through build.cc. These logs are significantly more useful on the client, both when debugging tvix itself and (more importantly) when building things with it. To make this all work, this commit also refactors the passing-around of the log sink in build.cc slightly - the main reference is now owned by the Worker class, which already stores pointers to itself on the Goals it creates, so those Goals can then load the log sink from the Worker. This change was originally made to allow for logging to the log_sink from instance methods of the worker, but actually feels significantly more natural than storing the sink on the goal anyway. Change-Id: I461685bc7f5c0a326054940ac5971b568dd43f8e Reviewed-on: https://cl.tvl.fyi/c/depot/+/1925 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-09-01 r/1758 refactor(tvix): Remove default buildMode arguments from libstoreGriffin Smith1-4/+4
These were not used anyway, and *may* have been causing a weird series of problems leading to things sometimes aborting. Or maybe it has nothing to do with it - the only correlation we've found is from disassembling the output of a core dump. Regardless, this feels stylistically like a good idea. Change-Id: I288b50945e74ea90c118a21767fa4387c9f47f7d Reviewed-on: https://cl.tvl.fyi/c/depot/+/1921 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-09-01 r/1756 refactor(tvix): getEnv(): Return std::optionalEelco Dolstra1-1/+1
This allows distinguishing between an empty value and no value. Patch ported from upstream at https://github.com/NixOS/nix/commit/ba87b08f8529e4d9f8c58d8c625152058ceadb75 Change-Id: I061cc8e16b1a7a0341adfc3b0edca1c0c51d5c97 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1884 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-23 r/1704 refactor(tvix/libutil): Mark single-argument constructors explicitVincent Ambo1-29/+31
This is the clang-tidy lint 'google-explicit-constructor'. There's a whole bunch of breakage that was introduced by this, and we had to opt out a few types of this (esp. the string formatting crap). In some cases minor other changes have been done to keep the code working, instead of converting between types (e.g. an explicit comparison operator implementation for nix::Pid). Change-Id: I12e1ca51a6bc2c882dba81a2526b9729d26988e7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1832 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com> Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-21 r/1699 fix(tvix): Add newlines to build logsVincent Ambo1-2/+2
.. turns out producing an entire build log on a single line is not useful. This does not use `<< std::endl` because we have run into buffering issues with the implementation of the logs->gRPC sink, but intend to replace this in the future using a structured sink for BuildEvent protos rather than a raw stream. Change-Id: Ia9b05fa804391d389e2ef53ab4436c0ec5cc452e Reviewed-on: https://cl.tvl.fyi/c/depot/+/1828 Reviewed-by: glittershark <grfn@gws.fyi> Tested-by: BuildkiteCI
2020-08-21 r/1697 feat(tvix): Write build logs into the build log sinkVincent Ambo1-3/+4
This *should* wire up the builder's logs all the way back through the gRPC client, where they are then conveniently discarded. Change-Id: I65f22526d0b5a8b8d90f28665bc1b4bc7f7c802a Reviewed-on: https://cl.tvl.fyi/c/depot/+/1825 Reviewed-by: glittershark <grfn@gws.fyi> Tested-by: BuildkiteCI
2020-08-21 r/1696 chore(tvix): Thread std::ostream through builder goalsVincent Ambo1-42/+66
This passes an output stream for build logs to almost all relevant functions inside of build.cc by threading it through the `Goal`-abstraction. Store calls that create goals but don't have a sink available use the DiscardLogsSink(). Change-Id: I2c0cb1aec1f9150f33113f4752055cea518ede8b Reviewed-on: https://cl.tvl.fyi/c/depot/+/1824 Reviewed-by: glittershark <grfn@gws.fyi> Tested-by: BuildkiteCI
2020-08-20 r/1689 chore(tvix): Thread a std::ostream through Store::buildPathsVincent Ambo1-4/+6
This part of the store API needs to carry a handle to the log sink from now on, so that it can be passed in as appropriate from the gRPC handlers. In all places where there is no such handler available at the moment, the discarding log sink has been inserted. This can be used as a convenient grep target in the future. Change-Id: I26628e30b4c6437dccdf8f722ca2e8ed827dfc19 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1797 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com> Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-20 r/1688 style(tvix): typedef -> using in build.cc/store-api.hhVincent Ambo1-9/+9
this shuts up a few lints ... Change-Id: I51ee9418767e97247f98b170cdad137e5b12b44d Reviewed-on: https://cl.tvl.fyi/c/depot/+/1796 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com> Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-14 r/1649 refactor(tvix): Make Store::buildPaths return a StatusGriffin Smith1-2/+8
Make Store::buildPaths return a Status with [[nodiscard]] rather than throwing exceptions to signal failure. This is the beginning of a long road to refactor the entire store API to be status/statusor based instead of using exceptions. Change-Id: I2e32371c95a25b87ad129987c217d49c6d6e0c85 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1745 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-10 r/1632 style(tvix): Sort out minor formatting errorsVincent Ambo1-2/+2
Invocations of the MakeError macro that were not followed by a semicolon messed up indentation in the next lines. Change-Id: I03d7d1443f062a38af2c7da3da8928e0ed05e274 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1708 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-08 r/1619 chore(3p/nix): apply google-readability-castingKane York1-14/+17
Command run: jq <compile_commands.json -r 'map(.file)|.[]' | grep -v '/generated/' | parallel clang-tidy -p compile_commands.json -checks=-*,google-readability-casting --fix Manual fixes applied in src/nix-env/nix-env.cc, src/libstore/store-api.cc Change-Id: I406b4be9368c557ca59329bf6f7002704e955f8d Reviewed-on: https://cl.tvl.fyi/c/depot/+/1557 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi> Reviewed-by: tazjin <mail@tazj.in>
2020-08-06 r/1613 fix(3p/nix): Use SkipEmpty in all calls to absl::StrSplitKane York1-1/+2
The behavior to return a list containing a single empty string when provided an empty string is a behavior that absl inherited from legacy code. However, the behavior expected by legacy code in Nix is the behavior provided by the SkipEmpty option. Switch all calls to use SkipEmpty, except for the call already using SkipWhitespace. See also commit 26a59482d2427f640893517f1b24dd650a5bd5da, with the partly-prophetic message: "there may be other places we need to fix this as well." Change-Id: I6e94856a12cfb1b7e4a3b4e221769ed446648861 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1687 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-06 r/1611 refactor(tvix): Use absl::btree_map for DerivationInputsVincent Ambo1-1/+1
Change-Id: If160ab1b09161969d9080d5d0d6f82e44a953e3c Reviewed-on: https://cl.tvl.fyi/c/depot/+/1684 Reviewed-by: glittershark <grfn@gws.fyi> Tested-by: BuildkiteCI
2020-08-04 r/1574 fix(3p/nix): pass SkipEmpty to StrSplit("", ...)Griffin Smith1-4/+8
When tokenizeString was changed to absl::StrSplit, there was a behavior change because tokenizeString on an empty string returned an empty vector - which the derivation builder (and likely a bunch of other stuff) was depending on. The canonical way of fixing this is by passing absl::SkipEmpty() to the function - there may be other places we need to fix this as well. This commit also includes some opportunistic absl::StrFormats and StrCats, because I was here anyway, but those have no semantic difference. Change-Id: Ibf9bb602284f793fa55728481f63b838fb7a41db Reviewed-on: https://cl.tvl.fyi/c/depot/+/1631 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2020-08-01 r/1534 fix(3p/nix): revert "apply all clang-tidy fixes"Kane York1-39/+36
This reverts commit ef54f5da9fa30b5c302f2a49595ee5d041f9706a. Resolved conflicts: third_party/nix/src/libexpr/eval.cc third_party/nix/src/libstore/builtins/fetchurl.cc third_party/nix/src/libstore/references.cc third_party/nix/src/libutil/hash.cc third_party/nix/src/nix-daemon/nix-daemon.cc Change-Id: Ib9cf6e96a79a23bde3983579ced3f92e530cb011 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1547 Reviewed-by: glittershark <grfn@gws.fyi> Tested-by: BuildkiteCI
2020-07-27 r/1495 fix(3p/nix): apply all clang-tidy fixesKane York1-36/+39
Change-Id: I265e763393422ee1881653527c91024458060825 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1432 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-07-24 r/1457 fix(3p/nix): do not call vforkKane York1-3/+0
The use of vfork() in Nix is entirely illegal. Quote: If the process created by vfork() returns from the function in which vfork() was called, or calls any other function before successfully calling _exit() or one of the exec*() family of functions, the behavior is undefined. -- Linux man-pages, release 5.05 Add a TODO to use the higher-performance variants of clone() on Linux when it is available. Change-Id: I42370e1568ad6e2d00d70d0b66c8aded8f1288bb Reviewed-on: https://cl.tvl.fyi/c/depot/+/1418 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: Alyssa Ross <hi@alyssa.is>
2020-07-23 r/1437 fix(3p/nix): remove usage of strcpyKane York1-1/+1
Change-Id: I86125609f433469a8722c780fd758234211d677e Reviewed-on: https://cl.tvl.fyi/c/depot/+/1381 Tested-by: BuildkiteCI Reviewed-by: Alyssa Ross <hi@alyssa.is> Reviewed-by: glittershark <grfn@gws.fyi>
2020-05-31 r/876 fix(3p/nix): passAsFile: leave out the hash prefixedef1-1/+1
Having a colon in the path may cause issues, and having the hash function indicated isn't actually necessary. We now verify the path format in the tests to prevent regressions. (cherry picked from commit c65a6fa86aef7bdf51fb4fba7bd31d265619ba3f)
2020-05-31 r/875 fix(3p/nix): passAsFile: hash the attribute name instead of numbering ↵Puck Meerburg1-2/+2
sequentially This makes the paths consistent without relying on ordering. Co-authored-by: edef <edef@edef.eu> (cherry picked from commit 515c0a263e137a00e82f7d981284dbe54db23247)
2020-05-27 r/865 refactor(3p/nix): Anchor local includes at src/Vincent Ambo1-16/+16
Previously all includes were anchored in one global mess of header files. This moves the includes into filesystem "namespaces" (if you will) for each sub-package of Nix. Note: This commit does not introduce the relevant build system changes.
2020-05-25 r/846 refactor(3p/nix): Replace tokenizeStrings with absl::StrSplitVincent Ambo1-2/+5
This function was a custom (and inefficient in the case of single-character delimiters) string splitter which was used all over the codebase. Abseil provides an appropriate replacement function.
2020-05-25 r/843 refactor(3p/nix/libutil): Replace string2Int & trim functionsVincent Ambo1-2/+4
Replaces these functions with corresponding functions from Abseil, namely absl::StripAsciiWhitespace and absl::SimpleAtoi. In the course of doing this some minor things I encountered along the way were also refactored. This also changes the signatures of the various custom readFile functions to use absl::string_view types.
2020-05-24 r/840 style(3p/nix): Remove 'using std::*' from types.hhVincent Ambo1-66/+65
It is considered bad form to use things from includes in headers, as these directives propagate to everywhere else and can make it confusing. types.hh (which is includes almost literally everywhere) had some of these directives, which this commit removes.
2020-05-24 r/835 refactor(3p/nix/libutil): Replace chomp() with absl::stringsVincent Ambo1-3/+6
2020-05-24 r/834 chore(3p/nix): Remove some OS X specific definesVincent Ambo1-184/+1
This project will be dropping OS X support until the core is simplified.
2020-05-20 r/789 refactor(3p/nix): Apply clang-tidy's performance-* fixesVincent Ambo1-19/+21
This applies the performance fixes listed here: https://clang.llvm.org/extra/clang-tidy/checks/list.html
2020-05-20 r/788 refactor(3p/nix): Apply clang-tidy's readability-* fixesVincent Ambo1-57/+58
This applies the readability fixes listed here: https://clang.llvm.org/extra/clang-tidy/checks/list.html
2020-05-20 r/787 refactor(3p/nix): Apply clang-tidy's modernize-* fixesVincent Ambo1-50/+51
This applies the modernization fixes listed here: https://clang.llvm.org/extra/clang-tidy/checks/list.html The 'modernize-use-trailing-return-type' fix was excluded due to my personal preference (more specifically, I think the 'auto' keyword is misleading in that position).
2020-05-19 r/778 refactor(3p/nix): Make all single-argument constructors explicitVincent Ambo1-11/+11
Implicit constructors can be confusing, especially in a codebase that is already as unintentionally obfuscated as this one. https://google.github.io/styleguide/cppguide.html#Explicit_Constructors
2020-05-19 r/777 style(3p/nix): Final act in the brace-wrapping sagaVincent Ambo1-201/+375
This last change set was generated by a full clang-tidy run (including compilation): clang-tidy -p ~/projects/nix-build/ \ -checks=-*,readability-braces-around-statements -fix src/*/*.cc Actually running clang-tidy requires some massaging to make it play nice with Nix + meson, I'll be adding a wrapper or something for that soon.
2020-05-19 r/772 style(3p/nix): Add braces around single-line for-loopsVincent Ambo1-17/+51
These were not caught by the previous clang-tidy invocation, but were instead sorted out using amber[0] as such: ambr --regex 'for (\(.+\))\s([a-z].*;)' 'for $1 { $2 }' [0]: https://github.com/dalance/amber
2020-05-19 r/771 style(3p/nix): Add braces around single-line conditionalsVincent Ambo1-81/+243
These were not caught by the previous clang-tidy invocation, but were instead sorted out using amber[0] as such: ambr --regex 'if (\(.+\))\s([a-z].*;)' 'if $1 { $2 }' [0]: https://github.com/dalance/amber
2020-05-19 r/767 style(3p/nix): Enforce braces around loops and conditionalsVincent Ambo1-16/+42
This change was generated with: fd -e cc -e hh | xargs -I{} clang-tidy {} -p ~/projects/nix-build/ \ --checks='-*,readability-braces-around-statements' --fix \ -fix-errors Some manual fixes were applied because some convoluted unbraced statements couldn't be untangled by clang-tidy. This commit still includes invalid files, but I decided to clean them up in a subsequent commit so that it becomes more obvious where clang-tidy failed. Maybe this will allow for a bug-report to clang-tidy.
2020-05-19 r/766 style(3p/nix): Reformat all includes to match new styleVincent Ambo1-10/+12
2020-05-19 r/756 refactor(3p/nix/libstore): Replace logging.h with glogVincent Ambo1-171/+125
2020-05-17 r/740 style(3p/nix): Reformat project in Google C++ styleVincent Ambo1-3901/+3792
Reformatted with: fd . -e hh -e cc | xargs clang-format -i
2020-05-17 r/724 Add 'third_party/nix/' from commit 'be66c7a6b24e3c3c6157fd37b86c7203d14acf10'Vincent Ambo1-0/+4708
git-subtree-dir: third_party/nix git-subtree-mainline: cf8cd640c1adf74a3706efbcb0ea4625da106fb2 git-subtree-split: be66c7a6b24e3c3c6157fd37b86c7203d14acf10
generated by cgit-pink 1.4.1 (git 2.44.1) at 2024-07-01T01·00+0000