Age | Commit message (Collapse) | Author | Files | Lines |
|
clang-tidy: google-build-using-namespace
Change-Id: I07ea10b03a6d9582c0508747698038f7106e8f63
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2181
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
|
|
To make it easier for future tests to use the arbitrary specialisations
we've defined for some nix types, centralize them all in a single
arbitrary.hh header file.
Change-Id: I767f27949cfe7ec55c79901f7d7aa538d2f98c6b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2182
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
Add missing break statements to the big switch block in
BuildResult::FromProto, and cover the whole thing with a rapidcheck
round-trip test.
Change-Id: I7a07ca398cc5f02ca4fd8e6256fd563c6f3aea9d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2178
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
Similarly to how we did for buildPaths, add a std::ostream& log_sink
parameter to the build_derivation method on Store, and pass it std::cerr
when called at the top level by nix commands - most notably, the
build-remote hook binary, so that we get build logs when using tvix as a
remote builder.
Change-Id: I0f8f729ba8429d4838a0a135a5c2ac1e1a95d575
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2176
Tested-by: BuildkiteCI
Reviewed-by: andi <andi@notmuch.email>
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
Upstream doesn't, and for good reason - this might not be a store path,
for example if it's /bin/sh
Fixes: b/73
Change-Id: If9aa96de2cd8ab941c098a9f122b8b302a92ec38
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2175
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Tested-by: BuildkiteCI
|
|
The --quiet and -v/--verbose flags were removed during the glog
conversion in d0c44425e147ab7d38410f400825ad20da15037b, which has
previously broken programs like e.g. home-manager, which passes --quiet
to nix-build.
A nix-build-specific workaround was added in
24f9354d5b8e1521d00f41a7c100b8e9ad327ce7, which manipulates the
FLAGS_stderrthreshold global variable from glog. This commit moves the
--quiet logic back into the argument handling code in libmain, and adds
corresponding handling for -v/--verbose.
Change-Id: I13d860ebbb78541d9f1236691a1efe8bd2163c67
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2170
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
|
|
This fmt call was particularly egregious
Change-Id: I2a3b1006c285170ab3374d1c8d81fc53e82a7b05
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2174
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
This was referencing a nonexistent note in buildPaths, for one, but for
another let's get log outputs when calling this RPC.
Change-Id: Ic9d17834b356ea84d69692ccc0249d09777e833b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2173
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
Subclasses of std::streambuf are expected to override overflow in
addition to xsputn, as it's called in certain cases by the non-virtual
methods. In our case, this was preventing endlines from getting sent
over the log stream.
Change-Id: I70d00f0c7cb8f8cf2f744f58974c21e7a70a715b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2172
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
- A couple of log messages had double-newlines still
- Remove the [ drvPath ] bit from the "building path" output, since the
derivation path is already present in the message itself.
Change-Id: Ib4b514a18fcb63c7191f4d7cf0a9b857aaa3f3fa
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2171
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
Backporting from upstream at d860295e116f9aa0f37e2637b5ec4b9c86fdf4d3
Change-Id: Iefde47ff6f24d21e9b37c9b52a615bd8c9f2fdb2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2141
Tested-by: BuildkiteCI
Reviewed-by: andi <andi@notmuch.email>
|
|
More #3377.
Backported from upstream at 9f46f54de4e55267df492456fc0393f74616366
Change-Id: I11bfca4ec56bd4e45291ce3f98a60f198dff0196
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2140
Tested-by: BuildkiteCI
Reviewed-by: andi <andi@notmuch.email>
|
|
Backported from upstream at a350d0beb0e13d0f58698510bd6a96d894cd06fd
Change-Id: Ib486bc1b36ef65041fab1a4634a0a82e13036fd4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2139
Tested-by: BuildkiteCI
Reviewed-by: andi <andi@notmuch.email>
|
|
Change-Id: I676dd4e9d8b1a7ff5389323e0462ba6681d62cf0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2168
Reviewed-by: andi <andi@notmuch.email>
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Tested-by: BuildkiteCI
|
|
The different signedness of level and withLevel was causing implicit conversions. Use a nullopt instead of a -1 sentinel value.
third_party/nix/src/libexpr/nixexpr.cc:242:21: warning: narrowing conversion from 'unsigned int' to signed type 'int' is implementation-defined [bugprone-narrowing-conversions]
Change-Id: I7c2cadb6fd6bbff6c5b84028651ad4ebba423297
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2157
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: glittershark <grfn@gws.fyi>
|
|
Implicit signedness conversion was causing warnings. Perform a static_cast; this value should never be out of range.
third_party/nix/src/libmain/shared.cc:310:12: warning: narrowing conversion from 'unsigned int' to signed type 'int' is implementation-defined [bugprone-narrowing-conversions]
Change-Id: I34fe53d186ae33f970147135bbe3894fb39a0ec7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2156
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: glittershark <grfn@gws.fyi>
|
|
third_party/nix/src/libstore/sqlite.cc:89:33: warning: narrowing conversion from 'unsigned int' to signed type 'int' is implementation-defined [bugprone-narrowing-conversions]
Change-Id: I1dbc01acaecc4f382f63e7108429cd814ff05f4f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2155
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: glittershark <grfn@gws.fyi>
|
|
backported from upstream at f1fac0b5c3b75efab781949fdff2b67ffdda2cb3
Change-Id: I788e3a9b930351118a5f248b356c351afd7f5391
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2138
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
|
|
This is passed by some third-party tools, notably home-manager. At some
point we should fully revamp argument parsing and log verbosity setting,
but for now this gets us to more of a working state
Fixes: b/78
Change-Id: I4134b17315e30145fd43d3659f407d7121ce8876
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2136
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
|
|
Change-Id: Id6e871a2b7fbb24fb5c8e84e1638bfe94e363b57
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2135
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
|
|
It seems like the amount of logging is causing things to move a little
slower - even if that's not really the case, it gets in the way of
debugging things.
Refs: b/76
Change-Id: I9ea99a3b16e3307a0b0371bad22d03b0e2175af6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2134
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
|
|
To aid in both debugging and (eventually) printing stacktraces, add a
Pos member to a few more Expr variants.
Change-Id: Ic1d2a056fc7e6c07bc3e79fa38845cb4a5da5ca5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2133
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
|
|
Included fixes for random breakage:
* 3p/awscli: pick from the stable channel; it is broken on unstable
* 3p/googletest: bumped version & removed patches that nixpkgs applies
* 3p/lisp/cffi: bumped library version for SBCL compat
* 3p/nix: fix libsystemd attribute
* 3p/nix: reformatted (clang-format handling of ternaries changed)
* glittershark/home: Use home-manager from nixkpgs
* glittershark/kernel: bumped linux-ck patch hash
* glittershark/kernel: removed "patch patch"
* multi/whitby: Use home-manager from nixpkgs
* tazjin/frog: drop Sourcetrail (it doesn't build currently)
Note that in addition to these changes, some previous CLs updated the
versions of git and cgit which was necessary for this channel bump,
but which could not be done in the same commit due to the nature of
the subtree merges.
Change-Id: If2563e8a68e2750c4b913a976ff7b93b42e8b7f3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2110
Tested-by: BuildkiteCI
Reviewed-by: multi <depot@in-addr.xyz>
Reviewed-by: glittershark <grfn@gws.fyi>
|
|
The Abseil version of `StatusOr` does not come with the status macros
or the `Consume*` family of functions.
This change modifies the existing code to use the common denominator
of the API that is available between Abseil's own implementation of
`StatusOr` and the one from Tensorflow that we are currently using.
Change-Id: I5c37f68636a1fd54d153f95d7303ab8644abb774
|
|
Passing a string directly to add_paths like this causes the proto class
to take ownership over the string, meaning when it is destructed it
will *explicitly* free the string. When the string's actual owner (the
derivation struct) then goes out of scope it'll get freed again, causing
a double-free. This fixes that to instead use the copy constructor to
assign to a pointer to a new path, and covers the whole to_proto method
with a rapidcheck test.
Fixes: b/64
Change-Id: I84235bed9104ff430a0acf686d4a96f1e2e9a897
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2106
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
|
|
This was accidentally using the proto arena API to assign the derivation
field of a BuildDerivationRequest. We *thought* this was causing a
double free, but even with this change that's still happening. That
said, this change is probably still a good idea since it's using the
proto API as intended.
References: b/64
Change-Id: I950a4eafb214e9113639ea54d2dfd4659b7be931
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2104
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
|
|
The code that calls queryPathInfoUncached explicitly catches the
InvalidPath exception and translates it into a null result - but the RPC
code was throwing a regular old Error for invalid paths. At some point
we should get rid of all the exception-driven control flow in this whole
thing, but in the meantime this gets us back to functional.
Change-Id: I2a38790ee0c691ab0c8394c7738d7693fa42aa10
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1980
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
Change-Id: I04cc42ae62d2ff71b07fde8e3710dcc9b4363a05
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1973
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
|
|
The static_assert is present to force us to change the constructor when a member grows.
Change-Id: Ifa4f4a03eb7ce13cff109cf26ecf0032045905e9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1972
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
|
|
Starting to reclaim memory. Fairly simple mechanical replacement.
Change-Id: I6b4c5c5596729470d1a049eba61e69e8097decf4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1971
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
|
|
Make all static std::strings constexpr std::string_views, and replace
concatenation with absl::StrCat where necessary.
Technically all of these are constant, so they really don't need to be
top-level statics - and since I'm trying to get rid of as much global
state as possible in preparation for making the nix daemon properly
multithreaded I figured I'd knock these out while I was at it.
Change-Id: Ibd3ad9ef68f0a0eacb135541b39fdb13dae042e1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1939
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
|
|
This is constant, so let's make it a constexpr.
Change-Id: I6a9eb9f740979740f730ebe142d2cbf23dbcfd70
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1961
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
Since the daemon is running in threads now rather than forking a process
per connection (thanks to grpc) this static flag to prevent accidentally
initializing a new worker during a build is getting stepped on by
multiple threads. This converts it to thread-local, and also adds an
actual message to the check so that if it hits in the future we know
what's going on.
Fixes: b/58
Change-Id: I07a2f1582e56709c104f79935e5405fa24888f59
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1940
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Reviewed-by: V <v@anomalous.eu>
|
|
Missed this one
Fixes: b/62
Change-Id: I6f3bbcd3e2a7db5786283761b57466da33a497f7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1936
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Tested-by: BuildkiteCI
|
|
This reverts commit a41c3dedb18201aa689206079e203f41c9bef389.
Reason for revert: I believe its time has come.
Change-Id: I4133dc8a3b57b40b280646d1622ed7a0409c5246
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1941
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
Write *all* build logs (essentially everything that wasn't a DLOG), not
just the stdout of the builder, to the client-bound log_sink_ that's
threaded through build.cc. These logs are significantly more useful on
the client, both when debugging tvix itself and (more importantly) when
building things with it.
To make this all work, this commit also refactors the passing-around of
the log sink in build.cc slightly - the main reference is now owned by
the Worker class, which already stores pointers to itself on the Goals
it creates, so those Goals can then load the log sink from the Worker.
This change was originally made to allow for logging to the log_sink
from instance methods of the worker, but actually feels significantly
more natural than storing the sink on the goal anyway.
Change-Id: I461685bc7f5c0a326054940ac5971b568dd43f8e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1925
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
|
|
These were not used anyway, and *may* have been causing a weird series
of problems leading to things sometimes aborting. Or maybe it has
nothing to do with it - the only correlation we've found is from
disassembling the output of a core dump. Regardless, this feels
stylistically like a good idea.
Change-Id: I288b50945e74ea90c118a21767fa4387c9f47f7d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1921
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
Rather than printing "Nix" in the version string, print "Tvix". This is
cosmetically nice, but also is very useful when making sure I actually
have installed tvix on a system successfully.
Change-Id: Idd1a9954bd66509327f52b7457cfba2f090ab30c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1924
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Tested-by: BuildkiteCI
|
|
This allows distinguishing between an empty value and no value.
Patch ported from upstream at
https://github.com/NixOS/nix/commit/ba87b08f8529e4d9f8c58d8c625152058ceadb75
Change-Id: I061cc8e16b1a7a0341adfc3b0edca1c0c51d5c97
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1884
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
When the nix daemon starts up, first check (using sd_listen_fds) whether
we have been systemd socket-activated. If so, instead of passing the nix
daemon socket path to grpc, start a manual accept(2) loop, passing the
client file descriptors to grpc via AddInsecureChannelFromFd. There's an
open grpc issue at https://github.com/grpc/grpc/issues/19133 for
building support into grpc to do this automatically, but as of right now
this appears to be the only way to make this happen.
Making this happen, by the way, was a bit of a journey - at one point I
attempted to ServerBuilder's experimental AddExternalConnectionAcceptor
API, and that didn't work either - it appears that the final missing
piece to getting this working was explicitly fcntl(2)ing the client file
descriptors to set O_NONBLOCK before passing them into gRPC. With that
set, this all works inside of the test vm.
Fixes: b/56
Change-Id: I5d2ab2b5b02eb570249b30a9674e115c61b0ab0e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1882
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
|
|
Rather than sending the entire AddTextToStore request along in a single
message, send it in a stream of chunks using the same metadata-first
approach we've been using for the other store gRPC requests. This fixes
a bug where certain builds could send more data than the maximum gRPC
request size (4194304 bytes, it would appear), resulting in a
RESOURCE_EXHAUSTED error.
The initial chunk size, which is currently constant but should be made
dynamic at some point in the future, has been chosen based on the IPC
bandwidth delay product for tazjin's desktop, rounded up.
Change-Id: I6f0232cdbc98653484816b39855126873fc59a03
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1835
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
The command line options --arg and --argstr that are used by a bunch of
CLI commands to pass arguments to top-level functions in files go
through the same code-path as auto-calling top-level functions with
their default arguments - this, however, was only passing the arguments
that were *explicitly* mentioned in the formals of the function - in the
case of an as-pattern with an ellipsis (eg args @ { ... }) extra passed
arguments would get omitted. This fixes that to instead pass *all*
specified auto args in the case that our function has an ellipsis.
Submitted upstream at https://github.com/NixOS/nix/pull/3965
Fixes: #46
Change-Id: I32b7ee0e5bacf75b2bc43a3f0796f533f4bd5959
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1863
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
|
|
Previously, MixEvalArgs (a generic data type used to handle --arg,
--argstr, and -I arguments to `nix-build`, `nix eval`, etc.) was storing
the difference between --arg and --argstr by prepending a single
character (either 'E' or 'S') to the value of the arg. This is messy and
un-type-safe, so this commit refactors that to use a proper enum and a
std::pair, which allows us to add a switch and get totality checking.
yay, types!
Change-Id: I883f8db3136d05dda190fac0a1b494386c2ff87b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1862
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
|
|
This is the clang-tidy lint 'google-explicit-constructor'.
There's a whole bunch of breakage that was introduced by this, and we
had to opt out a few types of this (esp. the string formatting crap).
In some cases minor other changes have been done to keep the code
working, instead of converting between types (e.g. an explicit
comparison operator implementation for nix::Pid).
Change-Id: I12e1ca51a6bc2c882dba81a2526b9729d26988e7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1832
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Reviewed-by: glittershark <grfn@gws.fyi>
|
|
The previous clang-tidy invocation missed some header files, which has
now been rectified.
Change-Id: I31547754fbf52f439dc7aeefb08ab90bd50c4156
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1831
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
|
|
Change (mostly) created via:
fd . -e hh -e cc | parallel clang-tidy -p ~/builds/nix --fix "'--checks=-*,modernize-use-using'"
Change-Id: I90ad6e64a5399f2005dfb9956074ec51c2c78cea
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1830
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
|
|
.. turns out producing an entire build log on a single line is not
useful.
This does not use `<< std::endl` because we have run into buffering
issues with the implementation of the logs->gRPC sink, but intend to
replace this in the future using a structured sink for BuildEvent
protos rather than a raw stream.
Change-Id: Ia9b05fa804391d389e2ef53ab4436c0ec5cc452e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1828
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
|
|
Introduces the actual log sink in several places where we actually
want the build logs to thread through correctly.
This should cover *most* build paths.
Change-Id: I735dff8a79f7e35a5874eb89b4abb980f9703dc2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1827
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
|
|
This *should* wire up the builder's logs all the way back through the
gRPC client, where they are then conveniently discarded.
Change-Id: I65f22526d0b5a8b8d90f28665bc1b4bc7f7c802a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1825
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
|
|
This passes an output stream for build logs to almost all relevant
functions inside of build.cc by threading it through the
`Goal`-abstraction.
Store calls that create goals but don't have a sink available use the
DiscardLogsSink().
Change-Id: I2c0cb1aec1f9150f33113f4752055cea518ede8b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1824
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
|