| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
Similarly to how we did for buildPaths, add a std::ostream& log_sink
parameter to the build_derivation method on Store, and pass it std::cerr
when called at the top level by nix commands - most notably, the
build-remote hook binary, so that we get build logs when using tvix as a
remote builder.
Change-Id: I0f8f729ba8429d4838a0a135a5c2ac1e1a95d575
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2176
Tested-by: BuildkiteCI
Reviewed-by: andi <andi@notmuch.email>
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
This was referencing a nonexistent note in buildPaths, for one, but for
another let's get log outputs when calling this RPC.
Change-Id: Ic9d17834b356ea84d69692ccc0249d09777e833b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2173
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
- A couple of log messages had double-newlines still
- Remove the [ drvPath ] bit from the "building path" output, since the
derivation path is already present in the message itself.
Change-Id: Ib4b514a18fcb63c7191f4d7cf0a9b857aaa3f3fa
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2171
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
This was accidentally using the proto arena API to assign the derivation
field of a BuildDerivationRequest. We *thought* this was causing a
double free, but even with this change that's still happening. That
said, this change is probably still a good idea since it's using the
proto API as intended.
References: b/64
Change-Id: I950a4eafb214e9113639ea54d2dfd4659b7be931
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2104
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
|
|
The code that calls queryPathInfoUncached explicitly catches the
InvalidPath exception and translates it into a null result - but the RPC
code was throwing a regular old Error for invalid paths. At some point
we should get rid of all the exception-driven control flow in this whole
thing, but in the meantime this gets us back to functional.
Change-Id: I2a38790ee0c691ab0c8394c7738d7693fa42aa10
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1980
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
Make all static std::strings constexpr std::string_views, and replace
concatenation with absl::StrCat where necessary.
Technically all of these are constant, so they really don't need to be
top-level statics - and since I'm trying to get rid of as much global
state as possible in preparation for making the nix daemon properly
multithreaded I figured I'd knock these out while I was at it.
Change-Id: Ibd3ad9ef68f0a0eacb135541b39fdb13dae042e1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1939
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
|
|
Rather than sending the entire AddTextToStore request along in a single
message, send it in a stream of chunks using the same metadata-first
approach we've been using for the other store gRPC requests. This fixes
a bug where certain builds could send more data than the maximum gRPC
request size (4194304 bytes, it would appear), resulting in a
RESOURCE_EXHAUSTED error.
The initial chunk size, which is currently constant but should be made
dynamic at some point in the future, has been chosen based on the IPC
bandwidth delay product for tazjin's desktop, rounded up.
Change-Id: I6f0232cdbc98653484816b39855126873fc59a03
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1835
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
This part of the store API needs to carry a handle to the log sink
from now on, so that it can be passed in as appropriate from the gRPC
handlers.
In all places where there is no such handler available at the moment,
the discarding log sink has been inserted. This can be used as a
convenient grep target in the future.
Change-Id: I26628e30b4c6437dccdf8f722ca2e8ed827dfc19
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1797
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Reviewed-by: glittershark <grfn@gws.fyi>
|
|
Introduces a new `BuildEvent` proto type which is streamed in response
to calls that trigger builds of derivations.
This type can currently supply build statuses, log lines and
information about builds starting.
This is in preparation for threading build logs through the processes.
Since we have nowhere to send the logs (yet), a null sink is used
instead.
Co-authored-by: Griffin Smith <grfn@gws.fyi>
Change-Id: If7332337b89506c7e404cd20174acdaa1a3be4e8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1793
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
Make Store::buildPaths return a Status with [[nodiscard]] rather than
throwing exceptions to signal failure. This is the beginning of a long
road to refactor the entire store API to be status/statusor based
instead of using exceptions.
Change-Id: I2e32371c95a25b87ad129987c217d49c6d6e0c85
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1745
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
Implement all remaining RPC calls on the RpcSstore client, remove a few
stub methods we had added that weren't actually present in the old
RemoteStore implementation, and add one more RPC call for getBuildLog
that is present in the store API, but that we hadn't added as a
stub *or* to the proto.
Fixes: #29
Change-Id: Id827f51a393ece4bc7bbecaf38aee9eb4b329770
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1692
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Tested-by: BuildkiteCI
|
|
Factor out the shared utilities for interacting with protobufs/grpc from
libstore to a proto.hh header in libproto.
Change-Id: I1cb8d94867d5d4b63a9994be0b53f8f612eb8e3a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1691
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
To aid in debugging RPC call failures, pass the name of the function
being called to the SuccessOrThrow util funcion in the RpcStore
Change-Id: I523dacfab896b85a3dbe6050c07ee6bd5906fa44
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1690
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
Implement both the client and server sides of AddToStoreNar, using a
templated generalization of the sources and sinks we were using for
AddToStore on both ends.
Change-Id: I73d0ed34118c711b125851dff99a7518ced4af35
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1686
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
Implement the RPC client and server handlers for the FindRoots and
CollectGarbage RPC calls
Change-Id: Ifa5d582c6a33bd1e7661ac2fc860505ef404dad0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1656
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
Implement the AddToStore RPC client method, which uses an
AddToStorePathWriterSink (the dual of the AddToStorePathReaderSource on
the server side) to hook into the dumpPath machinery (which we should
refactor not to use sinks or sources, but not yet) and write dumped
paths as binary data over gRPC. With this commit and sandboxing
disabled, the following derivation builds cleanly:
derivation {
name = "test";
builder = ./build.sh;
system = "x86_64-linux";
}
where build.sh has chmod +x and contains:
#!/bin/sh
echo 1 > $out
Change-Id: I94cab86f0825a3a9993262a9807130645c13bf44
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1638
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Tested-by: BuildkiteCI
|
|
Implement RPC handlers and client methods for the AddWithRoot and
SyncWithGC calls. With this and sandboxing disabled, the following
derivation is now building successfully:
derivation {
name = "test";
builder = "/bin/sh";
args = [ "-c" "echo 1 > $out" ];
system = "x86_64-linux";
}
Co-authored-by: Vincent Ambo <mail@tazj.in>
Change-Id: I245dca0dfca882e2eae5d93364287a8b79fead71
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1637
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Tested-by: BuildkiteCI
|
|
Without this, these errors print as "uncaught exception of type
std::string", but with this they print using nix's built-in error
printer, which will be nice to have as we bang out the remaining store
API methods.
Co-authored-by: Vincent Ambo <mail@tazj.in>
Change-Id: I9bd4f77f65b27c8f97b727b98ef62e2061999e71
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1636
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
|
|
We wrap every server-side proto handler with a macro that catches
exceptions and turns them into proper grpc error codes. For the
time being, most exceptions map to INTERNAL, the existing mapping.
Change-Id: Id6ed6a279b198ad185d32562f39000ccc15eadbf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1599
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
|
|
Implement AddTextToStore and BuildPaths both on the client and the
server
Refs: #29
Change-Id: I45294c3e1c1a7489e42099d36425b7acc04e0427
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1560
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Tested-by: BuildkiteCI
|
|
It turns out these aren't supposed to be reused after the first request
- reusing it was causing things to blow up.
Change-Id: I935f1ee82ed4e7f26528ae105c8fa2501fa81b97
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1559
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Tested-by: BuildkiteCI
|
|
Prepend the unix:// URI scheme to the daemon socket so that grpc knows
we want to connect to a unix socket rather than another type of URI. As
part of debugging this I made the failure message for the RPCStore
include the URI, which I'm leaving in since it'll be nice to have.
Change-Id: I6e70596895117b9a0d53fe2a61d8542ceb64c940
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1543
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Tested-by: BuildkiteCI
|
|
Implement the RPC client calls for QueryPathFromHashPart,
QuerySubstitutablePaths, and QuerySubstitutablePathInfos, and the
handler for QuerySubstitutablePathInfos.
Refs: #29
Change-Id: Idf383b771f159f267d8f65367bc4af3d239e32b7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1515
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
The use of `unwrap_throw` can be used as a later grep target.
Change-Id: I8c54ed90c4289f07aecb8a1393dd10204c8bce4e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1493
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
|
|
Add a stub class for wrapping a gRPC client to the new, proto-backed nix
store protocol, along with several methods implemented but several left
throwing a not implemented exception.
Paired-With: Vincent Ambo <mail@tazj.in>
Paired-With: Perry Lorier <isomer@tvl.fyi>
Change-Id: Id943d4f6d75084b8498786d580e6c9f7c92c104d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1436
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
