about summary refs log tree commit diff
path: root/third_party/nix/src/libstore/rpc-store.cc
AgeCommit message (Collapse)AuthorFilesLines
2022-05-19 r/4098 chore(3p/nix): unvendor tvix 0.1Vincent Ambo1-549/+0
Nothing is using this now, and we'll likely never pick this up again, but we learned a lot in the process. Every now and then this breaks in some bizarre way on channel bumps and it's just a waste of time to maintain that. Change-Id: Idcf2f5acd4ca7070ce18d7149cbfc0d967dc0a44 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5632 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org> Reviewed-by: lukegb <lukegb@tvl.fyi> Autosubmit: tazjin <tazjin@tvl.su>
2020-11-28 r/1946 feat(tvix): Thread a log sink through calls to buildDerivationGriffin Smith1-2/+3
Similarly to how we did for buildPaths, add a std::ostream& log_sink parameter to the build_derivation method on Store, and pass it std::cerr when called at the top level by nix commands - most notably, the build-remote hook binary, so that we get build logs when using tvix as a remote builder. Change-Id: I0f8f729ba8429d4838a0a135a5c2ac1e1a95d575 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2176 Tested-by: BuildkiteCI Reviewed-by: andi <andi@notmuch.email> Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-11-27 r/1942 feat(tvix): Write build logs to LOG(INFO) in buildDerivationGriffin Smith1-4/+1
This was referencing a nonexistent note in buildPaths, for one, but for another let's get log outputs when calling this RPC. Change-Id: Ic9d17834b356ea84d69692ccc0249d09777e833b Reviewed-on: https://cl.tvl.fyi/c/depot/+/2173 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-11-27 r/1940 fix(tvix): Remove more superfluous log outputGriffin Smith1-2/+1
- A couple of log messages had double-newlines still - Remove the [ drvPath ] bit from the "building path" output, since the derivation path is already present in the message itself. Change-Id: Ib4b514a18fcb63c7191f4d7cf0a9b857aaa3f3fa Reviewed-on: https://cl.tvl.fyi/c/depot/+/2171 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-11-19 r/1885 fix(tvix): Use mutable field for assigning proto fieldsGriffin Smith1-2/+2
This was accidentally using the proto arena API to assign the derivation field of a BuildDerivationRequest. We *thought* this was causing a double free, but even with this change that's still happening. That said, this change is probably still a good idea since it's using the proto API as intended. References: b/64 Change-Id: I950a4eafb214e9113639ea54d2dfd4659b7be931 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2104 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-09-16 r/1795 fix(tvix): Convert INVALID_ARGUMENT to InvalidPathGriffin Smith1-2/+6
The code that calls queryPathInfoUncached explicitly catches the InvalidPath exception and translates it into a null result - but the RPC code was throwing a regular old Error for invalid paths. At some point we should get rid of all the exception-driven control flow in this whole thing, but in the meantime this gets us back to functional. Change-Id: I2a38790ee0c691ab0c8394c7738d7693fa42aa10 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1980 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-09-14 r/1791 refactor(tvix): Make static strings constexpr string_viewsGriffin Smith1-2/+2
Make all static std::strings constexpr std::string_views, and replace concatenation with absl::StrCat where necessary. Technically all of these are constant, so they really don't need to be top-level statics - and since I'm trying to get rid of as much global state as possible in preparation for making the nix daemon properly multithreaded I figured I'd knock these out while I was at it. Change-Id: Ibd3ad9ef68f0a0eacb135541b39fdb13dae042e1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1939 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-08-29 r/1736 fix(tvix): Chunk the AddTextToStore requestGriffin Smith1-6/+27
Rather than sending the entire AddTextToStore request along in a single message, send it in a stream of chunks using the same metadata-first approach we've been using for the other store gRPC requests. This fixes a bug where certain builds could send more data than the maximum gRPC request size (4194304 bytes, it would appear), resulting in a RESOURCE_EXHAUSTED error. The initial chunk size, which is currently constant but should be made dynamic at some point in the future, has been chosen based on the IPC bandwidth delay product for tazjin's desktop, rounded up. Change-Id: I6f0232cdbc98653484816b39855126873fc59a03 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1835 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-20 r/1689 chore(tvix): Thread a std::ostream through Store::buildPathsVincent Ambo1-13/+8
This part of the store API needs to carry a handle to the log sink from now on, so that it can be passed in as appropriate from the gRPC handlers. In all places where there is no such handler available at the moment, the discarding log sink has been inserted. This can be used as a convenient grep target in the future. Change-Id: I26628e30b4c6437dccdf8f722ca2e8ed827dfc19 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1797 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com> Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-20 r/1685 feat(tvix): Introduce build event streams in worker protocolVincent Ambo1-6/+45
Introduces a new `BuildEvent` proto type which is streamed in response to calls that trigger builds of derivations. This type can currently supply build statuses, log lines and information about builds starting. This is in preparation for threading build logs through the processes. Since we have nowhere to send the logs (yet), a null sink is used instead. Co-authored-by: Griffin Smith <grfn@gws.fyi> Change-Id: If7332337b89506c7e404cd20174acdaa1a3be4e8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1793 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi> Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-14 r/1649 refactor(tvix): Make Store::buildPaths return a StatusGriffin Smith1-2/+4
Make Store::buildPaths return a Status with [[nodiscard]] rather than throwing exceptions to signal failure. This is the beginning of a long road to refactor the entire store API to be status/statusor based instead of using exceptions. Change-Id: I2e32371c95a25b87ad129987c217d49c6d6e0c85 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1745 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-09 r/1622 feat(tvix): Implement all remaining RPC callsGriffin Smith1-35/+62
Implement all remaining RPC calls on the RpcSstore client, remove a few stub methods we had added that weren't actually present in the old RemoteStore implementation, and add one more RPC call for getBuildLog that is present in the store API, but that we hadn't added as a stub *or* to the proto. Fixes: #29 Change-Id: Id827f51a393ece4bc7bbecaf38aee9eb4b329770 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1692 Reviewed-by: kanepyork <rikingcoding@gmail.com> Tested-by: BuildkiteCI
2020-08-09 r/1621 refactor(tvix): Factor-out proto utilitiesGriffin Smith1-136/+47
Factor out the shared utilities for interacting with protobufs/grpc from libstore to a proto.hh header in libproto. Change-Id: I1cb8d94867d5d4b63a9994be0b53f8f612eb8e3a Reviewed-on: https://cl.tvl.fyi/c/depot/+/1691 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-09 r/1620 feat(tvix): Pass function name to SuccessOrThrowGriffin Smith1-17/+28
To aid in debugging RPC call failures, pass the name of the function being called to the SuccessOrThrow util funcion in the RpcStore Change-Id: I523dacfab896b85a3dbe6050c07ee6bd5906fa44 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1690 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-08 r/1615 feat(tvix): Implement AddToStoreNarGriffin Smith1-18/+42
Implement both the client and server sides of AddToStoreNar, using a templated generalization of the sources and sinks we were using for AddToStore on both ends. Change-Id: I73d0ed34118c711b125851dff99a7518ced4af35 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1686 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-05 r/1600 feat(3p/nix): Implement FindRoots, CollectGarbageGriffin Smith1-2/+27
Implement the RPC client and server handlers for the FindRoots and CollectGarbage RPC calls Change-Id: Ifa5d582c6a33bd1e7661ac2fc860505ef404dad0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1656 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-05 r/1594 feat(3p/nix): Implement AddToStore rpc clientGriffin Smith1-1/+51
Implement the AddToStore RPC client method, which uses an AddToStorePathWriterSink (the dual of the AddToStorePathReaderSource on the server side) to hook into the dumpPath machinery (which we should refactor not to use sinks or sources, but not yet) and write dumped paths as binary data over gRPC. With this commit and sandboxing disabled, the following derivation builds cleanly: derivation { name = "test"; builder = ./build.sh; system = "x86_64-linux"; } where build.sh has chmod +x and contains: #!/bin/sh echo 1 > $out Change-Id: I94cab86f0825a3a9993262a9807130645c13bf44 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1638 Reviewed-by: kanepyork <rikingcoding@gmail.com> Tested-by: BuildkiteCI
2020-08-04 r/1583 feat(tvix): Implement AddIndirectRoot, SyncWithGCGriffin Smith1-2/+7
Implement RPC handlers and client methods for the AddWithRoot and SyncWithGC calls. With this and sandboxing disabled, the following derivation is now building successfully: derivation { name = "test"; builder = "/bin/sh"; args = [ "-c" "echo 1 > $out" ]; system = "x86_64-linux"; } Co-authored-by: Vincent Ambo <mail@tazj.in> Change-Id: I245dca0dfca882e2eae5d93364287a8b79fead71 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1637 Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: kanepyork <rikingcoding@gmail.com> Tested-by: BuildkiteCI
2020-08-04 r/1582 chore(tvix): Make not implemented errors nix::UnsupportedGriffin Smith1-22/+24
Without this, these errors print as "uncaught exception of type std::string", but with this they print using nix's built-in error printer, which will be nice to have as we bang out the remaining store API methods. Co-authored-by: Vincent Ambo <mail@tazj.in> Change-Id: I9bd4f77f65b27c8f97b727b98ef62e2061999e71 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1636 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2020-08-04 r/1573 feat(3p/nix/daemon): catch-all explicit Error-Status conversionKane York1-3/+95
We wrap every server-side proto handler with a macro that catches exceptions and turns them into proper grpc error codes. For the time being, most exceptions map to INTERNAL, the existing mapping. Change-Id: Id6ed6a279b198ad185d32562f39000ccc15eadbf Reviewed-on: https://cl.tvl.fyi/c/depot/+/1599 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-03 r/1567 feat(3p/nix): Implement two more RPC callsGriffin Smith1-3/+24
Implement AddTextToStore and BuildPaths both on the client and the server Refs: #29 Change-Id: I45294c3e1c1a7489e42099d36425b7acc04e0427 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1560 Reviewed-by: kanepyork <rikingcoding@gmail.com> Tested-by: BuildkiteCI
2020-08-02 r/1540 fix(3p/nix): Don't reuse ClientContextGriffin Smith1-1/+11
It turns out these aren't supposed to be reused after the first request - reusing it was causing things to blow up. Change-Id: I935f1ee82ed4e7f26528ae105c8fa2501fa81b97 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1559 Reviewed-by: kanepyork <rikingcoding@gmail.com> Tested-by: BuildkiteCI
2020-08-02 r/1538 fix(3p/nix): Prepend unix:// to daemon socketGriffin Smith1-2/+3
Prepend the unix:// URI scheme to the daemon socket so that grpc knows we want to connect to a unix socket rather than another type of URI. As part of debugging this I made the failure message for the RPCStore include the URI, which I'm leaving in since it'll be nice to have. Change-Id: I6e70596895117b9a0d53fe2a61d8542ceb64c940 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1543 Reviewed-by: kanepyork <rikingcoding@gmail.com> Tested-by: BuildkiteCI
2020-08-01 r/1529 feat(3p/nix): Implement a few more RPC callsGriffin Smith1-3/+32
Implement the RPC client calls for QueryPathFromHashPart, QuerySubstitutablePaths, and QuerySubstitutablePathInfos, and the handler for QuerySubstitutablePathInfos. Refs: #29 Change-Id: Idf383b771f159f267d8f65367bc4af3d239e32b7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1515 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-01 r/1515 chore(3p/nix/hash): prefer StatusOr over throwing constructorKane York1-1/+2
The use of `unwrap_throw` can be used as a later grep target. Change-Id: I8c54ed90c4289f07aecb8a1393dd10204c8bce4e Reviewed-on: https://cl.tvl.fyi/c/depot/+/1493 Reviewed-by: glittershark <grfn@gws.fyi> Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2020-07-25 r/1473 feat(3p/nix): Start implementing RPC store clientGriffin Smith1-0/+282
Add a stub class for wrapping a gRPC client to the new, proto-backed nix store protocol, along with several methods implemented but several left throwing a not implemented exception. Paired-With: Vincent Ambo <mail@tazj.in> Paired-With: Perry Lorier <isomer@tvl.fyi> Change-Id: Id943d4f6d75084b8498786d580e6c9f7c92c104d Reviewed-on: https://cl.tvl.fyi/c/depot/+/1436 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-11-01T04·01+0000