about summary refs log tree commit diff
path: root/third_party/nix/src/proto
AgeCommit message (Collapse)AuthorFilesLines
2020-08-29 r/1736 fix(tvix): Chunk the AddTextToStore requestGriffin Smith1-4/+11
Rather than sending the entire AddTextToStore request along in a single message, send it in a stream of chunks using the same metadata-first approach we've been using for the other store gRPC requests. This fixes a bug where certain builds could send more data than the maximum gRPC request size (4194304 bytes, it would appear), resulting in a RESOURCE_EXHAUSTED error. The initial chunk size, which is currently constant but should be made dynamic at some point in the future, has been chosen based on the IPC bandwidth delay product for tazjin's desktop, rounded up. Change-Id: I6f0232cdbc98653484816b39855126873fc59a03 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1835 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-20 r/1685 feat(tvix): Introduce build event streams in worker protocolVincent Ambo1-7/+28
Introduces a new `BuildEvent` proto type which is streamed in response to calls that trigger builds of derivations. This type can currently supply build statuses, log lines and information about builds starting. This is in preparation for threading build logs through the processes. Since we have nowhere to send the logs (yet), a null sink is used instead. Co-authored-by: Griffin Smith <grfn@gws.fyi> Change-Id: If7332337b89506c7e404cd20174acdaa1a3be4e8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1793 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi> Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-09 r/1622 feat(tvix): Implement all remaining RPC callsGriffin Smith1-0/+7
Implement all remaining RPC calls on the RpcSstore client, remove a few stub methods we had added that weren't actually present in the old RemoteStore implementation, and add one more RPC call for getBuildLog that is present in the store API, but that we hadn't added as a stub *or* to the proto. Fixes: #29 Change-Id: Id827f51a393ece4bc7bbecaf38aee9eb4b329770 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1692 Reviewed-by: kanepyork <rikingcoding@gmail.com> Tested-by: BuildkiteCI
2020-08-08 r/1615 feat(tvix): Implement AddToStoreNarGriffin Smith1-2/+8
Implement both the client and server sides of AddToStoreNar, using a templated generalization of the sources and sinks we were using for AddToStore on both ends. Change-Id: I73d0ed34118c711b125851dff99a7518ced4af35 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1686 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-05 r/1600 feat(3p/nix): Implement FindRoots, CollectGarbageGriffin Smith1-1/+1
Implement the RPC client and server handlers for the FindRoots and CollectGarbage RPC calls Change-Id: Ifa5d582c6a33bd1e7661ac2fc860505ef404dad0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1656 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-01 r/1529 feat(3p/nix): Implement a few more RPC callsGriffin Smith1-0/+2
Implement the RPC client calls for QueryPathFromHashPart, QuerySubstitutablePaths, and QuerySubstitutablePathInfos, and the handler for QuerySubstitutablePathInfos. Refs: #29 Change-Id: Idf383b771f159f267d8f65367bc4af3d239e32b7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1515 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-01 r/1519 feat(3p/nix): Implement AddToStore proto handlerGriffin Smith1-7/+2
Implement the proto handler for AddToStore, which adds a nix path to the store. This is implemented by adding a new (probably soon-to-be-generalized) Source concretion that wraps a grpc ServerReader for the stream of data we're receiving from the client - this is less than ideal, as it's perpetuating the source/sink thing that's going on and storing entire nars in memory, but is at the very worst an incremental step towards a functioning nix that we can refactor in the future. Paired-With: Perry Lorier <isomer@tvl.fyi> Paired-With: Vincent Ambo <mail@tazj.in> Change-Id: I48db734e7460a47aee4a85dd5137b690980859e3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1441 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com> Reviewed-by: tazjin <mail@tazj.in>
2020-07-25 r/1473 feat(3p/nix): Start implementing RPC store clientGriffin Smith1-0/+1
Add a stub class for wrapping a gRPC client to the new, proto-backed nix store protocol, along with several methods implemented but several left throwing a not implemented exception. Paired-With: Vincent Ambo <mail@tazj.in> Paired-With: Perry Lorier <isomer@tvl.fyi> Change-Id: Id943d4f6d75084b8498786d580e6c9f7c92c104d Reviewed-on: https://cl.tvl.fyi/c/depot/+/1436 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-07-18 r/1383 feat(3p/nix/nix-daemon): Implement Worker::AddSignatures handlerGriffin Smith1-1/+6
I got the message proto wrong on this one as well - it needs both a path and a signatures. Change-Id: I9a489b1285bda61c15b2a3b47d9cfc3b50e387da Reviewed-on: https://cl.tvl.fyi/c/depot/+/1270 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-07-18 r/1382 feat(3p/nix/nix-daemon): Implement Worker::BuildDerivation handlerGriffin Smith1-1/+16
Implement the proto handler on the server side for Worker::BuildDerivation. This includes several additions to the proto which I had missed on the first pass, including the actual proto definition for the Derivation itself and a few sequence number reorderings which are fine because this is all provisional and not deployed yet. A couple things to note - I implemented a couple constructors for nix classes that initialize themselves based on their proto variants, which felt nice and didn't end up causing any issues. - I've made the conversions between the enum types in nix and in proto explicit via switch statements rather than using a static_cast, out of an abundance of caution that the error would get mismatched in the future and we'd convert the wrong thing to the wrong thing - this is verbose, but exceptionally future proof. Change-Id: Iecf6b88e76bc37e49efa05fd65d6cd0cb0deffed Reviewed-on: https://cl.tvl.fyi/c/depot/+/1249 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: Kane York <rikingcoding@gmail.com>
2020-07-17 r/1361 feat(3p/nix/nix-daemon): Implement Worker::VerifyStore handlerGriffin Smith1-2/+3
Also adds the missing check_contents field to the VerifyStoreRequest proto message, since it was missed in the original pass. This is done using a renumbering, which is fine in this case since the proto hasn't been deployed yet Change-Id: I92bf4e48a71a25ae02ae02b3deaf6e7c71fe5da7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1237 Tested-by: BuildkiteCI Reviewed-by: isomer <isomer@tvl.fyi> Reviewed-by: Kane York <rikingcoding@gmail.com>
2020-07-17 r/1355 feat(3p/nix/nix-daemon): Implement Worker::QueryPathInfo handlerGriffin Smith1-1/+1
Change-Id: I580bd29356f7bcd0cc2050afda11d2e115d44c94 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1230 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-07-17 r/1350 chore(3p/nix/proto): Fix style violations in new proto defsKane York1-5/+5
This is a blocker to enabling the linter. Change-Id: I9f8d3cc19c7539086f53474a505362230fc56c04 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1245 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-07-16 r/1336 feat(3p/nix/proto): Add QueryMissing RPC callVincent Ambo1-0/+13
Change-Id: I6a29c56a0f945afdef7434da539b4b91b2e4d2e6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1222 Reviewed-by: glittershark <grfn@gws.fyi> Tested-by: BuildkiteCI
2020-07-16 r/1335 feat(3p/nix/proto): Add AddToStoreNar RPC callVincent Ambo1-15/+18
Change-Id: I335630644205adb280d83e006baf9d497ac1ac95 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1221 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-07-16 r/1334 feat(3p/nix): Add NarFromPathGriffin Smith1-0/+3
What it does I will never know. It takes a path and returns a path. Something happens in the middle. Change-Id: I499a9df700e5b954c9aaf6d694753ff34e773dfd Reviewed-on: https://cl.tvl.fyi/c/depot/+/1210 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-07-16 r/1333 feat(3p/nix): Add AddSignaturesGriffin Smith1-0/+8
Change-Id: Ic37409c7987c261dc3eb91382d40f458fae6ea0c Reviewed-on: https://cl.tvl.fyi/c/depot/+/1209 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2020-07-16 r/1332 feat(3p/nix): Add BuildDerivation protoGriffin Smith1-0/+31
Change-Id: Ib66b4ff2da72677e74b6a713186c72f66d9f6f88 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1208 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-07-16 r/1330 feat(3p/nix): Add VerifyStoreGriffin Smith1-0/+14
Change-Id: I6a0596e986867e94857da461c746a80750830f7b Reviewed-on: https://cl.tvl.fyi/c/depot/+/1206 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-07-16 r/1329 feat(3p/nix): Add OptimiseStoreGriffin Smith1-0/+4
Change-Id: I0d254504801283e018eef41bd5ea9850b17c0d80 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1205 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-07-16 r/1328 feat(3p/nix): Add QueryValidDeriversGriffin Smith1-0/+4
Change-Id: I52965bdbf143ae11201e1fafed1c25b3d8f1bee9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1204 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-07-16 r/1327 feat(3p/nix): Add path query opsGriffin Smith1-0/+6
Add QueryValidPaths and QuerySubstitutablePaths, both of which filter a list of paths based on a set of criteria. Change-Id: I6aa4647efe82b82dc9582a311643d5f9b6d521d5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1203 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-07-16 r/1326 feat(3p/nix): Add QueryPathFromHashPartGriffin Smith1-12/+19
Change-Id: I0a31557f4cf585d3e539e29ef1a07b443aeec9fc Reviewed-on: https://cl.tvl.fyi/c/depot/+/1202 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-07-16 r/1315 feat(3p/nix/proto): Add QueryDerivationOutputNamesGriffin Smith1-0/+7
Change-Id: I5452d0f76441a61d70031f567c9cb75249569c2a Reviewed-on: https://cl.tvl.fyi/c/depot/+/1201 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2020-07-16 r/1314 feat(3p/nix): Add proto for QueryPathInfoGriffin Smith1-0/+18
Change-Id: I10ec338ef2d5360954abdc5bb3d4789f34b031fc Reviewed-on: https://cl.tvl.fyi/c/depot/+/1200 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2020-07-16 r/1312 feat(3p/nix/worker.proto): Add QueryAllValidPathsGriffin Smith1-0/+3
takes nothing, returns paths Change-Id: I4087b9b5a25d9edc76b97d0a0f553259693cca3a Reviewed-on: https://cl.tvl.fyi/c/depot/+/1187 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-07-16 r/1311 feat(3p/nix/proto): Add additional worker-protocol operationsVincent Ambo1-48/+43
On the chopping block this time: * requests/responses that return one or more store paths, and contain nothing else, have been changed to use the same types (StorePath and StorePaths, respectively) * QuerySubstitutablePathInfos has been added. It should be noted that legacy Nix has two versions of this call, one that only queries a single info (deprecated) and one that queries multiple. We have only implemented the latter. * QueryDerivationOutputs has been added. Change-Id: Iccc9041e7064e141cf593467ecdcc327581c4056 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1186 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-07-10 r/1246 feat(3p/nix): Add additional worker protocol actions to protoVincent Ambo1-4/+114
This takes us about halfway through worker-protocol.hh I have left out the documentation strings for some of these items because I don't feel that I can currently write an unambigous description of them. For now I am just attempting to match the types. Change-Id: Iae64b1676152fe4ea069e2021b75ad76465cf368 Reviewed-on: https://cl.tvl.fyi/c/depot/+/960 Tested-by: BuildkiteCI Reviewed-by: isomer <isomer@tvl.fyi>
2020-07-05 r/1216 feat(3p/nix): Add protobuf & gRPC dependencies and generate sourcesVincent Ambo1-0/+37
Adds dependencies on the gRPC & protobuf libraries, and implements Nix code to generate the C++ sources from the included proto definitions. This is theoretically supported via CMake, but practically doesn't work and I don't care to debug why. Doing it like this lets us instead add a CMake library target for our proto definitions based on the sources generated by Nix. Pros: * no need to deal with the gRPC CMake mess * it works! Cons: * iteration requires nix-shell restart Change-Id: Ie1fe9807fc96c49cb8f7161ba59d093456062b15 Reviewed-on: https://cl.tvl.fyi/c/depot/+/927 Tested-by: BuildkiteCI Reviewed-by: isomer <isomer@tvl.fyi>
2020-07-05 r/1215 feat(3p/nix): Add gRPC definitions for worker protocolVincent Ambo1-0/+88
Adds initial gRPC definitions for the Nix worker protocol, which is currently defined messily across the following files: src/libstore/worker-protocol.hh src/libstore/remote-store.cc src/nix-daemon/nix-daemon.cc The protocol definition is basically a big enum with the signatures of the calls being implicit in the various client/server implementation functions. The definitions in this file are slowly reversed from these implicit signatures, and are likely to contain an error or two which will be weeded out when this is taken into use. Only a handful of the calls are included in this commit, it is intended to get us up and running first. Change-Id: Ibc9b2ab4b91a064c8935f09f7ac72bb8150fb476 Reviewed-on: https://cl.tvl.fyi/c/depot/+/926 Reviewed-by: isomer <isomer@tvl.fyi> Tested-by: BuildkiteCI
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-11-06T04·38+0000