about summary refs log tree commit diff
path: root/ops
diff options
context:
space:
mode:
authorFlorian Klink <flokli@flokli.de>2023-03-23T12·49+0100
committerclbot <clbot@tvl.fyi>2023-03-27T08·48+0000
commit367a5e9922264b787667fd5e750c8eadf8a7796f (patch)
treedd34d886aa0cbda5668f3126870552e6002a857e /ops
parentb919f297528b74d9eab95fe93aa87541d7dffc53 (diff)
feat(tvix/store/directorysvc): add gRPC client r/6044
This provides a GRPCDirectoryService struct implementing
DirectoryService, allowing a client to Directory objects from a (remote)
tvix-store.

Remote in this case is anything outside the current process, be it
another process, or an endpoint on the network.

To keep the sync interface in the `DirectoryService` trait, a handle to
some tokio runtime needs to be passed into the constructor, and the two
methods use `self.tokio_handle.spawn` to start an async function, and
`self.tokio_handle.block_on` to wait for its completion.

The client handle, called `grpc_client` itself is easy to clone, and
treats concurrent requests internally. This means, even though we keep
the `DirectoryService` trait sync, there's nothing preventing it from
being used concurrently, let's say from multiple threads.

There's still two limitations for now:

1) The trait doesn't make use of the `recursive` request, which
   currently leads to a N+1 query problem. This can be fixed
   by `GRPCDirectoryService` having a reference to another
   `DirectoryService` acting as the local side.
   I want to wait for general store composition code to pop up before
   manually coding this here.

2) It's currently only possible to put() leaf directory nodes, as the
   request normally requires uploading a whole closure. We might want
   to add another batch function to upload a whole closure, and/or do
   this batching in certain cases. This still needs some more thinking.

Change-Id: I7ffec791610b72c0960cf5307cefbb12ec946dc9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8336
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Diffstat (limited to 'ops')
0 files changed, 0 insertions, 0 deletions