about summary refs log tree commit diff
path: root/tvix
AgeCommit message (Collapse)AuthorFilesLines
2023-05-22 r/6173 refactor(tvix/eval): use &Path instead of PathBufFlorian Klink5-19/+19
This allows getting rid of some clones in eval/src/vm/generators.rs. Change-Id: I330390307d3bcfeef19c98954c753ee55b1ccee3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8604 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-22 r/6172 fix(tvix/eval): add path where useful to ErrorKind::IOFlorian Klink1-2/+10
These two places didn't add the path from the context to the ErrorKind, but simply relied on the impl From<std::io::Error> for tvix_eval::ErrorKind, which doesn't add the path. Change-Id: Ifc7dbbe305d24242b0705de1dea34e8923e9d2cb Reviewed-on: https://cl.tvl.fyi/c/depot/+/8603 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de>
2023-05-22 r/6171 refactor(tvix/eval/io): use io::Error instead of tvix_eval errorsFlorian Klink4-63/+70
We didn't return anything useful other than ErrorKind::IO anyways. We can use io::ErrorKind::Unsupported for DummyIO. Fixes b/271. Change-Id: Icb231e9b38168e8b6fa473bfa405d160357b317f Reviewed-on: https://cl.tvl.fyi/c/depot/+/8602 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-18 r/6166 fix(tvix/store): fix timing sensitivity in gRPC directorysvc testFlorian Klink1-24/+45
One test in here assumed the server was fast, but when very busy, a computer running these tests might not be fast. Treat both cases that can occur separately. Change-Id: Iba168ad39f2458c4fb8873211df33beeaff7c6c1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8595 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-05-18 r/6165 refactor(tvix/store): bump CalculateNARResponse.nar_size to u64Florian Klink5-9/+9
Change-Id: I5f41981820363a5eb9982c3d19830916207c62cc Reviewed-on: https://cl.tvl.fyi/c/depot/+/8591 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-18 r/6164 fix(tvix/store/proto/grpc_blobsvc): clippyFlorian Klink1-1/+1
Change-Id: I0473562793f5fa2211cb10329ba1bca14b1acab9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8594 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-05-18 r/6163 refactor(tvix/store/blobsvc/sled): clippyFlorian Klink1-5/+5
Change-Id: Icfea8ceb6b827a645c6f10efae741f6fa2114fad Reviewed-on: https://cl.tvl.fyi/c/depot/+/8590 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-18 r/6162 refactor(tvix/store/directorysvc/traverse): clippy, use NamedNodeFlorian Klink1-17/+6
Also, get rid of the explicit byte comparison here, but unwrap like in the rest of the codebase. We'll deal with this in a generic manner in b/267 and b/189. Change-Id: Ie5f3d27ab35b7e88d67a2796c29cdd7bc7df71f3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8589 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-18 r/6161 refactor(tvix/nix-compat): clippyFlorian Klink2-10/+8
Change-Id: Ie5277a5c15d9dfe543ca41fa7c6a1eedf22a9f64 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8593 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-05-18 r/6160 chore(tvix/cli): remove unnecessary RefCellVincent Ambo1-7/+5
This remained around from some previous time where this type was being cloned around directly, but it's not anymore. Change-Id: I2c61cf9cecb8e5d7d08644ed20642ee61357bc21 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8580 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-05-18 r/6159 refactor(tvix/cli): split CLI-specific IO logic into TvixIO typeVincent Ambo3-11/+84
This adds a wrapper type TvixIO<T: EvalIO>, which can wrap around an arbitrary EvalIO implementation and perform actions needed for the Tvix CLI (marking imported paths as known, and handling __corepkgs__). Change-Id: I5fc1ca199b9f94b21a89103b84575e0f8f58dff9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8579 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-05-18 r/6150 feat(tvix/store/directorysvc): add traverse_toFlorian Klink2-0/+238
This walks from a node further down until it reaches the requested path. Change-Id: I2f9a15a8601db4d06c95d7b47cd6153264e203e3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8568 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-05-17 r/6149 refactor(tvix/store): rename import::{import_path -> ingest_path}Florian Klink3-12/+13
This distinguishes it better from the EvalIO::import_path method. Also update the docstring to explain what it does (and what it doesn't). Change-Id: I32a8b2869fa67a894df28532b22bf170961a2abf Reviewed-on: https://cl.tvl.fyi/c/depot/+/8578 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-05-17 r/6148 fix(tvix/store/src/proto/mod.rs): fix error stringFlorian Klink1-1/+1
This was missed while renaming NixPath to StorePath. Change-Id: Ibcc929c43b111e4370e8222c1dd86d403548367f Reviewed-on: https://cl.tvl.fyi/c/depot/+/8577 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-05-16 r/6146 refactor(tvix/store/pathinfosvc): drop ByWhat, use digest directlyFlorian Klink5-72/+46
We currently only support querying by the output hash digest. This makes the interface a bit simpler. Change-Id: I80b285373f1923e85cb0e404c4b15d51a7f259ef Reviewed-on: https://cl.tvl.fyi/c/depot/+/8570 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-16 r/6145 feat(tvix/nix-compat): add StorePath::from_absolute_path_fullFlorian Klink1-2/+75
This allows decomposing a path consisting of a store path AND a suffix into a StorePath, and a PathBuf containing the rest. Change-Id: I81290e2fd804cdc9d1e88c71cb22c0fb882d7936 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8567 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-05-16 r/6144 docs(tvix/nix-compat): update docstringsFlorian Klink2-11/+18
Make it cleaner that StorePath only does encode the first path component after the STORE_DIR prefix. Also, move some of the comments around a bit, so it makes more sense what's using what. Change-Id: Ibb57373a13526e30c58ad561ca50e1336b091d94 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8566 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-05-14 r/6140 feat(tvix/eval/io): allow &mut self in EvalIOFlorian Klink5-29/+34
It's okay if these calls mutate some internal state inside an implementation. Change-Id: I12bb11bde0310778c3da1275696bf7de058863a3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8571 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-14 r/6139 refactor(tvix/nix-compat): update expect_err stringsFlorian Klink1-4/+4
Make it more explicit that we expect the from_string calls to fail here. Change-Id: Ib3d46fc0850e364125e3548670ef301eeea2e45c Reviewed-on: https://cl.tvl.fyi/c/depot/+/8565 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-05-14 r/6138 feat(tvix/store): add GRPCBlobServiceFlorian Klink2-0/+198
This connects to a (remote) tvix-store BlobService over gRPC. Change-Id: If31f706738a5c3445886c117feca8b61f3203e9e Reviewed-on: https://cl.tvl.fyi/c/depot/+/8552 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-05-12 r/6135 fix(tvix/eval): builtins.trace prints to stderrVincent Ambo1-1/+1
Change-Id: Icf577396035474d6977e627058aba5805c61985e Reviewed-on: https://cl.tvl.fyi/c/depot/+/8563 Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de>
2023-05-11 r/6134 chore(tvix/*): bump to smol_str 0.2.0Florian Klink4-8/+8
Change-Id: Ic9ac1b6fecb564eafb41b265bf317cd385fdc170 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8560 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-05-11 r/6133 refactor(tvix/store): remove ChunkServiceFlorian Klink26-1300/+563
Whether chunking is involved or not, is an implementation detail of each Blobstore. Consumers of a whole blob shouldn't need to worry about that. It currently is not visible in the gRPC interface either. It shouldn't bleed into everything. Let the BlobService trait provide `open_read` and `open_write` methods, which return handles providing io::Read or io::Write, and leave the details up to the implementation. This means, our custom BlobReader module can go away, and all the chunking bits in there, too. In the future, we might still want to add more chunking-aware syncing, but as a syncing strategy some stores can expose, not as a fundamental protocol component. This currently needs "SyncReadIntoAsyncRead", taken and vendored in from https://github.com/tokio-rs/tokio/pull/5669. It provides a AsyncRead for a sync Read, which is necessary to connect our (sync) BlobReader interface to a GRPC server implementation. As an alternative, we could also make the BlobReader itself async, and let consumers of the trait (EvalIO) deal with the async-ness, but this is less of a change for now. In terms of vendoring, I initially tried to move our tokio crate to these commits, but ended up in version incompatibilities, so let's vendor it in for now. Change-Id: I5969ebbc4c0e1ceece47981be3b9e7cfb3f59ad0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8551 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-10 r/6131 chore(tvix): bump cargo dependenciesFlorian Klink3-543/+875
Change-Id: I6b872a33885f4e29082c554062a60317db754188 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8550 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Johannes Kirschbauer <hsjobeki+github@gmail.com>
2023-05-10 r/6130 fix(tvix/store/nar/renderer): handle digest errorFlorian Klink1-3/+7
Change-Id: I183580732e1dd33ed079a2593097ec790def0a55 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8554 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-05-10 r/6129 feat(tvix/nix-compat/nar/writer): check for more data in readerFlorian Klink1-1/+14
We already returned UnexpectedEof in case the reader stopped returning bytes too early, but similarly we should also fail if there's still bytes left to be read in the reader passed. We normally use the NAR writer to produce new NAR files, so the readers point to the blobs we actually want to render, and having some data left in there should be an error. If for some reason the reader points to more data than just the blob, the `.take` method can be used to limit it to the (known) size. Change-Id: I9e8fa0a6dd9c794492abb6dc9e55995e619cb3bb Reviewed-on: https://cl.tvl.fyi/c/depot/+/8553 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-04-09 r/6088 refactor(nix-compat): Properly encapsulate store path constructionJohn Ericson9-125/+268
Before there was code scattered about (e.g. text hashing module and derivation output computation) constructing store paths from low level building blocks --- there was some duplication and it was easy to make nonsense store paths. Now, we have roughly the same "safe-ish" ways of constructing them as C++ Nix, and only those are exposed: - Make text hashed content-addressed store paths - Make other content-addressed store paths - Make input-addressed fixed output hashes Change-Id: I122a3ee0802b4f45ae386306b95b698991be89c8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8411 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-04-07 r/6077 docs(tvix/store): update comment regarding reachability checkFlorian Klink1-9/+6
This might be more understandable. Change-Id: I91371d304a9a316a2934ce41929998eaef67e11c Reviewed-on: https://cl.tvl.fyi/c/depot/+/8416 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-04-07 r/6076 docs(tvix/store/grpc_directorysvc_wrapper): add TODOFlorian Klink1-0/+3
The logic validating connectivity of Directory nodes should be moved to SimplePutter, and this use whatever DirectoryPutter the store comes with. Change-Id: Id68a86a96cc49ff73920017839788859ea9c5161 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8358 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-04-07 r/6074 refactor(tvix/store/import): use DirectoryPutter in import.rsFlorian Klink1-5/+7
This should allow import_path to communicate to a gRPC remote store, that actually verifies the Directory nodes are interconnected. Change-Id: Ic5d28c33518f50dedec15f1732d81579a3afaff1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8357 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-04-07 r/6073 feat(tvix/store/directorysvc): add put_multiple_startFlorian Klink5-12/+281
This provides a handle to upload multiple proto::Directory as part of the same closure. Change-Id: I9213dde257a260c8622239918ea541064b270484 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8356 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de>
2023-03-31 r/6069 refactor(nix-compat): Avoid encoding round tripJohn Ericson1-6/+10
When building store paths we can just construct the thing. Change-Id: Ife5d461d6a440ecbb22f32a86a6d51d212a2035b Reviewed-on: https://cl.tvl.fyi/c/depot/+/8409 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-03-31 r/6068 refactor(nix-compat) Make `nixhash_*` mods hierarchicalJohn Ericson4-4/+5
They can go under `nixhash` Change-Id: Ia15835c57130b66d58f5df80ae9595dceee00941 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8408 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-03-31 r/6067 refactor(tvix/nix-compat): Inline `texthash` modJohn Ericson3-44/+40
It is moved into `store_path::utils` with the other path builders. Change-Id: I3257170e442af5d83bcf79e63fa7387dd914597c Reviewed-on: https://cl.tvl.fyi/c/depot/+/8410 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-03-31 r/6066 refactor(tvix/eval): improve representation of chunk/span mappingVincent Ambo1-25/+23
This switches out the previous compressed representation (count of instructions per span) with a representation where the chunk's span list stores the index of the first operation that belongs to a span, and finds the right span by using a binary search when looking them up. This improves the lookup complexity from O(n) to O(log n). This improvement was suggested and (mostly) implemented by GPT-4. I only fixed up some names and updated the logic for deleting spans (which it only did not do because I didn't tell it about that). The code was verified by producing a complex error before/after the change and ensuring that all spans in the error match exactly. Co-Authored-By: GPT-4 Change-Id: Ibfa12cc6973af1c9b0ae55bb464d1975209771f5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8385 Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com> Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su>
2023-03-31 r/6065 refactor(tvix/store/directorysvc): move DirectoryTraverser to utilsFlorian Klink2-105/+108
Change-Id: Ie60a660e0fda7c80a6c7de20404c1965fe0e0d63 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8355 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-03-30 r/6060 refactor(tvix/nix-compat): move build_store_path out of derivationFlorian Klink10-155/+176
This doesn't have anything to do with ATerms, we just happen to be using the aterm representation of a Derivation as contents. Moving this into store_path/utils.rs makes these things much cleaner - Have a build_store_path_from_references function, and a build_store_path_from_fingerprint helper function that makes use of it. build_store_path_from_references is invoked from the derivation module which can be used to calculate the derivation path. In the derivation module, we also invoke build_store_path_from_fingerprint during the output path calculation. Change-Id: Ia8d61a5e8e5d3f396f93593676ed3f5d1a3f1d66 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8367 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-03-30 r/6059 refactor(tvix/nix-compat): use Sha256::new_with_prefixFlorian Klink1-2/+1
Change-Id: I328983008c0680e0448ccff604dbdae312f7988b Reviewed-on: https://cl.tvl.fyi/c/depot/+/8366 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-30 r/6058 refactor(tvix/nix-compat): drop is_derivation in build_store_pathFlorian Klink2-12/+7
This only added a suffix to the input argument, if build_store_path was building the path of a Derivation. As we need to also add the `.drv` suffix to the name we pass into text_hash_string inside calculate_derivation_path, we can simply add the suffix there and drop the parameter from build_store_path. Change-Id: Icd5343dd1458f112b9296b389e81ce2ebdd16a9f Reviewed-on: https://cl.tvl.fyi/c/depot/+/8365 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-03-30 r/6057 refactor(tvix/nix-compat): add text_hash_string functionFlorian Klink4-60/+58
Use it to calculate the text_hash_string, which is then used in the calculate_derivation_path and path_with_references functions. Relates to b/263. Change-Id: I7478825e2a23a11224212fea5e3fd06daa97d5e5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8364 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-03-30 r/6056 refactor(tvix/nix-compat): move HashAlgo to separate fileFlorian Klink3-38/+42
… and keep the pub exports as is. Change-Id: I2ad21660577553395f05b5ba71083626429b0dfc Reviewed-on: https://cl.tvl.fyi/c/depot/+/8363 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de>
2023-03-30 r/6055 refactor(tvix/nix-compat): move NixHashWithMode into separate fileFlorian Klink3-108/+114
… and keep the pub exports as is. Change-Id: I9f89a738c508c478ddba61303c21ea294f01ee9f Reviewed-on: https://cl.tvl.fyi/c/depot/+/8362 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-28 r/6052 refactor(views): move view export configuration to //viewsVincent Ambo1-11/+1
Change-Id: I064b996fd52134b7e83541ea1190774c51a8ba30 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8359 Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-03-27 r/6050 docs(tvix/store/directorysvc/grpc): fix commentFlorian Klink1-1/+1
We do compare for equality. This comment probably was when I tried to compare the `Result<T, E>`, and as `E` doesn't derive PartialEq it was annoying. Change-Id: I18bb19528c76af91c9d24d88d55dd46d0c092d20 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8354 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-27 r/6047 chore(tvix): bump Cargo dependenciesFlorian Klink2-214/+236
Change-Id: I421f7a9440bbf18646143cd9fbfe790db22a0e80 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8352 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-27 r/6046 feat(tvix/store/directorysvc): add DirectoryService::get_recursive()Florian Klink5-101/+320
This moves the recursive BFS traversal of Directory closures from the GRPCDirectoryServiceWrapper out into a a DirectoryTraverser struct implementing Iterator. It is then used from various implementors of DirectoryService in the `get_recursive()` method. This allows distinguishing between recursive requests and non-recursive requests in the gRPC client trait implementation. Change-Id: I50bfd4a0d9eb11832847329b78c587ec7c9dc7b1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8351 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-03-27 r/6045 refactor(tvix/eval): retain call frames when entering callsVincent Ambo1-6/+13
This grows the frame stack as the call stack grows, which yields *much* better user-facing error messages. I haven't measured the performance impact this has yet, for now I'm still just trying to add more information to errors and then cut down again where necessary. Change-Id: I89f058ef31979edacf4667775d460b60704ce4d7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8334 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su>
2023-03-27 r/6044 feat(tvix/store/directorysvc): add gRPC clientFlorian Klink5-4/+204
This provides a GRPCDirectoryService struct implementing DirectoryService, allowing a client to Directory objects from a (remote) tvix-store. Remote in this case is anything outside the current process, be it another process, or an endpoint on the network. To keep the sync interface in the `DirectoryService` trait, a handle to some tokio runtime needs to be passed into the constructor, and the two methods use `self.tokio_handle.spawn` to start an async function, and `self.tokio_handle.block_on` to wait for its completion. The client handle, called `grpc_client` itself is easy to clone, and treats concurrent requests internally. This means, even though we keep the `DirectoryService` trait sync, there's nothing preventing it from being used concurrently, let's say from multiple threads. There's still two limitations for now: 1) The trait doesn't make use of the `recursive` request, which currently leads to a N+1 query problem. This can be fixed by `GRPCDirectoryService` having a reference to another `DirectoryService` acting as the local side. I want to wait for general store composition code to pop up before manually coding this here. 2) It's currently only possible to put() leaf directory nodes, as the request normally requires uploading a whole closure. We might want to add another batch function to upload a whole closure, and/or do this batching in certain cases. This still needs some more thinking. Change-Id: I7ffec791610b72c0960cf5307cefbb12ec946dc9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8336 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-27 r/6043 feat(tvix/store/pathinfosvc): add gRPC clientFlorian Klink3-0/+80
Change-Id: Ie8e205c691bd11db99fcf097357c1e49161c6e19 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8349 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-03-27 r/6042 docs(tvix/store/blobreader): fix docstringFlorian Klink1-1/+1
Change-Id: Ia8ce1dd1663663903b9f60feada12e6a16fb4af2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8353 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-12-18T02·50+0000