about summary refs log tree commit diff
path: root/tvix/store/Cargo.toml
diff options
context:
space:
mode:
authorFlorian Klink <flokli@flokli.de>2023-03-01T17·30+0100
committerflokli <flokli@flokli.de>2023-03-10T11·53+0000
commit510927e43a950e727c31e100f1e88f0d8a80b6b9 (patch)
tree918fcd3fcee62b0bafc9637c8116faa8680eb935 /tvix/store/Cargo.toml
parent2ef60282b61a61496e642021f0ab8eab7569bbaa (diff)
feat(tvix/store): use rayon to upload chunks concurrently r/5939
Look at the data that's written to us, and upload all chunks but the
rest in parallel, using rayon. This required moving `upload_chunk`
outside the struct, and accepting a ChunkService to use for upload
(which it was previously getting from `self.chunk_service`).

This doesn't speed up things too much for now, because things are still
mostly linear.

Change-Id: Id785b5705c3392214d2da1a5b6a182bcf5048c8d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8195
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Diffstat (limited to 'tvix/store/Cargo.toml')
-rw-r--r--tvix/store/Cargo.toml3
1 files changed, 2 insertions, 1 deletions
diff --git a/tvix/store/Cargo.toml b/tvix/store/Cargo.toml
index 09a62bc79fba..65d3420ab7d3 100644
--- a/tvix/store/Cargo.toml
+++ b/tvix/store/Cargo.toml
@@ -14,10 +14,11 @@ count-write = "0.1.0"
 data-encoding = "2.3.3"
 fastcdc = "3.0.0"
 lazy_static = "1.4.0"
+nix-compat = { path = "../nix-compat" }
 prost = "0.11.2"
+rayon = "1.6.1"
 sha2 = "0.10.6"
 sled = { version = "0.34.7", features = ["compression"] }
-nix-compat = { path = "../nix-compat" }
 thiserror = "1.0.38"
 tokio-stream = "0.1.11"
 tokio = { version = "1.23.0", features = ["rt-multi-thread"] }