about summary refs log tree commit diff
path: root/tvix/glue
diff options
context:
space:
mode:
authorConnor Brewster <cbrewster@hey.com>2024-04-20T21·41-0500
committerclbot <clbot@tvl.fyi>2024-04-23T17·02+0000
commit79698c470cf9e043204741b727ce34041fcb1e32 (patch)
treec686f8e716e214b5f8f8f571825fe0c59efc10a5 /tvix/glue
parentfa69becf4d723c1549d8252eeabeb256423dbc19 (diff)
feat(tvix/castore): upload blobs concurrently when ingesting archives r/8001
Ingesting tarballs with a lot of small files is very slow because of the
round trip time to the `BlobService`. To mitigate this, small blobs can
be buffered into memory and uploaded concurrently in the background.

Change-Id: I3376d11bb941ae35377a089b96849294c9c139e6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11497
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Autosubmit: Connor Brewster <cbrewster@hey.com>
Diffstat (limited to 'tvix/glue')
-rw-r--r--tvix/glue/src/fetchers.rs4
1 files changed, 2 insertions, 2 deletions
diff --git a/tvix/glue/src/fetchers.rs b/tvix/glue/src/fetchers.rs
index 7981770eb39a..7560c447d8d2 100644
--- a/tvix/glue/src/fetchers.rs
+++ b/tvix/glue/src/fetchers.rs
@@ -168,7 +168,7 @@ async fn hash<D: Digest + std::io::Write>(
 
 impl<BS, DS, PS> Fetcher<BS, DS, PS>
 where
-    BS: AsRef<(dyn BlobService + 'static)> + Send + Sync,
+    BS: AsRef<(dyn BlobService + 'static)> + Clone + Send + Sync + 'static,
     DS: AsRef<(dyn DirectoryService + 'static)>,
     PS: PathInfoService,
 {
@@ -242,7 +242,7 @@ where
 
                 // Ingest the archive, get the root node
                 let node = tvix_castore::import::archive::ingest_archive(
-                    &self.blob_service,
+                    self.blob_service.clone(),
                     &self.directory_service,
                     archive,
                 )