diff options
| author | Connor Brewster <cbrewster@hey.com> | 2024-04-20T21·41-0500 |
|---|---|---|
| committer | clbot <clbot@tvl.fyi> | 2024-04-23T17·02+0000 |
| commit | 79698c470cf9e043204741b727ce34041fcb1e32 (patch) | |
| tree | c686f8e716e214b5f8f8f571825fe0c59efc10a5 /tvix/glue | |
| parent | fa69becf4d723c1549d8252eeabeb256423dbc19 (diff) | |
feat(tvix/castore): upload blobs concurrently when ingesting archives r/8001
Ingesting tarballs with a lot of small files is very slow because of the round trip time to the `BlobService`. To mitigate this, small blobs can be buffered into memory and uploaded concurrently in the background. Change-Id: I3376d11bb941ae35377a089b96849294c9c139e6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11497 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: Connor Brewster <cbrewster@hey.com>
Diffstat (limited to 'tvix/glue')
| -rw-r--r-- | tvix/glue/src/fetchers.rs | 4 |
1 files changed, 2 insertions, 2 deletions
diff --git a/tvix/glue/src/fetchers.rs b/tvix/glue/src/fetchers.rs index 7981770eb39a..7560c447d8d2 100644 --- a/tvix/glue/src/fetchers.rs +++ b/tvix/glue/src/fetchers.rs @@ -168,7 +168,7 @@ async fn hash<D: Digest + std::io::Write>( impl<BS, DS, PS> Fetcher<BS, DS, PS> where - BS: AsRef<(dyn BlobService + 'static)> + Send + Sync, + BS: AsRef<(dyn BlobService + 'static)> + Clone + Send + Sync + 'static, DS: AsRef<(dyn DirectoryService + 'static)>, PS: PathInfoService, { @@ -242,7 +242,7 @@ where // Ingest the archive, get the root node let node = tvix_castore::import::archive::ingest_archive( - &self.blob_service, + self.blob_service.clone(), &self.directory_service, archive, ) |