diff options
author | Connor Brewster <cbrewster@hey.com> | 2024-05-18T20·07-0500 |
---|---|---|
committer | Connor Brewster <cbrewster@hey.com> | 2024-05-20T15·21+0000 |
commit | e7be3422566b36e5bd3aeaaf7d47537dfd050a5c (patch) | |
tree | a1c529eddf255ae172980867d80bd2415c1e3cb3 /users/edef | |
parent | b0aaff25fa68a7b55b870b59024ca2b40c658f33 (diff) |
feat(tvix/store): concurrently upload small blobs during nar ingestion r/8159
Currently all blobs are uploaded serially when ingesting NARs. If a NAR contains many, small blobs, ingestion may become slow if there is a lot of round-trip latency to the blob service. This makes the NAR ingester use the ConcurrentBlobUploader which allows for buffering small blobs in memory so they can be uploaded concurrently to the blob service without blocking further deserialization. Change-Id: I093a73770232df12d9a11e5d901b99c08505c3cb Reviewed-on: https://cl.tvl.fyi/c/depot/+/11694 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
Diffstat (limited to 'users/edef')
0 files changed, 0 insertions, 0 deletions