about summary refs log tree commit diff
path: root/tvix/store/src/blobwriter.rs
AgeCommit message (Collapse)AuthorFilesLines
2023-03-13 r/5960 feat(tvix/store): drop BlobWriterFlorian Klink1-140/+0
All code initially using this has been replaced by the simpler and more performant implementation with StreamCDC and read_all_and_chunk. Change-Id: I08889e9a6984de91c5debcf2b612cb68ae5072d1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8265 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-03-13 r/5957 refactor(tvix/store): use update_hasher in blobwriterFlorian Klink1-7/+3
Make use of the helper function here as well. Change-Id: Ia0afd84eb3903bb897ee6aee884dc291f3e4371c Reviewed-on: https://cl.tvl.fyi/c/depot/+/8258 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-03-10 r/5948 refactor(tvix/store): move upload_chunk out of blobwriterFlorian Klink1-27/+2
This is useful not only in blobwriter contexts. Change-Id: I4c584b5264ff7b4bb3b1a9671affc39e18bf4ccf Reviewed-on: https://cl.tvl.fyi/c/depot/+/8245 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-03-10 r/5945 refactor(tvix/store/blobwriter): don't cloneFlorian Klink1-2/+1
This already is a &CS, so we don't need to clone(). Change-Id: I5397a5948ae7fe4781f18df760a79047f83dca01 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8243 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-10 r/5939 feat(tvix/store): use rayon to upload chunks concurrentlyFlorian Klink1-36/+57
Look at the data that's written to us, and upload all chunks but the rest in parallel, using rayon. This required moving `upload_chunk` outside the struct, and accepting a ChunkService to use for upload (which it was previously getting from `self.chunk_service`). This doesn't speed up things too much for now, because things are still mostly linear. Change-Id: Id785b5705c3392214d2da1a5b6a182bcf5048c8d Reviewed-on: https://cl.tvl.fyi/c/depot/+/8195 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-03-10 r/5927 refactor(tvix/store): move blob splitting into a BlobWriter structFlorian Klink1-0/+149
This will moves the chunking-as-we-receive logic that so far only lived in grpc_blobservice_wrapper.rs into a generic BlobWriter. Change-Id: Ief7d1bda3c6280129f7139de3f6c4174be2ca6ea Reviewed-on: https://cl.tvl.fyi/c/depot/+/8154 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>