about summary refs log tree commit diff
path: root/tvix/store/src/chunkservice
AgeCommit message (Collapse)AuthorFilesLines
2023-03-13 r/5958 refactor(tvix/store): add read_all_and_chunk methodFlorian Klink2-2/+42
This moves the logic from src/import.rs that - reads over the contents of a file - chunks them up and uploads individual chunks - keeps track of the uploaded chunks in a BlobMeta structure - returns the hash of the blob and the BlobMeta structure … into a generic read_all_and_chunk function in src/chunkservice/util.rs. It will work on anything implementing io::Read, not just files, which will help us in a bit. Change-Id: I53bf628114b73ee2e515bdae29974571ea2b6f6f Reviewed-on: https://cl.tvl.fyi/c/depot/+/8259 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de>
2023-03-11 r/5952 refactor(tvix/store): factor out hash update into functionFlorian Klink2-6/+23
We're using this in a bunch of places. Let's move it into a helper function. Change-Id: I118fba35f6d343704520ba37280e4ca52a61da44 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8251 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-03-11 r/5951 docs(tvix/store): make upload_chunk docs a docstringFlorian Klink1-1/+1
Change-Id: I6794e2a69b4907ab8d9ed2d55335c11db2c30c02 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8250 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-03-10 r/5948 refactor(tvix/store): move upload_chunk out of blobwriterFlorian Klink2-0/+33
This is useful not only in blobwriter contexts. Change-Id: I4c584b5264ff7b4bb3b1a9671affc39e18bf4ccf Reviewed-on: https://cl.tvl.fyi/c/depot/+/8245 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-03-10 r/5944 refactor(tvix/store): clippyFlorian Klink1-1/+1
Change-Id: Ib41c766a45e7d5467a4e5e90e1c00b009121959a Reviewed-on: https://cl.tvl.fyi/c/depot/+/8238 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de>
2023-03-10 r/5942 refactor(tvix/store): impl Default for MemoryChunkServiceFlorian Klink1-9/+1
Change-Id: If1ceb870e9e5d03b651923a3a786350a8663e990 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8236 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-10 r/5934 feat(tvix/store): add new_temporary for all Sled servicesFlorian Klink1-0/+7
This provides a service using /dev/shm, that's deleted once the reference is dropped. Refactor all tests to use these, which allows getting rid of most TempDir usage in the tests. The only place where we still use TempDir is in the importer tests, which work on a filesystem path. Change-Id: I08a950aa774bf9b46d9f5c92edf5efba36053242 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8193 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-03-06 r/5893 feat(tvix/store): add chunkserviceFlorian Klink3-0/+147
This adds the simpler ChunkService trait, and an implementation for it using sled, and one using a HashMap. Change-Id: Icb0fdc41b37b44e9e9e4f548d0f4acae1d83b71e Reviewed-on: https://cl.tvl.fyi/c/depot/+/8086 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI