about summary refs log tree commit diff
path: root/tvix/tools/crunch-v2
AgeCommit message (Collapse)AuthorFilesLines
2024-08-23 r/8568 feat(tvix/tools/crunch-v2): Add crate2nix-checkIlan Joselevich1-1/+3
Change-Id: I922caae2ec3f87a7f3a5dcc8e9d366f0e71ce900 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12300 Tested-by: BuildkiteCI Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com> Reviewed-by: flokli <flokli@flokli.de>
2024-08-21 r/8547 feat(nix-compat/nix_http): init parse_nar[info]_strFlorian Klink2-1/+53
This moves the URL component parsing code we had in nar-bridge to nix-compat. We change the function signature to return an Option, not a Result<_, StatusCode>. This allows returning more appropriate error codes, as we can ok_or(…) at the callsite, which we now do: on an upload to an invalid path, we now return "unauthorized", while on a GET/HEAD, we return "not found". This also adds support to parse compression suffixes. While not supported in nar-bridge, other users of nix-compat might very well want to parse these paths. Also fix the error message when parsing NAR urls, it mentioned 32, not 52, which is a copypasta error from the narinfo URL parsing code. Change-Id: Id1be9a8044814b54ce68b125c52dfe933c9c4f74 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12260 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-08-16 r/8499 chore(tvix/tools/crunch-v2): bump curve25519-dalekFlorian Klink2-30/+4
Fix for RUSTSEC-2024-0344. Change-Id: I11651ea56e52988ee7cef2e02d0cbf61062f5c7b Reviewed-on: https://cl.tvl.fyi/c/depot/+/12213 Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-08-16 r/8495 chore(tvix/tools/crunch-v2): update Cargo.{lock,nix}Florian Klink2-46/+245
These were out of date. Change-Id: Ideaf888c2851bb9ec36ae01b11d93165b62cd7a5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12209 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
2024-07-09 r/8362 chore(tvix/tools/crunch-v2): Regenerate Cargo.nix and filter srcIlan Joselevich3-68/+104
The Cargo.lock also got changed because of crate2nix using --freezed. Change-Id: I90245bad27f6fee01455557bbccc865301a4f1a0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11968 Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-05-08 r/8095 refactor(nix-compat/nar/reader): reuse prev_name allocationedef1-1/+1
We reuse the prev_name allocation for Entry, instead of allocating and returning a separate Vec. We encode the `prev_name: None` case as an empty vector, since we don't allow empty names anyway, and the sorting is equivalent. Change-Id: I975b37ff873805f5ff099bc82128706891052247 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11607 Reviewed-by: Brian Olsen <me@griff.name> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-05-08 r/8089 chore(tvix/tools/crunch-v2): upgrade tokioedef2-5/+20
This drops the Sized requirement on AsyncReadExt. Change-Id: I2a89c708922ca528cb493aefd0613f477f648e83 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11608 Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI
2024-01-27 r/7453 feat(tvix/tools/crunch-v2): add CLI argsFlorian Klink6-33/+884
Use clap derive to make the input and output files configurable, as well as the chunk size parameters. Change-Id: I02b29126f3bd2c13ba2c6e7e0aa4ff048ff803ed Reviewed-on: https://cl.tvl.fyi/c/depot/+/10691 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: edef <edef@edef.eu>
2024-01-27 r/7452 feat(tvix/tools/crunch-v2): initedef12-0/+15022
This is a tool for ingesting subsets of cache.nixos.org into its own flattened castore format. Currently, produced chunks are not preserved, and this purely serves as a way of measuring compression/deduplication ratios for various chunking and compression parameters. Change-Id: I3983af02a66f7837d76874ee0fc8b2fab62ac17e Reviewed-on: https://cl.tvl.fyi/c/depot/+/10486 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>