From 4f22203a3aecd070881ae9b4eabc47532d948f01 Mon Sep 17 00:00:00 2001 From: edef Date: Wed, 17 Jan 2024 16:04:03 +0000 Subject: feat(tvix/tools/crunch-v2): init This is a tool for ingesting subsets of cache.nixos.org into its own flattened castore format. Currently, produced chunks are not preserved, and this purely serves as a way of measuring compression/deduplication ratios for various chunking and compression parameters. Change-Id: I3983af02a66f7837d76874ee0fc8b2fab62ac17e Reviewed-on: https://cl.tvl.fyi/c/depot/+/10486 Tested-by: BuildkiteCI Reviewed-by: flokli --- tvix/tools/crunch-v2/.gitignore | 1 + 1 file changed, 1 insertion(+) create mode 100644 tvix/tools/crunch-v2/.gitignore (limited to 'tvix/tools/crunch-v2/.gitignore') diff --git a/tvix/tools/crunch-v2/.gitignore b/tvix/tools/crunch-v2/.gitignore new file mode 100644 index 000000000000..4bed5da93fb2 --- /dev/null +++ b/tvix/tools/crunch-v2/.gitignore @@ -0,0 +1 @@ +*.parquet -- cgit 1.4.1