depot - monorepo for the virus lounge

Age	Commit message (Collapse)	Author	Files	Lines
2024-10-19	r/8846 feat(users/edef/weave): use FxHashSet and dedupe early	edef	4	-4/+33
	Deduping early saves a fair bit of memory, but the extra hashing is costly. We switch to FxHash, since we don't need a DoS-proof hash, but we do need it to be fast. Change-Id: Ic6b7010874c417862baa9b882593208c8dd1d5e6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12648 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-10-19	r/8845 feat(users/edef/weave): ingest roots in Parquet format	edef	1	-13/+15
	Parsing of store-paths.xz is now handled by //users/edef/fetchroots. Change-Id: I78be5aada0c0a321ed79d80c9b615e5f997ac3e0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12670 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-10-19	r/8844 refactor(users/edef/weave/swizzle): use polars streaming	edef	5	-112/+70
	This vastly reduces the memory requirements, so we can run in ~40G RAM. Change-Id: I4952a780df294bd852a8b4682ba2fd59b9bae675 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12667 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-10-19	r/8843 refactor(users/edef/weave): simplify OwningRef handling	edef	2	-32/+12
	We switch to working with the underlying Arc<Bytes<u8>> type, since Buffer<u8> is already essentially an OwningRef-esque subslice. Because we're now working with an exposed Arc directly, we don't need to have our own `unsafe impl StableAddress` any more. Change-Id: I9ce2edc6899177145e15b72aa5380f708a62173c Reviewed-on: https://cl.tvl.fyi/c/depot/+/12668 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-10-19	r/8842 refactor(users/edef/weave): use swap rather than CAS	edef	1	-3/+5
	We only care about ordering on the same variable, and we rely on the release barrier provided by rayon's thread joining. The comparison failing is always an error path, and we're indifferent about which thread it fails on. Change-Id: I592a7eaae5b6935cf424c6576a49786f39909fb6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12666 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-10-19	r/8841 refactor(users/edef/weave): simplify DoubleEndedIterator bound	edef	1	-1/+1
	Thanks, Clippy! Change-Id: I116b478124b72e070e94150ee850532752f64d60 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12669 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-10-17	r/8828 fix(users/edef/weave): use safer_owning_ref	edef	4	-33/+35
	owning_ref has serious unsoundness. Change-Id: Ie760697cd6399e6bc75f1ad17c9bb74adc077a35 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12656 Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI Autosubmit: edef <edef@edef.eu> Reviewed-by: flokli <flokli@flokli.de>
2024-10-12	r/8800 chore(tvix/tools): move weave to //users/edef	Vincent Ambo	9	-0/+11508
	This is not a core Tvix tool, it's a tool that uses a Tvix component. Change-Id: I705f2c4ab87f1512e005007c933e16b84ed4279f Reviewed-on: https://cl.tvl.fyi/c/depot/+/12605 Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su>