about summary refs log tree commit diff
path: root/tvix/castore/src/directoryservice/closure_validator.rs (follow)
AgeCommit message (Collapse)AuthorFilesLines
2024-04-07 r/7864 refactor(tvix/castore): migrate closure_validator to rstestFlorian Klink1-13/+14
Change-Id: I6c594d2e670a681484b858c3e04bc25b9e5a2077 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11368 Tested-by: BuildkiteCI Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
2024-03-24 r/7774 refactor(tvix/castore/directory/grpc_wrapper): use ClosureValidatorFlorian Klink1-1/+1
This greatly simplifies the code in this function, replacing it with a much better tested (and more capable!) version of the validation logic. It also enables the gRPC server frontend to make use of the DirectoryPutter interface. While this might not be too visible in terms of latency thanks to gRPC streams bursting, it also enables further optimizations later (such as bucketing of directory closures). Change-Id: I21f805aa72377dd5266de3b525905d9f445337d6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11221 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-03-24 r/7772 feat(tvix/castore/directory): add ClosureValidatorFlorian Klink1-0/+254
This can be used to validate a Directory closure (connected DAG of Directories), and their insertion order. Directories need to be inserted (via `add`), in an order from the leaves to the root. During insertion, we validate as much as we can at that time: - individual validation of Directory messages - validation of insertion order (no upload of not-yet-known Directories) - validation of size fields of referred Directories Internally it keeps all received Directories (and their sizes) in a HashMap, keyed by digest. Once all Directories have been inserted, a drain() function can be called to get a (deduplicated and) validated list of directories, in from-leaves-to-root order (to be stored somewhere). While assembling that list, a check for graph connectivity is performed too, to ensure there's no separate components being sent (and only one root). It adds a test suite for these cases, which is much nicer to test than where we previously had these checks (only in the gRPC server wrapper). Followup CLs will move the existing putters to use this. Change-Id: Ie88c832924c170a24626e9e3e91d868497b5d7a4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11220 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de>