From beae3a4bf163c34b90530ab4601ce4dc753ed9e4 Mon Sep 17 00:00:00 2001 From: Florian Klink Date: Thu, 19 Oct 2023 15:01:53 +0100 Subject: chore(tvix/castore): move data model docs to here These describe the castore data model, so it should live in the castore crate. Also, some minor edits to //tvix/store/docs/api.md, to honor the move of the castore bits to tvix-castore. Change-Id: I1836556b652ac0592336eac95a8d0647599f4aec Reviewed-on: https://cl.tvl.fyi/c/depot/+/9893 Autosubmit: flokli Reviewed-by: tazjin Tested-by: BuildkiteCI --- tvix/castore/docs/data-model.md | 50 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 50 insertions(+) create mode 100644 tvix/castore/docs/data-model.md (limited to 'tvix/castore/docs/data-model.md') diff --git a/tvix/castore/docs/data-model.md b/tvix/castore/docs/data-model.md new file mode 100644 index 000000000000..2df6761aae8f --- /dev/null +++ b/tvix/castore/docs/data-model.md @@ -0,0 +1,50 @@ +# Data model + +This provides some more notes on the fields used in castore.proto. + +See `//tvix/store/docs/api.md` for the full context. + +## Directory message +`Directory` messages use the blake3 hash of their canonical protobuf +serialization as its identifier. + +A `Directory` message contains three lists, `directories`, `files` and +`symlinks`, holding `DirectoryNode`, `FileNode` and `SymlinkNode` messages +respectively. They describe all the direct child elements that are contained in +a directory. + +All three message types have a `name` field, specifying the (base)name of the +element (which MUST not contain slashes or null bytes, and MUST not be '.' or '..'). +For reproducibility reasons, the lists MUST be sorted by that name and also +MUST be unique across all three lists. + +In addition to the `name` field, the various *Node messages have the following +fields: + +## DirectoryNode +A `DirectoryNode` message represents a child directory. + +It has a `digest` field, which points to the identifier of another `Directory` +message, making a `Directory` a merkle tree (or strictly speaking, a graph, as +two elements pointing to a child directory with the same contents would point +to the same `Directory` message. + +There's also a `size` field, containing the (total) number of all child +elements in the referenced `Directory`, which helps for inode calculation. + +## FileNode +A `FileNode` message represents a child (regular) file. + +Its `digest` field contains the blake3 hash of the file contents. It can be +looked up in the `BlobService`. + +The `size` field contains the size of the blob the `digest` field refers to. + +The `executable` field specifies whether the file should be marked as +executable or not. + +## SymlinkNode +A `SymlinkNode` message represents a child symlink. + +In addition to the `name` field, the only additional field is the `target`, +which is a string containing the target of the symlink. -- cgit 1.4.1