about summary refs log tree commit diff
path: root/tvix/castore/src/import
diff options
context:
space:
mode:
authorConnor Brewster <cbrewster@hey.com>2024-04-20T14·43-0500
committerclbot <clbot@tvl.fyi>2024-04-20T16·46+0000
commit18ab59ed70c7d86e0d7d36379f5cbb81e82dac43 (patch)
treed51c2200a56ec8981b42ee8694fa7543b164c97e /tvix/castore/src/import
parent3107961428af2fe34fce7d6ac5085be55f7c239a (diff)
fix(tvix/castore): ensure all directories are present during ingestion r/7984
`ingest_entries` requires that all directories referenced by entries in
the ingestion stream have an explicit entry in the stream.

For example, if the stream contains a file with path `foo/bar`, there
must be an entry that comes later in the stream for the directory `foo`.

Change-Id: I61b4fbbb73ea7278715e04271d8073b484e05e61
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11488
Autosubmit: Connor Brewster <cbrewster@hey.com>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Diffstat (limited to 'tvix/castore/src/import')
-rw-r--r--tvix/castore/src/import/mod.rs8
1 files changed, 8 insertions, 0 deletions
diff --git a/tvix/castore/src/import/mod.rs b/tvix/castore/src/import/mod.rs
index c5887685bbdb..72d55e13e08e 100644
--- a/tvix/castore/src/import/mod.rs
+++ b/tvix/castore/src/import/mod.rs
@@ -42,6 +42,9 @@ pub mod fs;
 /// - The last entry must be the root node which must have a single path component.
 /// - Every entry should have a unique path, and only consist of normal components.
 ///   This means, no windows path prefixes, absolute paths, `.` or `..`.
+/// - All referenced directories must have an associated directory entry in the stream.
+///   This means if there is a file entry for `foo/bar`, there must also be a `foo` directory
+///   entry.
 ///
 /// Internally we maintain a [HashMap] of [PathBuf] to partially populated [Directory] at that
 /// path. Once we receive an [IngestionEntry] for the directory itself, we remove it from the
@@ -141,6 +144,11 @@ where
             .add(node);
     };
 
+    assert!(
+        directories.is_empty(),
+        "Tvix bug: left over directories after processing ingestion stream"
+    );
+
     // if there were directories uploaded, make sure we flush the putter, so
     // they're all persisted to the backend.
     if let Some(mut directory_putter) = maybe_directory_putter {