about summary refs log tree commit diff
path: root/tvix/castore/src/fs/mod.rs
AgeCommit message (Collapse)AuthorFilesLines
2024-04-15 r/7932 refactor(tvix/castore/fs): add parenthesis for readabilityFlorian Klink1-4/+4
As suggested in cl/11426. Change-Id: Ic2bb8cf2838bf0be09fb8bc62b8e598a3d153699 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11434 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-04-15 r/7931 refactor(tvix/castore/fs): remove From<Node> for InodeDataFlorian Klink1-64/+33
These were copying unnecessarily. Instead, have a InodeData::from_node(), which *consumes* the Node entirely, returns `InodeData` and the split-off name (which is not part of InodeData). Callers can then use the result in various helper functions, like: - InodeData::as_fuse_type - InodeData::as_fuse_file_attr - InodeData::as_fuse_entry … to prepare their replies to the kernel. This removes not only a bunch of clones, but also a lot of copy-pasted code. Change-Id: Idbca5f25cc29e96c1f4c614b33dff2becb0a8738 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11435 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-04-15 r/7921 refactor(tvix/castore/fs): add InodeData::as_fuse_{entry,file_attr}Florian Klink1-32/+8
Remove the now unused gen_file_attr (which had a wrong docstring). Change-Id: Ie86b14d1ad798e6233bc44c43ace3f8b95c67ea9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11430 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-04-15 r/7920 refactor(tvix/castore/fs): remove "add … handle" debug messagesFlorian Klink1-2/+0
Change-Id: Iac22bbef96a2afa0416f011d073934b52b19975d Reviewed-on: https://cl.tvl.fyi/c/depot/+/11433 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-04-15 r/7919 feat(tvix/castore/fs): assign read[dir*]/release[dir] ops to parent spanFlorian Klink1-13/+20
When a directory or file is open()'ed, we already put some data into a lookup table, and subsequent operations then use the returned handle id. By also adding the span that's been created during these calls into the lookup table, we can properly set the span parent for these requests, nicely connecting the individual operations to the bigger picture. Change-Id: Ia354842fccdbc7f45c2d3efda3acf058b2dbc48e Reviewed-on: https://cl.tvl.fyi/c/depot/+/11429 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Brian Olsen <me@griff.name>
2024-04-15 r/7918 fix(tvix/castore/fs): use record to add fields to current spanFlorian Klink1-19/+11
Instead of creating another child span, we can use `tracing::Span::current().record(k,v)` to add an additional field to the current span. Change-Id: I337faac0e73a0da6eb0a52cb75c2e8c026eff774 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11428 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-04-15 r/7917 feat(tvix/castore/fs): implement readdirplusFlorian Klink1-1/+126
This currently shares some code with readdir, except it's also providing a second `fuse_backend_rs::api::filesystem::Entry` argument to the `add_entry` function call. Refactoring this to reduce some duplication is left for a future CL. Change-Id: I282c8dfc6a711d00a4482c87cbb84d4950c0aee9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11426 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-04-15 r/7916 refactor(tvix/castore/fs): use consistent span field name for handleFlorian Klink1-5/+4
Use rq.handle in `release` too, and remove interpolating it into the log message itself. Also update the comment, we don't get ownership, just simply drop, and change the level to warn!, as suggested in cl/11425. Change-Id: If4e6cff6d8b580671b1548ae3862851db4af6694 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11427 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-04-15 r/7915 feat(tvix/castore/fs): implement opendir/releasedirFlorian Klink1-60/+142
Similar to how we already handle opening files, implement opendir/ releasedir, and keep a map of dir_handles. They point to the rx side of a channel. This greatly improves performance listing the root of the filesystem when used inside tvix-store, as we don't need to re-request the listing (and skip to the desired position) all the time. Change-Id: I0d3ec4cb70a8792c5a1343439cf47d78d9cbb1d6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11425 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-04-15 r/7914 refactor(tvix/castore/fs): use std::sync::MutexFlorian Klink1-21/+24
This allows us acquiring the lock in sync code still. Also, simplify some of the error handling a bit. Change-Id: I29e83b715f92808e95ecb0ae9de787339d1a371d Reviewed-on: https://cl.tvl.fyi/c/depot/+/11424 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-04-15 r/7913 refactor(tvix/castore/fs): simplify some separate spawn and blocksFlorian Klink1-13/+6
We can just pass an async move closure to `self.tokio_handle.block_on` and make this a bit shorter. Change-Id: Iba674f34f22ba7a7de7c5bae59d64584884cb17c Reviewed-on: https://cl.tvl.fyi/c/depot/+/11423 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-04-15 r/7912 feat(tvix/castore/fs): support extended attributesFlorian Klink1-1/+94
This exposes `user.tvix.castore.{blob,directory}.digest` xattr keys for files and directories: ``` ❯ getfattr -d /tmp/tvix/06jrrv6wwp0nc1m7fr5bgdw012rfzfx2-nano-7.2-info getfattr: Removing leading '/' from absolute path names user.tvix.castore.directory.digest="b3:SuYDcUM9RpWcnA40tYB1BtYpR0xw72v3ymhKDQbBfe4=" ❯ getfattr -d /tmp/tvix/156a89x10c3kaby9rgf3fi4k0p6r9wl1-etc-shells getfattr: Removing leading '/' from absolute path names user.tvix.castore.blob.digest="b3:pZkwZoHN+/VQ8wkaX0wYVXZ0tV/HhtKlSqiaWDK7uRs=" ``` It's currently mostly used for debugging, though it might be useful for tvix-castore-aware syncing programs using the filesystem too. Change-Id: I26ac3cb9fe51ffbf7f880519f26741549cb5ab6a Reviewed-on: https://cl.tvl.fyi/c/depot/+/11422 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Reviewed-by: Brian Olsen <me@griff.name>
2024-02-19 r/7558 feat(tvix/castore): Compile fix for DarwinPeter Kolloch1-0/+3
Towards https://b.tvl.fyi/issues/264 Change-Id: If8fa912ae3fb2987b761f649ab738529ebf3b2e8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10970 Autosubmit: Peter Kolloch <info@eigenvalue.net> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-01-09 r/7364 refactor(tvix/castore): do clone inside a scopeFlorian Klink1-6/+5
Make it clear this is only used inside the scope. Change-Id: Ie94f88d7f0fb58cd4bf9c2f1176000b272e6f2e6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10585 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-01-09 r/7359 refactor(tvix): use AsRef<dyn …> instead of Deref<Target= …>Florian Klink1-7/+6
Removes some more needs for Arcs. Change-Id: I9a9f4b81641c271de260e9ffa98313a32944d760 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10578 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-12-29 r/7270 feat(tvix/castore/fs): borrow some matchesFlorian Klink1-4/+4
We only do things with the reference, so we don't need to locally borrow it. Change-Id: I6073f7ec7aff717ae3069e28a00b1cb408a50ceb Reviewed-on: https://cl.tvl.fyi/c/depot/+/10455 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-12-24 r/7258 refactor(tvix/castore): add RootNode impl for BTreeMap, mv fs testsFlorian Klink1-0/+3
cl/10378 did already move store/fs to castore/fs, but we kept the tests in tvix-store, as they were populating a PathInfoService to make nodes appear in the mount root. Update these tests to now just insert root nodes into a BTreeMap, and ensure we can use that as a RootNodes too. Change-Id: Iad7d1ee4f9423eb6e3a1da33f433842c9ae0de1f Reviewed-on: https://cl.tvl.fyi/c/depot/+/10410 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-12-22 r/7256 chore(tvix): move store/fs to castore/fsFlorian Klink1-0/+627
With the recent introduction of the RootNodes trait, there's nothing in the fs module pulling in tvix-store dependencies, so it can live in tvix-castore. This allows other crates to make use of TvixStoreFS, without having to pull in tvix-store. For example, a tvix-build using a fuse mountpoint at /nix/store doesn't need a PathInfoService to hold the root nodes that should be present, but just a list. tvix-store now has a pathinfoservice/fs module, which contains the necessary glue logic to implement the RootNodes trait for a PathInfoService. To satisfy Rust orphan rules for trait implementations, we had to add a small wrapper struct. It's mostly hidden away by the make_fs helper function returning a TvixStoreFs. It can't be entirely private, as its still leaking into the concrete type of TvixStoreFS. tvix-store still has `fuse` and `virtiofs` features, but they now simply enable these features in the `tvix-castore` crate they depend on. The tests for the fuse functionality stay in tvix-store for now, as they populate the root nodes through a PathInfoService. Once above mentioned "list of root nodes" implementation exists, we might want to shuffle this around one more time. Fixes b/341. Change-Id: I989f664827a5a361b23b34368d242d10c157c756 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10378 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>