about summary refs log tree commit diff
path: root/tvix
AgeCommit message (Collapse)AuthorFilesLines
2023-10-05 r/6694 refactor(tvix/nar-bridge): let callbaks return calculated digestsFlorian Klink6-100/+109
This aligns behaviour more with how it should be - it's the responsibility of the callback functions to return digests of the things they consume(d). It allows further cleaning up the hasher struct. Change-Id: I9cbfc87e6abd4ff17fadf39eb6563ec3cb7fcc6f Reviewed-on: https://cl.tvl.fyi/c/depot/+/9528 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-10-05 r/6693 refactor(tvix/nar-bridge): drop reader packageFlorian Klink6-56/+41
Make the import function usable on any reader. Change-Id: I84d2004cb73cdd7a11fe8efb0f2efb6335d5e6b0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9527 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-10-05 r/6692 refactor(tvix/nar-bridge): rename item to stackItemFlorian Klink1-3/+3
Change-Id: I76c8db3b04d9f97e51e91f7d36a4dd8c6500d1a2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9526 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-10-05 r/6691 refactor(tvix/nar-bridge): inline assemblePathInfoFlorian Klink1-17/+14
This is only called once. Change-Id: I342443b8d04050929733fc84d5f36cd64060afe3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9525 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-10-04 r/6690 fix(tvix/nix-compat): reject dotfilesedef1-5/+7
Nix has historically rejected these. The current behaviour was accidentally introduced in Nix 2.4, and is considered a bug. Link: https://github.com/NixOS/nix/pull/9095 Change-Id: I38ffa911f0a413086479bd972de09671dbe85121 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9507 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: edef <edef@edef.eu>
2023-10-04 r/6689 fix(tvix/castore): explicitly name lifetimes in process_entryedef1-3/+3
Otherwise this produces absolutely inscrutable errors: note: hidden type `[async fn body@castore/src/import.rs:63:1: 63:94]` captures lifetime '_#24r Change-Id: If5d9626c9edf400de5bcec038bcaa5a3117561f0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9506 Tested-by: BuildkiteCI Autosubmit: edef <edef@edef.eu> Reviewed-by: flokli <flokli@flokli.de>
2023-10-03 r/6688 fix(tvix/nix-compat): bits are not bytesedef1-2/+2
Change-Id: Ib5744ea240895fe84454ad271ee5640b29cab838 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9505 Autosubmit: edef <edef@edef.eu> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-10-02 r/6687 fix(tvix/store): Fix FUSE support on MacOSBrian Olsen9-22/+42
This partially fixes b/312 and gets FUSE to work again on MacOS. It is mostly small type changes and an update to fuse-backend-rs because upstream currently doesn't work with MacFuse. It also sets the default FUSE thread count on MacOS to 1 because otherwise the mount command will hang when shutting down as only one thread gets ENODEV and all the others just keep blocking. Change-Id: Ifb3c4268caf296c487049c1dc4618acb32497f44 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9490 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com> Reviewed-by: flokli <flokli@flokli.de>
2023-10-02 r/6686 fix(tvix/store): FuseDaemon unmount is blocking async runtimeBrian Olsen1-1/+1
The unmount method in FuseDaemon calls join on a bunch of threads and that is a blocking call but it is called from an async context in the tvix-store binary. This change wraps the call to unmount in a spawn_blocking. Change-Id: If89183b4a3f890874e75f5faf90cd24cb18da1e1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9489 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-09-28 r/6667 docs(tvix/boot): Update command to reflect new tvix/boot directoryConnor Brewster1-5/+5
This used to be in tvix/test but was moved to tvix/boot, but not all the command were updated so they no longer work as documented. This updates the commands to use the new tvix/boot directory. Change-Id: I8658d928a152140df182c159bf98ea9d9ecd1302 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9482 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-09-26 r/6663 fix(tvix/store): rename Read method in ReadmeFlorian Klink1-1/+1
This call was renamed to `Read` a while ago, but I forgot to update the README. Change-Id: Ibcb97fe2520a5e9bb8444ec1adade6890dda4a91 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9476 Reviewed-by: Connor Brewster <cbrewster@hey.com> Reviewed-by: flokli <flokli@flokli.de> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-09-26 r/6662 feat(tvix/eval): test case for b/281Adam Joseph2-0/+2
This commit adds a test case for b/281. Change-Id: I8dfbfc0ff636184d7882530d8aefb329a3af9e5c Reviewed-on: https://cl.tvl.fyi/c/depot/+/9288 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: Adam Joseph <adam@westernsemico.com> Reviewed-by: flokli <flokli@flokli.de>
2023-09-26 r/6661 fix(tvix/store): allow building with Nix on DarwinFlorian Klink1-0/+2
Curently both fuse and virtiofs features fail to build on Darwin. Disable them for now until fixed. Change-Id: Ic99fa5e3ebec8a7072ba884101a5c18652ed409c Reviewed-on: https://cl.tvl.fyi/c/depot/+/9473 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-09-26 r/6660 chore(tvix/[ca]store): allow building without tonic-reflectionFlorian Klink8-18/+15
- rename the feature to `tonic-reflection` in both tvix-store and tvix- castore. - set it to disabled in tvix-castore by default - enable it in tvix-store, and pull in tvix-castore with the feature enabled. Change-Id: Ie22833d85569502cae55812f6eeb17a9c15b9e2c Reviewed-on: https://cl.tvl.fyi/c/depot/+/9472 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-09-26 r/6659 fix(tvix/cli): don't enable default features for tvix-storeFlorian Klink2-1/+2
These enable fuse and virtiofs features, which fail to build on MacOS. tvix-cli needs neither of these to be built, so we can disable them. Fixes b/240 (tvix-cli build on Darwin) Change-Id: I991d947f31d0185aedd1c8a341f714f4eedd03c3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9474 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-09-26 r/6658 chore(tvix/store): only depend on vhost-user-fs if virtiofs enabledFlorian Klink2-8/+12
Change-Id: Iaf88d43df27b87548794a8dddc9a1e08279b8ffb Reviewed-on: https://cl.tvl.fyi/c/depot/+/9471 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-09-26 r/6657 chore(third_party/crate2nix): bump crate2nix to current HEADFlorian Klink2-15/+25
This will bring conditional features support. Also invoke crate2nixgenerate with the --all-features argument, so all dependencies, including the ones for optional features are included in the Cargo.nix file. Change-Id: I3bbcb200c9b481f660db89efba650ea4f7418a63 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9470 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-09-26 r/6654 chore(tvix/store): move fuse-backend-rs back to upstreamFlorian Klink4-7/+7
It's in no release yet, but this is the current rev in master. Change-Id: I9e4bc2fde26e85c4182bd66b3e566c2d957072ed Reviewed-on: https://cl.tvl.fyi/c/depot/+/9469 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-09-25 r/6653 feat(tvix/store): Support listening on UNIX domain socketsConnor Brewster6-71/+388
This adds support for listening on UNIX domain sockets via the tokio-listener crate. The crate will automatically determine whether to start a TCP or UNIX domain socket server based on the listen address. Unfortunately, it's not compatible with tonic right out of the box so I added some wrapper types to implement the necessary traits to make things work. We should investigate upstreaming a `tonic` option to the tokio-listener crate which implements the relevant `tonic` traits. Example: ``` $ tvix-store daemon -l /run/tvix-store.sock INFO tvix_store: tvix-store listening on /run/tvix-store.sock $ tvix-store mount -l /mnt/tvix --blob-service-addr grpc+unix:///run/tvix-store.sock --directory-service-addr grpc+unix:///run/tvix-store.sock --path-info-service-addr grpc+unix:///run/tvix-store.sock $ ls /mnt/tvix ``` Change-Id: I91c4a4b0c5a177b3b90e6c01a4e5d263130e6bdb Reviewed-on: https://cl.tvl.fyi/c/depot/+/9429 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-09-25 r/6652 chore(tvix/tests): rename to //tvix/bootFlorian Klink4-41/+44
This is mostly boot tooling, the integration test is just one instance making use of it. Expose initrd, kernel and runVM as a separate target to CI, and move the tests to a subdirectory. Change-Id: I1d22cd68bf5af095bc11dd9d7117b62956c7f7f2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9465 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-09-25 r/6651 fix(tvix/tests/tvix-init): run as pid1, support init=Florian Klink3-53/+133
We were previously not running as PID1, but were executing things via uinitcmd (by u-root init). This had some problems - a real system pid1 really wants to be pid1. u-root init itself seems to be too complicated to configure as a chainloader, so instead, we just provide init/pid1 ourselves and do the little bit of mount setup / stage 1 that's necessary. `tvix.exec` is renamed to `tvix.run`, to further distinguish it from the `exec` bash function. Our init will now also power off the machine if the shell, or the custom tvix.run command finished. In the case of switch_root (`init=` being passed), we terminate before, so it's up to the real system anyways. Change-Id: If00c0c11ca9393968e76ae0d131594b5c135fa48 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9454 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2023-09-24 r/6650 fix(tvix/eval): fix b/281 by adding Value::CatchableAdam Joseph16-247/+320
This commit makes catchable errors a variant of Value. The main downside of this approach is that we lose the ability to use Rust's `?` syntax for propagating catchable errors. Change-Id: Ibe89438d8a70dcec29e016df692b5bf88a5cad13 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9289 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: Adam Joseph <adam@westernsemico.com> Tested-by: BuildkiteCI
2023-09-24 r/6649 refactor(tvix/eval): factor CatchableErrorKind out of ErrorKindAdam Joseph5-29/+57
This commit creates a separate enum for "catchable" errors (the kind that `builtins.tryEval` can detect). Change-Id: Ie81d1112526d852255d9842f67045f88eab192af Reviewed-on: https://cl.tvl.fyi/c/depot/+/9287 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: Adam Joseph <adam@westernsemico.com>
2023-09-24 r/6648 docs(tvix) sync readme and website component listing, extendFlorian Klink2-8/+12
These existed with slightly similar descriptions in both places, and castore and nar-bridge and castore were missing entirely. Change-Id: I4794c18665e25ee1f812975b526ff27ce197d0af Reviewed-on: https://cl.tvl.fyi/c/depot/+/9453 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-09-24 r/6646 chore(tvix): add /*.sled to .gitignoreFlorian Klink1-0/+2
The README suggests putting state here for some examples, so let's make sure it doesn't cause a dirty worktree. Change-Id: I3a191f5b2681ef6937a03d3a9686ae0d7aebd76e Reviewed-on: https://cl.tvl.fyi/c/depot/+/9450 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2023-09-24 r/6645 feat(tvix/tests): add basic plumbing for integration testsFlorian Klink3-0/+337
This adds a basic skeleton to test various tvix components in combination. As a first start, we test the virtiofs integration, by seeding the tvix- store with some data, booting a VM, and listing the contents from there. Change-Id: Ice6516d0eb508d19ea8fabefee860a0950046171 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9449 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-09-24 r/6644 feat(tvix/nar-bridge): provide a default for store-addrFlorian Klink1-3/+3
Defaulting to where `tvix-store daemon` listens to is a sane choice. Change-Id: Ic52fa856c5708e0e1a8d51618f8c9ad1894fd28f Reviewed-on: https://cl.tvl.fyi/c/depot/+/9452 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-09-24 r/6643 refactor(tvix/nar-bridge): rename binary to nar-bridgeFlorian Klink1-0/+0
nar_bridge is an odd name for the binary. Change-Id: I761ec5f986bde2f7e50e5a0c0b6182164a6cdc7f Reviewed-on: https://cl.tvl.fyi/c/depot/+/9451 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de>
2023-09-24 r/6642 feat(tvix/store/fs): Add support for virtiofs backendConnor Brewster6-5/+587
This adds a virtiofs daemon implementation which hooks into the existing tvix-store filesystem implementation that is used for FUSE. This allows adding the filesystem to a microvm without having to set up FUSE inside the guest. Change-Id: If80c36c9657f2289853e8d9a364bf4f1f7b7559c Reviewed-on: https://cl.tvl.fyi/c/depot/+/9344 Autosubmit: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-09-23 r/6641 feat(tvix/cli/derivation): reject derivations with empty namesFlorian Klink1-0/+27
As shown in the previous CLs, we can very well have store paths starting with periods, but we can't have derivations with an empty name: ``` nix-build -E 'derivation { name = ""; builder = "/bin/sh"; system = "x86_64-linux"; }' error: store path 'nr7i5pf18hw2zg487vkdyrbasdqylfcj-' has an empty name ``` I'm currently using ErrorKind::Abort here, because we don't have a Derivation- related error in tvix-eval (and probably don't want to). Change-Id: I0e9743cee98dbfa69e9caa2a58352176270f15bd Reviewed-on: https://cl.tvl.fyi/c/depot/+/9448 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-09-23 r/6640 fix(tvix/nix-compat): drop first character period checkFlorian Klink1-6/+10
Suggested in https://cl.tvl.fyi/c/depot/+/9108/5, but this disallows adding a .gitignore file to the store. Remove the check, and add a testcase. Change-Id: Ieb78c417934756b2dbeb493040fe79726d1b9079 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9447 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de>
2023-09-23 r/6639 feat(tvix/cli): add back a derivation testFlorian Klink1-0/+30
Setting up the evaluator is a bit annoying currently, might get easier with b/262, but it's better than no tests on that granularity at all. Change-Id: Ie8c61466768f37f4efbc19ad497d37f87ddc2044 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9446 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-09-23 r/6638 feat(tvix/nix-compat): fix and improve error messageFlorian Klink1-1/+6
Similar to cl/9350. Most of the times, this is still a regular string, so let's print it like that if we can, and resort to base64 encoding if we can't. We were also wrongly always outputting the second character in the string. Change-Id: Id0e2a9d9f1ad3d2d7b554893ecd89a7e6383e9c2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9445 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-09-22 r/6633 refactor(tvix/nar-bridge): update to new separated protosFlorian Klink16-158/+170
Change-Id: I5eaadc837a4d3a7f635574437127a22de88f556b Reviewed-on: https://cl.tvl.fyi/c/depot/+/9407 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de>
2023-09-22 r/6632 chore(tvix): re-enable {ca,}store-protos-goFlorian Klink3-30/+22
This couldn't be enabled before until go get was able to fetch the modules again. Change-Id: I82db7d22e497661ea2a0c8a45c0c865fcc0a1f18 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9406 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-09-22 r/6630 chore(tvix/castore): remove dependency on anyhowFlorian Klink4-9/+1
We don't use this here, except in a test, where we don't really use it either. Change-Id: Ia6c45fccf663fe328942b1e1a2cd1c3a1b7f9ae5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9376 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2023-09-22 r/6629 refactor(tvix): move castore into tvix-castore crateFlorian Klink87-1832/+2306
This splits the pure content-addressed layers from tvix-store into a `castore` crate, and only leaves PathInfo related things, as well as the CLI entrypoint in the tvix-store crate. Notable changes: - `fixtures` and `utils` had to be moved out of the `test` cfg, so they can be imported from tvix-store. - Some ad-hoc fixtures in the test were moved to proper fixtures in the same step. - The protos are now created by a (more static) recipe in the protos/ directory. The (now two) golang targets are commented out, as it's not possible to update them properly in the same CL. This will be done by a followup CL once this is merged (and whitby deployed) Bug: https://b.tvl.fyi/issues/301 Change-Id: I8d675d4bf1fb697eb7d479747c1b1e3635718107 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9370 Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-09-22 r/6628 docs(tvix/castore): rename traverse_to -> descend_toFlorian Klink3-18/+14
With the move of this code out into castore it has become apparent this is a general descent inside the castore. Concerns like making sure the whole Directory closure has been fetched/is fetched initially is nothing this layer needs to worry about. We can handle this during substitution of a new PathInfo, once there's store composition. Closes b/270. Change-Id: I661ed08e54bc81478e032cfb9abeb23e5b337fbe Reviewed-on: https://cl.tvl.fyi/c/depot/+/9373 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-09-22 r/6627 chore(tvix): remove some unused crate dependenciesFlorian Klink5-481/+26
Change-Id: I31b0e7be3a2ebc268c6f4045ba282e8fb1c72a64 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9375 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-09-22 r/6626 feat(tvix/shell): add cargo-macheteFlorian Klink1-0/+1
This helps spotting unused crate dependencies. It's preferred over cargo-udeps for the reasons described in https://blog.benj.me/2022/04/27/cargo-machete/ Change-Id: Ie801004485858741f2fa5ae6f33bd0ddfc292cd0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9374 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-09-22 r/6625 docs(tvix/store/pathinfosvc): fix copypasta errorFlorian Klink1-1/+0
This is not a simple get and put returning digest. You currently can get by output hash, and a put gives you back the (possibly modified) PathInfo message. Change-Id: I43174d87a764417b8620c4d5bdac058cc25b0373 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9371 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2023-09-22 r/6624 docs(tvix/eval): fix some broken docstr referencesFlorian Klink5-10/+11
There's some more left, but they've been renamed/refactored out of sight. Change-Id: I41579dedc74342b4c5f8cb39d2995b5b0c90b0f4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9372 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de>
2023-09-21 r/6623 refactor(tvix/store): Asyncify PathInfoService and DirectoryServiceConnor Brewster24-727/+776
We've decided to asyncify all of the services to reduce some of the pains going back and for between sync<->async. The end goal will be for all the tvix-store internals to be async and then expose a sync interface for things like tvix eval io. Change-Id: I97c71f8db1d05a38bd8f625df5087d565705d52d Reviewed-on: https://cl.tvl.fyi/c/depot/+/9369 Autosubmit: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-09-20 r/6622 refactor(tvix/store/fs): Separate FUSE and filesystem codeConnor Brewster10-133/+141
In prepration for adding virtiofs support, I thought it would make sense to split out the filesystem implementation from FUSE itself. The `fs` module holds the tvix-store filesystem implemetation and the `fuse` module holds the code to spawn a FUSE daemon backed by multiple threads. Change-Id: I8c58447b8c3aa016a613068f8e7ec166554e237c Reviewed-on: https://cl.tvl.fyi/c/depot/+/9343 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: Connor Brewster <cbrewster@hey.com>
2023-09-20 r/6621 refactor(tvix/store/fuse): Switch from fuser to fuse-backend-rsConnor Brewster11-584/+934
This switches the FUSE implementation from fuser to fuse-backend-rs. fuse-backend-rs is designed to work with both FUSE and virtiofs. Virtiofs support will make it possible to plug the tvix-store into a microvm and have `/nix/store` access without having to setup FUSE inside the guest. Additionally fuse-backend-rs has nice support for running multiple FUSE threads and has some async support. The goal of this commit is to mechanically switch over to fuse-backend-rs with minimal changes. I did have to add some locks here and there because fuse-backend-rs uses `&self` on all methods whereas fuser uses `&mut self`. `&self` is required for concurrent access to the FUSE server, so this makes sense. We can consider switching to concurrent maps and use some other techniques to reduce lock contention and critical section size. Issue: https://b.tvl.fyi/issues/305 Change-Id: Icde5a58c6eef98f8984c1e04e980b756dfb76b47 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9341 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-09-18 r/6616 feat(tvix/nar-bridge): graceful shutdownFlorian Klink2-16/+30
This gives existing clients 30s to finish their requests after receiving an interrupt. Change-Id: Ia9b0e662fd1ffbbb6c2d03f3dd6548b13cf3d241 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9365 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-09-18 r/6615 refactor(tvix/nar-bridge): rename contentReader to blobReaderFlorian Klink1-5/+5
More aligned with how it's called in other places Change-Id: I759ac7ca3b5b69c1101d2d51a569d76c183a6330 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9362 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-09-18 r/6614 refactor(tvix/nar-bridge): don't buffer blob in memoryFlorian Klink1-16/+25
Create a pipe, pass the read end, and have a goroutine write to the write end. Change-Id: I301c273355705e60113b018e7e84b76972200e8c Reviewed-on: https://cl.tvl.fyi/c/depot/+/9361 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-09-18 r/6613 refactor(tvix/nar-bridge): simplify CLI interfaceFlorian Klink8-138/+56
Only keep the `serve` subcommand, and make it appear at the root. Introduce a --log-level argument, and be a bit less noisy in normal operation. Change-Id: I86b8abde1869a5c0c947508bcc29f845222aac09 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9360 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2023-09-18 r/6612 fix(tvix/nar-bridge): don't log error on simple 404sFlorian Klink2-2/+2
`nix copy` checks if NARs and NARInfo files are present, before uploading them. That's not an error, but normal behaviour, so no need to log with level info for these cases. We only want to log if the error is not a 404, and log with Warn level. Change-Id: I762de3b862d070a0f18bc62e324e94ca5c7c3693 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9359 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-11-21T18·44+0000