about summary refs log tree commit diff
path: root/tvix/boot
AgeCommit message (Collapse)AuthorFilesLines
2024-06-29 r/8324 feat(tvix/boot/tests): add preStart parameterFlorian Klink1-0/+6
This allows running some code before tvix-daemon is started up, which is helpful to spin up some mock services. Change-Id: Ie7b6eaaf76c40def493f50879ee5255fd1ebd2fe Reviewed-on: https://cl.tvl.fyi/c/depot/+/11893 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de>
2024-06-29 r/8323 fix(tvix/boot/tests): set SSL_CERT_FILEFlorian Klink1-0/+2
This is necessary for http clients to successfully construct. Change-Id: If0d968e964f5c58d9956b92ee0ab3d97c39391d6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11892 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-06-29 r/8322 feat(tvix/boot): introduce useNarBridge parameterFlorian Klink1-2/+50
This is still useful to test the ingestion part of nar-bridge. Reintroduce the logic removed in cl/11398, but put it behind a `useNarBridge` parameter, defaulting to false. Add an additional target to tvix.boot.tests, `closure-nixos-nar-bridge`, which does the same as `closure-nixos`, except seeding the contents through nar-bridge. Change-Id: Iee87d5d9c7b7d02df11ee47fbc3f1fbe951935e9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11891 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de>
2024-06-29 r/8321 refactor(tvix/boot/tests): only use exportReferencesGraph when neededFlorian Klink1-5/+5
These fields don't need to be set if isClosure is false. Change-Id: Id29ed034e0757c61cef09885962dfc5507ca106c Reviewed-on: https://cl.tvl.fyi/c/depot/+/11890 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-05-15 r/8147 chore(tvix/nar-bridge): move to nar-bridge-goFlorian Klink1-1/+1
Make some space for the rust implementation. Change-Id: I924dc1657be10abe5a11951c3b9de50bae06db19 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11662 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: yuka <yuka@yuka.dev>
2024-05-11 r/8100 chore(tvix/castore/blob): remove sled implementationFlorian Klink1-7/+2
This never did any chunking, and sled (rightfully) performs really bad if values get too large. We switched the default to using the objectstore backend with the local filesystem a while ago, no need to keep this footgun around anymore. Change-Id: I2c12672f2ea6a22e40d0cbf9161560baddd73d4a Reviewed-on: https://cl.tvl.fyi/c/depot/+/11616 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-04-22 r/7992 feat(tvix/boot): default CH_NUM_CPUS=2, and set num_queues= to itFlorian Klink1-3/+3
See https://patchwork.kernel.org/project/qemu-devel/patch/20200706135650.438362-5-stefanha@redhat.com/ Change-Id: I3ba9a1c1f8b0be27f215da6b71cd3a0ded513a75 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11502 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-04-16 r/7951 feat(tvix/boot): boot nixos VMFlorian Klink1-0/+34
This makes use of the vmCmdline and assertVMOutput to boot a NixOS VM off of `tvix-store virtiofs` have it write a message on successful bootup, then have it shut down again. We check we see the message in the output. Seeding the store with the closure it's what's currently taking most of the time (as it's all linear still), so log a message before booting the VM. Change-Id: I6887d935f589eb0ad6507033c01337e1ec500a0b Reviewed-on: https://cl.tvl.fyi/c/depot/+/11191 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-04-13 r/7901 refactor(tvix/boot): use tvix-store copyFlorian Klink1-33/+6
Instead of using an additional Nix build producing a directory structure containing xz-compressed NAR files and NARInfos, and uploading these via `curl` to NAR-bridge, which then decomposes them and uploads them to tvix-store, have tvix-store interpret the exported reference graph directly, and deal with the uploading on its own. Change-Id: I44ee0b132944c53d11abb688d2d017f0cc9c3d97 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11398 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-03-31 r/7822 chore(tvix/boot): bump u-rootFlorian Klink1-4/+4
Bump to the latest u-root release. This seems to be compatible with go 1.22, too, which previously segfaulted the compiler in cl/11289. Change-Id: Ie8a0d3cb51a435d6a0b194b38a9c83ee260618df Reviewed-on: https://cl.tvl.fyi/c/depot/+/11291 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2024-03-18 r/7729 feat(tvix/boot): make VM cmdline and asserted VM output configurableFlorian Klink1-2/+8
This can be used in the future to boot proper NixOS VMs, which want a init= in their cmdline. Change-Id: Iad4a25d5081f3a6af1c7f62f15853f3afaae4a0c Reviewed-on: https://cl.tvl.fyi/c/depot/+/11190 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-03-18 r/7728 fix(tvix/boot/tests): disable otlpFlorian Klink2-3/+8
We don't have a OTLP collector inside the Nix build, this removes some noise from the log about not being able to reach it. Change-Id: I058839c8f214821b536306c8c30e1a8e7192153b Reviewed-on: https://cl.tvl.fyi/c/depot/+/11189 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-03-18 r/7727 feat(tvix/boot): support seeding closures via nar-bridgeFlorian Klink1-42/+99
This updates the parameters mkBootTest can be called with. It now accepts a `path`, and then either importPathName needs to be set, or isClosure needs to be set to true. The former activates the existing functionality, tvix-store import is used to import contents as a NAR-addressed store path. The latter uploads the path as a closure (so including its references, and keeping the store paths intact) to tvix-store. We use nar-bridge, and the HTTP interface it provides to do this. As `nix copy` can't be used inside a Nix build, we use `pkgs.mkBinaryCache` to come up with the .narinfo and .nar files that would be in a binary cache, and then use a bit of GNU Parallel and bash to upload store paths ourselves. Change-Id: Icfa5c0af0c22ab5418686947aa2c060f5987b873 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11188 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-03-18 r/7726 docs(tvix/boot): document we could use a smaller kernel here.Florian Klink1-0/+2
We currently use a pretty big kernel to boot the tvix VMs, with a lot of drivers. It can probably be slimmed down significantly, at least on the hardware driver and filesystem front. Change-Id: I4c7f73fb2dafbf2bcdec8057d2b14a9b0e9b3275 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11187 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de>
2024-03-18 r/7725 feat(tvix/boot/tests): make urls and paths configurableFlorian Klink1-16/+39
Fork off a long-living tvix-store daemon in the background, and have both the import, as well as the virtiofs invocation connect to that one, rather than use the underlying storage directly. This also opens up the potential to have other things seed the store (like nar-bridge). Change-Id: Ie13aedaf7ab31e5ebe78a7be42f144af3920fa9c Reviewed-on: https://cl.tvl.fyi/c/depot/+/11186 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-03-11 r/7686 feat(tvix/boot/tests): add simple objectstore testFlorian Klink1-25/+30
This makes BLOB_SERVICE_ADDR configurable, and creates a flavor setting it to objectstore+file://$PWD/blobs. Change-Id: I68c21367f83f68b4dee701c5678f438c7d8fbe43 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11137 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2023-10-05 r/6703 refactor(tvix/nar-bridge): mv nar-bridge{,-http}Florian Klink1-2/+2
Rename the nar-bridge CLI to nar-bridge-http, because it's the one spinning up an http server. Change-Id: I0fb75c50e4299272a128dd5ecaa4be8f06fa3dbe Reviewed-on: https://cl.tvl.fyi/c/depot/+/9538 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-09-28 r/6667 docs(tvix/boot): Update command to reflect new tvix/boot directoryConnor Brewster1-5/+5
This used to be in tvix/test but was moved to tvix/boot, but not all the command were updated so they no longer work as documented. This updates the commands to use the new tvix/boot directory. Change-Id: I8658d928a152140df182c159bf98ea9d9ecd1302 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9482 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-09-25 r/6652 chore(tvix/tests): rename to //tvix/bootFlorian Klink4-0/+420
This is mostly boot tooling, the integration test is just one instance making use of it. Expose initrd, kernel and runVM as a separate target to CI, and move the tests to a subdirectory. Change-Id: I1d22cd68bf5af095bc11dd9d7117b62956c7f7f2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9465 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI