about summary refs log tree commit diff
AgeCommit message (Collapse)AuthorFilesLines
2024-04-19 r/7976 chore(tvix/cli): remove unused dependencyFlorian Klink3-10/+0
Change-Id: I6a6cabde22d28bb9afc391b9bcd896a999090366 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11475 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-04-19 r/7975 chore(nix-compat): remove unused test-case and test-generatorFlorian Klink5-318/+133
This is especially nice, as it allows us getting rid of the other versions of proc-macro2 and quote. Change-Id: I9fdd012ee6c0ded3e18ec30504b83ac2032d1390 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11474 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-04-19 r/7974 chore(nix-compat/drv): migrate from test_{case,resources} to rstestFlorian Klink1-76/+88
Also, replace the useless read_file helper with std::fs::read. Change-Id: I704ab42d1ffb9089716533125ff390b7441a8591 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11473 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-04-19 r/7973 chore(nix-compat/derivation/parser): migrate from test_case to rstestFlorian Klink1-23/+37
Change-Id: I7c29e7d9e2b9cd66483ed44a9f2b3d291061f81d Reviewed-on: https://cl.tvl.fyi/c/depot/+/11472 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-04-19 r/7972 chore(nix-compat/nixhash): migrate from test_case to rstestFlorian Klink1-12/+14
Change-Id: I053f2c7fc9fdf59de5d8697eab32644b860e40be Reviewed-on: https://cl.tvl.fyi/c/depot/+/11471 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-04-19 r/7971 chore(nix-compat/aterm): migrate from test_case to rstestFlorian Klink2-22/+30
Change-Id: If05fa3d0f41d45e3574e0c5ddff406cb33dc52f4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11470 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de>
2024-04-19 r/7970 chore(tvix/nix-compat/store_path): migrate from test_case to rstestFlorian Klink1-16/+20
Change-Id: Ic466a27d61b95ca4d297abd6eb976c083e8b40af Reviewed-on: https://cl.tvl.fyi/c/depot/+/11469 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-04-19 r/7969 chore(nix-compat/nixbase32): migrate from test_case to rstestFlorian Klink1-14/+17
Change-Id: I9ed5e728ff0706a37d0a238210a8dd23eb306033 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11468 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-04-19 r/7968 chore(nix-compat/narinfo): migrate from test_case to rstestFlorian Klink2-33/+42
Change-Id: Ibd3cd1bc0d9deaf018a66748cea56af51451703c Reviewed-on: https://cl.tvl.fyi/c/depot/+/11467 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-04-19 r/7967 chore(tvix/build): migrate from test_case to rstestFlorian Klink5-39/+41
Change-Id: I75d8d61f836c76e8765e0e3b49022c056de84850 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11466 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-04-19 r/7966 chore(tvix/store): remove test-case dependencyFlorian Klink3-7/+2
Change-Id: I62c3b93af438fd6397ded85dd4fe5acc01e0093a Reviewed-on: https://cl.tvl.fyi/c/depot/+/11465 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-04-19 r/7965 chore(tvix/store): migrate import.rs and tests/pathinfo.rs to rstestFlorian Klink5-87/+72
Also, rename the DUMMY_NAME constant in the fixtures to DUMMY_PATH, which aligns more with the ToString representation and from_bytes conversions we have on StorePath[Ref]. Change-Id: I39763c9dfa84c5d86f2fd0171b3a4d36fd72f267 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11464 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-04-19 r/7964 chore(tvix/store): migrate from test_case to rstestFlorian Klink1-43/+37
Change-Id: Id673cd9c574b00a739de78aab2ee86f9d9b1ed40 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11463 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-04-19 r/7963 feat(web/panettone): Create users tableAspen Smith3-2/+41
Create a (currently unused) table to store information about users. I'll be manually migrating over all the information about users into this table, then will make a subsequent CL to make the rest of the tables foreign-key into this table Change-Id: I1b1c4b50c4a61326df3382809f701947a2caf536 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11411 Autosubmit: aspen <root@gws.fyi> Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2024-04-19 r/7962 feat(aspen/system): all desktops should have alacrittyAspen Smith1-0/+1
Change-Id: If93fda38c1f01f33a82db69aa42a5c601f6d633a Reviewed-on: https://cl.tvl.fyi/c/depot/+/11479 Tested-by: BuildkiteCI Reviewed-by: aspen <root@gws.fyi> Autosubmit: aspen <root@gws.fyi>
2024-04-19 r/7961 feat(aspen/system): use profiles in rebuild-homeAspen Smith1-5/+4
Change-Id: I07e6e16b89c864da85480d980fce98f6b10d2978 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11478 Tested-by: BuildkiteCI Reviewed-by: aspen <root@gws.fyi> Autosubmit: aspen <root@gws.fyi>
2024-04-19 r/7960 feat(aspen/lusca): tweak alacritty font size; install slackAspen Smith1-1/+3
Change-Id: I14dd872fb3b6f495e57f3e538486c5fad901ccc2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11477 Autosubmit: aspen <root@gws.fyi> Tested-by: BuildkiteCI Reviewed-by: aspen <root@gws.fyi>
2024-04-19 r/7959 chore(nix/buildGo): remove unused functionFlorian Klink1-2/+0
Spotted by deadnix. Change-Id: Ia5a9ec86fe7bbfe1b4fa90dc135c000b429df334 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11461 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su>
2024-04-19 r/7958 chore(nix/buildGo): cleanupsFlorian Klink1-4/+3
io/ioutil is deprecated, and the range expression can be simplified. Change-Id: I73b85991faafa333bddf90adbdefe2006f8d409c Reviewed-on: https://cl.tvl.fyi/c/depot/+/11460 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-04-18 r/7957 chore(users/tazjin/keys): update yubikey key on khamovnikVincent Ambo1-1/+1
Change-Id: I9b8ca783a994e862788fea7b180541f5febeb7b5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11459 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2024-04-18 r/7956 fix(views/kit): include dependency-analyzersterni2-1/+11
Since cl/11116, //nix/buildkite needs dependency-analyzer as an input. Change-Id: I2d924d6648b4ebf39388d8142a99496ef854d0c1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11451 Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com> Tested-by: BuildkiteCI
2024-04-18 r/7955 feat(tvix/nix-compat/nix_daemon/version): add ProtocolVersionFlorian Klink5-27/+160
This provides a nice wrapper struct to deal with versions. Change-Id: I6acc03bc9f8d84a0583196073b52776c45d3fe92 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11454 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
2024-04-18 r/7954 feat(tvix/nix-compat): enable more features by defaultFlorian Klink1-0/+3
There's no point in not enabling these, and rust-analyzer is quite useless too. nix-compat users outside of here can still explicitly disable default features if they don't want to pull in these features. Change-Id: I8f2f24e7734ad152d1733bc3421098fc91af290a Reviewed-on: https://cl.tvl.fyi/c/depot/+/11455 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
2024-04-18 r/7953 fix(ops/nixery-01): enable automatic GCVincent Ambo1-0/+9
This relates to nixery#167. Using our GC module is much more reliable than what we were doing previously. Change-Id: I1956457812a3a847a7c8a1f4e7e91e50fad08ac0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11453 Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de>
2024-04-18 r/7952 fix(ops): update DNS record for nixery-01Vincent Ambo2-2/+2
I accidentally deallocated the previous public IP, and had to make a new one :( Change-Id: Ie30305bdfdb8443e058270e5324baf555343441c Reviewed-on: https://cl.tvl.fyi/c/depot/+/11452 Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de>
2024-04-16 r/7951 feat(tvix/boot): boot nixos VMFlorian Klink1-0/+34
This makes use of the vmCmdline and assertVMOutput to boot a NixOS VM off of `tvix-store virtiofs` have it write a message on successful bootup, then have it shut down again. We check we see the message in the output. Seeding the store with the closure it's what's currently taking most of the time (as it's all linear still), so log a message before booting the VM. Change-Id: I6887d935f589eb0ad6507033c01337e1ec500a0b Reviewed-on: https://cl.tvl.fyi/c/depot/+/11191 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-04-16 r/7950 fix(tvix/castore/blobservice/grpc): don't use NaiveSeeker for nowFlorian Klink1-9/+17
Userland likes to seek backwards, and until we have store composition and can serve chunks from a local cache, we need to buffer the individual chunks in memory. Change-Id: I66978a0722d5f55ed4a9a49d116cecb64a01995d Reviewed-on: https://cl.tvl.fyi/c/depot/+/11448 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-04-16 r/7949 fix(tvix/castore/src): record rq.handle field in read()Florian Klink1-1/+1
This makes it easier to separate concurrent requests on the same inode. Change-Id: I7637c1d889336beeb0d186182ce22fbf60fd16c3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11447 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-04-16 r/7948 fix(tvix/castore/fs): use io::copy to fill kernel-provided bufferFlorian Klink1-3/+12
The docs state we must fill all of the buffer, except on EOF. Change-Id: Id977ba99c0b15132422474ebbf82bb92b79d55ba Reviewed-on: https://cl.tvl.fyi/c/depot/+/11446 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-04-16 r/7947 feat(tvix/castore/blob/naive_seeker): add some more tracingFlorian Klink1-10/+17
Change-Id: Iecf4a82a7d84008a8620825570b34e9094e6d590 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11445 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-04-16 r/7946 feat(tvix/castore/blob/chunked_reader): add some more tracesFlorian Klink1-2/+8
Change-Id: I2408707a7bc0e1c0cd8bd2933f8d68805b9e12c9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11444 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-04-16 r/7945 fix(tvix/castore/blobservice/chunk_rd): only skip *first* chunk bytesFlorian Klink1-3/+4
When (re)initializing a chunked reader, we were erroneously skipping the first n bytes from all chunks, not just the first one. Fix this, by passing in an enumerated list of chunks, and only calling SeekFrom::Start() on the first chunk in the stream. With this, I'm able to invoke b3sum on bin/bash successfully. Change-Id: I52ea480569267e093b0ac9d6bcd5c2d1b4db25f7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11443 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-04-16 r/7944 refactor(tvix/castore/blob/naive_seeker): rework skipping for clarityFlorian Klink1-42/+35
Increase the discard_buf to 4096 (as I've seen this size). Use the ready! macro to propagate pendings. Make it more clear what exactly should be skipped in total, and what during the current iteration. Also write down that poll_read call already takes care of updating self.pos, as I ran into that trap earlier (and added it here). Change-Id: I2d22e1c8a835c0f3dd0c648917009b2bad4fd57c Reviewed-on: https://cl.tvl.fyi/c/depot/+/11442 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-04-16 r/7943 feat(tvix/castore/blob/chunked_reader): only reassemble on real seekFlorian Klink1-16/+16
If the resulting offset equals to our current position, there's no need to recreate a reader. Change-Id: I855f0c79c514c16ca48a78e12978af2835fbbd6a Reviewed-on: https://cl.tvl.fyi/c/depot/+/11441 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-04-16 r/7942 docs(nix-1p): add (temporary) "ad" for Volga SprintVincent Ambo1-0/+5
Want to get a few more signups than we allocated, in case somebody drops out. Change-Id: I832c4fdfa900f117092f5f7e8a5d8b7d256a94e5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11450 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2024-04-16 r/7941 feat(tazjin/khamovnik): add VAAPI intel driverVincent Ambo1-0/+1
Change-Id: Ib67853015318e09de5ba43aaa0cbf1bd7bde421d Reviewed-on: https://cl.tvl.fyi/c/depot/+/11449 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2024-04-16 r/7940 feat(nix/buildkite): reflect deps between derivations in pipelinessterni1-3/+23
Most of the steps in our buildkite pipeline build derivations without doing anything else. A lot of those derivations depend on each other. Consequently, buildkite will schedule builds of derivations whose dependencies are still in the process of being built. The result is many buildkite agents doing nothing but blocking on other derivations being built. We can easily prevent this by using the dependency information we can get from the derivation (files) of the targets we want to build and translating them into buildkite step dependencies. The hard part of this has already been done for a while: //nix/dependency-analyzer finds the dependencies between a list of “known” derivations (even if they only depend on each other through intermediate derivations) without depending on a specific derivation builder convention, but rather relying on `.drv` files. It still has a few rough edges, but has been working reliably for our purposes. Since our steps are identified by derivation hashes, we can just directly use the available dependency data. Luckily, buildkite seems to just takes a step as if it was completed if it is skipped, so we don't even have to check whether dependencies have been skipped or not. On whitby it seems that the dependency analysis costs about a minute additionally (which is how long it takes to run //nix/dependency-analyzer in isolation just about). Supersedes cl/5063, cl/5060, cl/5064 and cl/5065. Change-Id: I91d2eb2b43d60811cac0d26fa94467298f622970 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11116 Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
2024-04-16 r/7939 feat(nix/buildkite): use keys based on drvPaths where possiblesterni2-3/+27
This will make it easier to emit intra pipeline dependencies based on the dependencies between derivations contained therein later. A consequence of this change is that it is no longer possible to have the same derivation be exposed as multiple steps in the pipeline. I doubt that having this is very useful, though. Keys for extraSteps are not changed significantly, but are distinguished from derivation based steps (with prefix `drv-*`) by their prefix `extra-step-*`. Change-Id: I4165900e512b3967fa3ca4cd5bffd44bc15915fc Reviewed-on: https://cl.tvl.fyi/c/depot/+/11115 Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com> Tested-by: BuildkiteCI
2024-04-15 r/7938 docs(tvix/castore/blobservice): explain open_read for small blobs moreFlorian Klink1-0/+1
State that this case applies if the blob is small enough to fit inside a single chunk. Change-Id: I0383514729e686799599b629cf1303b284147bb4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11440 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-04-15 r/7937 feat(tvix/castore/blobs/object_store): chunks() method for small blobsFlorian Klink1-4/+37
We previously returned Ok(None) when being asked for more granular chunking info, signalling the blob does not exist at all. This is however incorrect, we should return an empty Vec instead, as documented in the trait. Change-Id: I83ecc2027e0767134c7598792c2ee6d964853c66 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11439 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de>
2024-04-15 r/7936 fix(tvix/castore/blob/object_store): tweak log levelsFlorian Klink1-3/+3
Don't log with info! here, bug debug!. Change-Id: I57bd5f2a45276090b893a4051fd175e3948ddfa4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11438 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-04-15 r/7935 fix(tix/castore/blobservice): don't warn if chunk list is emptyFlorian Klink1-4/+1
It's perfectly normal if we ask for more granular chunking info and the backend responds it does not have it. Change-Id: I593ab3e53b4f4e70c99f39b266546d2ac8eb10c1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11437 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-04-15 r/7934 feat(tvix/castore/blob/grpc_wrapper): add blob.digest fieldFlorian Klink1-2/+3
We're receiving bytes over the wire, and encode them the same way B3Digest does internally, but don't use it for formatting, as we're discarding that string. In case the sent bytes don't have the right length, the string will be short, but it's better to still have it as a field, even if it's not a valid b3 digest. Change-Id: I6ef08275d51c8a0d98f5e46844b15dfd05d17cd8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11436 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-04-15 r/7933 fix(nix/dependency-analyzer): ignore non-drv paths for Nix < 2.6sterni1-24/+29
Looking for .drv file names in non .drv files doesn't make sense, as it less reliably a reference in those cases. Matches behavior of the function for Nix >= 2.6. Change-Id: I79fc1da3e55df869f03702fa5137d030790bc4eb Reviewed-on: https://cl.tvl.fyi/c/depot/+/11114 Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: aspen <root@gws.fyi>
2024-04-15 r/7932 refactor(tvix/castore/fs): add parenthesis for readabilityFlorian Klink1-4/+4
As suggested in cl/11426. Change-Id: Ic2bb8cf2838bf0be09fb8bc62b8e598a3d153699 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11434 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-04-15 r/7931 refactor(tvix/castore/fs): remove From<Node> for InodeDataFlorian Klink2-107/+76
These were copying unnecessarily. Instead, have a InodeData::from_node(), which *consumes* the Node entirely, returns `InodeData` and the split-off name (which is not part of InodeData). Callers can then use the result in various helper functions, like: - InodeData::as_fuse_type - InodeData::as_fuse_file_attr - InodeData::as_fuse_entry … to prepare their replies to the kernel. This removes not only a bunch of clones, but also a lot of copy-pasted code. Change-Id: Idbca5f25cc29e96c1f4c614b33dff2becb0a8738 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11435 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-04-15 r/7930 fix(tvix/castore/blobs): reply to has() for chunksFlorian Klink1-1/+8
We allow reading individual chunks via open_read(), it's inconsistent if a has() would return Ok(false). Change-Id: Ie713d968172ccd2687d2e6e0dfef89ee152ef511 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11420 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-04-15 r/7929 refactor(tvix/castore): relax trait bounds on BlobServiceFlorian Klink1-2/+2
We don't need to clone BlobService anymore. Change-Id: I2f3b9a595f604ec0f1e081f6e90cd8b67cbb8961 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11419 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-04-15 r/7928 fix(tvix/castore/directory): fix graph traversalFlorian Klink4-90/+109
Use a proper graph library to ensure all nodes are reachable from the root. We had a bit of that handrolled during add(), as well as later, which had an annoying bug: Redundant nodes were omitted during insert, but when returning the list during finalize, we did not properly account they need to be introduced before their parents are sent. We now simply populate a petgraph DiGraph during insert (skipping inserting nodes we already saw), and use petgraph's DfsPostOrder to traverse the graph during finalize. If the number of returned indices equals the total number of nodes in the graph, all nodes are reachable from the root, we can consume the graph and return the nodes as a vec, in the same order as the traversal (and insertion). Providing a regression test for the initial bug is challenging, as the current code uses a bunch of HashSets. I manually tested ingesting a full NixOS closure using this mechanism (via gRPC, which exposes this problem, as it validates twice), and it now works. Change-Id: Ic1d5e3e981f2993cc08c5c6b60ad895e578326dc Reviewed-on: https://cl.tvl.fyi/c/depot/+/11418 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-04-15 r/7927 refactor(tvix/castore/import): restructure directory uploader a bitFlorian Klink1-17/+51
Have a Option<Box<dyn DirectoryPutter>>, which is lazily initialized whenever we first want to upload a directory. Have the loop explicitly break when it encounters the root_node, and deal with the flushing after the loop. Deal with the FUTUREWORK (assertion for root directory digest matching what the DirectoryPutter returns). Change-Id: Iefc4904d8b8387e868fb752d40e3e4e4218c7407 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11417 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-11-16T04·51+0000