about summary refs log tree commit diff
path: root/tvix
AgeCommit message (Collapse)AuthorFilesLines
2024-04-19 r/7964 chore(tvix/store): migrate from test_case to rstestFlorian Klink1-43/+37
Change-Id: Id673cd9c574b00a739de78aab2ee86f9d9b1ed40 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11463 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-04-18 r/7955 feat(tvix/nix-compat/nix_daemon/version): add ProtocolVersionFlorian Klink4-22/+156
This provides a nice wrapper struct to deal with versions. Change-Id: I6acc03bc9f8d84a0583196073b52776c45d3fe92 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11454 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
2024-04-18 r/7954 feat(tvix/nix-compat): enable more features by defaultFlorian Klink1-0/+3
There's no point in not enabling these, and rust-analyzer is quite useless too. nix-compat users outside of here can still explicitly disable default features if they don't want to pull in these features. Change-Id: I8f2f24e7734ad152d1733bc3421098fc91af290a Reviewed-on: https://cl.tvl.fyi/c/depot/+/11455 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
2024-04-16 r/7951 feat(tvix/boot): boot nixos VMFlorian Klink1-0/+34
This makes use of the vmCmdline and assertVMOutput to boot a NixOS VM off of `tvix-store virtiofs` have it write a message on successful bootup, then have it shut down again. We check we see the message in the output. Seeding the store with the closure it's what's currently taking most of the time (as it's all linear still), so log a message before booting the VM. Change-Id: I6887d935f589eb0ad6507033c01337e1ec500a0b Reviewed-on: https://cl.tvl.fyi/c/depot/+/11191 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-04-16 r/7950 fix(tvix/castore/blobservice/grpc): don't use NaiveSeeker for nowFlorian Klink1-9/+17
Userland likes to seek backwards, and until we have store composition and can serve chunks from a local cache, we need to buffer the individual chunks in memory. Change-Id: I66978a0722d5f55ed4a9a49d116cecb64a01995d Reviewed-on: https://cl.tvl.fyi/c/depot/+/11448 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-04-16 r/7949 fix(tvix/castore/src): record rq.handle field in read()Florian Klink1-1/+1
This makes it easier to separate concurrent requests on the same inode. Change-Id: I7637c1d889336beeb0d186182ce22fbf60fd16c3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11447 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-04-16 r/7948 fix(tvix/castore/fs): use io::copy to fill kernel-provided bufferFlorian Klink1-3/+12
The docs state we must fill all of the buffer, except on EOF. Change-Id: Id977ba99c0b15132422474ebbf82bb92b79d55ba Reviewed-on: https://cl.tvl.fyi/c/depot/+/11446 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-04-16 r/7947 feat(tvix/castore/blob/naive_seeker): add some more tracingFlorian Klink1-10/+17
Change-Id: Iecf4a82a7d84008a8620825570b34e9094e6d590 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11445 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-04-16 r/7946 feat(tvix/castore/blob/chunked_reader): add some more tracesFlorian Klink1-2/+8
Change-Id: I2408707a7bc0e1c0cd8bd2933f8d68805b9e12c9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11444 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-04-16 r/7945 fix(tvix/castore/blobservice/chunk_rd): only skip *first* chunk bytesFlorian Klink1-3/+4
When (re)initializing a chunked reader, we were erroneously skipping the first n bytes from all chunks, not just the first one. Fix this, by passing in an enumerated list of chunks, and only calling SeekFrom::Start() on the first chunk in the stream. With this, I'm able to invoke b3sum on bin/bash successfully. Change-Id: I52ea480569267e093b0ac9d6bcd5c2d1b4db25f7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11443 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-04-16 r/7944 refactor(tvix/castore/blob/naive_seeker): rework skipping for clarityFlorian Klink1-42/+35
Increase the discard_buf to 4096 (as I've seen this size). Use the ready! macro to propagate pendings. Make it more clear what exactly should be skipped in total, and what during the current iteration. Also write down that poll_read call already takes care of updating self.pos, as I ran into that trap earlier (and added it here). Change-Id: I2d22e1c8a835c0f3dd0c648917009b2bad4fd57c Reviewed-on: https://cl.tvl.fyi/c/depot/+/11442 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-04-16 r/7943 feat(tvix/castore/blob/chunked_reader): only reassemble on real seekFlorian Klink1-16/+16
If the resulting offset equals to our current position, there's no need to recreate a reader. Change-Id: I855f0c79c514c16ca48a78e12978af2835fbbd6a Reviewed-on: https://cl.tvl.fyi/c/depot/+/11441 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-04-15 r/7938 docs(tvix/castore/blobservice): explain open_read for small blobs moreFlorian Klink1-0/+1
State that this case applies if the blob is small enough to fit inside a single chunk. Change-Id: I0383514729e686799599b629cf1303b284147bb4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11440 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-04-15 r/7937 feat(tvix/castore/blobs/object_store): chunks() method for small blobsFlorian Klink1-4/+37
We previously returned Ok(None) when being asked for more granular chunking info, signalling the blob does not exist at all. This is however incorrect, we should return an empty Vec instead, as documented in the trait. Change-Id: I83ecc2027e0767134c7598792c2ee6d964853c66 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11439 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de>
2024-04-15 r/7936 fix(tvix/castore/blob/object_store): tweak log levelsFlorian Klink1-3/+3
Don't log with info! here, bug debug!. Change-Id: I57bd5f2a45276090b893a4051fd175e3948ddfa4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11438 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-04-15 r/7935 fix(tix/castore/blobservice): don't warn if chunk list is emptyFlorian Klink1-4/+1
It's perfectly normal if we ask for more granular chunking info and the backend responds it does not have it. Change-Id: I593ab3e53b4f4e70c99f39b266546d2ac8eb10c1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11437 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-04-15 r/7934 feat(tvix/castore/blob/grpc_wrapper): add blob.digest fieldFlorian Klink1-2/+3
We're receiving bytes over the wire, and encode them the same way B3Digest does internally, but don't use it for formatting, as we're discarding that string. In case the sent bytes don't have the right length, the string will be short, but it's better to still have it as a field, even if it's not a valid b3 digest. Change-Id: I6ef08275d51c8a0d98f5e46844b15dfd05d17cd8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11436 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-04-15 r/7932 refactor(tvix/castore/fs): add parenthesis for readabilityFlorian Klink1-4/+4
As suggested in cl/11426. Change-Id: Ic2bb8cf2838bf0be09fb8bc62b8e598a3d153699 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11434 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-04-15 r/7931 refactor(tvix/castore/fs): remove From<Node> for InodeDataFlorian Klink2-107/+76
These were copying unnecessarily. Instead, have a InodeData::from_node(), which *consumes* the Node entirely, returns `InodeData` and the split-off name (which is not part of InodeData). Callers can then use the result in various helper functions, like: - InodeData::as_fuse_type - InodeData::as_fuse_file_attr - InodeData::as_fuse_entry … to prepare their replies to the kernel. This removes not only a bunch of clones, but also a lot of copy-pasted code. Change-Id: Idbca5f25cc29e96c1f4c614b33dff2becb0a8738 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11435 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-04-15 r/7930 fix(tvix/castore/blobs): reply to has() for chunksFlorian Klink1-1/+8
We allow reading individual chunks via open_read(), it's inconsistent if a has() would return Ok(false). Change-Id: Ie713d968172ccd2687d2e6e0dfef89ee152ef511 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11420 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-04-15 r/7929 refactor(tvix/castore): relax trait bounds on BlobServiceFlorian Klink1-2/+2
We don't need to clone BlobService anymore. Change-Id: I2f3b9a595f604ec0f1e081f6e90cd8b67cbb8961 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11419 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-04-15 r/7928 fix(tvix/castore/directory): fix graph traversalFlorian Klink4-90/+109
Use a proper graph library to ensure all nodes are reachable from the root. We had a bit of that handrolled during add(), as well as later, which had an annoying bug: Redundant nodes were omitted during insert, but when returning the list during finalize, we did not properly account they need to be introduced before their parents are sent. We now simply populate a petgraph DiGraph during insert (skipping inserting nodes we already saw), and use petgraph's DfsPostOrder to traverse the graph during finalize. If the number of returned indices equals the total number of nodes in the graph, all nodes are reachable from the root, we can consume the graph and return the nodes as a vec, in the same order as the traversal (and insertion). Providing a regression test for the initial bug is challenging, as the current code uses a bunch of HashSets. I manually tested ingesting a full NixOS closure using this mechanism (via gRPC, which exposes this problem, as it validates twice), and it now works. Change-Id: Ic1d5e3e981f2993cc08c5c6b60ad895e578326dc Reviewed-on: https://cl.tvl.fyi/c/depot/+/11418 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-04-15 r/7927 refactor(tvix/castore/import): restructure directory uploader a bitFlorian Klink1-17/+51
Have a Option<Box<dyn DirectoryPutter>>, which is lazily initialized whenever we first want to upload a directory. Have the loop explicitly break when it encounters the root_node, and deal with the flushing after the loop. Deal with the FUTUREWORK (assertion for root directory digest matching what the DirectoryPutter returns). Change-Id: Iefc4904d8b8387e868fb752d40e3e4e4218c7407 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11417 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-04-15 r/7926 refactor(tvix/castore/import): put invariant checker into a .inspect()Florian Klink1-18/+18
Separate this a bit stronger from the main application flow. Change-Id: I2e9bd3ec47cc6e37256ba6afc6e0586ddc9a051f Reviewed-on: https://cl.tvl.fyi/c/depot/+/11416 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Brian Olsen <me@griff.name> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-04-15 r/7925 refactor(tvix/*/import): rename direntry_stream, entries_per_depthsFlorian Klink2-8/+10
Align these names and comments with the two users, to make it more obvious we're doing the same thing here, just use a different method to come up with entries_per_depths. Change-Id: I42058e397588b6b57a6299e87183bef27588b228 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11415 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-04-15 r/7924 refactor(tvix/castore/import): inline process_entryFlorian Klink1-118/+65
This did very little, and especially the part of relying on the outside caller to pass in a Directory if the type is a directory required having per-entry-type specific logic anyways. It's cleaner to just inline it. Change-Id: I997a8513ee91c67b0a2443cb5cd9e8700f69211e Reviewed-on: https://cl.tvl.fyi/c/depot/+/11414 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-04-15 r/7923 refactor(tvix/castore/import): move process_entry to the end of the fileFlorian Klink1-95/+95
This makes it easier to understand the code. Change-Id: I0a9047433000551a6ba1f50a8c5c93527bc86216 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11413 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-04-15 r/7922 fix(tvix/castore/blobservice): update bytes_read only on successful readFlorian Klink2-22/+16
We previously updated this.pos also in case the underlying read returned an error. Also, use the ready! macro to remove the match block, and instrument errors returned during start_seek. Change-Id: Ic32e26579d964a76b45687134acc48d72d67c36f Reviewed-on: https://cl.tvl.fyi/c/depot/+/11421 Reviewed-by: Brian Olsen <me@griff.name> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-04-15 r/7921 refactor(tvix/castore/fs): add InodeData::as_fuse_{entry,file_attr}Florian Klink3-57/+46
Remove the now unused gen_file_attr (which had a wrong docstring). Change-Id: Ie86b14d1ad798e6233bc44c43ace3f8b95c67ea9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11430 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-04-15 r/7920 refactor(tvix/castore/fs): remove "add … handle" debug messagesFlorian Klink1-2/+0
Change-Id: Iac22bbef96a2afa0416f011d073934b52b19975d Reviewed-on: https://cl.tvl.fyi/c/depot/+/11433 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-04-15 r/7919 feat(tvix/castore/fs): assign read[dir*]/release[dir] ops to parent spanFlorian Klink1-13/+20
When a directory or file is open()'ed, we already put some data into a lookup table, and subsequent operations then use the returned handle id. By also adding the span that's been created during these calls into the lookup table, we can properly set the span parent for these requests, nicely connecting the individual operations to the bigger picture. Change-Id: Ia354842fccdbc7f45c2d3efda3acf058b2dbc48e Reviewed-on: https://cl.tvl.fyi/c/depot/+/11429 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Brian Olsen <me@griff.name>
2024-04-15 r/7918 fix(tvix/castore/fs): use record to add fields to current spanFlorian Klink1-19/+11
Instead of creating another child span, we can use `tracing::Span::current().record(k,v)` to add an additional field to the current span. Change-Id: I337faac0e73a0da6eb0a52cb75c2e8c026eff774 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11428 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-04-15 r/7917 feat(tvix/castore/fs): implement readdirplusFlorian Klink1-1/+126
This currently shares some code with readdir, except it's also providing a second `fuse_backend_rs::api::filesystem::Entry` argument to the `add_entry` function call. Refactoring this to reduce some duplication is left for a future CL. Change-Id: I282c8dfc6a711d00a4482c87cbb84d4950c0aee9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11426 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-04-15 r/7916 refactor(tvix/castore/fs): use consistent span field name for handleFlorian Klink1-5/+4
Use rq.handle in `release` too, and remove interpolating it into the log message itself. Also update the comment, we don't get ownership, just simply drop, and change the level to warn!, as suggested in cl/11425. Change-Id: If4e6cff6d8b580671b1548ae3862851db4af6694 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11427 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-04-15 r/7915 feat(tvix/castore/fs): implement opendir/releasedirFlorian Klink2-68/+144
Similar to how we already handle opening files, implement opendir/ releasedir, and keep a map of dir_handles. They point to the rx side of a channel. This greatly improves performance listing the root of the filesystem when used inside tvix-store, as we don't need to re-request the listing (and skip to the desired position) all the time. Change-Id: I0d3ec4cb70a8792c5a1343439cf47d78d9cbb1d6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11425 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-04-15 r/7914 refactor(tvix/castore/fs): use std::sync::MutexFlorian Klink1-21/+24
This allows us acquiring the lock in sync code still. Also, simplify some of the error handling a bit. Change-Id: I29e83b715f92808e95ecb0ae9de787339d1a371d Reviewed-on: https://cl.tvl.fyi/c/depot/+/11424 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-04-15 r/7913 refactor(tvix/castore/fs): simplify some separate spawn and blocksFlorian Klink1-13/+6
We can just pass an async move closure to `self.tokio_handle.block_on` and make this a bit shorter. Change-Id: Iba674f34f22ba7a7de7c5bae59d64584884cb17c Reviewed-on: https://cl.tvl.fyi/c/depot/+/11423 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-04-15 r/7912 feat(tvix/castore/fs): support extended attributesFlorian Klink7-5/+227
This exposes `user.tvix.castore.{blob,directory}.digest` xattr keys for files and directories: ``` ❯ getfattr -d /tmp/tvix/06jrrv6wwp0nc1m7fr5bgdw012rfzfx2-nano-7.2-info getfattr: Removing leading '/' from absolute path names user.tvix.castore.directory.digest="b3:SuYDcUM9RpWcnA40tYB1BtYpR0xw72v3ymhKDQbBfe4=" ❯ getfattr -d /tmp/tvix/156a89x10c3kaby9rgf3fi4k0p6r9wl1-etc-shells getfattr: Removing leading '/' from absolute path names user.tvix.castore.blob.digest="b3:pZkwZoHN+/VQ8wkaX0wYVXZ0tV/HhtKlSqiaWDK7uRs=" ``` It's currently mostly used for debugging, though it might be useful for tvix-castore-aware syncing programs using the filesystem too. Change-Id: I26ac3cb9fe51ffbf7f880519f26741549cb5ab6a Reviewed-on: https://cl.tvl.fyi/c/depot/+/11422 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Reviewed-by: Brian Olsen <me@griff.name>
2024-04-13 r/7911 feat(tvix/nix-compat/path_info): derive Hash for ExportedPathInfoIlan Joselevich1-1/+1
Change-Id: Icd2431f27043f120eee1d156cfcdc9bc85519ead Reviewed-on: https://cl.tvl.fyi/c/depot/+/11412 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-04-13 r/7910 feat(tvix/castore/import): remove copying in find_ancestorFlorian Klink1-8/+2
We don't need to copy if we explicitly say that the returned Option<Path> may hold onto bytes from the passed in &DirEntry. Change-Id: Ib46b6fd2f8f19a45f8bef79c4c1d2fa6b490cad7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11410 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-04-13 r/7909 fix(tvix/eval): fix the `unsafeGetAttrPos` stub return valueRyan Lahfa1-1/+1
It was returning `{ line, col, file }` instead of `{ line, column, file }`. Note this is not a regression in Nix, Nix always had `column` since 2013. Change-Id: I4871a201b32324dff0432526d0449806a535828f Signed-off-by: Ryan Lahfa <tvl@lahfa.xyz> Reviewed-on: https://cl.tvl.fyi/c/depot/+/11409 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-04-13 r/7908 fix(tvix): apply cbtemulator patch in tvix viewFlorian Klink1-1/+10
This tells josh to include the patch from third_party/overlays/patches/cbtemulator-uds.patch at nixpkgs/cbtemulator-uds.patch. We then use the `additionalOverlays` feature introduced earlier, and pass it an overlay adding the patch to `cbtemulator`. This only gets applied if tvix/shell.nix is not called with a pkgs argument, which is the case in the tvix view. Change-Id: I7bc5543460ddd30325ab0998e891c15fcaf8bcfe Reviewed-on: https://cl.tvl.fyi/c/depot/+/11408 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de>
2024-04-13 r/7907 feat(3p/nixpkgs): add additionalOverlays optional argFlorian Klink1-0/+1
This allows applying additional overlays to nixpkgs before instantiating it. Change-Id: Iee486086e13bb73e6bd20a817b1106c3cd99c935 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11407 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-04-13 r/7906 feat(tvix/*store/bigtable): limit retries connecting to cbtemulatorFlorian Klink2-2/+6
This kept retrying indefinitely if the socket didn't appear. Change-Id: I4d4ef61df73cef6abda698501432f370abc8a82c Reviewed-on: https://cl.tvl.fyi/c/depot/+/11406 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-04-13 r/7905 feat(tvix/nix-compat/wire/bytes): add #[inline] for ensure_* funcsFlorian Klink2-0/+2
While the compiler might realize it's a good idea to inline these, especially considering this is in the same crate [1], it doesn't hurt to add that annotation too. Suggested in cl/11385. [1]: https://nnethercote.github.io/perf-book/inlining.html Change-Id: If679fce0e1aab9ec681ba60f689d386d6fd92d61 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11403 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de>
2024-04-13 r/7903 feat(tvix/buildkite): apply a 10 minutes timeoutFlorian Klink1-0/+1
We should cap the time of a `cargo build && cargo test` to a reasonable amount of time. Change-Id: If618198459ac267f9f6d7eb60466534a95a95c2c Reviewed-on: https://cl.tvl.fyi/c/depot/+/11404 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-04-13 r/7902 refactor(tvix/castore/import): rename ingest_entries function argFlorian Klink1-2/+2
This is a stream of DirEntry, so let's call it direntry_stream. Change-Id: I5b3cb4efba899d746393f75f6ece7eaa79424717 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11401 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-04-13 r/7901 refactor(tvix/boot): use tvix-store copyFlorian Klink1-33/+6
Instead of using an additional Nix build producing a directory structure containing xz-compressed NAR files and NARInfos, and uploading these via `curl` to NAR-bridge, which then decomposes them and uploads them to tvix-store, have tvix-store interpret the exported reference graph directly, and deal with the uploading on its own. Change-Id: I44ee0b132944c53d11abb688d2d017f0cc9c3d97 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11398 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-04-13 r/7900 feat(tvix/store/bin): add "copy" commandFlorian Klink4-0/+99
This allows copying individual store paths from the local /nix/store into tvix-store. As we don't support getting this information from Nix yet, we currently expect metadata to be provided externally: Nix' `exportReferencesGraph` feature contains pretty much all data we need for this. Expect a list of this information at a well-known key (`closure`, similar to nixpkgs' `pkgs/build-support/binary-cache/ default.nix`). We currently simply upload all store paths sequentially, without any parallelism or awareness in how the reference graph looks like. As long as the connected stores don't enforce this, this is fine, at least for now. Change-Id: Ib83c998465adddfdb110db994843c44e26b3d3d8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11397 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-04-13 r/7899 feat(tvix/eval): contextful JSON operationsRyan Lahfa11-33/+102
`toJSON` transform a Nix structure into a JSON string. For each context in that Nix structure, the JSON string must possess it. Thus, it is necessary to take the union of all contexts and attach it to the final structure. Unfortunately, the return type of `into_json` is a serde's JSON object, not a string. Therefore, it is not possible to reuse `NixString` machinery. Context tests are reinforced as Nix does not test those behaviors. Fixes b/393. Change-Id: I5afdbc4e18dd70469192c1aa657d1049ba330149 Signed-off-by: Ryan Lahfa <tvl@lahfa.xyz> Reviewed-on: https://cl.tvl.fyi/c/depot/+/11266 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-11-25T00·28+0000