about summary refs log tree commit diff
path: root/tvix
AgeCommit message (Collapse)AuthorFilesLines
2024-05-20 r/8158 refactor(tvix/castore): extract concurrent blob uploaderConnor Brewster3-97/+190
The archive ingester has a mechanism for concurrently uploading small blobs to the blob service in order to hide round trip latency with the blob service when ingesting many small blobs. Other ingestion sources like NARs also need a similar mechanism, this extracts the concurrent blob uploading mechanism into its own struct to make it more reusable. Change-Id: I05020419ff4b9ad5829fbfb5cd08d36db983b8c0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11693 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-05-20 r/8157 refactor(tvix/store/pathinfo): test with PathInfoService directlyFlorian Klink4-71/+58
Since cl/…, a PathInfoService doesn't need to implement `calculate_nar` anymore, so most of them don't actually have a handle to a {Blob,Directory}Service anymore. This means, we can simplify the construction of them for test cases a lot. Change-Id: I100e9e1c9b00a049b4d6136c57aad4cdb04461c6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11691 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-05-20 r/8156 fix(tvix/eval): use fake values for __curPos, rather than an errorFlorian Klink1-2/+9
Have this return the same values as builtins.unsafeGetAttrsPos, rather than returning a CatchableErrorKind, which crashes the CLI if it bubbles up. The environment we're in doesn't allow emitting a warning, as we don't have `co` in scope, but that's probably OK as a stopgap solution. Alternative to cl/11665. Change-Id: I5b2c2530842547c93b6533ed9601ee9b2923b1bf Reviewed-on: https://cl.tvl.fyi/c/depot/+/11685 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2024-05-16 r/8154 docs(tvix): Begin documenting the Nix daemon protocolBrian Olsen5-0/+1527
This adds rough notes documenting the history of the Nix daemon protocol, how logging works as well as begins documenting inputs and outputs for all operations. Change-Id: Id24a9a658c3e4e7c350ca1e4622f63ed96ccef5a Reviewed-on: https://cl.tvl.fyi/c/depot/+/11594 Reviewed-by: Adam Joseph <adam@westernsemico.com> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-05-16 r/8153 fix(castore/directory/objectstore): fix responses for deduplicated dirsYureka1-7/+13
Using remove_node messed up the extraction of nodes from the graph. Use into_nodes_edges() instead, to remove the nodes without cloning. Change-Id: Id76c7935d082d6f26192cc3cd490483594f1d1e2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11684 Tested-by: BuildkiteCI Autosubmit: yuka <yuka@yuka.dev> Reviewed-by: flokli <flokli@flokli.de>
2024-05-16 r/8152 chore(tvix): drop json feature for tracing-subscriberFlorian Klink5-58/+5
Drop the (unused) feature flags here, and get rid of some crate dependencies. Change-Id: Id64852b498725467d56abb12b548301cfba6a760 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11678 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-05-16 r/8151 feat(tvix/build): drop json loggingFlorian Klink3-21/+6
This has already been dropped for tvix-store, drop it here as well. Change-Id: Ib3aa37bbe9fd4c498b5ba1213f7d922d0c64ffc7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11677 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-05-16 r/8150 feat(tvix/store): add ObjectStoreDirectoryServiceYureka8-11/+332
Change-Id: I1636012be2e8ee3ae64f7bc62fd28bfe0cb2bca5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11668 Autosubmit: yuka <yuka@yuka.dev> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-05-15 r/8149 chore(tvix): bump to tokio-listener 0.4.1Florian Klink4-840/+74
This gives multi-listener support, as well as listening on named socket FDs. Change-Id: I92b441f854e2faf80074463d0ca6bdc23cbd890a Reviewed-on: https://cl.tvl.fyi/c/depot/+/11663 Tested-by: BuildkiteCI Reviewed-by: yuka <yuka@yuka.dev> Autosubmit: flokli <flokli@flokli.de>
2024-05-15 r/8148 chore(tvix/store): bump otlp stackFlorian Klink3-361/+101
This aligns the tonic version we use directly in Tvix to the one pulled in by the OTLP stack. Change-Id: I658528c8dabb7cd6948f1207ddcdef1984e82037 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11666 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: yuka <yuka@yuka.dev> Tested-by: BuildkiteCI
2024-05-15 r/8147 chore(tvix/nar-bridge): move to nar-bridge-goFlorian Klink32-14/+13
Make some space for the rust implementation. Change-Id: I924dc1657be10abe5a11951c3b9de50bae06db19 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11662 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: yuka <yuka@yuka.dev>
2024-05-13 r/8144 feat(tvix/store/pathinfo): add a Cache combinatorFlorian Klink2-0/+113
This allows querying two PathInfoService implementations sequentially, and inserts into the "near" one if it's not there yet. There is no negative cache, and put / listing is not implemented (for now). Change-Id: I24c3d0e0c3c2f0524a6cc7b2f3cbc33eb20cf92b Reviewed-on: https://cl.tvl.fyi/c/depot/+/11636 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-05-13 r/8143 fix(tvix/castore/grpc): don't use explicit channelFlorian Klink1-47/+36
We can just use the `BoxStream` directly, or a `once` with the single `Directory`. In the recursive case, we also did not properly close the channel after the first error. Change-Id: Ifad56d307fc7861107b6d3cffd28d35631d526e6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11635 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de>
2024-05-13 r/8142 feat(tvix/castore/directory/get_recursive): add 'static to BoxStreamFlorian Klink5-6/+6
Necessary to directly use this in the GRPC DirectoryService wrapper directly. Change-Id: Ic6a0038a40dc30071d145af5035345fcd93288ae Reviewed-on: https://cl.tvl.fyi/c/depot/+/11634 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-05-13 r/8141 feat(tvix/castore/directory/traverse_directory): simplifyFlorian Klink1-48/+43
Use try_stream! rather than stream!, and a bit more map_err and ok_err to make things a bit more concise. Once we have proper error types here, and impl Froms, a lot of the error mapping would disappear entirely. Change-Id: I5240a6b0ff7818b94c151322774242b2c142e33b Reviewed-on: https://cl.tvl.fyi/c/depot/+/11633 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-05-13 r/8140 feat(tvix/castore/directory/traverse): simplify codeFlorian Klink1-53/+42
Replace the loop manually driving the iterator with a for … in, and some of the match with ok_or_else. Change-Id: I6d7b3ef1bf1c7aa128bd6adef09390b54f79479e Reviewed-on: https://cl.tvl.fyi/c/depot/+/11632 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-05-13 r/8139 chore(tvix/clippy): configure clippy allow block_in_conditionsFlorian Klink5-3/+15
… in Cargo.toml. This gets an imperative `cargo clippy` run to pick up that config, so `-A clippy::blocks_in_conditions` doesn't need to be explicitly specified anymore. Change-Id: I32b6cc50c77c22cba0d816d0db508c2f94b2c383 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11659 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI
2024-05-12 r/8117 feat(tvix/store/pathinfo/*): add more instrumentationFlorian Klink6-11/+20
Add instrumentation to the get() and put() implementations of all PathInfoService. Use the nixbase32 representation of the digest, not the base64 one. Change-Id: Iea79bbd363bf20f23985e877c6fc1793bbee6a7e Reviewed-on: https://cl.tvl.fyi/c/depot/+/11630 Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-05-11 r/8115 refactor(tvix/castore/error): drop From<PoisonError<T>> for ErrorFlorian Klink1-7/+0
We don't produce these erorrs anymore, no need to provide a conversion to it. Change-Id: I37933e436ad15c5d90b3ac270c4ef5742980513d Reviewed-on: https://cl.tvl.fyi/c/depot/+/11614 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-05-11 r/8114 refactor(tvix/castore/directoryservice/memory): use tokio RwLockFlorian Klink1-3/+4
We don't want to block here, and this also means there's no poisoning to deal with. Change-Id: Ic375571970c48beace0005ae2c012135086a4d67 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11613 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-05-11 r/8113 refactor(tvix/castore/blobservice/memory): use parking_lot RwLockFlorian Klink5-31/+27
This one doesn't require us to deal with poisoning, is upgradeable and the right thing to use when locking access to data, not IO resources. Change-Id: I78634953a73404500d28f51f1d93a87e215c8149 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11612 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-05-11 r/8112 refactor(tvix/store/pathinfo/memory): tokio RwLock, improve list()Florian Klink1-14/+14
We don't want to use the std::sync::RwLock here, as it blocks. This also means we don't need to deal with the error cases anymore. The list() implementation is updated to use try_stream, which means we can now avoid collecting everything into a Vec before returning from it. Change-Id: I9057dcc410dc553e6b1be3f20d5ee830569e8218 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11611 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-05-11 r/8111 docs(tvix/docs): async nar reader is doneFlorian Klink1-4/+1
Change-Id: I4e8fd03d521b7d30904f84a6afcf842f1505ec7b Reviewed-on: https://cl.tvl.fyi/c/depot/+/11628 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de>
2024-05-11 r/8110 docs(tvix/todo): add section on [ca]store errorsFlorian Klink1-0/+7
Change-Id: I832eed8abb62b7e13972ed58649a96e31b17f033 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11610 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-05-11 r/8109 docs(tvix/todo): update test suite sectionFlorian Klink1-4/+13
nix_oracle is quite misleading, and fooled me into thinking this validates all .exp files in the repo to match Nix output (which it doesn't, that's done by verify-lang-tests). Also, the whole test suite structure thing can use a bit more explanation. Change-Id: I2fadcc871843143270ad2ed9ac98de8287280e6f Reviewed-on: https://cl.tvl.fyi/c/depot/+/11609 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-05-11 r/8108 chore(tvix/narinfo2parquet): bump dependenciesFlorian Klink2-27/+135
Get rid of the RUSTSEC warnings. Also upgrade tokio to drop the Sized requirement, similar to cl/11608. Change-Id: Idc6fece23c79eb30fd1dfc5fe64fa2e4c08cd412 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11624 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-05-11 r/8107 chore(tvix): update dependenciesFlorian Klink2-57/+57
Get rid of the RUSTSEC warnings. Change-Id: I949f5173eb3e807f2de497f2eb94493f7a331e18 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11623 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de>
2024-05-11 r/8106 feat(tvix/store): add LruPathInfoServiceFlorian Klink5-1/+341
This provides an implementation of PathInfoService storing PathInfo in memory up to a certain capacity, then evicting these that have been used the least recently. Change-Id: I9d738687caf4f181a957f72245f26b92832313cd Reviewed-on: https://cl.tvl.fyi/c/depot/+/11622 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de>
2024-05-11 r/8105 refactor(tvix/store/pathinfo/sled): drop {blob,directory}_serviceFlorian Klink2-31/+10
These are not used anymore. Change-Id: I9c348391c9600e9319f171faf3eda7175ebf7076 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11621 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-05-11 r/8104 refactor(tvix/store/pathinfo/memory): drop {blob,directory}_serviceFlorian Klink3-26/+6
These are not used anymore. Change-Id: I6c16b4d80ddaabcb75fec3ea3e32b923b7719485 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11620 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de>
2024-05-11 r/8103 refactor(tvix/store): drop calculate_nar from PathInfoServiceFlorian Klink20-187/+241
This shouldn't be part of the PathInfoService trait. Pretty much none of the PathInfoServices do implement it, and requiring them to implement it means they also cannot make use of this calculation already being done by other PathInfoServices. Move it out into its own NarCalculationService trait, defined somewhere at tvix_store::nar, and have everyone who wants to trigger nar calculation use nar_calculation_service directly, which now is an additional field in TvixStoreIO for example. It being moved outside the PathInfoService trait doesn't prohibit specific implementations to implement it (like the GRPC client for the `PathInfoService` does. This is currently wired together in a bit of a hacky fashion - as of now, everything uses the naive implementation that traverses blob and directoryservice, rather than composing it properly. I want to leave that up to a later CL, dealing with other parts of store composition too. Change-Id: I18d07ea4301d4a07651b8218bc5fe95e4e307208 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11619 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-05-11 r/8102 fix(tvix/store/pathinfo/sled): use spawn_blockingFlorian Klink1-22/+45
This does IO, which might take a longer amount of time than what we want to be blocking the normal executor. Use spawn_blocking instead. I didn't add it for the constructors, as we only call these once. Change-Id: I9a1063099bac9582ca9681043c58c1edc780c5ff Reviewed-on: https://cl.tvl.fyi/c/depot/+/11618 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-05-11 r/8101 fix(tvix/castore/directory/sled): use spawn_blockingFlorian Klink1-39/+60
This does IO, which might take a longer amount of time than what we want to be blocking the normal executor. Use spawn_blocking instead. I didn't add it for the constructors, as we only call these once. Change-Id: I96231fcff8d10abe90cafde25a099a2db6ea9414 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11617 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-05-11 r/8100 chore(tvix/castore/blob): remove sled implementationFlorian Klink5-202/+3
This never did any chunking, and sled (rightfully) performs really bad if values get too large. We switched the default to using the objectstore backend with the local filesystem a while ago, no need to keep this footgun around anymore. Change-Id: I2c12672f2ea6a22e40d0cbf9161560baddd73d4a Reviewed-on: https://cl.tvl.fyi/c/depot/+/11616 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-05-11 r/8099 refactor(tvix/store/pathinfo/grpc): simplify pingpongFlorian Klink2-76/+7
We already have the same code in make_grpc_path_info_service_client. Change-Id: Ibcd60831af8a061a2c3bb2f960f52a43d06cf6fa Reviewed-on: https://cl.tvl.fyi/c/depot/+/11615 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-05-09 r/8096 feat(tvix/cli): Use the same TvixStoreIO for the full runtimeAspen Smith1-16/+26
This specifically allows subsequent fetch calls in the REPL to avoid fetching the same path twice (because they share the same PathInfoService). Change-Id: Ieda089080174913a8c014f13d6852cac599a7e17 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11484 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: aspen <root@gws.fyi>
2024-05-08 r/8095 refactor(nix-compat/nar/reader): reuse prev_name allocationedef8-50/+73
We reuse the prev_name allocation for Entry, instead of allocating and returning a separate Vec. We encode the `prev_name: None` case as an empty vector, since we don't allow empty names anyway, and the sorting is equivalent. Change-Id: I975b37ff873805f5ff099bc82128706891052247 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11607 Reviewed-by: Brian Olsen <me@griff.name> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-05-08 r/8094 feat(nix-compat/wire/bytes): read bytes into an existing bufferedef1-1/+60
For small bytestrings (like NAR names), we can read into a preallocated fixed-size buffer, instead of allocating a Vec every time. Change-Id: Id8da9e9cea99c814361230c0ec02606b731c79a3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11606 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-05-08 r/8089 chore(tvix/tools/crunch-v2): upgrade tokioedef2-5/+20
This drops the Sized requirement on AsyncReadExt. Change-Id: I2a89c708922ca528cb493aefd0613f477f648e83 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11608 Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI
2024-05-08 r/8088 refactor(nix-compat/wire/bytes): style fixesedef1-13/+10
Change-Id: I65c3c43df83e0c364a4b7f1f3054c5b676bd07d5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11605 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-05-08 r/8087 fix(nix-compat/store_path): use Box<str>edef1-8/+5
We don't actually build up names in place here, so we don't need a capacity field. Saves 8 bytes. Change-Id: Icb01b45561e28fd525b726612f56d4640bc834c7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11604 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-05-08 r/8086 feat(nix-compat/wire/bytes/reader): support buffered readingedef3-6/+195
If our underlying reader supports AsyncBufRead, then we can too. Change-Id: If4b948c983400ca591c1c475bbcf7dc00d562040 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11545 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-05-08 r/8085 fix(nix-compat/wire/bytes/reader): always read trailer tag dataedef3-16/+73
Even if we have an aligned (or zero) size (and thus have no padding), we may still have a non-padding trailer. Since an exact read of only the user data bytes must always read the trailer bytes as well, this has to happen for aligned sizes too. For zero-sized trailers, we solve this by reading immediately, since no read is required of the user at all. user_len thus becomes a NonZeroU64, and the definition of "body data" is adjusted to always exclude the final block. Change-Id: I2c6bb51270fb22ad72a65076a3468673ab56ef68 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11597 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-05-08 r/8084 fix(nix-compat/wire/bytes/reader): drop allow(dead_code)edef1-1/+0
We're using this in the NAR reader now. Change-Id: I28f17b1ccedd62ffcaf2fa32b517f16bcd036d94 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11603 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-05-08 r/8083 refactor(nix-compat/wire/bytes/reader): drop prev_fillededef1-2/+1
We specifically structured the code this way to allow using `this.filled` as-is, so we should use it. Change-Id: I7e11bddceb4d5f37b1dd4c453a9d53b85fc1f6c8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11602 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-05-06 r/8082 refactor(tvix): remove usage of async-recursionConnor Brewster8-68/+19
Rust 1.77 supports async recursion as long as there is some form of indirection (ie. `Box::pin`). This removes the need to use the async-recursion crate. Change-Id: Ic9613ab7f32016f0103032a861edff92e2fb8b41 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11596 Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-05-06 r/8081 fix(tvix/store/pathinfo/nix_http): update error messageFlorian Klink1-1/+1
Autosubmit was too fast and submitted this before my push went through. Flagged in https://cl.tvl.fyi/c/depot/+/11580/comment/40a56824_7be73334/ Change-Id: I1f835ae60c2c6fd2db6654c1b1c71d90bee141af Reviewed-on: https://cl.tvl.fyi/c/depot/+/11595 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-05-06 r/8080 feat(tvix/store): use async nar reader for ingest_narFlorian Klink7-351/+212
Rename read_nar to ingest_nar, and have it use the async nar reader version, and the ingest_entries machinery. This means we can now drop all code dealing with manually assembling castore nodes. Update our consumer, NixHTTPPathInfoService to use the new API. As we now accept an AsyncRead, we don't need to do any blocking here anymore, and can use the same async-compression crate as in the fetching logic (and support some more compression formats out of the box). Change-Id: I8646d20bd8603f8da47b5c84bc9e4ac236eb7f1a Reviewed-on: https://cl.tvl.fyi/c/depot/+/11580 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-05-06 r/8079 test(tvix-castore/import): add tests for ingest_entriesFlorian Klink1-3/+138
Change-Id: Ia7906533868fd948509419e0d64b64582575a7fa Reviewed-on: https://cl.tvl.fyi/c/depot/+/11591 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-05-06 r/8078 fix(tvix/castore/import): assert end of streamFlorian Klink1-0/+5
Once we break out with the root node, there may be no more elements in the stream. Change-Id: I6f5fc5662095aa2b2a56bcad506d25520d9ad00c Reviewed-on: https://cl.tvl.fyi/c/depot/+/11592 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-12-14T04·57+0000