about summary refs log tree commit diff
path: root/tvix
AgeCommit message (Collapse)AuthorFilesLines
2024-10-13 r/8804 refactor(tvix/build): remove use of lazy_staticFlorian Klink4-12/+2
This is now supported in the standard library via std::sync::LazyLock, but requires some manual shuffling around of code. Change-Id: Ie2af74beda9fcf8aa19fca7d844bcbe732f05bf8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12610 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2024-10-13 r/8803 refactor(tvix/glue): remove use of lazy_staticVincent Ambo6-50/+64
This is now supported in the standard library via std::sync::LazyLock, but requires some manual shuffling around of code. Change-Id: Ibb3be8458b8a8912ea04c9360d64c5cf914254d4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12609 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-10-13 r/8802 refactor(tvix/nix-compat): remove use of lazy_staticVincent Ambo9-110/+92
This is now supported in the standard library via std::sync::LazyLock, but requires some manual shuffling around of code. I found at least one dead variable along the way, which I deleted. Change-Id: I8600c87c49078fb5ff72671994c77b919259e67b Reviewed-on: https://cl.tvl.fyi/c/depot/+/12608 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su>
2024-10-12 r/8801 chore(tvix/tools): move narinfo2parquet to //users/edefVincent Ambo6-11345/+0
This is not a core Tvix tool, it's a tool that uses a Tvix component. Change-Id: I81d2b2374da23489df0097dcabb8295c82652fc1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12606 Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su>
2024-10-12 r/8800 chore(tvix/tools): move weave to //users/edefVincent Ambo9-11508/+0
This is not a core Tvix tool, it's a tool that uses a Tvix component. Change-Id: I705f2c4ab87f1512e005007c933e16b84ed4279f Reviewed-on: https://cl.tvl.fyi/c/depot/+/12605 Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su>
2024-10-12 r/8799 chore(tvix/tools): move turbofetch to //users/edefVincent Ambo9-9375/+0
This was introduced in cl/9925 without any commit message, but this is clearly not relevant to Tvix itself (it even says so in a comment in Cargo.toml). Change-Id: I84f12d5145c3f53c9df23863f887bad913856c50 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12604 Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: edef <edef@edef.eu>
2024-10-12 r/8798 chore(tvix/tools): move crunch-v2 to //users/edefVincent Ambo12-15454/+0
This is not a core Tvix tool, it's some sort of one-off analysis thing. Change-Id: I05fcbed45abad27d6b5cfd49db1727249dad3971 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12603 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: edef <edef@edef.eu>
2024-10-12 r/8797 refactor(tvix/eval): remove usage of lazy_staticVincent Ambo6-38/+26
Equivalent logic is now in the standard library, and this dependency is no longer needed for eval. Change-Id: Iaa4410d89fdaa5b84cbd9e6bc6ae479c659d92f2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12602 Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI
2024-10-12 r/8796 refactor(tvix/eval): Make `strict` an EvalMode enumAspen Smith8-31/+50
Refactor the `strict` boolean passed into evaluation at the top-level to be a (two-variant, so far) EvalMode enum of Lazy and Strict. This is more explicit than a boolean, and if we ever add more EvalModes it's a simple extension of the enum. Change-Id: I3de50e74ec971011664f6cd0999d08b792118410 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12186 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: aspen <root@gws.fyi>
2024-10-12 r/8794 chore(tvix/tools/crunch-v2): bump crate dependenciesFlorian Klink2-1974/+1266
Change-Id: Ib67e0121cebf6b2f049490125c7ded9637955928 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12599 Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-10-12 r/8793 chore(tvix/tools/weave): bump crate dependenciesFlorian Klink2-1208/+903
Change-Id: I31e5f3d46321265d662d41aa79aa4178e121818f Reviewed-on: https://cl.tvl.fyi/c/depot/+/12598 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
2024-10-12 r/8792 chore(tvix/tools/turbofetch): bump crate dependenciesFlorian Klink2-685/+1354
Change-Id: I81988571344829708881021527b83e7d99b598c3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12597 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
2024-10-12 r/8791 chore(tvix): bump crate dependenciesFlorian Klink5-1541/+1650
tonic-build deprecated their `compile()` function, it's now called `compile_protos()`. Change-Id: I8cacd7f01a251c207401e4e226b0e880744e96e8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12596 Tested-by: BuildkiteCI Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com> Autosubmit: flokli <flokli@flokli.de>
2024-10-11 r/8790 test(tvix/store/signing_wrapper): restructureFlorian Klink1-14/+16
Move things around a bit to make it easier to understand what's going on: - We first validate our fixture invariants - We then insert into the PathInfoService - Do all comparisons and checks we can on the returned PathInfo struct - Only convert to the NarInfo variant to calculate the fingerprint, and don't keep intermediate let bindings for this Before cl/12588, this was arguably much harder to do that way, as we relied on some of the conversions done in the to_narinfo() function. Change-Id: Iaddbf1079f73ce566ef6d56f69a823e080b2e006 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12595 Reviewed-by: Marijan Petričević <marijan.petricevic94@gmail.com> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Reviewed-by: sinavir <tvix@sinavir.fr>
2024-10-11 r/8789 refactor(tvix/glue/register_in_path_info_service): return only PathInfoFlorian Klink3-33/+42
The store path is already contained in the PathInfo, and the ca bits is already passed into the function, so known to the caller - there's no need to duplicate this. We can also avoid having two separate block_on in our import builtin - we already know the content hash before constructing, as we pass it in via ca_hash. There's still some room to unclutter some more of the code around importing - we still do NAR calculation twice in some cases, and some of the code might be share-able from other places producing PathInfo too. Log a TODO for this cleanup. Change-Id: I6a5fc427d15bc9293a396310143c7694dd2996c0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12592 Reviewed-by: Marijan Petričević <marijan.petricevic94@gmail.com> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-10-11 r/8788 refactor(nix-compat/store_path): consistently use SP as type paramFlorian Klink3-10/+13
We also use S in other places in the same file, but that's for the string-like references. SP is now consistently used as the type parameter for StorePath<_> (and build_output_path) gets support for it). By being a bit more careful in the order of assignments in nix-compat/ src/derivation, we can nudge the compiler to use the type we want. Change-Id: Ia7c298e110dff98d3b113d2388674ce9e22b80e8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12590 Reviewed-by: flokli <flokli@flokli.de> Reviewed-by: Marijan Petričević <marijan.petricevic94@gmail.com> Tested-by: BuildkiteCI
2024-10-11 r/8787 refactor(tvix/store): use strictly typed PathInfo structMarijan Petričević26-1042/+726
This switches the PathInfoService trait from using the proto-derived PathInfo struct to a more restrictive struct, and updates all implementations to use it. It removes a lot of the previous conversion and checks, as invalid states became nonrepresentable, and validations are expressed on the type level. PathInfoService implementations consuming protobuf need to convert and do the verification internally, and can only return the strongly typed variant. The nix_compat::narinfo::NarInfo conversions for the proto PathInfo are removed, we only keep a version showing a NarInfo representation for the strong struct. Converting back to a PathInfo requires the root node now, but is otherwise trivial, so left to the users. Co-Authored-By: Florian Klink <flokli@flokli.de> Change-Id: I6fdfdb44063efebb44a8f0097b6b81a828717e03 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12588 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-10-11 r/8786 chore(tvix/build): Bump oci-spec dependencyIlan Joselevich4-202/+149
Our oci-spec was a bit oudated and there were some renamings in one of the release, which made building tvix-build fail if it's a dependency. I encountered this issue while working on tvix-eval-jobs. Change-Id: I6d982965176b83170a07445e351d3f5e5679ed2e Reviewed-on: https://cl.tvl.fyi/c/depot/+/12586 Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com> Tested-by: BuildkiteCI
2024-10-10 r/8785 feat(tvix/composition): allow urls as anonymous storesYureka9-91/+151
This allows specifying an url in place of a named reference to another composition entry, if the castore crate has been compiled with the xp-store-composition feature. Example: `--directory-service-addr cache://?near=memory://&far=memory://` This would be equivalent to the instantiation via toml file: ```toml [memory1] type = "memory" [memory2] type = "memory" [default] type = "cache" near = "memory1" far = "memory2" ``` Note that each anonymous url causes a distinct instance to be created. Change-Id: Iee5a07a94b063b5e767c704d9cad0114fa843164 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12146 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-10-10 r/8784 feat(tvix/dirsvc/Cache): support building from urlYureka1-5/+6
Change-Id: I80121319795319bb977427efeca3666c6b87a1b7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12147 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-10-10 r/8783 feat(tvix/pathinfo/nixhttp): use ingest stores from urlYureka1-2/+18
This still defaults to the "default" services, but allows users to tell the nix+http pathinfoservice to ingest the castore nodes into a non-default blob-/directoryservice when used with the experimental store composition. Change-Id: I5c0f683ce95d888eadf3f302520a47f42f1a481d Reviewed-on: https://cl.tvl.fyi/c/depot/+/12148 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-10-09 r/8782 refactor(tvix/cli/args): remove log-level argumentMarijan Petričević3-12/+0
RUST_LOG allows for a more granular tracing configuration Change-Id: Iea29fc2b154345eb7479dfd8bbea6abfd4716341 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12585 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-10-04 r/8756 feat(tvix/[ca]store): Add logging support to redbIlan Joselevich4-2/+11
We wanted to enable this earlier but the log level of many of the messages in redb were too high, they've now been downgraded, so we can enable logs from redb. Context on the fix and release: - https://github.com/cberner/redb/pull/828 - https://github.com/cberner/redb/releases/tag/v2.1.2 Change-Id: I8635e8a0bcb01a7d0b580387ac9134ccdd0205f3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12568 Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-10-03 r/8752 chore(tvix/[ca]store): Drop sled support completely in favor of redbIlan Joselevich14-812/+18
Over the past couple of months we've been using redb instead of sled as the default filesystem-based database in PS and DS. I am confident that we can get rid of sled completely now, and just keep redb. Change-Id: I11fa1e4453e280253855f8eade990b37eb6965ae Reviewed-on: https://cl.tvl.fyi/c/depot/+/12567 Reviewed-by: yuka <yuka@yuka.dev> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
2024-10-01 r/8748 test(tvix/castore/refscan): add empty pattern regression testYureka1-0/+16
Change-Id: I165261170edaabfc56f6ac0a6baae388332b3f73 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12559 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: yuka <yuka@yuka.dev>
2024-10-01 r/8747 fix(tvix/castore/refscan): don't panic on empty patternsYureka1-1/+9
Previously, the overlap calculation would underflow when the pattern is empty. Change-Id: I1f6bf49fafc4b8183a3a5e5e491a5a5bfc41ca97 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12558 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: yuka <yuka@yuka.dev>
2024-10-01 r/8746 docs(tvix/TODO): add PathInfo data types and ca reference itemsFlorian Klink1-0/+43
With https://cl.tvl.fyi/12533 in, we still need to lookup references to properly populate `BuildRequest`. It currently fails as the reference to h9lc1dpi14z7is86ffhl3ld569138595-audit-tmpdir.sh is not propagated. We should prevent Frankenbuilds from the go, so let's update our PathInfo type to accomodate for that. Change-Id: I26f9215312c258bba222efd390bc135f1a3a3d6d Reviewed-on: https://cl.tvl.fyi/c/depot/+/12560 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-10-01 r/8745 feat(tvix/build/oci): wire up refscanningYureka1-20/+47
Change-Id: I07d016f831dcc596b4627f1d8f33909e632be416 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12533 Autosubmit: yuka <yuka@yuka.dev> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-10-01 r/8744 feat(tvix/build), add OciBuildService, the old wayFlorian Klink14-18/+1270
This is just patchset 10 of CL10855, before the color_eyre changes, rebased to the tvix_castore api. Change-Id: If4b42412ff8568058908cda971ad7d6f2d9f9b7b --- This provides a build service invoking runc. It can be used by using the `oci://$path_to_some_tempdir` builder URL for now. For now, it can be tested as such: ``` BUILD_SERVICE_ADDR=oci://$PWD/bundles target/debug/tvix let pkgs = (import <nixpkgs> {}); in builtins.readDir pkgs.perl ``` readDir is to actually trigger IO into the store path (which triggers the builds). For now it fails due to missing reference scanning (see followup CLs). Change-Id: I09b40e410114ce69966a41a0e3c33281b859e443 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12526 Autosubmit: yuka <yuka@yuka.dev> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-10-01 r/8743 feat(tvix/glue): wire up nix refscanningYureka2-12/+73
After this, attempting to build the nixpkgs still fails in the same way, because the references are not yet properly used by the code at `tvix/glue/src/tvix_store_io.rs`. Change-Id: I8a59ef8ef3c9a6f6aa7b05106dd9eef2e9ac0d0f Reviewed-on: https://cl.tvl.fyi/c/depot/+/12532 Reviewed-by: Brian Olsen <me@griff.name> Autosubmit: yuka <yuka@yuka.dev> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-10-01 r/8742 feat(tvix/build): add refscanning interfaceYureka3-42/+153
This provides a generic interface to let the builder search for needles in the output, as described in the notes at `docs/src/build/index.md`. Change-Id: Ic2c5bd563e9aa2e766c157f2b13cdb19aede12f8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12531 Autosubmit: yuka <yuka@yuka.dev> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Reviewed-by: Brian Olsen <me@griff.name>
2024-10-01 r/8741 feat(castore/fs): optional refscanner for ingestYureka5-22/+60
Change-Id: Ieca06de4c2e2680d89fe05a380079fafa5454837 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12529 Autosubmit: yuka <yuka@yuka.dev> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-10-01 r/8740 feat(tvix/castore/refscan): share the scanner between readersYureka1-39/+30
This changes the only actual state the ReferenceScanner has to use atomic bools, so it no longer requires a mutable borrow for .scan(). This allows passing an immutable borrow of a reference scanner to multiple threads which might be ingesting blobs in parallel, and using them in the ReferenceReader or calling .scan() there. Change-Id: Id5c30bcebb06bf15eae8c4451d70eb806cab722e Reviewed-on: https://cl.tvl.fyi/c/depot/+/12528 Autosubmit: yuka <yuka@yuka.dev> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-09-30 r/8738 feat(tvix/nar-bridge): treat HEAD requests explicitlyFlorian Klink2-4/+13
We don't need to access castore for HEAD requests. Change-Id: I9365d9520d5a9e52ed92897d3c4972ec5b6e11fb Reviewed-on: https://cl.tvl.fyi/c/depot/+/12547 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2024-09-30 r/8737 feat(tvix/nar-bridge): implement range request for NARsFlorian Klink5-21/+352
With an implementation of AsyncRead + AsyncSeek, axum-range can answer range requests. We only use it if a range has been requested, as it uses more memory than the linear variant. Change-Id: I0072b0a09b328f3e932f14567a2caa3a49abcbf7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12509 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz> Reviewed-by: yuka <yuka@yuka.dev>
2024-09-28 r/8726 chore(tvix/castore/fuse): impl Layer for TvixStoreFsYureka1-1/+12
Allows using a TvixStoreFs in overlay filesystems Change-Id: I10e63c92776b1c783947d92070f360865ef2883c Reviewed-on: https://cl.tvl.fyi/c/depot/+/12539 Autosubmit: yuka <yuka@yuka.dev> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-09-28 r/8725 chore(tvix/castore/fuse): update fuse-backend-rsYureka4-13/+20
This release includes support for overlay filesystems Change-Id: I946cbf346df045209afaea2d720bb57fc2f2659f Reviewed-on: https://cl.tvl.fyi/c/depot/+/12538 Reviewed-by: flokli <flokli@flokli.de> Autosubmit: yuka <yuka@yuka.dev> Tested-by: BuildkiteCI
2024-09-27 r/8724 refactor(tvix): move refscan module to castoreYureka7-13/+19
This is required to add the optional refscanner parameter to the ingest functions. Change-Id: Ib40a7287cf857eb55e31e0df309a79474fefb518 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12527 Autosubmit: yuka <yuka@yuka.dev> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-09-26 r/8717 fix(tvix/castore): don't return object_store::has with INFO levelFlorian Klink1-1/+1
This otherwise spams the logs quite a bit, for example when uploading to nar-bridge with the new check from cl/12497. Change-Id: Idc2bcc513caea6fae38ae04489e4e3ee7be64bce Reviewed-on: https://cl.tvl.fyi/c/depot/+/12510 Reviewed-by: yuka <yuka@yuka.dev> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-09-26 r/8716 feat(tvix/store): Add a signing PathInfoServicesinavir6-0/+225
- Add a new PathInfoService implementation that wraps transparently around another except that it dynamically signs all the incoming path-infos with the provided signer. - Add a ServiceBuilder for this PathInfoService that provides a SigningPathInfoService with a keyfile signer Change-Id: I845ddfdf01d14c503c796b2b80c720dab98be091 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12032 Reviewed-by: flokli <flokli@flokli.de> Autosubmit: sinavir <tvix@sinavir.fr> Tested-by: BuildkiteCI
2024-09-25 r/8715 feat(tvix/store): seekable nar rendererYureka4-0/+535
Co-authored-by: edef <edef@edef.eu> Change-Id: I233206e8aae35504ca0519ac88178dfc5596bedb Reviewed-on: https://cl.tvl.fyi/c/depot/+/12439 Reviewed-by: flokli <flokli@flokli.de> Autosubmit: yuka <yuka@yuka.dev> Tested-by: BuildkiteCI
2024-09-25 r/8714 refactor(tvix/store/tests): combine tests into one parametrized functionYureka2-179/+103
Change-Id: I9ff43b29be68b9840c58286da96fa52927691804 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12507 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-09-25 r/8713 feat(tvix/eval): Use thiserror for ErrorKind and CatchableErrorKindIlan Joselevich4-242/+100
thiserror is much more easier to maintain than manually implementing Error and Display. Change-Id: Ibf13e2d8a96fba69c8acb362b7515274a593dfd6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12452 Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com> Tested-by: BuildkiteCI
2024-09-24 r/8711 fix(tvix/nar-bridge): Remove name check for root node in nar generationsinavir4-16/+27
Nar-bridge tried to parse the name of the protobuf node encoded in the URL into a PathComponent but this name was empty, leading to an error when the user tried to retrieve the nar file. This was an oversight from the conversion to stricter types (some of the CLs in the serious containing cl/12217). We need a version converting a protobuf without a name to our stricter types, but an empty PathComponent cannot be constructed. So we need a into_name_and_node() version that returns the name as Bytes, not PathComponent. Change-Id: I2996cdd2e0107133e502748947298f512f1cc521 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12504 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-09-23 r/8710 feat(tvix/nix-compat/nar/writer/sync): add file_manual_writeYureka1-0/+68
This is useful for building other NAR writers which use custom (async or optimized) I/O to write the blob parts of the NAR. Change-Id: I447c09914fb0c99044e2fa910d4213660dc51c64 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12437 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-09-19 r/8704 fix(tvix/castore/import): check small blobs firstFlorian Klink1-0/+13
ConcurrentBlobUploader buffers small blobs in memory, and then uploads them to the BlobService in the background. In these cases, we know the hash of the whole blob, so we could check if it exists first before, uploading it. We were however not, and this caused rate limiting issues in GCS, as it has an update limit of one write per second on the same key, which we ran into especially frequently with the empty blob. This reduces the amount of writes of the same blob considerably. In the future, we might be able to drop this, as our chunked blob uploading protocol gets smarter and covers these cases. Change-Id: Icf482df815812f80a0b65cec0426f8e686308abb Reviewed-on: https://cl.tvl.fyi/c/depot/+/12497 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-09-08 r/8666 docs(tvix/nix-compat/wire/bytes/reader): None case doesn't existedef1-2/+0
We always read the length before returning a BytesReader, so len() cannot be called before the length is read. Change-Id: Ifa1cbc2c0923dd24d59e0af5e135ab8ed7314aa9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12453 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-09-07 r/8663 feat(tvix/nix-compat/nar/writer/sync): trait objects -> genericsYureka1-13/+10
This will later be used to allow the user to access the original writer. Change-Id: I1734600c5d580d18b16727f892cd2f6335cc9459 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12442 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-09-07 r/8662 feat(tvix/castore): expose ValidatedDirectoryGraph typeYureka1-1/+1
Change-Id: Id994258e74f75df8790159ed3180ce360874d0de Reviewed-on: https://cl.tvl.fyi/c/depot/+/12438 Reviewed-by: benjaminedwardwebb <benjaminedwardwebb@gmail.com> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-09-05 r/8653 chore(3p/sources): bump channels & overlays (2024-09-01)Vincent Ambo5-11/+12
Included changes: * users/aspen: explicitly use open-source nvidia driver This now has to be specified explicitly, otherwise evaluation fails with an error. * users/aspen: nixfmt -> nixfmt-classic * users/aspen: fixes for renamed packages & options * users/tazjin: fixes for renamed packages & options * 3p/overlays: remove cbtemulator patch (merged upstream) * tvix/shell: remove unnecessary patches (merged upstream) * 3p/rust-crates: mark libgit2_sys as broken * users/Profpatsch: mark git-db as broken * 3p/overlays: pick `mypaint` from stable channel * tvix: fix comments that clippy doesn't like anymore * tvix/glue: disable a misfiring clippy lint (applying its suggestion breaks code below) Change-Id: I6d3fc027694bbe7425a2d25dc53d65467a44f3b0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12403 Tested-by: BuildkiteCI Reviewed-by: aspen <root@gws.fyi> Reviewed-by: Profpatsch <mail@profpatsch.de> Autosubmit: tazjin <tazjin@tvl.su>
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-11-25T07·02+0000