about summary refs log tree commit diff
path: root/tvix
AgeCommit message (Collapse)AuthorFilesLines
2023-12-09 r/7137 refactor(nix-compat/nixbase32): address clippyFlorian Klink1-1/+1
Change-Id: I07b53ead10b0efb31e88c6ae93c124c5f35261d3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10230 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-12-09 r/7136 refactor(nix-compat/narinfo): address clippy lintsFlorian Klink3-3/+3
Change-Id: I00f28624ec0469525e52f408f00caa20d3b701b8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10229 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-12-09 r/7135 refactor(nix-compat/derivation): address clippyFlorian Klink1-3/+3
Change-Id: I5d7057347838f733d7dc1331c7b7ef8d27244412 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10228 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de>
2023-12-09 r/7134 refactor(tvix/eval): address clippy lintsFlorian Klink2-2/+2
Change-Id: Ic2bd4e8291b30ceac9fa0e88a4f56e61ae99b603 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10227 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-12-09 r/7133 chore(tvix/nix_cli): drop crateFlorian Klink7-186/+0
The only thing this was doing was invoking nix-store --add, which is not gonna help us populate the tvix-store at all (and we now have `tvix-store import`). This is also (rightfully) causing clippy warnings, because of some fields being unused. It's more of a skeleton, and rather than shelling out to Nix for some usecases, we might introduce a "compatible" Nix CLI frontend for a subset of commands. Drop this for now, to decrease the noise and confusion. Change-Id: I2fd399e9320260f08893b685561755af9c7c961c Reviewed-on: https://cl.tvl.fyi/c/depot/+/10226 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-12-09 r/7132 refactor(nix-compat/store_path): from_absolute_path to StorePathRefFlorian Klink3-19/+20
The only non-test usage was only checking for the error case, and we can still convert this to an owned StorePath by calling to_owned() on StorePathRef. Change-Id: I9f67a759e580c9c429c96896bcdd295392aa5a2a Reviewed-on: https://cl.tvl.fyi/c/depot/+/10225 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-12-09 r/7131 refactor(nix-compat/derivation): rename input derivation componentsFlorian Klink1-3/+3
Match the naming in parse_input_derivations, call the keys "input_derivation", and the values "output_names". Change-Id: I7d1974819028f8ea543dc3ad78afb803ff9db865 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10224 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-12-09 r/7130 feat(tvix/eval): impl DoubleEndedIter for OwnedAttrsIteratorAdam Joseph1-0/+10
Change-Id: I4bd85dbe9c27047f4abbdeff4e2b796e9bcab3a1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10211 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: Adam Joseph <adam@westernsemico.com> Tested-by: BuildkiteCI
2023-12-09 r/7129 chore(store/src/tests): make clippy shut upAdam Joseph1-12/+6
Change-Id: I6dfceaa32a8c01f8395b7889ae19847fabf95ed3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10214 Tested-by: BuildkiteCI Autosubmit: Adam Joseph <adam@westernsemico.com> Reviewed-by: flokli <flokli@flokli.de>
2023-12-09 r/7128 fix(tvix/glue): do not panic if PathInfoService returns ErrAdam Joseph1-1/+1
Nixpkgs tries to `import` the value returned by `builtins.unsafeGetAttrPos`, which in our case is the file `/deep/thought`. Since that doesn't exist, tvix-glue panics, but there's no interpreter backtrace to follow. Let's return an Err instead of panicking. ------------------------------------------------------------------------------ Before: thread 'tokio-runtime-worker' panicked at /source/src/import.rs:164:27: called `Result::unwrap()` on an `Err` value: Error { depth: 0, inner: Io { path: Some("/deep/thought:42"), err: Os { code: 2, kind: NotFound, message: "No such file or directory" } } } note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace thread 'main' panicked at src/tvix_store_io.rs:276:58: called `Result::unwrap()` on an `Err` value: JoinError::Panic(Id(41580), ...) Command exited with non-zero status 101 ------------------------------------------------------------------------------ After: thread 'tokio-runtime-worker' panicked at /source/src/import.rs:164:27: called `Result::unwrap()` on an `Err` value: Error { depth: 0, inner: Io { path: Some("/deep/thought:42"), err: Os { code: 2, kind: NotFound, message: "No such file or directory" } } } note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace note: while evaluating this Nix code --> [code]:1:1 | 1 | (import /nix/store/7xii7xcl0iliqxfq8hp577wdq5j0mikr-kp8vf3gzk1pff9r40j5p0y8kiwhkkqw1-nixpkgs-src {}).pkgsCross.aarch64-multiplatform.rocmPackages_5.hipblas.outPath | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ note: while evaluating this as native code (getAttr) --> <src-builtins/derivation.nix>:26:19 | 26 | outPath = builtins.getAttr outputName strict; | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ note: while evaluating this as native code (derivationStrict) --> <src-builtins/derivation.nix>:14:12 | 14 | strict = derivationStrict drvAttrs; | ^^^^^^^^^^^^^^^^^^^^^^^^^ note: while evaluating this as native code (getAttr) --> <src-builtins/derivation.nix>:26:19 | 26 | outPath = builtins.getAttr outputName strict; | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ note: while evaluating this as native code (derivationStrict) --> <src-builtins/derivation.nix>:14:12 | 14 | strict = derivationStrict drvAttrs; | ^^^^^^^^^^^^^^^^^^^^^^^^^ note: while evaluating this as native code (getAttr) --> <src-builtins/derivation.nix>:26:19 | 26 | outPath = builtins.getAttr outputName strict; | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ note: while evaluating this as native code (derivationStrict) --> <src-builtins/derivation.nix>:14:12 | 14 | strict = derivationStrict drvAttrs; | ^^^^^^^^^^^^^^^^^^^^^^^^^ note: while evaluating this as native code (getAttr) --> <src-builtins/derivation.nix>:26:19 | 26 | outPath = builtins.getAttr outputName strict; | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ note: while evaluating this as native code (derivationStrict) --> <src-builtins/derivation.nix>:14:12 | 14 | strict = derivationStrict drvAttrs; | ^^^^^^^^^^^^^^^^^^^^^^^^^ note: while evaluating this as native code (getAttr) --> <src-builtins/derivation.nix>:26:19 | 26 | outPath = builtins.getAttr outputName strict; | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ note: while evaluating this as native code (derivationStrict) --> <src-builtins/derivation.nix>:14:12 | 14 | strict = derivationStrict drvAttrs; | ^^^^^^^^^^^^^^^^^^^^^^^^^ note: while evaluating this as native code (getAttr) --> <src-builtins/derivation.nix>:26:19 | 26 | outPath = builtins.getAttr outputName strict; | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ note: while evaluating this as native code (derivationStrict) --> <src-builtins/derivation.nix>:14:12 | 14 | strict = derivationStrict drvAttrs; | ^^^^^^^^^^^^^^^^^^^^^^^^^ note: while evaluating this as native code (getAttr) --> <src-builtins/derivation.nix>:26:19 | 26 | outPath = builtins.getAttr outputName strict; | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ note: while evaluating this as native code (derivationStrict) --> <src-builtins/derivation.nix>:14:12 | 14 | strict = derivationStrict drvAttrs; | ^^^^^^^^^^^^^^^^^^^^^^^^^ note: while evaluating this Nix code --> /nix/store/7xii7xcl0iliqxfq8hp577wdq5j0mikr-kp8vf3gzk1pff9r40j5p0y8kiwhkkqw1-nixpkgs-src/pkgs/development/rocm-modules/5/llvm/stage-2/bintools-unwrapped.nix:6:78 | 6 | runCommand "rocm-llvm-binutils-${llvm.version}" { preferLocalBuild = true; } '' | ______________________________________________________________________________^ 7 | | mkdir -p $out/bin 8 | | 9 | | for prog in ${lld}/bin/*; do ... | 27 | | ln -s ${lld}/bin/lld $out/bin/ld 28 | | '' | |__^ note: while evaluating this as native code (coerce_to_string) --> /nix/store/7xii7xcl0iliqxfq8hp577wdq5j0mikr-kp8vf3gzk1pff9r40j5p0y8kiwhkkqw1-nixpkgs-src/pkgs/development/rocm-modules/5/llvm/stage-2/bintools-unwrapped.nix:27:9 | 27 | ln -s ${lld}/bin/lld $out/bin/ld | ^^^^^^ note: while evaluating this as native code (getAttr) --> <src-builtins/derivation.nix>:26:19 | 26 | outPath = builtins.getAttr outputName strict; | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ note: while evaluating this as native code (derivationStrict) --> <src-builtins/derivation.nix>:14:12 | 14 | strict = derivationStrict drvAttrs; | ^^^^^^^^^^^^^^^^^^^^^^^^^ note: while evaluating this Nix code --> /nix/store/7xii7xcl0iliqxfq8hp577wdq5j0mikr-kp8vf3gzk1pff9r40j5p0y8kiwhkkqw1-nixpkgs-src/lib/customisation.nix:249:23 | 249 | outPath = assert condition; drv.${outputName}.outPath; | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ note: while evaluating this as native code (force) --> /nix/store/7xii7xcl0iliqxfq8hp577wdq5j0mikr-kp8vf3gzk1pff9r40j5p0y8kiwhkkqw1-nixpkgs-src/lib/customisation.nix:249:30 | 249 | outPath = assert condition; drv.${outputName}.outPath; | ^^^^^^^^^ note: while evaluating this as native code (throw) --> /nix/store/7xii7xcl0iliqxfq8hp577wdq5j0mikr-kp8vf3gzk1pff9r40j5p0y8kiwhkkqw1-nixpkgs-src/pkgs/stdenv/generic/check-meta.nix:262:8 | 262 | in handler msg; | ^^^^^^^^^^^ note: while evaluating this Nix code --> /nix/store/7xii7xcl0iliqxfq8hp577wdq5j0mikr-kp8vf3gzk1pff9r40j5p0y8kiwhkkqw1-nixpkgs-src/pkgs/stdenv/generic/check-meta.nix:254:14 | 254 | else '' | ______________^ 255 | | Package ‘${getName attrs}’ in ${pos_str meta} ${errormsg}, refusing to evaluate. 256 | | 257 | | '' + (builtins.getAttr reason remediation) attrs; | |________________________________________________________^ note: while evaluating this as native code (force) --> /nix/store/7xii7xcl0iliqxfq8hp577wdq5j0mikr-kp8vf3gzk1pff9r40j5p0y8kiwhkkqw1-nixpkgs-src/pkgs/stdenv/generic/check-meta.nix:254:14 | 254 | else '' | ______________^ 255 | | Package ‘${getName attrs}’ in ${pos_str meta} ${errormsg}, refusing to evaluate. 256 | | 257 | | '' + (builtins.getAttr reason remediation) attrs; | |__________^ error[E029]: I/O error: /deep/thought:42: task panicked --> /nix/store/7xii7xcl0iliqxfq8hp577wdq5j0mikr-kp8vf3gzk1pff9r40j5p0y8kiwhkkqw1-nixpkgs-src/pkgs/stdenv/generic/check-meta.nix:255:41 | 255 | Package ‘${getName attrs}’ in ${pos_str meta} ${errormsg}, refusing to evaluate. | ^^^^^^^^^^^^^^^ Command exited with non-zero status 1 Benchmark: {"pkgsCross.aarch64-multiplatform.rocmPackages_5.hipblas.outPath":{"kbytes":"26613180","system":"22.35","user":"140.62"}} Change-Id: I587b57e9e49d1f3ecdc0fc9cf996d179a3548f34 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10223 Autosubmit: Adam Joseph <adam@westernsemico.com> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-12-09 r/7127 chore(nix_cli): make clippy shut upAdam Joseph1-0/+1
Change-Id: I0c243baf4deecfd81bd939da86d685e2cbc59d69 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10215 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Adam Joseph <adam@westernsemico.com>
2023-12-09 r/7126 fix(tvix/nar-bridge): fix FOD hashAdam Joseph1-1/+1
The golang mothership seems to be monkeying with hashes again. Change-Id: I7430b4cde84fa51be2b572fba02e3567864bb87a Reviewed-on: https://cl.tvl.fyi/c/depot/+/10209 Tested-by: BuildkiteCI Autosubmit: Adam Joseph <adam@westernsemico.com> Reviewed-by: flokli <flokli@flokli.de>
2023-12-07 r/7125 feat(tvix): run crate2nix generate in CIAdam Joseph1-5/+57
This runs `crate2nix generate` in CI and then runs `depotfmt` on the result to ensure that our machine-generated code is really, really readable and pretty. Then it checks that the result of all that is identical to the committed Cargo.nix. A self-hashing FOD is used to allow network access. No magic hashes are involved. Co-Authored-By: Florian Klink <flokli@flokli.de> Change-Id: I68ec5003dbc6a40894a5a4d6e902f138c99f6719 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10194 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Adam Joseph <adam@westernsemico.com> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-12-06 r/7123 refactor(tvix): Make benchmark script directory-agnosticAspen Smith1-1/+1
Allow running the benchmark script from any directory - primarily so Windtunnel can point to the /tvix josh workspace rather than the depot overall Change-Id: Ie5fc3ef995bf8114277298ae5c5010e6a0bf13ac Reviewed-on: https://cl.tvl.fyi/c/depot/+/10205 Reviewed-by: grfn <grfn@gws.fyi> Autosubmit: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2023-12-06 r/7120 feat(tvix/eval): rewrite Thunk::force() in nonrecursive formAdam Joseph3-64/+97
This commit rewrites Thunk::force() so that it is not (directly) self-recursive. It maintains a Vec of all the previously-encountered thunks which point to the one it is currently forcing, rather than recursively calling itself. Benefits: - Short term: This commit saves the cost of a round-trip through the generator machinery for the generators::request_force() which is removed by this commit. - Medium term: Once a similar transformation has been applied to nix_cmp(), nix_add(), nix_eq(), and coerce_to_string(), those four functions, along with Thunk::force(), will make non-tail calls only to each other. They can then be merged into a single tail-recursive function which does not use the generator machinery at all: enum Task { Cmp, Add, Eq, CoerceToString, Force}; fn Value::walk(task:Task, v1:Value, v2:Value) { // ... - Long term: The long-term goal here is to use generators **only for builtins** and [Marionette]-style remote control of the VM. In other words: use `async` for things that actually involve concurrency. Calls from the VM to builtins can then be blocking calls, because even cppnix will overflow the stack if you make a MAX_STACK_DEPTH-deep recursive call which passes through a builtin at every stack frame (e.g. `{ func = builtins.sort (a: b: ... func ...) ...}`). This way the inner "tight loop" of the interpreter doesn't pay the costs of `async` and generators. These costs manifest in terms of: performance, complex nonlocal control flow, and language impediments (async Rust is a restricted subset of real Rust, and is missing things like traits). [Marionette]: https://firefox-source-docs.mozilla.org/testing/marionette/Intro.html Change-Id: I6179b8abb2ea0492180fcb347f37595a14665777 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10039 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-12-05 r/7119 fix(tvix/eval): Return error rather than panicking on bad substringAspen Smith2-1/+10
If builtins.substring is invoked with (byte!!) offsets that aren't at codepoint boundaries, return an error rather than panicking. This is still incorrect (see b/337) but pushes the incorrectness forward a step. Change-Id: I5a4261f2ff250874cd36489ef598dcf886669d04 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10199 Tested-by: BuildkiteCI Autosubmit: grfn <grfn@gws.fyi> Reviewed-by: sterni <sternenseemann@systemli.org>
2023-12-05 r/7118 refactor(tvix): move src into let bindingFlorian Klink1-4/+6
Change-Id: Ida2a3ac722fb2445745759323975884dfeef3e87 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10193 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su>
2023-12-05 r/7115 feat(tvix): Add script for running benchmarks in WindtunnelAspen Smith1-0/+10
Currently this just uses a Docker container, which is gross but works fine for now since we don't have the ability to build benchmarks in nix as of cl/7538 Change-Id: I48e317f44bc2c73533d7663403786a3a37c7952f Reviewed-on: https://cl.tvl.fyi/c/depot/+/10189 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi> Autosubmit: grfn <grfn@gws.fyi>
2023-11-28 r/7087 fix(tvix/nar-bridge): drop pathinfoserviceFlorian Klink3-356/+0
This now exists in tvix-store directly, as NixHTTPPathInfoService, and contrary to this version, also validates signatures. Change-Id: Ib6ca161e40d627b7d9741839fc849f2392f422da Reviewed-on: https://cl.tvl.fyi/c/depot/+/10155 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-11-28 r/7086 feat(tvix/store/pathinfosvc/nix_http): allow configuring pubkeysFlorian Klink1-6/+28
This allows setitng the trusted-public-keys URL parameter to a (whitespace-separated) list of public keys. NARInfo files retrieved need to contain a valid signature. Change-Id: Ifd6580b723cbae3182e9cadfa54f1ca2b41d6599 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10153 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-11-28 r/7085 feat(tvix/store/pathinfosvc): add signature verificationFlorian Klink1-1/+33
Introduce an Option<Vec<narinfo::PubKey>>, configurable with a `set_public_keys` method. If set, this configures NixHTTPPathInfoService to validate signatures. Change-Id: I157c5e13c41fc9bfd40b0655381fb4cf33900868 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10152 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-11-28 r/7084 feat(nix-compat/narinfo): add PubKey::verify(fp, sig)Florian Klink1-0/+32
This makes it easy for each PubKey to check if a given Signature is correct for a given fingerprint. Change-Id: I56e6211d133f74f390fd1ae3ae799eef12221904 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10151 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-11-28 r/7083 feat(nix-compat/narinfo): add PubKeyFlorian Klink2-0/+119
This represents a ed25519 public key and "name". These are normally passed in the `trusted-public-keys` Nix config option, and consist of a name and base64-encoded ed25519 pubkey, separated by a `:`. Change-Id: I9ab4b3e0e5821805ea6faf2499626630fc5a3f0a Reviewed-on: https://cl.tvl.fyi/c/depot/+/10150 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-11-28 r/7082 refactor(nix-compat/narinfo/signature): rename Error typeFlorian Klink2-9/+7
Just call this Error, we can infer from the package what error this is. Change-Id: I5df25d2873ec739c49c08804f35562c84c222e06 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10149 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-11-28 r/7081 feat(tvix/store/pathinfosvc/nix_http): check Nar{Size,Hash} matchesFlorian Klink1-13/+84
Ensure the initially communicated NarHash/NarSize from the NarInfo matches what we read, and don't return a PathInfo message if there's a mismatch. Also move the buffering layer around a bit. Change-Id: I68c60ecfaf0f9cd5edacea648437ecb0c9729251 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10148 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-11-28 r/7080 fix(tvix/store/fs): don't panic on PathInfoService errorFlorian Klink1-11/+9
An error in the PathInfoService request can appear in case the underlying request returns an error. We shouldn't panic and bork the fuse mount, but instead return an IO error. Change-Id: I2daeae629e1627d06adcd7b82ddb76c50c602212 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10154 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-11-27 r/7076 chore(3p/sources): bump channels & overlays (2023-11-27)Vincent Ambo1-2/+2
* picked avrdude from stable channel * removed override for texlive, as the upstream fix is merged * picked awscli2 from stable channel * bump tdlib to 1.8.21 (new minimum for telega.el) * tvix/turbofetch: switch to nixpkgs-native mechanism for CARGO_MANIFEST_LINKS (whatever that is) Change-Id: Ic695721b5ca750b89d21cab7a257e1db682b23c0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10083 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su>
2023-11-27 r/7075 fix(nix-compat/narinfo/signature): validate name fieldFlorian Klink1-2/+15
We should restrict this to alphanumeric mostly, and we definitely don't want newlines. Not entirely sure about the exact additionally allowed characters outside of alphanumeric, but this can always be extended further. Change-Id: I1357e79e553f2df2fa97792889f63f0f35d50ed5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10147 Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-11-27 r/7074 fix(nix-compat/narinfo): don't panic trying to parse signaturesFlorian Klink1-6/+13
BASE64.decode_mut panics if we're passing data that has the wrong size. Do the size check first and error out there. Also update the error, and talk about b64-encoded sizes. Change-Id: I290f80a37d48526a30bf1df9d1d9fe34865008eb Reviewed-on: https://cl.tvl.fyi/c/depot/+/10146 Tested-by: BuildkiteCI Reviewed-by: edef <edef@edef.eu> Autosubmit: flokli <flokli@flokli.de>
2023-11-27 r/7073 refactor(nix-compat): use ed25519_dalek::SIGNATURE_LENGTHFlorian Klink1-8/+9
No need to hardcode magic numbers here, we have a constant for that. Change-Id: I67b671c0c4bb7c3bfb001e9c36499f31873ee717 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10145 Reviewed-by: edef <edef@edef.eu> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-11-25 r/7065 refactor(tvix/eval): use `or_default` helper in entry APIVincent Ambo1-3/+1
This fixes a future clippy lint. Change-Id: Ic830e94ef23595580c1037f10878c76bbb546dd9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10110 Tested-by: BuildkiteCI Reviewed-by: Adam Joseph <adam@westernsemico.com>
2023-11-25 r/7064 fix(tvix): ensure PartialOrd/Ord agree for StorePath & NixStringVincent Ambo2-2/+2
This fixes a *future* clippy lint: https://rust-lang.github.io/rust-clippy/master/index.html#/incorrect_partial_ord_impl_on_ord_type In essence, because the implementation of *both* Ord and PartialOrd implies that ordering is not partial, all results of PartialOrd should simply be those of Ord. This is to avoid subtle bugs in future refactorings. Change-Id: I8fc6694010208752dd47746a2aaaeca0c788d574 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10109 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-11-25 r/7055 refactor(tvix/eval): add ThunkRepr::is_forced()Adam Joseph1-5/+9
Change-Id: I4eab5c81fb82337da06327248845cd2f3a4490d3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10038 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: Adam Joseph <adam@westernsemico.com>
2023-11-25 r/7054 feat(tvix/eval): add Thunk::unwrap_or_clone()Adam Joseph1-1/+34
This commit adds Thunk::unwrap_or_clone(), which uses Rc::try_unwrap() to avoid cloning the Value out of a an Rc which has only one strong reference. Change-Id: Icacefe0c823dcddf046d90c0c5cd5ed59fe976d4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10037 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: Adam Joseph <adam@westernsemico.com> Tested-by: BuildkiteCI
2023-11-24 r/7053 fix(tvix/castore): correctly flag unreachable codesterni1-1/+1
Change-Id: Id09afa4b77c3c70fb5695f253f6df4aa88b61e19 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10113 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-11-24 r/7052 docs(tvix/eval): optimization potential for inherit (from) exprssterni1-0/+19
Change-Id: Ibddaa111a5b7a86c42dbe153ae8e53f9a5601a54 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10112 Tested-by: BuildkiteCI Reviewed-by: Adam Joseph <adam@westernsemico.com>
2023-11-22 r/7051 feat(tvix/store): add as_narinfo() for PathInfoFlorian Klink2-2/+106
This allows seeing a PathInfo as a nix_compat::narinfo::NarInfo<'_>. It doesn't allocate any new data, but the NarInfo<'_> view allows us to access things like signature verification, or rendering out (alternations of this) as strings. Change-Id: Id0d8d7feeb626ee02c3d8a4932f24ace77022619 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10108 Reviewed-by: edef <edef@edef.eu> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-11-22 r/7050 feat(nix-compat/narinfo/signature): add new() constructorFlorian Klink1-0/+5
This is useful when creating a new Signature struct where the individual elements are already parsed. Change-Id: Ie33c66287641951e7a030aaa1e7ff0a86b2628ac Reviewed-on: https://cl.tvl.fyi/c/depot/+/10111 Reviewed-by: edef <edef@edef.eu> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-11-22 r/7049 refactor(tvix/store): impl From<&nar_info::Ca> for nixhash::CAHashFlorian Klink1-0/+62
Change-Id: I637a4cff5a5ca29c4d86e0b76a2f20f8741f5628 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10107 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI
2023-11-22 r/7048 refactor(tvix/store): impl From<nixhash::CAHash> for nar_info::CaFlorian Klink1-16/+25
Change-Id: Iaa68044d3b469f15a932aa3b59548505eaa6b8bb Reviewed-on: https://cl.tvl.fyi/c/depot/+/10106 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI
2023-11-22 r/7047 refactor(tvix/nix-compat): move from_name_and_digest to StorePathRefFlorian Klink2-13/+11
We can simply use .to_owned() on that thing afterwards if we want to construct an owned StorePath. Change-Id: I0f3e2e4434b99ee522f2a7dbfa391e13a987479c Reviewed-on: https://cl.tvl.fyi/c/depot/+/10105 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI
2023-11-22 r/7046 feat(nix-compat/narinfo): drop .drv from Narinfo.deriver fieldFlorian Klink2-16/+16
We always know this needs to end with a .drv, and fail parsing if it doesn't, so there's no need to hang onto these 4 bytes. This will make it much easier to synthesize a NarInfo<'_> later on from a PathInfo proto, because we don't have to make this ".drv" appear out of thin air. Change-Id: Id95e7fd937d7c9a420a39b5a4bab73985640ca3b Reviewed-on: https://cl.tvl.fyi/c/depot/+/10084 Tested-by: BuildkiteCI Reviewed-by: edef <edef@edef.eu> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de>
2023-11-22 r/7045 refactor(tvix/nix-compat): cleanup parse_{ca,hash} and fmt structsFlorian Klink6-97/+83
These were used to format to and parse from strings. Move this to the CAHash and NixHash structs directly, and be explicit in the name about which encoding for digests is used. For output path calculation, nix encodes the nixpaths in hex, but for writing out NARInfos, it's using nixbase32. Change-Id: Ia585a76a3811b2609e7ce259fda66a29403b7e07 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10079 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-11-19 r/7043 feat(tvix/nix-compat/narinfo): add fingerprintFlorian Klink2-0/+64
This adds support to compute the fingerprint string, which is what's ed25519-signed in binary caches. Change-Id: I8947239c609896acfd7261f110450014bedf465a Reviewed-on: https://cl.tvl.fyi/c/depot/+/10080 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-11-19 r/7042 feat(tvix/nix-compat): add narinfo::Signature::verifyFlorian Klink6-211/+1388
This adds support for verifying signatures found in NARInfo files. This still needs to be hooked together with the nix+http[s] backend. Change-Id: Ic1c8ddbdecfb05cefca2492808388b0f7f3f2637 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10081 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-11-19 r/7040 refactor(nix-compat/narinfo): move signature into separate fileFlorian Klink2-52/+59
Change-Id: Ic257475e2afebf059c5317c1cc5b04ba63d5d318 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10078 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-11-19 r/7039 refactor(tvix/nix-compat): move narinfo into separate modFlorian Klink1-2/+4
Change-Id: Id85f979e46946da0345483cbbc6de3dd29c94c63 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10077 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-11-19 r/7038 feat(tvix/store/pathinfoservice): implement NixHTTPPathInfoServiceFlorian Klink6-10/+1043
NixHTTPPathInfoService acts as a bridge in between the Nix HTTP Binary cache protocol provided by Nix binary caches such as cache.nixos.org, and the Tvix Store Model. It implements the [PathInfoService] trait in an interesting way: Every [PathInfoService::get] fetches the .narinfo and referred NAR file, inserting components into a [BlobService] and [DirectoryService], then returning a [PathInfo] struct with the root. Due to this being quite a costly operation, clients are expected to layer this service with store composition, so they're only ingested once. The client is expected to be (indirectly) using the same [BlobService] and [DirectoryService], so able to fetch referred Directories and Blobs. [PathInfoService::put] and [PathInfoService::nar] are not implemented and return an error if called. This behaves very similar to the nar-bridge-pathinfo code in nar-bridge, except it's now in Rust. Change-Id: Ia03d4fed9d0657965d100299af97cd917a03f2f0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10069 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-11-19 r/7037 refactor(tvix/castore/blobservice): rm AsyncBufRead from BlobReaderFlorian Klink1-4/+1
There's no need to already require this to be buffered here. Change-Id: Ib9a11b194e0754d87ab8d2ef0b8cb0f4edc01229 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10074 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-11-19 r/7035 feat(nix-compat/nar/reader): provide passthrough buffered I/Oedef2-18/+80
Allow taking advantage of the buffer of the underlying reader to avoid unnecessary copies of file data. We can't easily implement the methods of BufRead directly, since we have some extra I/O to perform in the final consume() invocation. That could be resolved at the cost of additional bookkeeping, but this will suffice for now. Change-Id: I8100cf0abd79e7469670b8596bd989be5db44a91 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10089 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-11-24T03·20+0000