about summary refs log tree commit diff
path: root/tvix/nix-compat/src/derivation
AgeCommit message (Collapse)AuthorFilesLines
2024-05-02 r/8064 refactor(nix-compat): derivation_or_fod_hash -> hash_derivation_moduloFlorian Klink2-37/+40
There's no need for us to come up with our own names for this. Also update the comments/docstrings a bit, and inline the intermediate hash_derivation_modulo calculation. Change-Id: I09dab8ffe1ebfb6601841e98119eee4ff25d8f39 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11578 Reviewed-by: edef <edef@edef.eu> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-04-19 r/7974 chore(nix-compat/drv): migrate from test_{case,resources} to rstestFlorian Klink1-76/+88
Also, replace the useless read_file helper with std::fs::read. Change-Id: I704ab42d1ffb9089716533125ff390b7441a8591 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11473 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-04-19 r/7973 chore(nix-compat/derivation/parser): migrate from test_case to rstestFlorian Klink1-23/+37
Change-Id: I7c29e7d9e2b9cd66483ed44a9f2b3d291061f81d Reviewed-on: https://cl.tvl.fyi/c/depot/+/11472 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-04-13 r/7890 feat(tvix/nix-compat): implement Serialize, Deserialize for NixHashFlorian Klink1-1/+1
We use the (slightly more tolerant) from_str to deserialize, and serialize out as SRI. Change-Id: If76b0ed2d4e243904f02df34f6c90b976c0bab8c Reviewed-on: https://cl.tvl.fyi/c/depot/+/11393 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-03-14 r/7694 refactor(nix-compat/derivation): emphasize aterm_bytesFlorian Klink1-12/+13
derivation_or_fod_hash constructs ATerm bytes and feeds them to sha256. input_derivations being slightly modified is an implementation detail, so move the BTreeMap construction inline, and have aterm_bytes in a let binding (and feed it to the hash function directly while constructing it). This makes it a bit more understandable what's going on. Change-Id: I2f5cfbd1c964fd39ac731ca39e76cfc168f4c7d7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11147 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: John Ericson <git@johnericson.me>
2024-03-14 r/7691 refactor(nix-compat/store_path): take [u8;32] for outer fingerprintFlorian Klink3-22/+18
The outer fingerprint used for store path calculation is always a sha256 digest. This includes both input and output-addressed store paths. We used a NixHash here, which can also represent other hash types, and that had a bunch of annoyances: - Whenever we had the bytes, we had to wrap them in a NixHash::Sha256(). - Things like AtermWriteable had to be implemented on NixHash, even though we then had an assertion it was only called in the NixHash::Sha256 case. Change-Id: Ic895503d9b071800d2e52ae057666f44bd0ab9d6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11142 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: John Ericson <git@johnericson.me> Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
2024-03-03 r/7633 fix(tvix/nix-compat): Make CAHash deserialize more formatsBrian Olsen1-0/+30
Currently CAHash only deserializes the hash in hex code while the serializer outputs a nixbase32 hash. This means that you can't currently deserialize what has been serialized. This change makes deserialize support any digest format (so hex, nixbase32 and base64) as well as flattens the deserialize code and error handling. It also implements serde methods of HashAlgo directly using Display and TryFrom implementations because otherwise these would get serialized as eg. Sha256 instead of sha256 which also broke CAHash serialize/deserialize. Change-Id: I1941a72eaec741e4956292adaaf0115b97f260ba Reviewed-on: https://cl.tvl.fyi/c/depot/+/11082 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-03-03 r/7632 refactor(tvix/nix-compat): rename NixHash::{digest,hash}Florian Klink2-3/+3
This doesn't return the digest, but the internal NixHash. To get the digest, you use `ca_hash.hash().digest_as_bytes()` (as done in tvix/nix-compat/src/derivation/write.rs). Change-Id: Ib07918dec63ae42ddf8c8d3f1d10510f9c4df255 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11070 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-02-21 r/7585 feat(tvix/nix-compat): Use `StorePath` in `Output`Peter Kolloch7-61/+109
https: //b.tvl.fyi/issues/264 Change-Id: Icb09be9643245cc68d09f01d7723af2d44d6bd1a Reviewed-on: https://cl.tvl.fyi/c/depot/+/11001 Autosubmit: Peter Kolloch <info@eigenvalue.net> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-02-21 r/7584 feat(tvix/nix-compat): input_sources as StorePathPeter Kolloch5-29/+48
https: //b.tvl.fyi/issues/264 Change-Id: I7a235734dc1f8e93e387a04ba369f3b702c6d5b6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10992 Autosubmit: Peter Kolloch <info@eigenvalue.net> Reviewed-by: flokli <flokli@flokli.de> Reviewed-by: Peter Kolloch <info@eigenvalue.net> Tested-by: BuildkiteCI
2024-02-21 r/7583 feat(tvix/nix-compat): input_derivations with StorePathsPeter Kolloch5-47/+48
...in `Derivation`. This is more type-safe and should consume less memory. This also removes some allocations in the potentially hot path of output hash calculation. https: //b.tvl.fyi/issues/264 Change-Id: I6ad7d3cb868dc9f750894d449a6065608ef06e8c Reviewed-on: https://cl.tvl.fyi/c/depot/+/10957 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Peter Kolloch <info@eigenvalue.net> Reviewed-by: Peter Kolloch <info@eigenvalue.net>
2024-02-21 r/7582 feat(tvix/nix-compat): generalize aterm writing for derivationPeter Kolloch2-9/+70
...so that we can also use `StorePath`s in derivation.input_derivations. Towards https://b.tvl.fyi/issues/264 Change-Id: I71d296ca273979c70f277a7f4f88a5f76de3d8be Reviewed-on: https://cl.tvl.fyi/c/depot/+/10973 Reviewed-by: Peter Kolloch <info@eigenvalue.net> Autosubmit: Peter Kolloch <info@eigenvalue.net> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-02-19 r/7559 refactor(tvix/nix-compat): Restrict visibilityPeter Kolloch1-7/+13
Towards https://b.tvl.fyi/issues/264 Change-Id: I323907cb842ed10ddd05b72c285c598b60d213e0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10972 Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Peter Kolloch <info@eigenvalue.net> Tested-by: BuildkiteCI
2024-02-19 r/7556 feat(tvix/nix-compat): Extend parse errorsPeter Kolloch1-1/+4
...so that we can parse (and at the same time validate) store paths directly to `StorePath`s. https: //b.tvl.fyi/issues/264 Change-Id: Ide4cb5403f8e3400f7d2a5838c652ac35dbd1a4f Reviewed-on: https://cl.tvl.fyi/c/depot/+/10968 Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Peter Kolloch <info@eigenvalue.net> Tested-by: BuildkiteCI
2024-02-17 r/7538 refactor(tvix/nix-compat): derivation_or_fod_hash w/o self.clone()Peter Kolloch1-22/+36
Instead of constructing a completely new derivation for hashing, allow to call hashing with substituted input_derivations. This 1. reduces the number of allocations, 2. prepares substituting `String`s for store paths with proper `StorePath`s without needing a separate derivation struct that allows hashes without `/nix/store` in input_derivations, and 3. keeps the change local to the `Derivation` implementation. Change-Id: I36732c78f98fc59f0925b65823773222782017b0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10935 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Peter Kolloch <info@eigenvalue.net>
2024-02-17 r/7536 feat(nix-compat/derivation): make fod_digest publicFlorian Klink1-1/+1
This will be used inside glue, to populate the CA field. Change-Id: I52350d66ad09c90c4e9f3bc6ea44fba4427f1576 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10946 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su>
2024-02-17 r/7532 docs(tvix/nix-compat): on store paths in fieldsPeter Kolloch2-0/+5
Change-Id: Ie78b1c49d7386c70833a3b751799993ad9121aa6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10933 Tested-by: BuildkiteCI Autosubmit: Peter Kolloch <info@eigenvalue.net> Reviewed-by: flokli <flokli@flokli.de>
2024-02-17 r/7531 feat(nix-compat/derivation): simplify derivation_or_fod_hashFlorian Klink1-29/+22
Populate the input_derivations struct field in-place, using self.input_derivations.from_iter, which seems a bit more readable. Change-Id: I4aa12cfd66e08486ef27aedab1bb731d94b63657 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10934 Reviewed-by: flokli <flokli@flokli.de> Reviewed-by: Peter Kolloch <info@eigenvalue.net> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-01-19 r/7422 chore(3p/sources): Bump channels & overlayssterni1-2/+2
- Adjust to ecl 23.9.9 release - Regenerate go protos after protoc-gen-go update - Drop dhall fork which hasn't kept up with 1.42.* - Address new clippy warnings: - Variant naming of Error::ValidationError - Simplify .try_into().unwrap() - Drop unnecessary identity function - Test module must be last in file - Drop unused `pub use` - Update agenix to 0.15.0. Current master has a installCheckPhase that doesn't work with C++ Nix 2.3.*: https://github.com/ryantm/agenix/commit/a23aa271bec82d3e962bafb994595c1c4a62b133#commitcomment-137185861 Change-Id: Ic29eef20d6fd1362ce1031364a5ca6b4edf195bd Reviewed-on: https://cl.tvl.fyi/c/depot/+/10615 Reviewed-by: aspen <root@gws.fyi> Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org>
2024-01-16 r/7389 refactor(nix-compat): use StorePathRef for hash derivation moduloFlorian Klink2-12/+17
Rather than passing strings around, use a StorePathRef. This makes things a bit more typesafe, and more aligned with what we want to do in b/264. Change-Id: Ib7080addf27e7f1a9c8da1d8aaa66744468e3b5a Reviewed-on: https://cl.tvl.fyi/c/depot/+/10633 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-12-21 r/7250 docs(nix-compat/derivation): fix docstringFlorian Klink1-2/+2
Change-Id: I76200aa474b86cf655308f6710a2298b77fa4b38 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10404 Reviewed-by: edef <edef@edef.eu> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-12-21 r/7249 docs(nix-compat/derivation): fix referenceFlorian Klink1-1/+1
Change-Id: I0ebe256c62435901c12ca1895f30e380280e9ea8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10403 Reviewed-by: edef <edef@edef.eu> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-12-21 r/7239 docs(nix-compat/derivation): fix referencesFlorian Klink1-2/+2
Change-Id: If6962931350edfbae206fa739a673e7ccbc3ee89 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10393 Reviewed-by: edef <edef@edef.eu> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-12-21 r/7236 refactor(tvix/nix-compat): have helpers interact with StorePathRefFlorian Klink1-0/+1
In most case, we don't actually need an owned `StorePath` struct, but a `StorePathRef<'_>` is sufficient. The lifetime is only due to it holding onto the name, but that one is mostly left untouched. `Derivation::calculate_derivation_path` still needs to return `StorePath`, as its name has a `.drv` appended. Change-Id: Ie0d52f369d785711bb0658ea2b0bd2617fd9f45e Reviewed-on: https://cl.tvl.fyi/c/depot/+/10389 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: edef <edef@edef.eu>
2023-12-09 r/7135 refactor(nix-compat/derivation): address clippyFlorian Klink1-3/+3
Change-Id: I5d7057347838f733d7dc1331c7b7ef8d27244412 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10228 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de>
2023-12-09 r/7132 refactor(nix-compat/store_path): from_absolute_path to StorePathRefFlorian Klink2-5/+5
The only non-test usage was only checking for the error case, and we can still convert this to an owned StorePath by calling to_owned() on StorePathRef. Change-Id: I9f67a759e580c9c429c96896bcdd295392aa5a2a Reviewed-on: https://cl.tvl.fyi/c/depot/+/10225 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-12-09 r/7131 refactor(nix-compat/derivation): rename input derivation componentsFlorian Klink1-3/+3
Match the naming in parse_input_derivations, call the keys "input_derivation", and the values "output_names". Change-Id: I7d1974819028f8ea543dc3ad78afb803ff9db865 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10224 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-11-22 r/7045 refactor(tvix/nix-compat): cleanup parse_{ca,hash} and fmt structsFlorian Klink2-2/+2
These were used to format to and parse from strings. Move this to the CAHash and NixHash structs directly, and be explicit in the name about which encoding for digests is used. For output path calculation, nix encodes the nixpaths in hex, but for writing out NARInfos, it's using nixbase32. Change-Id: Ia585a76a3811b2609e7ce259fda66a29403b7e07 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10079 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-11-05 r/6956 refactor(tvix/nix-compat): check presence with btree_map's entry APIVincent Ambo1-8/+9
Walking a btree_map twice is more expensive than copying a string, especially because the cloning only happens in the (non-hot) error path. This fixes a clippy lint, so it's related to b/321. Change-Id: I2ccfd0bc46792a45d277f47564e595b87107d8be Reviewed-on: https://cl.tvl.fyi/c/depot/+/9962 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-11-05 r/6955 chore(tvix): fix trivial clippy lintsVincent Ambo2-4/+4
Relates to b/321. Change-Id: I37284f89b186e469eb432e2bbedb37aa125a6ad4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9961 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: tazjin <tazjin@tvl.su>
2023-11-04 r/6939 refactor(tvix/glue): move builtins into separate directoryFlorian Klink1-0/+1
Change-Id: I25b7197458dbfbde8623545dc0a0286eb2744f10 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9911 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-10-27 r/6886 fix(tvix/nix-compat): don't box CAHash::Textedef1-1/+1
Change-Id: I31df3909bc21c9038f9fb831879e60e541242819 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9853 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-10-27 r/6883 refactor(tvix): condense long bytestringsedef1-5/+3
Change-Id: I3bea0827ec2c8db835334ce378a7bf3a39e9b1a3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9849 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-10-23 r/6872 fix(nix-compat/src/derivation/validate): remove breakFlorian Klink1-2/+31
If the output was fixed, we broke out of the for loop too early, before actually validating individual outputs. Change-Id: I2259697dfa2a157764358f6d326a1f7f6610647c Reviewed-on: https://cl.tvl.fyi/c/depot/+/9815 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-10-23 r/6871 refactor(tvix/nix-compat): rename NixHashWithMode -> CAHashFlorian Klink6-57/+103
This specific struct is only used to represent content-addressed paths (in case a Derivation has a fixed-output hash, for example). Rename `Output`'s `hash_with_mode` to `ca_hash`. We now also include `CAHash::Text`, and update the `validate` function of the `Output` struct to reject text hashes there. This allows cleaning up the various output path calculation functions inside nix-compat/src/store_path/utils.rs, as they can now match on the type. `make_type` is renamed to `make_references_string`, `build_regular_ca_path` is renamed to `build_ca_path`, and `build_text_path` has a disclaimer added, because you might not actually want to use it. Change-Id: I674d065f2ed5c804012ddfed56e161ac49d23931 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9814 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-10-16 r/6841 refactor(tvix/nix-compat/derivation): remove some clonesFlorian Klink1-3/+9
Change-Id: Ie464467edfafd2050aa9e5fd0e07ff745cb15df8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9753 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-10-16 r/6834 refactor(nix-compat/derivation): drop non_unicode testFlorian Klink1-81/+1
The previous CLs did already absorb all the logic into the common tests, no need to write this here again. Change-Id: I7ba84ba86d5445ed247e5d11d5e59b7fa815670e Reviewed-on: https://cl.tvl.fyi/c/depot/+/9732 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2023-10-16 r/6833 refactor(nix-compat/derivation): use ATerm for output_paths testFlorian Klink1-5/+13
Due to the lack of a ATerm parser, we were previously loading the JSON fixtures to construct our Derivation structs to run the output path calculations with. However, as we now have a ATerm parser, we can load the ATerm representation directly. This also means we can test the output path calculation for non-UTF8 Derivations. Change-Id: I0e53f41a23566b5ad5f0fed12724e02a10b02707 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9731 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-10-16 r/6832 fix(nix-compat/derivation): handle dupsFlorian Klink2-23/+173
This now properly checks for duplicate output names in input derivation output names, and duplicate input sources. Change-Id: I0053854bfbf504f4f511fb3fe1a77a82b3aa61dd Reviewed-on: https://cl.tvl.fyi/c/depot/+/9738 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-10-16 r/6831 feat(nix-compat/{aterm,derivation}): init parserFlorian Klink7-29/+480
This provides a nom-based parser for Nix derivations in ATerm format, which can be reached via `Derivation::from_aterm_bytes`. Some of the lower-level ATerm primitives are moved into a (new) aterm module, and some more higher-level ones that construct derivation- specific types. Also, move the escape_bytes function into there, this is a generic ATerm thing. Change-Id: I2b03b8a1461c7ea2fcb8640c2fc3d1fa3ea719fb Reviewed-on: https://cl.tvl.fyi/c/depot/+/9730 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-10-16 r/6828 refactor(nix-compat/derivation): fix function nameFlorian Klink2-2/+2
typo: write_enviroment -> write_environment. Change-Id: I1a2dc826c0bf71f6ba97fe41c4d44619b5e6beff Reviewed-on: https://cl.tvl.fyi/c/depot/+/9739 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-10-15 r/6815 refactor(nix-compat/derivation): make write_environment more genericFlorian Klink1-5/+7
We don't actually care if it's a BTreeMap of strings, bstrings or any of that sort. The only thing we want to be able to do is get a reference to the bytes from the keys and values. Change-Id: I21b85811a9ea47fe06afa3108836ef9295e5d89b Reviewed-on: https://cl.tvl.fyi/c/depot/+/9737 Tested-by: BuildkiteCI Reviewed-by: edef <edef@edef.eu>
2023-10-15 r/6814 refactor(nix-compat/derivation): move test fixtures to subdirFlorian Klink21-10/+10
We use test_resources and globbing for some of the test cases, so adding additional files in there will also create new test cases, which we don't always want. Move it down one level to make some more space. Change-Id: I619867dc80a4ced59d45096d0703678663b559cd Reviewed-on: https://cl.tvl.fyi/c/depot/+/9729 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2023-10-14 r/6808 refactor(tvix/nix-compat/derivation): fod_digest return [u8; 32]Florian Klink1-4/+4
This is of known length. Change-Id: Iba48ccc486d5bf9e38ec1a2da6e7b80997d2c6ca Reviewed-on: https://cl.tvl.fyi/c/depot/+/9723 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2023-10-14 r/6807 refactor(tvix/nix-compat): make NixHash an enum with fixed-len bytesFlorian Klink4-26/+19
Less Vec<u8> passed around. Change-Id: Ie153a6bfaa084d7490ffa38634efdf5f3c31a768 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9722 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-08-20 r/6512 refactor(tvix/nix-compat/nixhash): validate digest lengthsFlorian Klink2-10/+19
There was a NixHash::new() before, which didn't perform any validation of the digest length. We had some length validation when parsing nix hashes or SRI hashes, but some places didn't perform validation and/or constructed the struct directly. Replace NixHash::new() with a `impl TryFrom<(HashAlgo, Vec<u8>)> for NixHash`, which does do this validation, and update constructing code to use that, rather than populating structs directly. In some rare cases where we're sure the digest length is correct we still populate the struct manually. Fixes b/291. Change-Id: I7a323c5b18d94de0ec15e391b3e7586df42f4229 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9109 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-08-20 r/6511 refactor(tvix/nix-compat): cargo clippyFlorian Klink1-8/+8
Change-Id: I1c1608a6e75e451940fe1c61dc5ace5f0e7d7752 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9111 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-08-20 r/6494 feat(tvix/nix-compat): don't swallow hash validation errorsRyan Lahfa1-1/+111
Previously, Output deserialization would silence validation errors and provide `None` for `hash_with_mode` as soon as a validation error would happen inside of the `NixHashWithMode` deserialization, e.g. invalid hash length would not provide an validation error but a silent `None` value. This is problematic, we workaround a serde limitation here by writing our own Deserializer. As you can see, we write some boilerplate code unfortunately, as, for example: - `#[serde(fail_if_unparsed_as="Option::is_none")]` is not a thing, otherwise, we could have been able to just bubble up errors in case of "not fully parsed" (and not missing) values. - `From<&serde_json::Value> for serde::de::Unexpected` is not a thing, otherwise, we could just map invalid type errors and reuse the existing types instead of doing extremely bizarre things with `serde::de::Unexpected::Other`, note: this is a not problem for expected, we know what we expect, we don't know what we received in practice. I decided to write a `NixHashWithMode::from_map` which will eat a map deserialized via `serde_json`, so our serde magic is not totally "data model" agnostic. I wanted to go for data model agnosticity and enable maximal performance, e.g. building the structure as the map values are streamed in the Visitor, this is needlessly painful because `Output` and `NixHashWithMode` are in different files and this really makes sense only if we write the full implementation in one file, indeed, otherwise, you end up duplicating the work or having strange coupling. So, for now, we will allocate a full map of the fields inside the `Output`, i.e. if any "unknown field" is in that map, we will deserialize it for no reason. Doing it properly, as I repeat it in the code and to flokli at C3Camp 2023, requires to patch serde upstream IMHO. Change-Id: I46fe6ccb8c390c48d6934fd3e3f02a0dfe59557b Reviewed-on: https://cl.tvl.fyi/c/depot/+/9107 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-08-19 r/6491 refactor(tvix/nix-compat/derivation): cargo clippyFlorian Klink1-3/+2
This `.into_iter()` call is equivalent to `.iter()` and will not consume the `BTreeMap`. Change-Id: Ie26637ebecb0bea5b09c447cc45ed207f8b50913 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9088 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-08-19 r/6487 refactor(tvix/nix-compat/derivation/escape): move to Vec<u8>Florian Klink2-7/+7
We only need bstr::ByteSlice to be able to use replace, it doesn't need to return a BString. Change-Id: I811948436fb89652e880970c2c05356183f3e439 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9084 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-12-01T09·59+0000