about summary refs log tree commit diff
path: root/tvix/nix-compat
AgeCommit message (Collapse)AuthorFilesLines
2023-05-18 r/6161 refactor(tvix/nix-compat): clippyFlorian Klink2-10/+8
Change-Id: Ie5277a5c15d9dfe543ca41fa7c6a1eedf22a9f64 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8593 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-05-16 r/6145 feat(tvix/nix-compat): add StorePath::from_absolute_path_fullFlorian Klink1-2/+75
This allows decomposing a path consisting of a store path AND a suffix into a StorePath, and a PathBuf containing the rest. Change-Id: I81290e2fd804cdc9d1e88c71cb22c0fb882d7936 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8567 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-05-16 r/6144 docs(tvix/nix-compat): update docstringsFlorian Klink1-10/+18
Make it cleaner that StorePath only does encode the first path component after the STORE_DIR prefix. Also, move some of the comments around a bit, so it makes more sense what's using what. Change-Id: Ibb57373a13526e30c58ad561ca50e1336b091d94 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8566 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-05-14 r/6139 refactor(tvix/nix-compat): update expect_err stringsFlorian Klink1-4/+4
Make it more explicit that we expect the from_string calls to fail here. Change-Id: Ib3d46fc0850e364125e3548670ef301eeea2e45c Reviewed-on: https://cl.tvl.fyi/c/depot/+/8565 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-05-10 r/6129 feat(tvix/nix-compat/nar/writer): check for more data in readerFlorian Klink1-1/+14
We already returned UnexpectedEof in case the reader stopped returning bytes too early, but similarly we should also fail if there's still bytes left to be read in the reader passed. We normally use the NAR writer to produce new NAR files, so the readers point to the blobs we actually want to render, and having some data left in there should be an error. If for some reason the reader points to more data than just the blob, the `.take` method can be used to limit it to the (known) size. Change-Id: I9e8fa0a6dd9c794492abb6dc9e55995e619cb3bb Reviewed-on: https://cl.tvl.fyi/c/depot/+/8553 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-04-09 r/6088 refactor(nix-compat): Properly encapsulate store path constructionJohn Ericson7-112/+257
Before there was code scattered about (e.g. text hashing module and derivation output computation) constructing store paths from low level building blocks --- there was some duplication and it was easy to make nonsense store paths. Now, we have roughly the same "safe-ish" ways of constructing them as C++ Nix, and only those are exposed: - Make text hashed content-addressed store paths - Make other content-addressed store paths - Make input-addressed fixed output hashes Change-Id: I122a3ee0802b4f45ae386306b95b698991be89c8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8411 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-03-31 r/6069 refactor(nix-compat): Avoid encoding round tripJohn Ericson1-6/+10
When building store paths we can just construct the thing. Change-Id: Ife5d461d6a440ecbb22f32a86a6d51d212a2035b Reviewed-on: https://cl.tvl.fyi/c/depot/+/8409 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-03-31 r/6068 refactor(nix-compat) Make `nixhash_*` mods hierarchicalJohn Ericson4-4/+5
They can go under `nixhash` Change-Id: Ia15835c57130b66d58f5df80ae9595dceee00941 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8408 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-03-31 r/6067 refactor(tvix/nix-compat): Inline `texthash` modJohn Ericson3-44/+40
It is moved into `store_path::utils` with the other path builders. Change-Id: I3257170e442af5d83bcf79e63fa7387dd914597c Reviewed-on: https://cl.tvl.fyi/c/depot/+/8410 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-03-30 r/6060 refactor(tvix/nix-compat): move build_store_path out of derivationFlorian Klink7-142/+156
This doesn't have anything to do with ATerms, we just happen to be using the aterm representation of a Derivation as contents. Moving this into store_path/utils.rs makes these things much cleaner - Have a build_store_path_from_references function, and a build_store_path_from_fingerprint helper function that makes use of it. build_store_path_from_references is invoked from the derivation module which can be used to calculate the derivation path. In the derivation module, we also invoke build_store_path_from_fingerprint during the output path calculation. Change-Id: Ia8d61a5e8e5d3f396f93593676ed3f5d1a3f1d66 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8367 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-03-30 r/6059 refactor(tvix/nix-compat): use Sha256::new_with_prefixFlorian Klink1-2/+1
Change-Id: I328983008c0680e0448ccff604dbdae312f7988b Reviewed-on: https://cl.tvl.fyi/c/depot/+/8366 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-30 r/6058 refactor(tvix/nix-compat): drop is_derivation in build_store_pathFlorian Klink2-12/+7
This only added a suffix to the input argument, if build_store_path was building the path of a Derivation. As we need to also add the `.drv` suffix to the name we pass into text_hash_string inside calculate_derivation_path, we can simply add the suffix there and drop the parameter from build_store_path. Change-Id: Icd5343dd1458f112b9296b389e81ce2ebdd16a9f Reviewed-on: https://cl.tvl.fyi/c/depot/+/8365 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-03-30 r/6057 refactor(tvix/nix-compat): add text_hash_string functionFlorian Klink4-60/+58
Use it to calculate the text_hash_string, which is then used in the calculate_derivation_path and path_with_references functions. Relates to b/263. Change-Id: I7478825e2a23a11224212fea5e3fd06daa97d5e5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8364 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-03-30 r/6056 refactor(tvix/nix-compat): move HashAlgo to separate fileFlorian Klink3-38/+42
… and keep the pub exports as is. Change-Id: I2ad21660577553395f05b5ba71083626429b0dfc Reviewed-on: https://cl.tvl.fyi/c/depot/+/8363 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de>
2023-03-30 r/6055 refactor(tvix/nix-compat): move NixHashWithMode into separate fileFlorian Klink3-108/+114
… and keep the pub exports as is. Change-Id: I9f89a738c508c478ddba61303c21ea294f01ee9f Reviewed-on: https://cl.tvl.fyi/c/depot/+/8362 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-15 r/6012 refactor(tvix/nix-compat): remove get_fixed_output() funcFlorian Klink1-66/+52
The behaviour of this function is a bit unintuitive, and cl/8310 already inlined the other consumer of it. Rewrite the last consumer of the function, so we can drop it. Change-Id: I59c8486037ce3f777667d1d9e4f4a9316d5a0cb9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8311 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-15 r/6011 refactor(tvix/nix-compat): simplify fingerprint calculationFlorian Klink1-76/+54
Instead of having two very similar match branches for the FOD and non- FOD case, detect the FOD case while looping over all outputs. In the case of anything other than recursive sha256 FODs, the fingerprint and output path calculation is exactly the same. Change-Id: Ieb6995653d008766e595cf29d7cd4fb1334e33dd Reviewed-on: https://cl.tvl.fyi/c/depot/+/8310 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de>
2023-03-15 r/6009 refactor(tvix/nix-compat): -derivation::Hash, +NixHashFlorian Klink6-76/+187
This stops using our own custom Hash structure, which was mostly only used because we had to parse the JSON representation somehow. Since cl/8217, there's a `NixHash` struct, which is better suited to hold this data. Converting the format requires a bit of serde labor though, but that only really matters when interacting with JSON representations (which we mostly don't). Change-Id: Idc5ee511e36e6726c71f66face8300a441b0bf4c Reviewed-on: https://cl.tvl.fyi/c/depot/+/8304 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-14 r/6000 refactor(tvix/nix-compat): rename helper funcFlorian Klink1-3/+3
This only trims the output paths from a Derivation struct, not the output hashes. Change-Id: I9250fec4602ed05bb64540c4a89ddb6fb052be1f Reviewed-on: https://cl.tvl.fyi/c/depot/+/8303 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de>
2023-03-14 r/5999 refactor(tvix/nix-compat): remove manual mapFlorian Klink1-8/+6
Change-Id: I1652e24c9be28112d98683de2d2db51dc46001ed Reviewed-on: https://cl.tvl.fyi/c/depot/+/8302 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-03-14 r/5998 refactor(tvix/nix-compat): rename test nameFlorian Klink1-1/+1
Change-Id: Ia6af8d4b53fbd4d9ea26f0094cd98dde66a91f78 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8301 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-03-14 r/5997 fix(tvix/store/derivation): fix commentFlorian Klink1-1/+1
The digest is encoded in LOWERHEX here, not nixbase32. Change-Id: I8e14742f0135966424073bfdb93ada8cd7bd52e2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8300 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-14 r/5995 refactor(tvix/nix-compat): replace calculate_drv_replacement_strFlorian Klink2-51/+68
Call this function derivation_or_fod_hash, and return a NixHash. This is more in line with how cppnix calls this, and allows using to_nix_hash_string() in some places. Change-Id: Iebf5355f08ed5c9a044844739350f829f874f0ce Reviewed-on: https://cl.tvl.fyi/c/depot/+/8293 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-03-04 r/5882 feat(tvix/nix-compat/derivation): make use of NixPath in some placesFlorian Klink2-6/+14
Some of these strings are actually just the nix hash representation of the hash calculated earlier. There's another one passed around via calculate_drv_replacement_str, but that's left for a followup. Change-Id: Id99a2a926a980d679eb49c34ee6a36bf224699b0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8218 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de>
2023-03-04 r/5881 feat(tvix/nix-compat): add NixHash::{new, to_nix_hash_string}Florian Klink1-1/+14
This provides a way to construct a NixHash struct without parsing strings. Change-Id: I947d96e15e51e72d5b02929cda8c5fc31d81253a Reviewed-on: https://cl.tvl.fyi/c/depot/+/8217 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-04 r/5880 feat(tvix/nix-compat): introduce parsing for Nix Hash stringsFlorian Klink1-178/+245
We ironically didn't add support parsing for the "native" format that Nix uses under the hood. This extends the from_str method to peek at the prefix of the string to determine whether to try decoding as SRI, Nix string, or whether it should be a bare digest. Change-Id: I33efd24968b16f86eff18305b4ca8f112c7131d7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8216 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-03-04 r/5879 refactor(tvix/nix-compat): move digest decoding into helper functionFlorian Klink1-35/+38
This will be used for both Nix hash strings and hash strings without the algo specified. Change-Id: Iedfe5494fba5f2be00614ba0fc38bf659eafd447 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8215 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-04 r/5878 refactor(tvix/nix-compat): address clippy in nixbase32.rs testsFlorian Klink1-1/+1
Change-Id: If8820cba4cf19bf0f7aa27e0d93b70eb7b4ee81f Reviewed-on: https://cl.tvl.fyi/c/depot/+/8221 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-04 r/5877 refactor(tvix/nix-compat): address clippy in derivation/tests/mod.rsFlorian Klink1-2/+2
Change-Id: I3f3f7cb590c900abf8b39533ed73fe9135d58f0b Reviewed-on: https://cl.tvl.fyi/c/depot/+/8220 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-03-04 r/5876 refactor(tvix/nix-compat): address clippy in store_path.rsFlorian Klink1-2/+2
Change-Id: Ib10bd93bbb23696d7048e7ed8e405953db94693f Reviewed-on: https://cl.tvl.fyi/c/depot/+/8219 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-03 r/5875 chore(tvix/nix-compat): include invalid hashes in SRI hash errorVincent Ambo1-3/+3
Change-Id: I379b18cb07da0988f8ce4934976ae7d6566d6bb5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8210 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-02-08 r/5843 feat(tvix/nix-compat/nixhash): support BASE64_NOPAD SRIFlorian Klink1-19/+61
Nix accepts SRI hashes that are missing their padding characters in base64, as seen in https://github.com/NixOS/nixpkgs/blob/7e49471316373c471a3bf4b78c130ebc907ae2d2/pkgs/development/libraries/kerberos/krb5.nix . It only seems to work in the SRI case, not with `sha256` being set to a (nopad) base64 string. Add regression tests for this, and document why we don't want to support *additional* characters afterwards. Reported in https://b.tvl.fyi/issues/252 Change-Id: I9ffc2b417501b426ced1894a9cbf95ff5f0e5159 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8037 Reviewed-by: Alyssa Ross <hi@alyssa.is> Tested-by: BuildkiteCI
2023-02-08 r/5842 refactor(tvix/nix-compat/nixhash): fix test function nameFlorian Klink1-1/+1
Change-Id: I3b089758ce8c01df544064422e56a97a8402513d Reviewed-on: https://cl.tvl.fyi/c/depot/+/8036 Tested-by: BuildkiteCI Reviewed-by: Alyssa Ross <hi@alyssa.is>
2023-02-07 r/5841 fix(tvix): fix InvalidEncodedDigestLength messageAlyssa Ross1-1/+1
Before: error[E997]: invalid output hash: invalid encoded digest length '43' for algo 43 After: error[E997]: invalid output hash: invalid encoded digest length '43' for algo sha256 Change-Id: Icb533839586b47f90b4e8d22083cce397f9f3655 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8035 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-02-01 r/5809 feat(tvix/nix-compat/derivation): Display -> to_aterm_string()Florian Klink2-11/+19
Instead of implementing `std::fmt::Display for Derivation` and relying on the `to_string` method, introduce a `to_aterm_string()` method, which does the same thing, but makes it clearer what we're producing, rather than just calling `to_string()``. Change-Id: I21823de9096a0f2c2eb6f4591e48c1aa9fd94161 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7998 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-02-01 r/5806 feat(tvix/nix-compat): add nixhash moduleFlorian Klink2-0/+359
This module takes care of parsing various hashes and algorithms. It will get used to modify derivation output hashes in the next CL. Change-Id: Idc07c401dbb7510f49883ac02b8379b9a5d930c7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7990 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-02-01 r/5805 refactor(tvix/nix-compat/derivation): remove module inceptionFlorian Klink3-350/+356
The module `src/derivation/derivation.rs` is a sign of module inception. Move the Derivation struct definiton up into `src/derivation/mod.rs`, and some of the helpers in a `util.rs`. Change-Id: Ib24a5f8a27bdd45df8b1fa2b3482a79b33cab8d5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7997 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-02-01 r/5804 refactor(tvix/nix-compat): operator precedence can trip the unwaryFlorian Klink1-1/+1
warning: operator precedence can trip the unwary --> nix-compat/src/nixbase32.rs:41:23 | 41 | c |= ((input[i + 1] as u16) << 8 - j as u16) as u8 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: consider parenthesizing your expression: `(input[i + 1] as u16) << (8 - j as u16)` | = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#precedence = note: `#[warn(clippy::precedence)]` on by default Change-Id: I091071d649abf4ed38f5f4e39a0c5d21a0459bff Reviewed-on: https://cl.tvl.fyi/c/depot/+/7996 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-02-01 r/5803 refactor(tvix/derivation): don't create deref'd immediately refFlorian Klink2-3/+3
clippy says: > This expression creates a reference which is immediately dereferenced > by the compiler Change-Id: Ic2c093b043ebee9ae80912075083107e4d216cf1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7995 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su>
2023-02-01 r/5802 refactor(tvix/nix-compat): remove unneeded returnFlorian Klink1-2/+2
Change-Id: I97b9fb3ad33d2f51baf29486d4eff11f8dedcbab Reviewed-on: https://cl.tvl.fyi/c/depot/+/7994 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de>
2023-02-01 r/5801 refactor(tvix/nix-compat): const is always 'staticFlorian Klink1-1/+1
Change-Id: If33bbf47a6cc376f2c0935ea4331a819223b2f00 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7993 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su>
2023-02-01 r/5799 feat(tvix/cli): implement builtins.placeholderVincent Ambo1-0/+18
This doesn't require any other corresponding handling *yet*, as the actual replacements happen in the builder logic (which we delegate to cppnix at the moment). Change-Id: I034147c933f05ae427c7a8794647132d108d0ede Reviewed-on: https://cl.tvl.fyi/c/depot/+/7972 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-01-31 r/5796 docs(tvix/nix-compat/store_path): fix docstringsFlorian Klink1-2/+2
Change-Id: I19261233e338b406a87fe680675eb4dfd853fe11 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7970 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de>
2023-01-31 r/5793 refactor(tvix/nix-compat): absorb nar writerFlorian Klink4-0/+254
Expose it at `nix_compat::nar::writer`. Change-Id: I98a8acfa6277074f24608d64fb6c0082b41d10f5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7969 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-01-31 r/5792 docs(tvix/nix-compat/derivation): fix docstringsFlorian Klink2-2/+5
Change-Id: I3fd02e62c8a24fa7f27461b17cdd824b060d9bdb Reviewed-on: https://cl.tvl.fyi/c/depot/+/7968 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de>
2023-01-31 r/5791 refactor(tvix/nix-compat): absorb //tvix/derivationFlorian Klink26-0/+1330
Put this in its src/derivation. Change-Id: Ic047ab1c2da555a833ee454e10ef60c77537b617 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7967 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-01-31 r/5788 refactor(tvix): introduce nix-compat crateFlorian Klink5-0/+388
Move nixbase32 and store_path into this. This allows //tvix/cli to not pull in //tvix/store for now. Change-Id: Id3a32867205d95794bc0d33b21d4cb3d9bafd02a Reviewed-on: https://cl.tvl.fyi/c/depot/+/7964 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-11-08T15·59+0000