about summary refs log tree commit diff
path: root/tvix/derivation
AgeCommit message (Collapse)AuthorFilesLines
2023-01-31 r/5791 refactor(tvix/nix-compat): absorb //tvix/derivationFlorian Klink26-1351/+0
Put this in its src/derivation. Change-Id: Ic047ab1c2da555a833ee454e10ef60c77537b617 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7967 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-01-31 r/5788 refactor(tvix): introduce nix-compat crateFlorian Klink6-7/+7
Move nixbase32 and store_path into this. This allows //tvix/cli to not pull in //tvix/store for now. Change-Id: Id3a32867205d95794bc0d33b21d4cb3d9bafd02a Reviewed-on: https://cl.tvl.fyi/c/depot/+/7964 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-30 r/5781 fix(tvix/store/nixbase32): fix encoder/decoderFlorian Klink3-7/+7
Replace our data_encoding usage with the implementation taken from https://github.com/nix-community/go-nix/tree/master/pkg/nixbase32 Also uncomment some of the unit tests, and add a regression test for a NIXBASE32.encode with a 32 bytes sequence. The previous implementation of NIXBASE32.encode is wrong in that case: ``` ❯ nix hash to-base32 sha256-s6JN6XqP28g1uYMxaVAQMLiXcDG8tUs7OsE3QPhGqzA= 0c5b8vw40dy178xlpddw65q9gf1h2186jcc3p4swinwggbllv8mk ❯ echo -n s6JN6XqP28g1uYMxaVAQMLiXcDG8tUs7OsE3QPhGqzA= | base64 -d | hexdump -C 00000000 b3 a2 4d e9 7a 8f db c8 35 b9 83 31 69 50 10 30 |..M.z...5..1iP.0| 00000010 b8 97 70 31 bc b5 4b 3b 3a c1 37 40 f8 46 ab 30 |..p1..K;:.7@.F.0| 00000020 ``` Change-Id: I0468b62bbbab390f8d7d3812e657e5d59dceed59 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7934 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Adam Joseph <adam@westernsemico.com> Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-27 r/5771 feat(tvix/derivation): derive PartialEq for errorFlorian Klink1-2/+2
Allows easier to use these errors in test cases. Change-Id: I8abee3b522909ed4aa4b066499cc48eececbc036 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7937 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-01-26 r/5762 refactor(tvix/derivation): align error messages with rust styleFlorian Klink1-13/+13
Change-Id: I9ccd4c043bdddefee98a2c0b3d6eb7d9cb53c454 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7935 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-01-23 r/5740 refactor(tvix/derivation): relax bounds for reference iteratorVincent Ambo2-4/+4
This makes it easier to pass owned strings if that's what we have. Change-Id: Ia7351ff2681292d16534ec50fe60b926b683bb9a Reviewed-on: https://cl.tvl.fyi/c/depot/+/7908 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-01-23 r/5739 refactor(tvix/derivation): remove DOT_FILE_EXT constFlorian Klink3-13/+3
This is used in few enough places to just inline it. It felt a bit alien in the ATerm construction aswell. write.rs now pleasantly only includes tokens that occur in the ATerm representation. Change-Id: I524f8d6c1ce9057ff7fd16c6c3efd98467040a44 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7911 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-01-23 r/5738 refactor(tvix/derivation): pass fingerprint to build_store_pathFlorian Klink2-66/+51
All of these occurences are actually sha256 digests of some fingerprint - which means there's not a lot of need to do the hashing outside. Instead of passing in a digest, keep the sha256 hasher in the function, and pass in a fingerprint. This makes it much easier to construct fingerprints using format!() in all consumers, because we need don't need to juggle with the hasher anymore. Change-Id: I2dc3af2cab6cf06f55ae6cbd9a8be95faf2a07b6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7907 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-23 r/5737 feat(tvix/derivation): add `path_with_references`Vincent Ambo3-1/+63
This allows the calculation of a store path for a plain string that potentially contains references. These paths are used for `builtins.toFile` (and potentially other features of C++ Nix). Change-Id: Ic507c7f264f362b5e6e628255869e5a4fbe4d788 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7906 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-01-23 r/5736 refactor(tvix/derivation): minor structure simplificationVincent Ambo4-18/+27
Fixes some module comments and embeds the `compress_hash` function in the `derivation` module, as it was not used outside of this module anyways. Change-Id: I6c5c92b3f0c03c2cdcbcfc2f813909a968c4d44c Reviewed-on: https://cl.tvl.fyi/c/depot/+/7905 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-01-18 r/5701 feat(tvix/derivation): more checks for output hashes and encodingFlorian Klink3-2/+28
Change-Id: Idebad60c3bf9daf94d04a36bb73ac0dd767f9e79 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7856 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-01-18 r/5700 feat(tvix/derivation): add validate_output_paths flag for validateFlorian Klink3-6/+12
This allows calling validate() on a derivation that doesn't have its output paths already calculated yet. It allows offloading some of the error checking in builtins.derivation* to be offloaded to that function. Change-Id: Ib4aeadc0eb6583ef8cd765f33e9a9ec32be62729 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7848 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-01-18 r/5699 feat(tvix/derivation): check for other invalid output namesFlorian Klink1-6/+11
This uses the exposed StorePath::validate_name method to check for other invalid output names (for which it would not be possible to construct a store path of). Change-Id: Ia3f65e19a07ef164f9f64013a5f37cbac99eb8e0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7855 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-01-18 r/5696 refactor(tvix/derivation): use DerivationError in Output::validateFlorian Klink4-12/+18
Change-Id: I7dbd3b8ff9ef92acddde2e579fb24b8311c34d8f Reviewed-on: https://cl.tvl.fyi/c/depot/+/7852 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-18 r/5695 refactor(tvix/derivation): rename {ValidateDerivation,}ErrorFlorian Klink4-34/+23
This is now used in more than just validate(). Change-Id: I69c3ad6cb5f3ad60a636fe2ea05d432aebe8e53b Reviewed-on: https://cl.tvl.fyi/c/depot/+/7851 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-01-18 r/5694 feat(tvix/derivation): also fail if output name is called `drv`Florian Klink3-10/+32
`drv` is an invalid output name too, as this would cause a `builtins.derivation` call to return an attrset with a `drvPath` key (which already exists) and has a different meaning. Also handle errors during store path construction, and return our own error type, instead of the ParseStorePathError. Change-Id: Ib7952dde1d5cf18a0e210928df7c57b5939b7678 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7850 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de>
2023-01-16 r/5668 refactor(tvix/derivation): expose proper ValidateDerivationErrorFlorian Klink5-30/+104
Use proper errors, instead of anyhow. Change-Id: I6db14c72a6319b389b0136aac7b84f50a30fb366 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7847 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-16 r/5667 feat(tvix/derivation): make input_sources a BTreeSetFlorian Klink3-14/+5
These need to be sorted anyways, so let's use the correct data structure for it. Change-Id: I009c9989d7647dc1df716170f3680c981db6e4b2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7846 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-16 r/5666 chore(tvix/derivation): drop TODOs related to stripping outputsFlorian Klink1-2/+0
Different to the go-nix implementation, we don't have a custom "writing mode" where output paths are omitted from output, but we simply run the serialization with these fields unpopulated (during construction). This means, there's no need for a custom writing mode that strips outputs, so we can drop the comments. Change-Id: Ic0aafd4e34e0294603490cfce2b1aef4085ff34b Reviewed-on: https://cl.tvl.fyi/c/depot/+/7845 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de>
2023-01-13 r/5657 refactor(tvix/derivation): use BTreeSet for derivation outputsVincent Ambo4-16/+13
When constructing derivations inside builtins.derivationStrict, we'd have to very frequently check whether certain outputs have already been inserted into the derivation inputs. Using a set type is much easier, especially as this has to be ordered and the source data that is being inserted also comes from a set, which might let us pass this more efficiently in the future. Note that the validate function no longer checks the order of the entries, as that is now guaranteed by the type. Change-Id: I2fbb984facba3e668075f6f8df8992092368c63d Reviewed-on: https://cl.tvl.fyi/c/depot/+/7826 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-01-09 r/5635 chore(tvix/derivation): expose necessary types in public APIVincent Ambo1-0/+5
Change-Id: Ia05b8eb3054a39897d3ec5783daecfe4bcf054e7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7779 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su>
2023-01-07 r/5623 feat(tvix/derivation): derive Default for DerivationFlorian Klink2-32/+21
Some of the fields in a Derivation struct stay empty, and manually creating BTreeMap or vec for it is annoying. Derive Default instead, so we can use the defaults instead of writing more by hand. Change-Id: I5d41b4b55c8187cb101eb4266451a470008e0067 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7788 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-01-06 r/5617 refactor(tvix/derivation): use StorePath::to_absolute_path()Florian Klink1-7/+3
Instead of concatenating STORE_DIR manually, use StorePath::to_absolute_path() that does it for us. Change-Id: Ia288851a05b4e339679db268f3dd7924e7b65586 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7782 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-01-06 r/5616 fix(tvix/derivation): fix build after StorePath renameVincent Ambo6-17/+18
This project was not previously covered by CI (fixed in this commit), so we didn't catch breakage due to a renamed module. This was noticed while rebasing a CL that has a dependency on this crate in its Nix build. Change-Id: Ic48570b9313e5f73e14daab50cf7ea70918c94d1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7778 Reviewed-by: flokli <flokli@flokli.de> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-01-06 r/5613 refactor(tvix/store): rename NixPath to StorePathFlorian Klink4-15/+15
As discussed in #tvl, this is a more common term for it. Change-Id: I9b904222b8c076f82192c9b7f0b42be171614ab7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7776 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-01-06 r/5612 test(tvix/derivation): add output_path_construction testFlorian Klink1-1/+156
This exercises the output path calculation functions like a constructing client (an implementation of builtins.derivation) would do. It first assembles the bar derivation, does the output path calculation on it, then continues with the foo derivation. The code ensures the resulting Derivations match our fixtures. Change-Id: If93f89c6622fac1c1941085083931b6f657c04bc Reviewed-on: https://cl.tvl.fyi/c/depot/+/7775 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-01-06 r/5610 feat(tvix/derivation): implement output path calculationFlorian Klink2-2/+262
This implement output path calculation for fixed outputs, both fixed- output and non-fixed-output. Change-Id: I0a77b99f2ba6b39467cc5dd589ce152a40387f9a Reviewed-on: https://cl.tvl.fyi/c/depot/+/7761 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: jrhahn <mail.jhahn@gmail.com>
2023-01-06 r/5609 feat(tvix/derivation): add get_fixed_output() helper functionFlorian Klink1-0/+21
This will return the fixed output of a derivation (and its hash), or None if the Derivation is not fixed-output. It will simplify the logic in the output path calculation a bit. Change-Id: I1066cc18ee4fc419421a8c5995c93ba91b35588f Reviewed-on: https://cl.tvl.fyi/c/depot/+/7760 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-06 r/5608 refactor(derivation): return NixPath in calculate_derivation_pathFlorian Klink2-17/+34
This moves all the hash compression logic into a common helper function. Also update the docstring, which said "path" here, which could have been confused with output paths. Change-Id: Iedfb59aeb24f7638afac669dcd18d57b6cfaece2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7759 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-01-04 r/5592 feat(tvix/derivation): derive Clone, Debug, Eq and PartialEqFlorian Klink2-3/+3
This allows juggling with Derivation structs in unit tests, and makes it very easy to compare them for equality. Change-Id: I1faf2ec1eefb1a40fcee3c29f04ec47d29f22691 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7758 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-04 r/5591 feat(tvix/derivation): implement Derivation::validate()Florian Klink5-2/+123
Change-Id: I87dfadda872439e108e5f678a5da63dd5b1915d1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7732 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-01-04 r/5590 chore(tvix/derivation): align json serialization with nix outputFlorian Klink11-103/+113
Use the #[serde(rename = "…")] field attributes to match the field names that Nix uses in its JSON output (nix show-derivation). This allows us to just re-use the exact same fixtures from go-nix, without manual post-massaging. Change-Id: Ifd5c08e43cd4f50d5e02903eccd8cb37230b70a6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7757 Reviewed-by: jrhahn <mail.jhahn@gmail.com> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-04 r/5589 refactor(tvix/derivation): make output hashes an Option<Hash>Florian Klink2-29/+35
This conveys better if an output of a Derivation is fixed-output or not, and provides a Hash struct that can be used to store the algo and digest. In case it's not, this can simply be None. The serde field attributes have been updated to still accept the same JSON. We currently still store the hash algo and digest as strings, mostly because the only thing populating it so far is the example JSONs. We might want to update this, once actual Nix code populates this. While updating write.rs, I pushed some of the Vec<String> to [&str] conversions inline, and made it a Vec<&str>, because it was annoying to juggle with. Change-Id: Ia9cd0568fe179ac22a4a636237f22ab4ad92b95b Reviewed-on: https://cl.tvl.fyi/c/depot/+/7746 Tested-by: BuildkiteCI Reviewed-by: jrhahn <mail.jhahn@gmail.com> Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-04 r/5588 feat(tvix/derivation): make struct members publicFlorian Klink1-7/+7
Allow others to peek into values. We should probably still restrict this further at a later point. Change-Id: I2831432038aa87c3c7dcc85af4fa76a4fe0eadff Reviewed-on: https://cl.tvl.fyi/c/depot/+/7745 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-01-04 r/5587 refactor(tvix/derivation): use tvix_store::nixpath::STORE_DIRFlorian Klink2-2/+4
There's now a common constant for this, so we don't need to redefine it here. Change-Id: I48b096f5e623e5fc5bbd355db83b674766935f94 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7753 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-01-04 r/5580 feat(tvix/derivation): add nix drv path generation to DerivationJürgen Hahn6-4/+114
This adds a function to generate the derivation path. The computation is based on the Go implementation. Change-Id: Iae89db4976f5fd9208f0453f73688689a245cd66 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7729 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-01-04 r/5579 docs(feat/derivation): add docstring to formatterJürgen Hahn1-0/+1
This is just to clarify that the formatted result is the ATerm representation Change-Id: I98fd0b1d2daa3cf6fdbe526ae1e0bd100ff62df3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7742 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-01-04 r/5577 feat(tvix/derivation) Add fmt::Display implementation for DerivationJürgen Hahn3-22/+42
This adds the implementation of fmt::Display for Derivation so that we can easily store the formatted content as a string. Internally, we use the serialization function to generate the string. Change-Id: I6caca0d6c1bea3ca44b6c535c5b1d5d66d8413b7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7741 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-01-02 r/5564 refactor(tvix/derivation): refactor the derivation serializationJürgen Hahn6-215/+248
This refactors the code to serialize a derivation. The original code has beed moved to seperate crates for better code structure. Change-Id: I3b1a6b134428fcbc9930c330bced8ec3610cfb4c Reviewed-on: https://cl.tvl.fyi/c/depot/+/7733 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2022-12-29 r/5536 feat(tvix/derivation): serialize Nix DerivationJürgen Hahn19-0/+444
This adds a Derivation structure and allows to write it to a structure that implements std::fmt:Write. The implementation is based on the go-nix version. Change-Id: Ib54e1202b5c67f5d206b21bc109a751e971064cf Reviewed-on: https://cl.tvl.fyi/c/depot/+/7659 Reviewed-by: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-11-02T14·41+0000