about summary refs log tree commit diff
path: root/tvix/nix-compat
AgeCommit message (Collapse)AuthorFilesLines
2024-04-08 r/7878 refactor(tvix/nix-compat/wire): mv Bytes{WriterState,PacketPosition}Florian Klink1-24/+25
This is perfectly fine to track the position inside a reader too, so rename it to reflect that. Also make the docstring a bit less write-specific. Change-Id: I831b0a8fe44a2477d4af96fefc692b9aabc378f1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11382 Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-04-08 r/7877 fix(tvix/nix-compat/wire): fix BytesWriter test payloadsFlorian Klink1-4/+4
We actually did construct other payload sizes than the test function was intended to, because I'm too stupid to count. Fix that, and also update a leftover _debug in a test name. Change-Id: I1bff8551f2078afe82f789646b851f2fcff3e82c Reviewed-on: https://cl.tvl.fyi/c/depot/+/11381 Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-04-08 r/7876 refactor(tvix/nix-compat): move handshake function to nix-compatPicnoir1-6/+100
Moving out the handshake route out of tvix-daemon to nix-compat. We're bumping the protocol to version 37. It does not change anything for us. Version 37 introduced a change in the build result serialization. We do not implement that for now. Tested tvix-daemon against Nix via the integration test to make sure we did not break anything in the process. Taking advantage of this refactor to remove the unused anyhow dependency in tvix-daemon. Change-Id: I8a2aad0b6dbb0aaa997446a612a365fd7d70336c Reviewed-on: https://cl.tvl.fyi/c/depot/+/11379 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-04-08 r/7875 refactor(tvix/nix-compat/wire): express magics as u64Florian Klink1-8/+5
This allows using read_u64, write_u64, which is a bit easier to juggle with. Also, update names to align with the nix codebase, which makes it easier to spot both the constant name as well as the value. Leave the ASCII interpretation as a comment afterwards. Change-Id: I0b9ab187acd22807e2785b0722aa4300dab37c51 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11378 Tested-by: BuildkiteCI Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr> Autosubmit: flokli <flokli@flokli.de>
2024-04-08 r/7874 refactor(tvix/nix-compat/wire): move magic bytes to worker_protocolFlorian Klink2-7/+7
`primitive.rs` implements reading and writing primitive (fixed-length) types in the wire format, used in the the nix daemon protocol and NAR format. Move worker-protocol specific magic bytes to worker_protocol.rs (and possibly further split there once needed) Change-Id: If681c01e9460294619f1d000229b81f0ac745810 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11377 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
2024-04-08 r/7873 feat(tvix/nix-compat/wire): have write_bytes accept AsRef<[u8]> payloadsFlorian Klink1-4/+20
This includes String, &str etc. An example testcase with &str is provided. Change-Id: I900186d6ceb52f52bd41ef4596524c1f5b52470b Reviewed-on: https://cl.tvl.fyi/c/depot/+/11376 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
2024-04-08 r/7872 refactor(tvix/nix-compat/wire): simplify write_bytes a bitFlorian Klink2-10/+14
Use the same EMPTY_BYTES trick from BytesWriter to write out the padding, rather than allocating a Vec. Change-Id: Ifb4ba1b45b7388adbc135fc8e46fd3d3cedd30aa Reviewed-on: https://cl.tvl.fyi/c/depot/+/11375 Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-04-08 r/7871 docs(tvix/nix-compat/wire): update docstringsFlorian Klink1-19/+21
Only describe the format once (in `read_bytes`, and simplify the other docstrings a bit. Change-Id: Iff898f3c4173d506a357bc14bdffbf69c4c6e0e0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11374 Tested-by: BuildkiteCI Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr> Autosubmit: flokli <flokli@flokli.de>
2024-04-07 r/7870 refactor(tvix/nix-compat): drop read_bytes_uncheckedFlorian Klink1-15/+19
This isn't used anywhere so far, and if someone really wants to, they can simply make the range open-ended. Change-Id: Iae9bcaa1f7ea032dd3ee76c8c142a38b6b72894d Reviewed-on: https://cl.tvl.fyi/c/depot/+/11373 Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr> Tested-by: BuildkiteCI
2024-04-07 r/7869 fix(nix-compat/store_path): fix Deserialize with borrowFlorian Klink1-3/+37
We were wrongly using `'de` as a lifetime for both `Deserializer` and `StorePathRef`. This prevented Deserializing into a struct containing `StorePathRef`. See https://serde.rs/lifetimes.html#the-deserializede-lifetime, the last part of the paragraph: The 'de lifetime should not appear in the type to which the Deserialize impl applies. - // Do not do this. Sooner or later you will be sad. - impl<'de> Deserialize<'de> for Q<'de> { + // Do this instead. + impl<'de: 'a, 'a> Deserialize<'de> for Q<'a> { This fixes it, and adds a test, deserializing into a `Container` struct. It also fixes the existing test cases, deserialize_ref was actually deserialize_owned, and deserialize_owned didn't exist yet - but they alone are not enough to provoke the lifetime issues. Change-Id: Iaed2832998cae5f192eafe7fd5243e82ff6e051e Reviewed-on: https://cl.tvl.fyi/c/depot/+/11372 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
2024-04-07 r/7867 feat(tvix/nix-compat): read client setting from wirePicnoir2-3/+225
Add the primitives necessary to read the client settings from the Nix daemon wire protocol. Introducing the read_string primitive. This trivial primitive parses a read_bytes call, check the bytes are valid utf-8 bytes and wraps the result in a String. Change-Id: Ie1253523a6bd4e31e7924e9898a0898109da2fa0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11358 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-04-06 r/7857 feat(tvix/nix-compat/wire): introduce BytesWriterFlorian Klink3-1/+524
This deals with writing byte packets of larger sizes to an underlying AsyncWrite. Its constructor receives the expected size. It also deals with writing padding if flush/shutdown is called after writing all the payload. Change-Id: I8acbf992467f3862ffb8c7d669e8c0c8eced14c1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11355 Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr> Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Brian Olsen <me@griff.name> Tested-by: BuildkiteCI
2024-04-04 r/7852 refactor(tvix/nix-compat): make padding_len pub(crate)Florian Klink1-1/+1
Let's make this usable for the entire crate. Change-Id: I754408908a00296ee80dd52680f84b8a7cb22317 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11354 Tested-by: BuildkiteCI Reviewed-by: Brian Olsen <me@griff.name> Autosubmit: flokli <flokli@flokli.de> Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
2024-04-03 r/7846 refactor(tvix/nix-compat): drop read_u32Picnoir1-33/+1
Actually these are all u64 LE encoded on the wire. Change-Id: I5ca22c7639607ac47117cd946e036a444271885a Reviewed-on: https://cl.tvl.fyi/c/depot/+/11348 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-04-03 r/7845 feat(users/picnoir/tvix-daemon): parse up to the operationPicnoir1-0/+2
Using all the primitives recently implemented to nix-compat to reach the point where the Nix client start to send us operation requests. Using a small integration test script (or the VM test, but let's face it, it's too slow to be useful), we manage to reach the point where we're able to read a store operation: 2024-03-21T18:53:27.624876Z INFO tvix_daemon: Incoming connection addr=unix 2024-03-21T18:53:27.625312Z INFO worker:perform_init_handshake: tvix_daemon: Trust sent conn=Connection(unix) conn=Connection(unix) 2024-03-21T18:53:27.625406Z INFO worker: tvix_daemon: Client hanshake succeeded conn=Connection(unix) 2024-03-21T18:53:27.625488Z INFO worker: tvix_daemon: Operation received op=SetOptions conn=Connection(unix) We had to take some shortcuts wrt. stderr/log management. The CPP Nix codebase is a bit confusing in that area. I'll need to spend more time reading this to fully understand what's happening there. For now, sending the STDERR_LAST command to the client does the trick. Change-Id: I9b0e20a52d885e64fe29188496aac5334de61edd Reviewed-on: https://cl.tvl.fyi/c/depot/+/11233 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-04-03 r/7844 feat(tvix/nix-compat): introduce write_worker_trust_levelPicnoir1-0/+22
This is used by the nix client to determine whether or not the daemon trust it. The trust conditions check are daemon-specific, hence not part of nix-compat. Change-Id: Icbcba2f7f1fd58f67e7da72d22a264f5a3f3619d Reviewed-on: https://cl.tvl.fyi/c/depot/+/11231 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-04-03 r/7843 feat(tvix/nix-compat): introduce write_bytesPicnoir1-9/+62
Write counterpart of read_bytes. Despite its name, we mostly use it to write strings (as in ascii strings) to the wire. We also extract the padding calculation in its own function. Change-Id: I8d936e989961107261b3089e4275acbd2c093a7f Reviewed-on: https://cl.tvl.fyi/c/depot/+/11230 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-03-27 r/7785 feat(tvix/nix-compat/store_path): derive Hash for StorePathRefIlan Joselevich1-1/+1
This allows StorePathRef as keys in a hashmap, by deriving Hash. The same is already done for StorePath. Change-Id: I3fc54c45787948116dcb27dfb5dc806b9b505835 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11269 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-03-27 r/7784 feat(tvix/nix-compat): worker protocol operation parserPicnoir3-0/+88
Change-Id: I7776635b17c44534223603d28cf59c7eebd976e0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11229 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-03-23 r/7763 fix(nix-compat/nixhash): fix SRI string parsing with superfluous suffixFlorian Klink1-41/+38
We tried to be more strict than Nix, actually detecting if multiple hashes were specified, or other garbage at the end. However, Nix seems to just chop off at the end, so happily accepts anything afterwards. Example: https://github.com/NixOS/nixpkgs/pull/298041 Example: https://github.com/NixOS/nixpkgs/pull/298052 Change-Id: I2c1a49f51c8f8589a84df2fbf148e67e7380b550 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11234 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-03-22 r/7758 feat(tvix/nix-compat): add read_u32 wire primitivePicnoir1-1/+35
Change-Id: I87a40f79ee7e91bc4fe3dc0ee7818a533c729373 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11225 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-03-19 r/7741 feat(tvix/nix-compat): introduce magic hello/responsePicnoir2-1/+6
Setting the wire module as public to re-use it from tvix-daemon. Change-Id: I570cffc480c7b784d813663f77572bbe9d4e8259 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11197 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-03-15 r/7700 docs(tvix): fix some docstringsFlorian Klink1-2/+2
Change-Id: Ife599387d0472cd746b992bd6755a2fb6a0e0dc4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11158 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-03-15 r/7697 feat(nix-compat/wire): add read_bytes[_unchecked]Florian Klink2-0/+133
This introduces a version reading sized byte packets. Both read_bytes, accepting a range of allowed sizes, as well as read_bytes_unchecked, which doesn't care, are added, including tests. Co-Authored-By: picnoir <picnoir@alternativebit.fr> Change-Id: I9fc1c61eb561105e649eecca832af28badfdaaa8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11150 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr> Tested-by: BuildkiteCI
2024-03-15 r/7696 docs(nix-compat/wire): update docstringsFlorian Klink1-5/+4
These are not streams, but AsyncRead and AsyncWrite. Change-Id: I7d988fa0490800b72862f4f0fcac3dceac70ec26 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11149 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
2024-03-15 r/7695 feat(nix-compat/wire): add low-level wire format primitives codeFlorian Klink4-1/+88
This brings some initial Nix wire format parsing code, used in the nix daemon protocol, remote store/builder protocol, as well as the NAR format itself (note we already have more specialized code for the last one). Thanks to embr, this code already exists, in https://codeberg.org/gorgon/gorgon/src/branch/main/nix-daemon/src/wire.rs, and we can vendor it into here, as EUPL is compatible with GPL (in that direction). The code uses the tokio::io Reader and Writer traits, not the ones from the `futures` crate, as they provide some more convenient `read_u64_le` functions. More application-specific parsing code, as well as code to read strings, or bytes are left out for now, as we want to be be more restrictive w.r.t allowed max sizes, and need to parse bytes, not strings. The code slightly diverges, as we have clippy looped into CI. `Ok(…?)` can be turned into just the inner expression, and some .and_then can be expressed in a simpler fashion. Change-Id: Ie3adcb485e9d66786673b1962a08d4e5df3781d9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11148 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
2024-03-14 r/7694 refactor(nix-compat/derivation): emphasize aterm_bytesFlorian Klink1-12/+13
derivation_or_fod_hash constructs ATerm bytes and feeds them to sha256. input_derivations being slightly modified is an implementation detail, so move the BTreeMap construction inline, and have aterm_bytes in a let binding (and feed it to the hash function directly while constructing it). This makes it a bit more understandable what's going on. Change-Id: I2f5cfbd1c964fd39ac731ca39e76cfc168f4c7d7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11147 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: John Ericson <git@johnericson.me>
2024-03-14 r/7693 refactor(nix-compat/store_path): add from_name_and_digest_fixedFlorian Klink2-5/+17
Allow constructing a StorePath with a fixed-size digest. Change-Id: Id7d0b0152f6c55660a8973a02c84afa9188ce3ba Reviewed-on: https://cl.tvl.fyi/c/depot/+/11144 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: John Ericson <git@johnericson.me> Tested-by: BuildkiteCI
2024-03-14 r/7692 refactor(nix-compat/store_path/utils): move helper function inFlorian Klink1-8/+7
This is only used inside this function, in 2 of the match cases. Change-Id: Ib361f5ee0e3b203802f7d05b9a7f332d14bbcf80 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11143 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: John Ericson <git@johnericson.me> Tested-by: BuildkiteCI
2024-03-14 r/7691 refactor(nix-compat/store_path): take [u8;32] for outer fingerprintFlorian Klink4-42/+37
The outer fingerprint used for store path calculation is always a sha256 digest. This includes both input and output-addressed store paths. We used a NixHash here, which can also represent other hash types, and that had a bunch of annoyances: - Whenever we had the bytes, we had to wrap them in a NixHash::Sha256(). - Things like AtermWriteable had to be implemented on NixHash, even though we then had an assertion it was only called in the NixHash::Sha256 case. Change-Id: Ic895503d9b071800d2e52ae057666f44bd0ab9d6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11142 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: John Ericson <git@johnericson.me> Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
2024-03-11 r/7678 feat(tvix/glue): Implement builtins.fetchurlAspen Smith1-4/+10
Implement the fetchurl builtin, and lay the groundwork for implementing the fetchTarball builtin (which works very similarly, and is implemented using almost the same code in C++ nix). An overview of how this works: 1. First, we check if the store path that *would* result from the download already exists in the store - if it does, we just return that 2. If we need to download the URL, TvixStoreIO has an `http_client: reqwest::Client` field now which we use to make the request 3. As we're downloading the blob, we hash the data incrementally into a SHA256 hasher 4. We compare the hash against the expected hash (if any) and bail out if it doesn't match 5. Finally, we put the blob in the store and return the store path Since the logic is very similar, this commit also implements a *chunk* of `fetchTarball` (though the actual implementation will likely include a refactor to some of the code reuse here). The main thing that's missing here is caching of downloaded blobs when fetchurl is called without a hash - I've opened b/381 to track the TODO there. Adding the `SSL_CERT_FILE` here is necessary to teach reqwest how to load it during tests - see 1c16dee20 (feat(tvix/store): use reqwests' rustls-native-roots feature, 2024-03-03) for more info. Change-Id: I83c4abbc7c0c3bfe92461917e23d6d3430fbf137 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11017 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: aspen <root@gws.fyi>
2024-03-09 r/7658 chore(tvix/nix-compat): bump zstd dependencyFlorian Klink1-1/+1
Otherwise this causes conflicts when adding a more recent zstd version to another crate. Change-Id: I02d1b1a37f18711c0969cb04e761e967fc4655dc Reviewed-on: https://cl.tvl.fyi/c/depot/+/11100 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-03-03 r/7634 feat(tvix/nix-compat): add HashMode and CAHash::mode()Florian Klink2-0/+17
This allows peeking at the mode without matching on all enum kinds of CAHash directly. Change-Id: Icca147a876f7f4cff0bbaa067ac4dae40d05bee9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11069 Reviewed-by: flokli <flokli@flokli.de> Reviewed-by: Brian Olsen <me@griff.name> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-03-03 r/7633 fix(tvix/nix-compat): Make CAHash deserialize more formatsBrian Olsen3-67/+230
Currently CAHash only deserializes the hash in hex code while the serializer outputs a nixbase32 hash. This means that you can't currently deserialize what has been serialized. This change makes deserialize support any digest format (so hex, nixbase32 and base64) as well as flattens the deserialize code and error handling. It also implements serde methods of HashAlgo directly using Display and TryFrom implementations because otherwise these would get serialized as eg. Sha256 instead of sha256 which also broke CAHash serialize/deserialize. Change-Id: I1941a72eaec741e4956292adaaf0115b97f260ba Reviewed-on: https://cl.tvl.fyi/c/depot/+/11082 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-03-03 r/7632 refactor(tvix/nix-compat): rename NixHash::{digest,hash}Florian Klink3-4/+4
This doesn't return the digest, but the internal NixHash. To get the digest, you use `ca_hash.hash().digest_as_bytes()` (as done in tvix/nix-compat/src/derivation/write.rs). Change-Id: Ib07918dec63ae42ddf8c8d3f1d10510f9c4df255 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11070 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-02-22 r/7594 refactor(nix-compat/store_path): simplify build_ca_pathFlorian Klink1-14/+12
Move the the `fixed:out:[r:]{}:` generation to a helper function, use matches! for more clarity. Change-Id: I4e930c42aacbf5c7451d1f8c8c80ccb4c45389f0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11006 Tested-by: BuildkiteCI Reviewed-by: aspen <root@gws.fyi> Autosubmit: flokli <flokli@flokli.de>
2024-02-21 r/7585 feat(tvix/nix-compat): Use `StorePath` in `Output`Peter Kolloch7-61/+109
https: //b.tvl.fyi/issues/264 Change-Id: Icb09be9643245cc68d09f01d7723af2d44d6bd1a Reviewed-on: https://cl.tvl.fyi/c/depot/+/11001 Autosubmit: Peter Kolloch <info@eigenvalue.net> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-02-21 r/7584 feat(tvix/nix-compat): input_sources as StorePathPeter Kolloch5-29/+48
https: //b.tvl.fyi/issues/264 Change-Id: I7a235734dc1f8e93e387a04ba369f3b702c6d5b6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10992 Autosubmit: Peter Kolloch <info@eigenvalue.net> Reviewed-by: flokli <flokli@flokli.de> Reviewed-by: Peter Kolloch <info@eigenvalue.net> Tested-by: BuildkiteCI
2024-02-21 r/7583 feat(tvix/nix-compat): input_derivations with StorePathsPeter Kolloch6-57/+117
...in `Derivation`. This is more type-safe and should consume less memory. This also removes some allocations in the potentially hot path of output hash calculation. https: //b.tvl.fyi/issues/264 Change-Id: I6ad7d3cb868dc9f750894d449a6065608ef06e8c Reviewed-on: https://cl.tvl.fyi/c/depot/+/10957 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Peter Kolloch <info@eigenvalue.net> Reviewed-by: Peter Kolloch <info@eigenvalue.net>
2024-02-21 r/7582 feat(tvix/nix-compat): generalize aterm writing for derivationPeter Kolloch3-9/+100
...so that we can also use `StorePath`s in derivation.input_derivations. Towards https://b.tvl.fyi/issues/264 Change-Id: I71d296ca273979c70f277a7f4f88a5f76de3d8be Reviewed-on: https://cl.tvl.fyi/c/depot/+/10973 Reviewed-by: Peter Kolloch <info@eigenvalue.net> Autosubmit: Peter Kolloch <info@eigenvalue.net> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-02-19 r/7560 feat(tvix/nix-compat): Extract to_plain_hex_stringPeter Kolloch1-5/+6
Towards https://b.tvl.fyi/issues/264 Change-Id: Ibde971bfb6baa97b5c678d84ce1941189bc59f6f Reviewed-on: https://cl.tvl.fyi/c/depot/+/10969 Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Peter Kolloch <info@eigenvalue.net> Tested-by: BuildkiteCI
2024-02-19 r/7559 refactor(tvix/nix-compat): Restrict visibilityPeter Kolloch1-7/+13
Towards https://b.tvl.fyi/issues/264 Change-Id: I323907cb842ed10ddd05b72c285c598b60d213e0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10972 Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Peter Kolloch <info@eigenvalue.net> Tested-by: BuildkiteCI
2024-02-19 r/7556 feat(tvix/nix-compat): Extend parse errorsPeter Kolloch1-1/+4
...so that we can parse (and at the same time validate) store paths directly to `StorePath`s. https: //b.tvl.fyi/issues/264 Change-Id: Ide4cb5403f8e3400f7d2a5838c652ac35dbd1a4f Reviewed-on: https://cl.tvl.fyi/c/depot/+/10968 Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Peter Kolloch <info@eigenvalue.net> Tested-by: BuildkiteCI
2024-02-17 r/7544 feat(tvix/nix-compat): serde for StorePath[Ref]sPeter Kolloch1-0/+94
Necessary, if we want to use it inside of `Derivation` etc. Change-Id: I8888060417b2ee83ac52d7ec3e7b27c393271d8b Reviewed-on: https://cl.tvl.fyi/c/depot/+/10947 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: Peter Kolloch <info@eigenvalue.net> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-02-17 r/7538 refactor(tvix/nix-compat): derivation_or_fod_hash w/o self.clone()Peter Kolloch1-22/+36
Instead of constructing a completely new derivation for hashing, allow to call hashing with substituted input_derivations. This 1. reduces the number of allocations, 2. prepares substituting `String`s for store paths with proper `StorePath`s without needing a separate derivation struct that allows hashes without `/nix/store` in input_derivations, and 3. keeps the change local to the `Derivation` implementation. Change-Id: I36732c78f98fc59f0925b65823773222782017b0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10935 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Peter Kolloch <info@eigenvalue.net>
2024-02-17 r/7536 feat(nix-compat/derivation): make fod_digest publicFlorian Klink1-1/+1
This will be used inside glue, to populate the CA field. Change-Id: I52350d66ad09c90c4e9f3bc6ea44fba4427f1576 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10946 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su>
2024-02-17 r/7532 docs(tvix/nix-compat): on store paths in fieldsPeter Kolloch2-0/+5
Change-Id: Ie78b1c49d7386c70833a3b751799993ad9121aa6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10933 Tested-by: BuildkiteCI Autosubmit: Peter Kolloch <info@eigenvalue.net> Reviewed-by: flokli <flokli@flokli.de>
2024-02-17 r/7531 feat(nix-compat/derivation): simplify derivation_or_fod_hashFlorian Klink1-29/+22
Populate the input_derivations struct field in-place, using self.input_derivations.from_iter, which seems a bit more readable. Change-Id: I4aa12cfd66e08486ef27aedab1bb731d94b63657 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10934 Reviewed-by: flokli <flokli@flokli.de> Reviewed-by: Peter Kolloch <info@eigenvalue.net> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-01-20 r/7429 fix(nix-compat): accept SRI hashes of invalid lengthRyan Lahfa1-2/+36
In cl/10468, we accepted SRI hashes of invalid padding while checking their trailing bits. In this commit, we accept SRI hashes of invalid padding and invalid length, as Nix does. Real world example: `pkgs.javaPackages.openjfx11.deps` <https://github.com/NixOS/nixpkgs/blob/849e4dc5ff0ae70fb7a0df19dbbf633e408c0f32/pkgs/development/compilers/openjdk/openjfx/11.nix#L71> in nixpkgs. Change-Id: I834437e7b94dab9fbb030163f7a2741f52bbf03a Reviewed-on: https://cl.tvl.fyi/c/depot/+/10668 Autosubmit: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-01-19 r/7422 chore(3p/sources): Bump channels & overlayssterni2-32/+32
- Adjust to ecl 23.9.9 release - Regenerate go protos after protoc-gen-go update - Drop dhall fork which hasn't kept up with 1.42.* - Address new clippy warnings: - Variant naming of Error::ValidationError - Simplify .try_into().unwrap() - Drop unnecessary identity function - Test module must be last in file - Drop unused `pub use` - Update agenix to 0.15.0. Current master has a installCheckPhase that doesn't work with C++ Nix 2.3.*: https://github.com/ryantm/agenix/commit/a23aa271bec82d3e962bafb994595c1c4a62b133#commitcomment-137185861 Change-Id: Ic29eef20d6fd1362ce1031364a5ca6b4edf195bd Reviewed-on: https://cl.tvl.fyi/c/depot/+/10615 Reviewed-by: aspen <root@gws.fyi> Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org>