about summary refs log tree commit diff
path: root/tvix (unfollow)
AgeCommit message (Collapse)AuthorFilesLines
2023-10-12 r/6791 refactor(tvix/store): clippy lintFlorian Klink4-4/+4
Change-Id: I03aa5c6cb0167ff6cb7312d81d1e43116fcb6d70 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9710 Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-10-12 r/6790 refactor(tvix/castore/utils): clippyFlorian Klink1-8/+4
Change-Id: Ib0fa1e1fb90d7f315f943a3f28266c0fb3714a31 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9708 Tested-by: BuildkiteCI Reviewed-by: edef <edef@edef.eu>
2023-10-12 r/6789 feat(tvix/store): enable tls[-roots] feature in tonicFlorian Klink3-5/+564
This teaches tonic to speak https, so `grpc+https` becomes a working connection string. Change-Id: I9c550ea665b829725b545a164cf6f78587bd5213 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9714 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Brian McGee <brian@bmcgee.ie> Tested-by: BuildkiteCI
2023-10-12 r/6788 fix(tvix/store/fs): fix typoedef1-3/+3
Change-Id: I132987dda1073dcc0b30e50e26682d6c8724d988 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9689 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-10-12 r/6787 refactor(tvix/nar-bridge): panic on inval PathInfo.NarInfo.NarSha256Florian Klink4-9/+6
Bump code.tvl.fyi/tvix/store/protos past cl/9649, where Validate() already ensures the NarSha256 has the correct size. Change-Id: I774668822f4d9dbd4dea47dde6e4745dc95e8e7f Reviewed-on: https://cl.tvl.fyi/c/depot/+/9665 Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI
2023-10-11 r/6786 feat(tvix/store): validate nar_sha256Florian Klink2-2/+26
Change-Id: I4c4dcdb75ea7748f2ab01a0bab218596b90b7b58 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9648 Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI
2023-10-11 r/6785 feat(tvix/store/protos): validate NarSha256Florian Klink2-1/+17
Change-Id: I016d1b020b04850f0dca68c2ea96643230dada10 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9649 Reviewed-by: Brian McGee <brian@bmcgee.ie> Tested-by: BuildkiteCI
2023-10-11 r/6784 docs(tvix/nar-bridge): remove redundant commentsFlorian Klink1-2/+0
These really only describe what the code is doing, remove. Pointed out in https://cl.tvl.fyi/c/depot/+/9652/comment/987b3b7f_c7b5e47b/ Change-Id: I512a274295244f96744a489871861fed9523bb37 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9654 Reviewed-by: Brian McGee <brian@bmcgee.ie> Tested-by: BuildkiteCI
2023-10-11 r/6783 refactor(tvix/nar-bridge): use mh.SHA2_256Florian Klink2-2/+5
As correctly mentioned in https://cl.tvl.fyi/c/depot/+/9652/comment/03b9b96e_bbb337fd/, we shouldn't be using these magic constants, but pull them from where they're defined. This already is a dependency of go-nix, and pkg/pathinfosvc/server.go, so no changes in go.mod. Change-Id: I0cc41ce040fcbddf4b6171417bc9b0de55af4991 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9653 Tested-by: BuildkiteCI Reviewed-by: Brian McGee <brian@bmcgee.ie>
2023-10-11 r/6782 docs(tvix/store/protos): explain a bit more what we're doingFlorian Klink1-1/+3
Explain a bit better what we do in the codeblock below, and add a newline to make it clear we don't just document the first branch. Change-Id: Ifb142d9984e6d2cbca648525c10298f2fcbdfedf Reviewed-on: https://cl.tvl.fyi/c/depot/+/9645 Tested-by: BuildkiteCI Reviewed-by: Brian McGee <brian@bmcgee.ie> Autosubmit: flokli <flokli@flokli.de>
2023-10-11 r/6781 feat(tvix/nar-bridge): stop parsing nixbase32 manually, validateFlorian Klink2-10/+24
We have nixhash.FromHashTypeAndDigest now. Also, run Validate() on the PathInfo received from the remote PathInfoService. Change-Id: I14db0d9356c539c084afc9dd712314b56da2587e Reviewed-on: https://cl.tvl.fyi/c/depot/+/9652 Tested-by: BuildkiteCI Reviewed-by: Brian McGee <brian@bmcgee.ie>
2023-10-11 r/6780 refactor(tvix/nar-bridge): have Export return root nodeFlorian Klink10-297/+211
… and nar size / sha256 digest. Instead of producing sparse PathInfo messages when NARs are sent to nar-bridge, the nar-bridge http server now keeps a lookup table (narsha256) -> (rootNode, narSize) This removes a whole bunch of noise, because we don't need to keep sparse fields around. A convenience function `GenPathInfo(rootNode *castorev1pb.Node, narInfo *narinfo.NarInfo)` is added, which is used to produce PathInfo messages, either when receiving a NAR file over http and uploading it to a remote PathInfoService, or to synthesize the PathInfoMessage to return to the client, if nar-bridge is acting as a PathInfoService for a remove Nix HTTP Binary cache. Change-Id: Ibba1ab6238a050816c4fab29cb21ae88877d8613 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9651 Tested-by: BuildkiteCI Reviewed-by: Brian McGee <brian@bmcgee.ie>
2023-10-11 r/6779 refactor(tvix/nar-bridge): pass root node to storev1pb.ExportFlorian Klink5-6/+6
Bumps the go module past cl/9604 and update the consumer side. Change-Id: Id44245017f1dc2f8aac28051cdbb45b83bdc5be3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9650 Reviewed-by: Brian McGee <brian@bmcgee.ie> Tested-by: BuildkiteCI
2023-10-10 r/6778 fix(tvix/castore): use bstr for formatting names in errorsedef4-4/+10
Much friendlier than either Base64 or raw byte slices. Change-Id: I9b4cdd57c83ddc76c0be8103da4320207657a72b Reviewed-on: https://cl.tvl.fyi/c/depot/+/9622 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-10-10 r/6777 fix(tvix/castore): handle Directory::size overflow explicitlyedef2-9/+90
We use checked arithmetic for computing the total size, and verify that size is in-bounds in Directory::validate. If an out-of-bounds size makes it to the "unchecked" size method, we either panic (in debug mode), or silently saturate to u32::MAX. No new panic sites are added, since overflows in debug mode already panic at the language level. Change-Id: I95b8c066a42614fa447f08b4f8fe74e16fbe8bf9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9616 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-10-10 r/6776 test(tvix/castore): demonstrate that Directory::size may panicedef1-0/+31
Change-Id: Iebfa12df51c8c9d39b4d9eb2fccc6ffa312639cd Reviewed-on: https://cl.tvl.fyi/c/depot/+/9620 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-10-10 r/6775 fix(third_party/crate2nix): run tests in debug modeedef3-0/+18
Behave like `cargo test` rather than `cargo test --release`. Change-Id: Ie013d04ac68d7dec2a3b870fa9f0060a70a9635d Reviewed-on: https://cl.tvl.fyi/c/depot/+/9621 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-10-10 r/6774 refactor(tvix/store/protos): have Export accept root nodeFlorian Klink2-43/+34
We don't need the full PathInfo message, only the root node. Change-Id: I667045ed766875dfbf8ac126a50b02baa2df67a4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9604 Tested-by: BuildkiteCI Reviewed-by: edef <edef@edef.eu>
2023-10-10 r/6773 chore(tvix/nar-bridge): move to Export from storev1pbFlorian Klink8-335/+49
This removes the Export method in nar-bridge, and updates all users to the version now in storev1pb. It moves the roundtrip test to the importer crate, and some of the utility functions into a separate util_test.go file. Change-Id: I81d9e0b35dfd78ef1042bed307281eecd2aaa2a8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9603 Reviewed-by: Brian McGee <brian@bmcgee.ie> Tested-by: BuildkiteCI
2023-10-10 r/6772 feat(tvix/store/protos): add ExportFlorian Klink10-79/+387
Export will traverse a given PathInfo structure, and write the contents in NAR format to the passed Writer. It uses directoryLookupFn and blobLookupFn to resolve references. This is being moved over from nar-bridge. We need to keep the code there around until we can bump go.mod to storev1 with this merged, but the tests can already be moved entirely. Change-Id: Ie0de3077b09344cafa00ff1e2ddb8b52e9e631bc Reviewed-on: https://cl.tvl.fyi/c/depot/+/9602 Tested-by: BuildkiteCI Reviewed-by: Brian McGee <brian@bmcgee.ie> Autosubmit: flokli <flokli@flokli.de>
2023-10-10 r/6771 refactor(tvix/nar-bridge): use castorev1pb.RenamedNodeFlorian Klink4-100/+35
We can use the helper to rename the node. Change-Id: Id8defea7e5ebbd43d7b7a9b2992c62084e1828ec Reviewed-on: https://cl.tvl.fyi/c/depot/+/9601 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Brian McGee <brian@bmcgee.ie> Tested-by: BuildkiteCI
2023-10-10 r/6770 chore(tvix/nar-bridge): bump google.golang.org/protobufFlorian Klink3-4/+4
… to 1.28.1 as well. Change-Id: Ieaaf9363a8dd63c2437c34d78a0ab369e7492f19 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9600 Reviewed-by: Brian McGee <brian@bmcgee.ie> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-10-10 r/6769 feat(tvix/store): use tvix_compat::nar::writer::asyncedef5-72/+156
Change-Id: Iad36872244df6f2225a2884f6b20cacd8f918b31 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9619 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: edef <edef@edef.eu>
2023-10-10 r/6768 feat(tvix/nix-compat): add nar::writer::asyncedef5-2/+264
Change-Id: Ib962f889836659fd278beec461ee329b00e64ea5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9615 Autosubmit: edef <edef@edef.eu> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-10-10 r/6767 refactor(tvix/nix-compat): move nar::writer to nar::writer::syncedef2-223/+227
This is preparation for adding an async port. Change-Id: Id638ec1f6f46e2f3935448184eed51e2233263fe Reviewed-on: https://cl.tvl.fyi/c/depot/+/9618 Tested-by: BuildkiteCI Autosubmit: edef <edef@edef.eu> Reviewed-by: flokli <flokli@flokli.de>
2023-10-10 r/6766 fix(tvix/nix-compat): clean up the debug assertions a bitedef1-12/+11
Consistent error messages, and slightly nicer code layout. We avoid printing the input data, since we primarily want to point out the specific violated invariant. In the one place where we do want to, we use BStr's Debug implementation, since byte slices don't print nicely. Change-Id: I3a9a0c37be270ea5f16cf124922c254608fb849e Reviewed-on: https://cl.tvl.fyi/c/depot/+/9617 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: edef <edef@edef.eu>
2023-10-10 r/6765 docs(nix-1p): add section about the merge operatorVincent Ambo1-2/+32
Without the examples, some behaviour of the merge operator might not be clear from the previous description. Due to how pervasively the operator is used, I think it warrants a separate section. This fixes https://github.com/tazjin/nix-1p/issues/16 Change-Id: Iecba5f1cb749bef0a4987b3fc5642832a92c18d5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9599 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-10-10 r/6764 docs(nix-1p): fix formatting of operator tableVincent Ambo1