about summary refs log tree commit diff
path: root/tvix
AgeCommit message (Collapse)AuthorFilesLines
2023-03-10 r/5912 feat(tvix/store): add directoryserviceFlorian Klink4-0/+182
This adds a DirectoryService trait, and an implementation for it using sled, and one using a HashMap. Change-Id: Ida61524b2ca949e1b3a78089a5aa7d9f9800c8d7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8093 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-03-10 r/5911 feat(tvix/store): add tests for GRPCBlobServiceWrapperFlorian Klink2-0/+211
Change-Id: I0c9a2d887b5b94c43afee97a8a1e701cc9f74f23 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8098 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-03-10 r/5910 feat(tvix/store/blobservice): add GRPCBlobServiceWrapperFlorian Klink3-0/+244
This takes a BlobService and ChunkService in the constructor, and provides a [proto::blob_service_server::BlobService] trait for it. Implementing proto::blob_service_server::BlobService is a lot of surface to cover, and providing this wrapper will make individual implementations taking care of how to store chunks or chunking information much simpler. Change-Id: Ia7b46484fb3ac9104354d496ff2922dca96ff7b9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8092 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-03-10 r/5909 refactor(tvix/store): remove anyhow::Result in directory it testFlorian Klink1-3/+1
There's no `?` being used. Change-Id: Ifbfd88f6056e0064d11458cb335d05b333ec7b6b Reviewed-on: https://cl.tvl.fyi/c/depot/+/8127 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-03-10 r/5908 refactor(tvix/store): remove useless clone in testFlorian Klink1-1/+1
Change-Id: If62887cbc3338110cb7c815ec2ee2160fae04353 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8111 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-03-10 r/5907 chore(tvix/store): move protos into separate mod.rsFlorian Klink6-3/+6
This allows adding more stuff into this namespace, from different files. Also move tests on proto-related code from src/tests to src/proto/tests. Change-Id: I49e066fce90efbc18e16d68f94497b32ed5625c0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8091 Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-03-10 r/5906 feat(tvix/store): add blobreaderFlorian Klink2-0/+386
A BlobReader can be used to read a blob, which might consist out of multiple chunks. Chunks are fetched from a ChunkService. Change-Id: I1806225b0052adaa4a2320b79b744e554e524ee3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8088 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz> Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-07 r/5899 feat(tvix/eval): introduce generators moduleVincent Ambo6-9/+821
This module contains the request/response types for generators requesting actions from the VM. For most of these, an async helper function is added that will be used inside of generator functions to make use of these requests/responses instead of constructing them directly. Change-Id: I1e085f88adaf784a34867957a0e82532d3a83d7c Reviewed-on: https://cl.tvl.fyi/c/depot/+/8148 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-03-07 r/5898 refactor(tvix/eval): merge OpCall & OpTailCallVincent Ambo3-24/+0
As applies are thunked, there was no situation where OpCall could be emitted. In practice, all calls were already tail calls. Change-Id: Id0d441dcdd86f804d7cddd0cc14f589bbfc75e5b Reviewed-on: https://cl.tvl.fyi/c/depot/+/8147 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-03-06 r/5894 feat(tvix/store): add blobserviceFlorian Klink4-0/+146
This adds a BlobService trait, and an implementation for it using sled, and one using a HashMap. Change-Id: Id6bc1b629195d0b26fc503bd7d2dc9e43c41c317 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8087 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-06 r/5893 feat(tvix/store): add chunkserviceFlorian Klink4-0/+148
This adds the simpler ChunkService trait, and an implementation for it using sled, and one using a HashMap. Change-Id: Icb0fdc41b37b44e9e9e4f548d0f4acae1d83b71e Reviewed-on: https://cl.tvl.fyi/c/depot/+/8086 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-03-06 r/5892 feat(tvix/store): impl From<PoisonError> for ErrorFlorian Klink1-0/+7
Change-Id: Ib61e276b45c0102e383a7e7e641172b151369b03 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8207 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-06 r/5891 feat(tvix/store): add errorsFlorian Klink2-0/+15
Change-Id: I7be0a4eaa29840e1a3329bccb791975de22828ef Reviewed-on: https://cl.tvl.fyi/c/depot/+/8085 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-03-05 r/5890 docs(tvix/store): add document describing why we don't use git treesFlorian Klink1-0/+57
This came up recently again, it makes sense to document the reasoning behind the decision. Change-Id: Ic51d5bc7998c70e8b070b6f42877d8e88613935b Reviewed-on: https://cl.tvl.fyi/c/depot/+/8223 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-03-05 r/5889 docs(tvix/store): move link hrefs to bottomFlorian Klink1-2/+3
Change-Id: Id8e32ee4de98fbe48266d417279194cd93e968cb Reviewed-on: https://cl.tvl.fyi/c/depot/+/8222 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-03-04 r/5888 refactor(tvix/eval): remove VM argument from suspended native thunksVincent Ambo3-13/+7
Because they do not use it, and it can not be passed with the coming generator refactoring. Change-Id: I0d96f2357a7ee79cd8a0f401583d4286230d4a6b Reviewed-on: https://cl.tvl.fyi/c/depot/+/8146 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-03-04 r/5887 refactor(tvix/eval): insert storeDir "builtin" in eval startupVincent Ambo2-10/+5
Instead of using a suspended native thunk, calculate and optionally insert the storeDir builtin when the VM is constructed. We already have the IO handle available at this point and can just check whether a storeDir is present, and insert its absolute value as a builtin. Change-Id: If966eee6ff26dc888b6e888e7c46170c0c346b05 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8145 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-03-04 r/5886 feat(tvix/eval): add SharedThunkSetVincent Ambo1-0/+11
This is a ThunkSet wrapped to be shareable, which will be required once ThunkSets are embedded in futures. Change-Id: I5a067b7972ac86e4d354c75ef05c86b2284c1137 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8144 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-03-04 r/5885 fix(tvix/eval): ThunkSet does not need mutable pointersVincent Ambo1-2/+2
Change-Id: Iea248870a0ea5d38cb02ff059c968fbd563570b6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8143 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-03-04 r/5884 chore(tvix/eval): implement From<OrdMap<..>> for NixAttrsVincent Ambo1-0/+6
Again simplifying some code down the line, where bits of code that construct attribute sets already have the final structure available. Change-Id: I0bb7a1daa63298122b51be73d35d695a4f73f8b0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8140 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-03-04 r/5883 refactor(tvix/eval): implement From<Span> for LightSpanVincent Ambo1-0/+6
This simplifies some code down the line. Change-Id: I58dd71e796e11479f44516cf24932f8061843d23 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8139 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-03-04 r/5882 feat(tvix/nix-compat/derivation): make use of NixPath in some placesFlorian Klink2-6/+14
Some of these strings are actually just the nix hash representation of the hash calculated earlier. There's another one passed around via calculate_drv_replacement_str, but that's left for a followup. Change-Id: Id99a2a926a980d679eb49c34ee6a36bf224699b0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8218 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de>
2023-03-04 r/5881 feat(tvix/nix-compat): add NixHash::{new, to_nix_hash_string}Florian Klink1-1/+14
This provides a way to construct a NixHash struct without parsing strings. Change-Id: I947d96e15e51e72d5b02929cda8c5fc31d81253a Reviewed-on: https://cl.tvl.fyi/c/depot/+/8217 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-04 r/5880 feat(tvix/nix-compat): introduce parsing for Nix Hash stringsFlorian Klink1-178/+245
We ironically didn't add support parsing for the "native" format that Nix uses under the hood. This extends the from_str method to peek at the prefix of the string to determine whether to try decoding as SRI, Nix string, or whether it should be a bare digest. Change-Id: I33efd24968b16f86eff18305b4ca8f112c7131d7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8216 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-03-04 r/5879 refactor(tvix/nix-compat): move digest decoding into helper functionFlorian Klink1-35/+38
This will be used for both Nix hash strings and hash strings without the algo specified. Change-Id: Iedfe5494fba5f2be00614ba0fc38bf659eafd447 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8215 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-04 r/5878 refactor(tvix/nix-compat): address clippy in nixbase32.rs testsFlorian Klink1-1/+1
Change-Id: If8820cba4cf19bf0f7aa27e0d93b70eb7b4ee81f Reviewed-on: https://cl.tvl.fyi/c/depot/+/8221 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-04 r/5877 refactor(tvix/nix-compat): address clippy in derivation/tests/mod.rsFlorian Klink1-2/+2
Change-Id: I3f3f7cb590c900abf8b39533ed73fe9135d58f0b Reviewed-on: https://cl.tvl.fyi/c/depot/+/8220 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-03-04 r/5876 refactor(tvix/nix-compat): address clippy in store_path.rsFlorian Klink1-2/+2
Change-Id: Ib10bd93bbb23696d7048e7ed8e405953db94693f Reviewed-on: https://cl.tvl.fyi/c/depot/+/8219 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-03-03 r/5875 chore(tvix/nix-compat): include invalid hashes in SRI hash errorVincent Ambo1-3/+3
Change-Id: I379b18cb07da0988f8ce4934976ae7d6566d6bb5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8210 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-03-03 r/5870 chore(tvix/eval): fix clippy warningsVincent Ambo4-21/+21
Change-Id: I4c02f0104c455ac00a3f299c1fbf75cbb08e8972 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8142 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su>
2023-03-03 r/5869 refactor(tvix/eval): remove useless map callVincent Ambo1-5/+1
Change-Id: Ifb59ef148ea4fab613f2e4efb133c04baafa3a98 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8141 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-03-03 r/5868 refactor(tvix/eval): enhance debug output for bytecode dumpsVincent Ambo2-1/+25
This adds addresses of thunk and closure chunks to the debug output displayed when dumping bytecode. This makes it possible to see in the dump which thunks are referenced by constants in other thunks. Change-Id: I2c98de5227e7cb415666cd3134c947a56979dc80 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8137 Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-02-16 r/5858 chore(tvix/cli): iter instead of into_iter for referencesAaqa Ishtyaq1-1/+1
This CL removes calling into_iter on a reference, as it will not move out it's content into resulting iterator. Change-Id: Ifcc10b7cf33b98453570cbcec3eb82ffaba2ffcb Signed-off-by: Aaqa Ishtyaq <aaqaishtyaq@gmail.com> Reviewed-on: https://cl.tvl.fyi/c/depot/+/8126 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-02-16 r/5857 refactor(tvix/eval): remove redundant cloneAaqa Ishtyaq3-6/+6
This CL removes redundant clone from value which is going to be dropped without further use. Change-Id: Ibd2a724853c5cfbf8ca40bf0b3adf0fab89b9be5 Signed-off-by: Aaqa Ishtyaq <aaqaishtyaq@gmail.com> Reviewed-on: https://cl.tvl.fyi/c/depot/+/8125 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-02-16 r/5856 docs(tvix/eval): add proposal for VM loop restructuringVincent Ambo1-0/+76
Change-Id: Ib991d68724a73886a8343d7f785b5b3aedd637ed Reviewed-on: https://cl.tvl.fyi/c/depot/+/8103 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2023-02-15 r/5854 refactor(tvix/store): remove needless borrow in build.rsFlorian Klink1-1/+1
Change-Id: I2c2a70e5ffbb080702f57218aa9dcfe53d08870f Reviewed-on: https://cl.tvl.fyi/c/depot/+/8110 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-02-15 r/5853 refactor(tvix/store/proto): use .cloned()Florian Klink1-10/+4
Instead of using an explicit closure to clone elements, use .cloned(). Change-Id: I31f0f0bad2b4935e1a8d91fa0d14163c94182e1b Reviewed-on: https://cl.tvl.fyi/c/depot/+/8109 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-02-15 r/5852 refactor(tvix/store): avoid casting to the same typeFlorian Klink1-1/+1
The size field already is u32, we don't need to convert here. Change-Id: Ie29819aa2d1d8022e9bd73fcf05b140e45c967a9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8107 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-02-14 r/5851 fix(tvix/eval): correctly print lambda address in observerVincent Ambo1-1/+1
We want the address that the Rc is pointing to, not the address of the Rc. Change-Id: I8eba21677f242bbe4166c74d4aa4269c316076e3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8045 Reviewed-by: flokli <flokli@flokli.de> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-02-13 r/5850 chore(tvix/eval): use writeln for newline stringAaqa Ishtyaq1-2/+2
This CL address clippy warning which expects to use `writeln` instead of `write` for strings with new line. Change-Id: Ia72a07502c60cfd489ecf1e3833b9d42d44a8b17 Signed-off-by: Aaqa Ishtyaq <aaqaishtyaq@gmail.com> Reviewed-on: https://cl.tvl.fyi/c/depot/+/8030 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-02-13 r/5849 chore(tvix/eval): clippy warn is length zeroAaqa Ishtyaq1-1/+1
This CL address clippy warning about finding the zero length of something using `is_empty()` instead of `len() == 0`. Change-Id: I2b36c7c7b65b733609fc0dcd33be06f9d772bc9b Signed-off-by: Aaqa Ishtyaq <aaqaishtyaq@gmail.com> Reviewed-on: https://cl.tvl.fyi/c/depot/+/8029 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-02-13 r/5848 fix(tvix/eval): skip runtime completely on compiler errorsVincent Ambo2-0/+18
This branch was missing, and an assumption elsewhere just executed the returned (broken) bytecode. This fixes b/253. Change-Id: I015023ba921bc08ea03882167f1f560feca25e50 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8090 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: tazjin <tazjin@tvl.su>
2023-02-13 r/5847 fix(tvix/eval): make fields of eval's Error type publicVincent Ambo1-3/+3
These should be inspectable by callers. Change-Id: Ia9ef871aa63958d06066aaea61b2aecbd217369b Reviewed-on: https://cl.tvl.fyi/c/depot/+/8089 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-02-08 r/5843 feat(tvix/nix-compat/nixhash): support BASE64_NOPAD SRIFlorian Klink1-19/+61
Nix accepts SRI hashes that are missing their padding characters in base64, as seen in https://github.com/NixOS/nixpkgs/blob/7e49471316373c471a3bf4b78c130ebc907ae2d2/pkgs/development/libraries/kerberos/krb5.nix . It only seems to work in the SRI case, not with `sha256` being set to a (nopad) base64 string. Add regression tests for this, and document why we don't want to support *additional* characters afterwards. Reported in https://b.tvl.fyi/issues/252 Change-Id: I9ffc2b417501b426ced1894a9cbf95ff5f0e5159 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8037 Reviewed-by: Alyssa Ross <hi@alyssa.is> Tested-by: BuildkiteCI
2023-02-08 r/5842 refactor(tvix/nix-compat/nixhash): fix test function nameFlorian Klink1-1/+1
Change-Id: I3b089758ce8c01df544064422e56a97a8402513d Reviewed-on: https://cl.tvl.fyi/c/depot/+/8036 Tested-by: BuildkiteCI Reviewed-by: Alyssa Ross <hi@alyssa.is>
2023-02-07 r/5841 fix(tvix): fix InvalidEncodedDigestLength messageAlyssa Ross1-1/+1
Before: error[E997]: invalid output hash: invalid encoded digest length '43' for algo 43 After: error[E997]: invalid output hash: invalid encoded digest length '43' for algo sha256 Change-Id: Icb533839586b47f90b4e8d22083cce397f9f3655 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8035 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-02-07 r/5840 docs(store/protos): update comment on blobstore Read and PutFlorian Klink3-8/+56
Further emphasize Read() can be used to ask for blobs OR chunks, and that clients usually want to stat and then request (smaller) chunks, rather than reading whole blobs. Also clarify that the chunking used to send BlobChunks over has nothing to do with the chunk sizes communicated in a Stat() request. Change-Id: Ia615d190aae570611de2655b11342a14d0b75976 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8028 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-02-04 r/5837 fix(tvix/eval): fix the default case for path parsingVincent Ambo1-10/+4
Plain paths like `foo/bar.nix` are also allowed, so we can not determine this based on the prefix. The upstream PR that is referenced in a comment here has a significantly different interface than we expected, so I'm not touching that comment yet in this CL before I've had more time to digest it. Change-Id: Iea33bbb35de9c00a7d7fedf64d02253c75c1cc9e Reviewed-on: https://cl.tvl.fyi/c/depot/+/8032 Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: Alyssa Ross <hi@alyssa.is> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-02-04 r/5836 fix(tvix/cli): use tvlfyi/wu-manber fork for refscannerVincent Ambo4-43/+443
Our fork fixes a small bug (https://github.com/jneem/wu-manber/pull/1) but it's not clear whether upstream will accept patches, so for now lets point this directly at our fork. Change-Id: Iccdcedae3e9a8b783241431787c952561d032694 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8031 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-02-03 r/5833 chore(tvix/eval): only use Rc with impure featureFlorian Klink1-2/+3
We only use Rc in `impl EvalIO for StdIO`, which is only included when building with the "impure" feature. Change-Id: Id29d647c899cbfcdda11abfb9fabd5aa7e24299f Reviewed-on: https://cl.tvl.fyi/c/depot/+/8025 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-12-18T05·18+0000