Age | Commit message (Collapse) | Author | Files | Lines |
|
Have `tvix/castore/protos` only contain the protos, no go noise.
Make the `.pb.go` file generation a pure Nix build
at `//tvix/castore/protos:go-bindings`, and have a script at
`//tvix:castore-go-generate` (TBD) that copies the results to
`tvix/castore-go`.
`//tvix:castore-go`, with sources in `tvix/castore-go` now contains the
tooling around the generated bindings, and the generated bindings
themselves (So go mod replace workflows still work).
An additional CI step is added from there to ensure idempotenty of
the .pb.go files.
The code.tvl.fyi webserver config is updated to the new source code
path. I'm still unsure if we want to also update the go.mod name. While
being a backwards-incompatible change, it'll probbaly make it easier
where to find these files, and the amount of external consumers is still
low enough.
Part of b/323.
Change-Id: I2edadd118c22ec08e57c693f6cc2ef3261c62489
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9787
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
|
|
After finishing the ingestion, the directory putter was not being
closed. This caused a race where the root directory node was accessed
before the directory node had been flushed to the server.
This patch makes it so we close the putter before returning the root
node which should ensure that the root node exists on the directory
service server before the `ingest_path` function returns.
Fixes b/326
Change-Id: Id16cf46bc48962121dde76d3c9c23a845d87d0f1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9761
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
|
|
Change-Id: Ie464467edfafd2050aa9e5fd0e07ff745cb15df8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9753
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
|
|
That's needed to build nar-bridge, as well as things in tvix/*store/
protos.
Change-Id: If5f71b6c528a27689faa3c5ab3489cc528f9e6e5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9759
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
|
|
We need to invoke protoc at build time (only).
Change-Id: Ib4c101d2ccdbdbb078725fc98c30d6c00086667d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9754
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
|
|
On Darwin, some crates producing binaries need to be able to link
against security.
Change-Id: I5bdd69247c12729b9efd5c4f18527d361ef99e87
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9758
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
Also, make some of them buildInputs, not nativeBuildInputs.
We likely need to link against libiconv, so it shouldn't be a native
build input.
Change-Id: I757bc7f5d8c864ecaa3db26988d00b496c085c8f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9757
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Autosubmit: flokli <flokli@flokli.de>
|
|
It looks like we need Security Framework in the shell to be able to
imperatively run `cargo build`.
Change-Id: Ia5df9052bafca3610d4ed235b180b812d33d0d15
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9756
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Autosubmit: flokli <flokli@flokli.de>
|
|
Change-Id: Ie9416f0046b6ee0a02096a7712cdba9cc84db4f6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9733
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Reviewed-by: Gabriel Doriath Döhler <gabriel.doriath.dohler@ens.fr>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
|
|
The previous CLs did already absorb all the logic into the common tests,
no need to write this here again.
Change-Id: I7ba84ba86d5445ed247e5d11d5e59b7fa815670e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9732
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
|
|
Due to the lack of a ATerm parser, we were previously loading the JSON
fixtures to construct our Derivation structs to run the output path
calculations with.
However, as we now have a ATerm parser, we can load the ATerm
representation directly.
This also means we can test the output path calculation for non-UTF8
Derivations.
Change-Id: I0e53f41a23566b5ad5f0fed12724e02a10b02707
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9731
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
|
|
This now properly checks for duplicate output names in input derivation
output names, and duplicate input sources.
Change-Id: I0053854bfbf504f4f511fb3fe1a77a82b3aa61dd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9738
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
|
|
This provides a nom-based parser for Nix derivations in ATerm format,
which can be reached via `Derivation::from_aterm_bytes`.
Some of the lower-level ATerm primitives are moved into a (new) aterm
module, and some more higher-level ones that construct derivation-
specific types.
Also, move the escape_bytes function into there, this is a generic ATerm
thing.
Change-Id: I2b03b8a1461c7ea2fcb8640c2fc3d1fa3ea719fb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9730
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
|
|
typo: write_enviroment -> write_environment.
Change-Id: I1a2dc826c0bf71f6ba97fe41c4d44619b5e6beff
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9739
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
|
|
Change-Id: I35a28a320c890f62e6a76641dfdf99e33e839c02
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9749
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
|
|
Change-Id: I31ba268bda21c1f07c63520a269dd62f7b309e99
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9748
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
Change-Id: I56282d9754b42e0e6cdbb08e9ae71f5700d28dd1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9747
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
|
|
Change-Id: I8a792d6b238190a78390c5d8f638669a2681f70f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9746
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
|
|
Change-Id: I11247e24475f0e49dc22f2d10bf3e1e276737899
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9709
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
|
|
We don't actually care if it's a BTreeMap of strings, bstrings or any of
that sort.
The only thing we want to be able to do is get a reference to the bytes
from the keys and values.
Change-Id: I21b85811a9ea47fe06afa3108836ef9295e5d89b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9737
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
|
|
We use test_resources and globbing for some of the test cases, so adding
additional files in there will also create new test cases, which we
don't always want.
Move it down one level to make some more space.
Change-Id: I619867dc80a4ced59d45096d0703678663b559cd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9729
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
|
|
Change-Id: Ieae628fab1926a498ae5c3eb27df1b722e6151d7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9728
Reviewed-by: edef <edef@edef.eu>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
Change-Id: Id0248047e9642d38afc106629957a2e7608f8c78
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9727
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
|
|
Limit the amount of memory consumed on the stack for NixHash. Sha512
isn't used that often, so it's fine if we heap-allocate it.
Change-Id: I4a9eecd20c6184610124dc130c41bfa5d0dc04c5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9726
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
|
|
Tonic support was added to tokio-listener upstream which removes the
need for use to have tonic compatibility wrapper types around it.
See: https://github.com/vi/tokio-listener/pull/2
Fixes b/311
Change-Id: I04a2dbb3bc3c8bfe9339583c0b46070c7ec97811
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9721
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
|
|
This is of known length.
Change-Id: Iba48ccc486d5bf9e38ec1a2da6e7b80997d2c6ca
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9723
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
|
|
Less Vec<u8> passed around.
Change-Id: Ie153a6bfaa084d7490ffa38634efdf5f3c31a768
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9722
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
This updates the code to make use of the new Validate() function defined
on a Node.
Change-Id: I9b6ed694661f41e700f19cc78d53d2224b61852d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9718
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
|
|
This uses the newly introduced StorePath message type to add a Deriver
field to the PathInfo message.
Support for validation is added to both the golang and rust
implementation. This includes extending unit tests.
Change-Id: Ifc3eb3263fa25b9eec260db354cd74234c40af7e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9647
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
|
|
This encodes a store path a bit more concise, which is used in the
Deriver field.
Change-Id: Ibfb54d3b206917e51970d1d5fe94fcedb901704b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9646
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
|
|
To make this easier, move it outside of TvixStoreFs, and accept the
DirectoryService as a function argument, so we don't need to worry about
the lifetime of self.
This also aligns with how we spawn async tasks inside the rest of
TvixStoreFs.
Change-Id: I3b95072209d32039f05aed122240f2d6db7ad172
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9713
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
This moves the repetitive code to parse a URL and create a channel
connected to it into `tvix_castore::channel::from_url`.
Part of b/308
Change-Id: Idd342cd71cad5e78a9b258b38c1b227993e75310
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9707
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
|
|
Move the channel creation depending on the string-based URL into its
own block.
Change-Id: I546b769acd2296b548eb966b62c495f910266df5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9706
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
|
|
We had to have these all while the traits where sync, and there was a
lot of spawning and moving.
Most of this can now be removed in favor of some inline `.clone()`.
Change-Id: Id5466c32a403100bc3347866b3172e06a792e311
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9705
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
|
|
Similar to cl/9715, this makes the validation checks more granular,
introducing a Validate on all *Node.
A check for symlink targets is added too.
Once merged, it can also be used from tvix/store/protos.
Change-Id: I0909a89fadcd74b74ef0c9a8a1f22658fccc83b0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9716
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
|
|
We can just check the digest length to be correct, and move the symlink
target checks to a single line.
Change-Id: I41d2e3a50e7990ef6c04f02acd754b1e17b43e77
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9717
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
|
|
Change-Id: I7620d2abe01675ea7028a478d4f8447e36d5768b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9605
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
|
|
Change-Id: I79a73de0cb06abcf16a6579ae298876004ab0ba8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9719
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
|
|
Implement `validate()` on `node::Node`, and call it from PathInfo's
validate() too. Node-related errors are moved to a ValidateNodeError
error type.
This additionally adds some more validations for symlink targets (they
must not be empty, and not contain null bytes).
Change-Id: Ib9b89f1c9c795e868a1533281239bc8a36d97c5d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9715
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
|
|
Suggested by clippy.
Change-Id: I686b1abcb40fab42017972a3f814fae29fa70c25
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9712
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
|
|
Change-Id: I3daca008dff5527169f5916f4845234e8f3263cd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9711
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
|
|
Change-Id: I03aa5c6cb0167ff6cb7312d81d1e43116fcb6d70
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9710
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
|
|
Change-Id: Ib0fa1e1fb90d7f315f943a3f28266c0fb3714a31
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9708
Tested-by: BuildkiteCI
Reviewed-by: edef <edef@edef.eu>
|
|
This teaches tonic to speak https, so `grpc+https` becomes a working
connection string.
Change-Id: I9c550ea665b829725b545a164cf6f78587bd5213
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9714
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Brian McGee <brian@bmcgee.ie>
Tested-by: BuildkiteCI
|
|
Change-Id: I132987dda1073dcc0b30e50e26682d6c8724d988
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9689
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
Bump code.tvl.fyi/tvix/store/protos past cl/9649, where Validate()
already ensures the NarSha256 has the correct size.
Change-Id: I774668822f4d9dbd4dea47dde6e4745dc95e8e7f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9665
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
|
|
Change-Id: I4c4dcdb75ea7748f2ab01a0bab218596b90b7b58
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9648
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
|
|
Change-Id: I016d1b020b04850f0dca68c2ea96643230dada10
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9649
Reviewed-by: Brian McGee <brian@bmcgee.ie>
Tested-by: BuildkiteCI
|
|
These really only describe what the code is doing, remove.
Pointed out in
https://cl.tvl.fyi/c/depot/+/9652/comment/987b3b7f_c7b5e47b/
Change-Id: I512a274295244f96744a489871861fed9523bb37
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9654
Reviewed-by: Brian McGee <brian@bmcgee.ie>
Tested-by: BuildkiteCI
|
|
As correctly mentioned in
https://cl.tvl.fyi/c/depot/+/9652/comment/03b9b96e_bbb337fd/,
we shouldn't be using these magic constants, but pull them from where
they're defined.
This already is a dependency of go-nix, and pkg/pathinfosvc/server.go,
so no changes in go.mod.
Change-Id: I0cc41ce040fcbddf4b6171417bc9b0de55af4991
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9653
Tested-by: BuildkiteCI
Reviewed-by: Brian McGee <brian@bmcgee.ie>
|