Age | Commit message (Collapse) | Author | Files | Lines |
|
This provides a GRPCDirectoryService struct implementing
DirectoryService, allowing a client to Directory objects from a (remote)
tvix-store.
Remote in this case is anything outside the current process, be it
another process, or an endpoint on the network.
To keep the sync interface in the `DirectoryService` trait, a handle to
some tokio runtime needs to be passed into the constructor, and the two
methods use `self.tokio_handle.spawn` to start an async function, and
`self.tokio_handle.block_on` to wait for its completion.
The client handle, called `grpc_client` itself is easy to clone, and
treats concurrent requests internally. This means, even though we keep
the `DirectoryService` trait sync, there's nothing preventing it from
being used concurrently, let's say from multiple threads.
There's still two limitations for now:
1) The trait doesn't make use of the `recursive` request, which
currently leads to a N+1 query problem. This can be fixed
by `GRPCDirectoryService` having a reference to another
`DirectoryService` acting as the local side.
I want to wait for general store composition code to pop up before
manually coding this here.
2) It's currently only possible to put() leaf directory nodes, as the
request normally requires uploading a whole closure. We might want
to add another batch function to upload a whole closure, and/or do
this batching in certain cases. This still needs some more thinking.
Change-Id: I7ffec791610b72c0960cf5307cefbb12ec946dc9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8336
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
|
|
Change-Id: Ie8e205c691bd11db99fcf097357c1e49161c6e19
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8349
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
|
|
Change-Id: Ia8ce1dd1663663903b9f60feada12e6a16fb4af2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8353
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
|
|
We query the blob service for detailled blob info, not the chunk
service.
Change-Id: I85a6a57b1dae74a950f734be7d4455c5c35ae355
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8348
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
|
|
Change-Id: I091c0d5decc0a1eb3d24e81b713434ab391c677d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8347
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
|
|
Change-Id: Ia0fb7638f94cfa7aaeac3044939b9efa20ef3507
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8337
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
|
|
This toggles whether tvix will evaluate the top-level value and
deep-force it, or return it potentially still containing thunks.
Change-Id: Ie910941e3b6a0f16c5c0cb896d73947626335f4b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8326
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
|
|
This makes it possible for callers to control whether they can receive
partially evaluated values from an evaluation or not.
We're actually flipping the default behaviour to non-strict top-level
evaluation, which means that callers have to set `strict = true` on
the Evaluation to get the previous behaviour.
Change-Id: Ic048e9ba09c88866d4c3177d5fa07db11c4eb20e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8325
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
|
|
Change-Id: Ie4c563e933f571f45cb4f4efe650d1b65f119e8d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8324
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
|
|
Change-Id: I4d002c933929508baab12d3802ac67d42813507b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8333
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
|
|
Starting with 1.18.1 we no longer need to pass an extra flag to work
around the log4j CVE, so baseJvmOpts can be empty.
Change-Id: I6d6c5a366ecbb499b2e3945db81ca0a8b2e2dcbf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8332
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
|
|
Change-Id: If9f6ee1c49a6c427530e56ceff60a2508f6fd9c6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8331
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
|
|
Change-Id: I5e27eb7cfb87cd20a87aa49af114f43aaadc32d5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8330
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
|
|
Change-Id: I3393c16db76fba839ea72612f54e6a2a72d0b69d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8329
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
|
|
Change-Id: I6f833c882e6a5428c9bd94951a5ef5b62ca7c6b6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8328
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: sterni <sternenseemann@systemli.org>
|
|
* //users/wpcarro/emacs: use top level (ELPA) version of eglot, as it
was removed from MELPA:
https://github.com/melpa/melpa/commit/dc2ead17a8e5d5df59fc3729c8f0435cfcbf55ef
* //3p/nixpkgs:tdlib: 1.8.11 -> 1.8.12
* //tvix/store: regenerate protos after buf update
Change-Id: I782a8d91fda5ed461788055a3721104e8c032207
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8327
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: wpcarro <wpcarro@gmail.com>
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: sterni <sternenseemann@systemli.org>
|
|
After ingestion of the contents into the store, this will use the
NonCachingNARCalculationService to create a NAR stream or the contents
of the path, and use our Derivation output path calculation machinery to
determine the output path (using recursive hashing strategy).
In a real-world scenario, we obviously want to cache these calculations,
but this should be sufficient to tinker around with it.
Change-Id: I9b2e69384414f0be1bdcb5a99a4bfd46e8db9932
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8317
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
|
|
Emits the span of the `set` that is being accessed in the `force`
operation of an attribute access.
Looking at traces, it's a lot more useful to get information about
*what* is being forced, as in cases like `foo.bar` it can be
misleading to have an error highlight `bar`, when the error occured
while forcing `foo` to be able to access `bar` in the first place.
Change-Id: Id46ff28f20c67cb4971727ac52cc4811795cea2d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8272
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
|
|
This reports the span
1. of the code within a thunk,
2. of the place where the thunk was instantiated,
3. of the place where the thunk was first forced,
4. of the place where the thunk was forced again,
when yielding an infinite recursion error, which hopefully makes it
easier to debug them.
The spans are tracked in the ThunkRepr::Blackhole variant when putting
a thunk under evaluation.
Note that we currently have some loss of span precision in the VM loop
when switching between frame types, so spans 3/4 are currently a bit
wonky. Working on it.
Change-Id: Icbd2a9df903d00e8c2545b3fc46dcd2a9e3e3e55
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8270
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
|
|
This is step 1 towards being able to use all 4 spans that we know when
dealing with infinite recursion. It tracks the span at which the
force of a thunk was first requested when constructing a blackhole, so
that we can highlight the spans of the first and second forces.
These are actually the least relevant spans, but the easiest to put in
place, more coming soon.
Change-Id: I4c7e82f6211b98756439d4148a4191457cc46807
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8269
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
|
|
This produces traces in which we can see what kind of native code was
run. Note that these "names" are named after the generator message, so
these aren't *really* intended for end-user consumption, but we can
give them saner names later.
Example:
https://gist.github.com/tazjin/82b24e92ace8e821008954867ee05057
This already makes the traces a little easier to parse.
Change-Id: Idcd601baf84f492211b732ea0f04b377112e10d0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8268
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
|
|
When emitting an error at runtime, the VM will now use the new
`NativeError` and `BytecodeError` error kinds (which just wrap inner
errors) to create a set of diagnostics to emit.
The primary diagnostic is emitted last, with `error` type (so it will
be coloured red in terminals), the other ones will be emitted with
`note` type, highlighting the causal chain.
Example:
https://gist.github.com/tazjin/25feba7d211702453c9ebd5f8fd378e4
This is currently quite verbose, and we can cut down on this further,
but the purpose of this commit is to surface more information first of
all before worrying about the exact display.
Change-Id: I058104a178c37031c0db6b4b3e4f4170cf76087d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8266
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
This actually uses coercion under the hood in C++ Nix. See the test
for an example.
Change-Id: Id56b364acf269225b6829d0b600e0222f8b3608d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8322
Reviewed-by: andi <andi@notmuch.email>
Tested-by: BuildkiteCI
|
|
Change-Id: Iba14dcec42aba3195dbe28a1cf31acd66919f1b7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8323
Autosubmit: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: grfn <grfn@gws.fyi>
|
|
This was commented out and forgotten during the generator refactor, oh
well.
Change-Id: I474b685159a955a846db462da0dd0067af177b04
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8321
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
Passing in a &proto::node::Node into all this allows us consumers to
keep ownership of the proto::node::Node.
Change-Id: I44882a86c46826b06a8a8a0b24c18adfc7052662
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8316
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
|
|
Change-Id: Idb78e0417a962599cdfdef5e7346f7fa41e3fa1b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8320
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
|
|
Change-Id: I2073408fe6e87ffadbeb35ef2f9aaa0316c54966
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8319
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
|
|
Change-Id: I4c85425782878295277abe6dc91a1b570768f7fa
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8318
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
Change-Id: Ie2b94aa5d69ff2c61fb77e13ae844f81f6270273
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8314
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
|
|
Also, simplify the trait interface, only allowing lookups of Directory
objects by their digest.
Change-Id: I6eec28a8cb0557bed9b69df8b8ff99a5e0f8fe35
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8313
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
|
|
Change-Id: I710f0920da118cb1b94cd8ff364adadbeef998bb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8315
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
|
|
The behaviour of this function is a bit unintuitive, and cl/8310 already
inlined the other consumer of it.
Rewrite the last consumer of the function, so we can drop it.
Change-Id: I59c8486037ce3f777667d1d9e4f4a9316d5a0cb9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8311
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
|
|
Instead of having two very similar match branches for the FOD and non-
FOD case, detect the FOD case while looping over all outputs.
In the case of anything other than recursive sha256 FODs, the
fingerprint and output path calculation is exactly the same.
Change-Id: Ieb6995653d008766e595cf29d7cd4fb1334e33dd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8310
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
|
|
Walking the arguments might encounter an `outputs` output, which might
explicitly (for whatever reason) specify the `out` output.
To prevent dropping FOD settings in this case, we have to populate
that part of the configuration after walking the other attributes.
Change-Id: Iee6a7f0a71e9c9699e79d35e6cb19e1ddb49395d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8312
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
This stops using our own custom Hash structure, which was mostly only
used because we had to parse the JSON representation somehow.
Since cl/8217, there's a `NixHash` struct, which is better suited to
hold this data. Converting the format requires a bit of serde labor
though, but that only really matters when interacting with JSON
representations (which we mostly don't).
Change-Id: Idc5ee511e36e6726c71f66face8300a441b0bf4c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8304
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
|
|
Change-Id: Ibb8061be75d6fd16e8f813bc94e3bacaae890454
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8309
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
|
|
Apparently, having multiple packages with the same path is a bad thing:
```
The bin target `tvix-store` in package `tvix-store-bin v0.1.0 (/home/flokli/tvl/tvix/store)` has the same output filename as the lib target `tvix_store` in package `tvix-store-bin v0.1.0 (/home/flokli/tvl/tvix/store)`.
Colliding filename is: /home/flokli/tvl/tvix/target/doc/tvix_store/index.html
The output filenames should be unique.
This is a known bug where multiple crates with the same name use
the same path; see <https://github.com/rust-lang/cargo/issues/6313>.
```
Change-Id: Ic785c0349070783baf5e8fd23f5fb60603a3c995
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8308
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
|
|
There's still some shadowing going on, but that's left for a followup
CL.
Change-Id: I02992f1eb494faca99857a3a5ee4dcd47f1b9fd0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8306
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
Change-Id: I198ea197867f9b9a48e51665d0665f722202e02e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8299
Reviewed-by: flokli <flokli@flokli.de>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
|
|
This will be published on docs.tvix.dev
Change-Id: I348e057351d5295ad20953c9e9a32c257abab089
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8298
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
|
|
Change-Id: Ie1994ac4d14344c82ae184f4e3cd9f5292d96c84
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8297
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
The website itself contains no useful information that we don't
already have published anywhere, but it's a decent landing place.
Mostly I want to have something on the root of tvix.dev, so that we
can start using it for other purposes.
Change-Id: Id2b4dabc7f6e4dd26b61484b86dbde2f39aa1719
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8296
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
|
|
Change-Id: Ied022e6c1a8039a9db375a8593afb76edcaa6dbd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8295
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
|
|
This only trims the output paths from a Derivation struct, not the
output hashes.
Change-Id: I9250fec4602ed05bb64540c4a89ddb6fb052be1f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8303
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
|
|
Change-Id: I1652e24c9be28112d98683de2d2db51dc46001ed
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8302
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
Change-Id: Ia6af8d4b53fbd4d9ea26f0094cd98dde66a91f78
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8301
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
|
|
The digest is encoded in LOWERHEX here, not nixbase32.
Change-Id: I8e14742f0135966424073bfdb93ada8cd7bd52e2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8300
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
|
|
Since cl/8213, tvix bundles corepkgs.
Change-Id: I17fa4452a6a1d554ae67b4aed9d9b1e77cb495f5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8305
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
|
|
Call this function derivation_or_fod_hash, and return a NixHash.
This is more in line with how cppnix calls this, and allows using
to_nix_hash_string() in some places.
Change-Id: Iebf5355f08ed5c9a044844739350f829f874f0ce
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8293
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
|