Age | Commit message (Collapse) | Author | Files | Lines |
|
This provides a Nix HTTP Binary Cache interface in front of a tvix-store
that's reachable via gRPC.
TODOs:
- remove import command, move serve up to toplevel. We have nix-copy-
closure and tvix-store commands.
- loop into CI. We should be able to fetch the protos as a third-party
dependency.
- Check if we can test nar-bridge slightly easier in an integration
test.
- Ensure we support connecting to unix sockets and grpc+http at least,
using the same syntax as tvix-store.
- Don't buffer the entire blob when rendering NAR
Co-Authored-By: Connor Brewster <cbrewster@hey.com>
Co-Authored-By: Márton Boros <martonboros@gmail.com>
Co-Authored-By: Vo Minh Thu <noteed@gmail.com>
Change-Id: I6064474e49dfe78cea67676957462d9f28658d4a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9339
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
|
|
This shuts off the following warning:
```
warning: some crates are on edition 2021 which defaults to `resolver = "2"`, but virtual workspaces default to `resolver = "1"`
note: to keep the current resolver, specify `workspace.resolver = "1"` in the workspace root's manifest
note: to use the edition 2021 resolver, specify `workspace.resolver = "2"` in the workspace root's manifest
```
Stuff still seems to work with it, so no need to stick with "1".
See https://doc.rust-lang.org/edition-guide/rust-2021/default-cargo-resolver.html
for more details.
Change-Id: I6056d95cd5cb793f37ef843ed43009a27ad36367
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9342
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
|
|
WrongSorting(Vec<u8>) actually encodes the name of a node, so if we can,
we want this to display it as a string (and fall back to the base64
encoding if we can't).
Before:
> rpc error: code = InvalidArgument desc = directory b3:yfwNlpPm8MkrRvshGHfgMtKLoSDtX2pKliVuVWmUt5g= failed validation: [108, 111, 99, 97, 108, 101] is not sorted
After:
> rpc error: code = InvalidArgument desc = directory b3:yfwNlpPm8MkrRvshGHfgMtKLoSDtX2pKliVuVWmUt5g= failed validation: locale is not sorted
Change-Id: I68420c53a89cb1aa96e4bdce414366cebcb7915f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9350
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
|
|
Some of the tonic-generated types create new warnings.
Change-Id: Ib538bfd7d90d857d80a129fbbdf3a38c89cc04ab
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9349
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Tested-by: BuildkiteCI
|
|
This imports the docs folder into the tvix store, and ensures it comes
up with the same store path as Nix.
While we validate hashes in general through tvix-cli output path checks
already, it doesn't include the tvix-store CLI itself.
See https://cl.tvl.fyi/c/depot/+/9329/comment/339f0720_524f0104/
for context.
Change-Id: I239ce5b6a07cb962b242142ab716693359b8674c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9338
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
|
|
This allows comparing the output in an integration test.
Change-Id: I8bb2254e18e90005a4f1b30fd47ef69642e3732e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9337
Tested-by: BuildkiteCI
Reviewed-by: Adam Joseph <adam@westernsemico.com>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
|
|
This leaves some space in stdout to provide actual meaningful output.
Also, rename print_node to log_node because that's what it does, it's
using the logger to log out more detailed info.
Change-Id: Ic64a6330dbfcdc63eb4198067a5c5e47b841b9a5
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9336
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Reviewed-by: Adam Joseph <adam@westernsemico.com>
|
|
In cppnix 2.17, commit b72bc4a972fe568744d98b89d63adcd504cb586c, the
libexpr pretty-printing routine was fixed so that it would no longer
pretty-print attrsets with keywords in their attrnames incorrectly.
This commit implements the corresponding fix for tvix, fixes our
tests to work with cppnix>=2.17 oracles, and expands our test cases
to cover all the keywords.
Change-Id: I4b51389cd3a9c44babc8ab2a84b383b7b0b116ca
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9283
Autosubmit: Adam Joseph <adam@westernsemico.com>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
|
|
We were asserting absolute_offset > self.pos, but that's not true for
both being zero.
Ramp up the tracing bits a bit, so we actually can see this in the debug
logs.
Change-Id: I21693bcafab227549b19cd6f1215d2f8dee77ecc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9292
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Reviewed-by: Connor Brewster <cbrewster@hey.com>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
|
|
Change-Id: I9f7021ce08d0259be06932b71a83ee0361e5096d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9296
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
|
|
On Darwin rust crates sometimes needs iconv manually added to
compile successfully. There is currently also a bug in strip that
requires that you set dontStrip on buildRustCrate for it to work.
See: https://github.com/NixOS/nixpkgs/issues/218712
Change-Id: I13555c7bbee1d34f08fc51a668d2067dbbe550ce
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9291
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
|
|
Change-Id: I2445b977c5bed3e2d4b90e8ad81119d316b29b50
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9281
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: Adam Joseph <adam@westernsemico.com>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
|
|
Also expose both formats, then use it from
users/tazjin/presentations/tvix-eval-2023.
Change-Id: Id906e8aff5510a7a4f33336326472e86db18ea32
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9280
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
|
|
Previously, compare_strict_eval_tests() was using Strictness::Lazy.
This appears to be a minor oversight from
0ab6494286e77abcdc0bba1c92386fc7e654ca12.
This commit corrects that, by changing Strictness::Lazy to
Strictness::Strict.
Change-Id: Ia2389a5d30481cd322ed55230731340b795e5d87
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9282
Tested-by: BuildkiteCI
Autosubmit: Adam Joseph <adam@westernsemico.com>
Reviewed-by: sterni <sternenseemann@systemli.org>
|
|
This target builds and runs the go tests in that directory, allowing us
to spot regressions as the one fixed in cl/9285.
Change-Id: Ia16c0622f29db343eae7c0386e715b292703bd4f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9286
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Reviewed-by: Brian McGee <brian@bmcgee.ie>
Tested-by: BuildkiteCI
|
|
The underlying protos were changed to return `[]byte` instead of `string`.
Change-Id: I5d3e5d8de0ed7200325f7ab0d62e3c10d8eb1b7d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9285
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
|
|
Change-Id: Iad8a5f78930872719b6481ddf1bbad8532bfa888
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7981
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
Change-Id: Ib98a9fe8c9aa3f0e61c27d10285c0926cda7a969
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7979
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
|
|
Change-Id: I42e610740b3687e1fd897d36694cce425751a8bc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/7975
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
Reviewed-by: flokli <flokli@flokli.de>
|
|
Forgot to squash this into cl/9217.
Change-Id: Id5a87cbe729592f3b53e8c329b6890519f5da319
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9272
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
|
|
Included changes:
* tvix/eval: enable some lang tests on nix_latest
Nix 2.16 contains some breaking language changes which Tvix does
not yet implement, but the existing tests for them are now passed by
Nix 2.16 (but not yet by Tvix).
* tvix/eval: disable a lang test on nix_latest
In Nix 2.17, the identifier formatting test fails because some
behaviour changed. We have not investigated further yet.
* 3p/overlays: use version of ihp-hsx that works with GHC 9.4
Originally from the separate cl/9185.
* top-level: introduce a mechanism to exclude build targets from CI in
the top level. This fixes b/296.
* users/grfn: disable builds of xanthous (and dependents) until the
CLs fixing its build are submitted
* 3p/overlays: build nixos-option against Nix 2.15, the only version
with which it builds
* 3p/overlays: bump tdlib to 1.8.16
Change-Id: Ia377f39dbdb08ac45ff830a615e64babc091e5ee
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9125
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Autosubmit: tazjin <tazjin@tvl.su>
|
|
This makes the inside code a bit less verbose.
I wasn't able to describe the type of the async move closure itself,
which would allow us to remove the JoinHandle<_> type annotation
entirely.
Change-Id: I06193982a0c7010bd72d3ffa4f760bea1b097632
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9268
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
|
|
This enables the tokio `signal` feature, and registers a ctrl_c signal
handler, which will use the unmount handle to unmount in case a ctrl-c
signal is received.
This avoids having disconnected mountpoints when Ctrl-C'ing a
`tvix-store mount` invocation.
In case the filesystem is unmounted externally (via `umount /path/to/
mountpoint`), the future is waiting for the signal is never resolved and
the task is stopped.
Change-Id: I149f705a6cb50188177f2a6c6a5fcd77218e2a3f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9218
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
This provides an additional configuration flag to the tvix-store mount
subcommand, and logic in the fuse module to request listing for the
root of the mountpoint.
Change-Id: I05a8bc11f7991b574696f27a30afe0f4e718a58c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9217
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: adisbladis <adisbladis@gmail.com>
Tested-by: BuildkiteCI
|
|
This provides an additional method in the PathInfoService trait, as
well as an RPC method on the gRPC layer to list all PathInfo objects in
a PathInfoService.
Change-Id: I7378f6bbd334bd6ac4e9be92505bd099a1c2b19a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9216
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
We don't need to explicitly describe the type of the task itself,
describing the return type of the async closure is sufficient.
Also, use io::Result<_> instead of Result<_, io::Error>.
Change-Id: I9ab3f990eb49929b0aea335b2bb07da392ab631f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9267
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
|
|
https://github.com/BurntSushi/walkdir/pull/170 got merged, meaning we
don't need to keep our own logic in here anymore.
Our test cases already cover this.
Change-Id: Ied3043ee651c8aafa10271c1e1ca5d460fb6c0b8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9269
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
|
|
This trait is eval-specific, there's no point in dealing with these
things in tvix-store.
This implements the EvalIO interface for a Tvix store.
The proper place for this glue code (for now) is tvix-cli, which knows
about both tvix-store and tvix-eval.
There's one annoyance with this move: The `tvix-store import` subcommand
previously also used the TvixStoreIO implementation (because it
conveniently did what we wanted).
Some of this code had to be duplicated, mostly logic to calculate the
NAR-based output path and create the PathInfo object.
Some, but potentially more of this can be extracted into helper
functions in a shared crate, and then be used from both TvixStoreIO in
tvix-cli as well as the tvix-store CLI entrypoint.
Change-Id: Ia7515e83c1b54f95baf810fbd8414c5521382d40
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9212
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
|
|
Fix some broken link references.
Change-Id: I69c9b2b62af35bb777e4df1a01ba3181a368be47
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9214
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
Don't repeat the name of the method in the description, don't repeat
things already described in request message comments.
Change-Id: I180e4792577419050947eea8fea7043861aba463
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9213
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
|
|
This returns a node with a new name.
Change-Id: Iebcab537f8dd63d826b9841d4d0181fcb941afdd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9211
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
Since the refactor to use URIs for all three services, this actually
does talk to a daemon by default.
Change-Id: Ied296772b77eef514bfcae0a9dfc50f848a1c2f3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9210
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
|
|
replaceStrings would previously fail to replace the last character
in a string.
Change-Id: I43a7c960945350b2e7a5b731b7fdb617723eb38f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9151
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
|
|
We already have these hashes accessible in the Cargo.nix file created
by cargo2nix, so there's no need to also manually maintain them here.
It removes one potential footgun I ran into while updating wu-manber to
a different rev, without updating it here.
Change-Id: I93932ac8ba55f83746ee38571b7740af2d49bbdf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9090
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
|
|
Change-Id: I46660da84065fd6938f581e14d67e231dca3c3ea
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9112
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
|
|
There was a NixHash::new() before, which didn't perform any validation
of the digest length. We had some length validation when parsing nix
hashes or SRI hashes, but some places didn't perform validation and/or
constructed the struct directly.
Replace NixHash::new() with a
`impl TryFrom<(HashAlgo, Vec<u8>)> for NixHash`, which does do this
validation, and update constructing code to use that, rather than
populating structs directly. In some rare cases where we're sure the
digest length is correct we still populate the struct manually.
Fixes b/291.
Change-Id: I7a323c5b18d94de0ec15e391b3e7586df42f4229
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9109
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
Change-Id: I1c1608a6e75e451940fe1c61dc5ace5f0e7d7752
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9111
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
|
|
This question popped up every once in a while. While already explained
quite well at
https://inbox.tvl.su/depot/20230316120039.j4fkp3puzrtbjcpi@tp/T/#t,
it's not easily accessible.
Lift it from there into tvix/docs for better visibility.
Change-Id: I5f2d4aff31ab4adc421e06a7d36c871f45e09100
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9080
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
|
|
Change-Id: I0b2fdb418c2e36280d5c551a81634e1742193903
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9105
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
|
|
HashMap already is on the heap.
Change-Id: I53763e17469359e85862f297b5c2e7c0d8c3a980
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9104
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
|
|
… instead of new(). Suggested by clippy.
Change-Id: Iac7be733392afefc2b4ff2e38386eee95f3bce94
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9103
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
|
|
Change-Id: I11d7d55f31ddd27952f201a0abbed7445013f434
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9102
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
|
|
This passes a unit value to the function.
Change-Id: I4df3ad8fb0f35c0f110cee3349971ae28ce2878c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9101
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
|
|
Change-Id: Ifec2a4fc93c482e41ff5993183643b5126a7728b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9100
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
|
|
Change-Id: I9d931ffcc03c6df7c0392dbc1c9a4ae0e3804213
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9099
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
|
|
Change-Id: I6b1b902ccbc12bf2acdb0fdf406d6ef336ff0b2f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9098
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
|
|
Change-Id: Iaeb9ed7024c2ce85373f8aec0d223f52e1a3a539
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9097
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
|
|
Change-Id: I3a80560d036e7ed08036b5e9f0974080d1a30ded
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9096
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
|
|
Change-Id: Ib48063660432bea9de2fc9aff5914ef76db3ad64
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9095
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
Change-Id: I723bc9b29733f5ee4e3094b92696fbdeb4b15a33
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9094
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|