Age | Commit message (Collapse) | Author | Files | Lines |
|
Moving out the handshake route out of tvix-daemon to nix-compat.
We're bumping the protocol to version 37. It does not change anything
for us. Version 37 introduced a change in the build result
serialization. We do not implement that for now.
Tested tvix-daemon against Nix via the integration test to make sure
we did not break anything in the process.
Taking advantage of this refactor to remove the unused anyhow
dependency in tvix-daemon.
Change-Id: I8a2aad0b6dbb0aaa997446a612a365fd7d70336c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11379
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
|
|
This allows using read_u64, write_u64, which is a bit easier to juggle
with.
Also, update names to align with the nix codebase, which makes it easier
to spot both the constant name as well as the value.
Leave the ASCII interpretation as a comment afterwards.
Change-Id: I0b9ab187acd22807e2785b0722aa4300dab37c51
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11378
Tested-by: BuildkiteCI
Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
Autosubmit: flokli <flokli@flokli.de>
|
|
`primitive.rs` implements reading and writing primitive (fixed-length)
types in the wire format, used in the the nix daemon protocol and NAR
format.
Move worker-protocol specific magic bytes to worker_protocol.rs (and
possibly further split there once needed)
Change-Id: If681c01e9460294619f1d000229b81f0ac745810
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11377
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
|
|
This includes String, &str etc. An example testcase with &str is
provided.
Change-Id: I900186d6ceb52f52bd41ef4596524c1f5b52470b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11376
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
|
|
The protocol is more stateful than I initially thought. We need to
keep track to a bunch of things, including but not limited to: the
client settings, the client version. I moved things around a bit to
keep this state along with the client socket.
Change-Id: Ibd34fbe7821c20a460934ea1af0719f5de46e491
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11359
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
Change-Id: I36f7a772e70be8711c03150b4b6dfcd6d4e299f2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11371
Reviewed-by: aspen <root@gws.fyi>
Autosubmit: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
|
|
- agenix has not been updated
(https://github.com/ryantm/agenix/pull/241).
- Re-enable now fixed dependency of flokli/archeology-ec2.
Change-Id: I4e0399e5b5dbaf5e504076e029013f165dd4d191
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11363
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
|
|
This deals with writing byte packets of larger sizes to an underlying
AsyncWrite.
Its constructor receives the expected size. It also deals with writing
padding if flush/shutdown is called after writing all the payload.
Change-Id: I8acbf992467f3862ffb8c7d669e8c0c8eced14c1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11355
Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: Brian Olsen <me@griff.name>
Tested-by: BuildkiteCI
|
|
Change-Id: I105aad32cfcebd7a1b9108e4fd847073fb30da15
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11360
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
|
|
Change-Id: Ia30cb6d533abd73f896b5a303049b2cb9cc26bf1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11357
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
|
|
Remove potential partial reads/writes and instead read/write the full
buffer size: we want those to be 64 bits.
Change-Id: I1f767baf23fa80c2babb8113f61d1a9e72a8d8dd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11350
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
Adding a verbosity flag available through the CLI/ENV variable.
Change-Id: If04cc2e6e26e7cb3c2df7821fce222da2b85a95a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11349
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
Actually these are all u64 LE encoded on the wire.
Change-Id: I5ca22c7639607ac47117cd946e036a444271885a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11348
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
Using all the primitives recently implemented to nix-compat to reach
the point where the Nix client start to send us operation requests.
Using a small integration test script (or the VM test, but let's face
it, it's too slow to be useful), we manage to reach the point where
we're able to read a store operation:
2024-03-21T18:53:27.624876Z INFO tvix_daemon: Incoming connection addr=unix
2024-03-21T18:53:27.625312Z INFO worker:perform_init_handshake: tvix_daemon: Trust sent conn=Connection(unix) conn=Connection(unix)
2024-03-21T18:53:27.625406Z INFO worker: tvix_daemon: Client hanshake succeeded conn=Connection(unix)
2024-03-21T18:53:27.625488Z INFO worker: tvix_daemon: Operation received op=SetOptions conn=Connection(unix)
We had to take some shortcuts wrt. stderr/log management. The CPP Nix
codebase is a bit confusing in that area. I'll need to spend more time
reading this to fully understand what's happening there. For now,
sending the STDERR_LAST command to the client does the trick.
Change-Id: I9b0e20a52d885e64fe29188496aac5334de61edd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11233
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
|
|
Write counterpart of read_bytes. Despite its name, we mostly use it to
write strings (as in ascii strings) to the wire.
We also extract the padding calculation in its own function.
Change-Id: I8d936e989961107261b3089e4275acbd2c093a7f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11230
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
Temporarily remove parquet-tools, so this builds again.
Can be re-added once https://github.com/NixOS/nixpkgs/pull/301032 landed
upstream.
Change-Id: Ie74f014eb8158d5f529a5f1c55788a4edc5c805d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11347
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
|
|
Update all 3p/sources as we do normally except
- agenix which is still pinned to 0.15.0
- nixpkgs (unstable) which we bump to the HEAD of the staging-next
branch. This branch includes the downgrade of xz from 5.6.1 to
5.4.6 (https://github.com/nixos/nixpkgs/commit/d6dc19adbd). It
also includes the second haskell-updates rotation with GHC 9.6.4
which contains a few build fixes that seem to be required to get
our Haskell targets to work.
Note that this only reverts xz to a version that doesn't contain the now
known backdoor (CVE-2024-3094) which may or may not actually affect
NixOS. Additionally reverting to a version before the malicious
contributor's involvement may be difficult, but prudent:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1068024
Changes required by the updates:
- //3p/overlays/haskell:
- Update ihp-hsx to latest master to fix build with Stackage LTS 22.
- Update tmp-postgres to latest master to work around failure with
ansi-wl-pprint >= 1.
- Patch punycode for mtl >= 2.3.
- //users/Profpatsch:
- Clean up some warnings, mostly about unused dependencies
- my-prelude: Fix build with ghc-boot-9.6.4
- cas-serve: Use crypton over unmaintained cryptonite
- ical-smolify: skip in ci, iCalendar would require heavy patching to
work with Stackage LTS 22.
- //users/{wpcarro,aspen,flokli}:
Disable home-manager / nixos configuration builds that seem to have
transient failures that should disappear as we move away from
staging-next and closer to an actual channel release.
Change-Id: I5cca48e101041c3aedc1d9932dbca2cac885fcc1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11289
Tested-by: BuildkiteCI
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
|
|
Change-Id: I63d9d1cc7221bd3f18b0157f423cf3e369482a2f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11327
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
|
|
Change-Id: I86a18fb3fa135c4324064e6acde06bd4dcfbb94d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11294
Reviewed-by: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
Autosubmit: aspen <root@gws.fyi>
|
|
Change-Id: I07b8cea035050ff90fbb851e4f6401285514db1a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11293
Reviewed-by: aspen <root@gws.fyi>
Autosubmit: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
|
|
Change-Id: Ib2572fea5fc81d6b4d321a2e317a02398d0ae612
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11292
Tested-by: BuildkiteCI
Reviewed-by: aspen <root@gws.fyi>
Autosubmit: aspen <root@gws.fyi>
|
|
A Framework laptop
Change-Id: I646e705d12b76c83e8cdcf11c618d07db3a21f0c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11235
Reviewed-by: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
|
|
My new laptop has the battery at index 1, not index 0
Change-Id: I64fa16e7f0eecf6b623361645d73b1176e1e5d48
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11311
Tested-by: BuildkiteCI
Autosubmit: aspen <root@gws.fyi>
Reviewed-by: aspen <root@gws.fyi>
|
|
Change-Id: I37bb2ea71d8dd07c7294b5c534fb3d4a834d98c1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11310
Autosubmit: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: aspen <root@gws.fyi>
|
|
Change-Id: I664f97b2321625482ef64218b51c93ba28599643
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11309
Autosubmit: aspen <root@gws.fyi>
Reviewed-by: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
|
|
Only those that want them
Change-Id: Id1caf5e89f69c51a7da90ff2387a9384abbf4f91
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11308
Autosubmit: aspen <root@gws.fyi>
Reviewed-by: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
|
|
Change-Id: Ib512c0a8b5c69b20c3fd21bd9bcdcf318702e272
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11307
Autosubmit: aspen <root@gws.fyi>
Reviewed-by: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
|
|
Change-Id: I9b1a8c0cf10676a260b521dbc6a84179c386d772
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11306
Tested-by: BuildkiteCI
Autosubmit: aspen <root@gws.fyi>
Reviewed-by: aspen <root@gws.fyi>
|
|
For the rare times I still occasionally use vim
Change-Id: Ie3059ab0ec5d5ace729e78b765f251d1fafdfac1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11305
Reviewed-by: aspen <root@gws.fyi>
Autosubmit: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
|
|
Change-Id: Ie3e5aeba7026724dacbe4d0dc15a1112a22c3670
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11304
Autosubmit: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: aspen <root@gws.fyi>
|
|
This is not necessary now that I'm using tailscale everywhere
Change-Id: I36d8060d4ad5f6955cd18c68b5fc667e64669004
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11303
Reviewed-by: aspen <root@gws.fyi>
Autosubmit: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
|
|
Change-Id: I10369a20db3f1be92caf0bace94e8beb6af1dbaa
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11302
Tested-by: BuildkiteCI
Autosubmit: aspen <root@gws.fyi>
Reviewed-by: aspen <root@gws.fyi>
|
|
I never got this working
Change-Id: Iacb68a7d6d5fe1d1f1f390d09f106c4eb9a51b6a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11301
Reviewed-by: aspen <root@gws.fyi>
Autosubmit: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
|
|
Change-Id: I9a0464f326f842895cf11a373c6c4f1a42546a3a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11300
Reviewed-by: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
Autosubmit: aspen <root@gws.fyi>
|
|
These are noisy, but more importantly I'm worried I might accidentally
commit a secret at some point
Change-Id: If6f2c358f2803af25ea27ef34d39c7f2108d4186
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11299
Reviewed-by: aspen <root@gws.fyi>
Autosubmit: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
|
|
Change-Id: I67be0eaa48c6b2a3df59fd4b384409c3952eff3a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11298
Autosubmit: aspen <root@gws.fyi>
Reviewed-by: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
|
|
Change-Id: I42fc3887a0eb95404de29efba98dc26cb3f21aed
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11297
Autosubmit: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: aspen <root@gws.fyi>
|
|
whatever dude
Change-Id: Ie0e01ccc49e82125d77dbf05fce5b1c53f630783
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11296
Reviewed-by: aspen <root@gws.fyi>
Autosubmit: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
|
|
Change-Id: Iddcee013e6d187f9a71329f0cccc6a0205aeb497
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11288
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
|
|
This is currently unused, and even if it would be used, it'd probably
be with `default-features = false`, explicitly enabling what's needed.
Change-Id: I3175a19adf8d65f0b805e5234267e460cd5e68d8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11287
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: picnoir picnoir <picnoir@alternativebit.fr>
Tested-by: BuildkiteCI
|
|
Change-Id: I7776635b17c44534223603d28cf59c7eebd976e0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11229
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
Implementing the full connection handshake.
The integration test is a bit naive, but there's not much to test yet.
Tested this against cpp nix. We reach the stage where cppnix sends the
opcode.
Change-Id: I98322832848ee5b048f22105731b0adeb44b2ce0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11227
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
Change-Id: I0686c802b902ed69334221367a2f00be758daec1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11256
Reviewed-by: aspen <root@gws.fyi>
Autosubmit: aspen <root@gws.fyi>
Tested-by: BuildkiteCI
|
|
Change-Id: I2959c201809ef0d77934992f809756a16108b96a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11255
Tested-by: BuildkiteCI
Reviewed-by: aspen <root@gws.fyi>
Autosubmit: aspen <root@gws.fyi>
|
|
First stab at factoring out handlers into a generalized handler
function.
This is still kind of confusing, but can be simplified later.
Change-Id: I42da047de83f6d489337d57059f85f793313443a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11245
Autosubmit: Profpatsch <mail@profpatsch.de>
Reviewed-by: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
|
|
Move the http calls into their own module, so we can trace the request
and provide a simple copy-to-replay command.
We have to work around a bug in the otel library, which would limit
our attribute value length to 128 bytes because it uses the wrong
option value.
~~~
`ifExists` is finally made more useful for dealing with optional
attributes in e.g. lists.
Change-Id: Iafab523e9ec4b00136db43f31fdc12aeefb7f77c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11241
Tested-by: BuildkiteCI
Autosubmit: Profpatsch <mail@profpatsch.de>
Reviewed-by: Profpatsch <mail@profpatsch.de>
|
|
Change-Id: Id774963178ba358447699d0297a6a1fbef5ac8fe
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11240
Autosubmit: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
|
|
Change-Id: Ia2bd60b8449592ef1f79ac4877554958eb0b0407
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11239
Reviewed-by: Profpatsch <mail@profpatsch.de>
Autosubmit: Profpatsch <mail@profpatsch.de>
Tested-by: BuildkiteCI
|
|
Move the functionality into two coarse modules.
There’s still the question about whether functions that change the
database tables should be in their own storage module, but let’s see
if it gets too confusing.
Change-Id: Ied1d47b353dd4597ffea35f111f440aad22e981d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11238
Tested-by: BuildkiteCI
Reviewed-by: Profpatsch <mail@profpatsch.de>
Autosubmit: Profpatsch <mail@profpatsch.de>
|
|
Change-Id: I16c25b43a92ba650f9862ab46d11f29e940cb7ae
Reviewed-on: https://cl.tvl.fyi/c/depot/+/11226
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
|