Tvix: Status update

2023-09-09

Florian Klink

Whoami

flokli
nixpkgs contributor since 2018, maintaining systemd, nsncd and some more stuff
Freelance Nix/DevOps consultant
I spend too much time on computers :-)

What is Tvix?

A new implementation of Nix
modular
written in Rust
developed in the TVL monorepo
subtree exported to github:tvlfyi/tvix

Structure

strong separation between Evaluator, Store and Builder
Defined interfaces between different components (Protobuf/gRPC)
- Allows adding to/combining with your own components
A lot of helper code for some of the Nix internals in the `nix-compat` crate

Note: Derivation types, serializers. NAR writers, nixbase32 enc/dec, Nix Hash function, stringparsing etc.

Evaluator: Design

Nix code is parsed via rnix
AST traversal, generate bytecode (with some transformations)
Bytecode is executed by an abstract machine with a Nix-specific instruction set

Evaluator: Design

Builtins are separate from the "evaluator core"
- inject your own builtins
- this includes builtins.derivation!
IO is nicely abstracted away
- We can run a Nixlang subset without IO in wasm (see tvixbolt), or parse Nix into a config struct with tvix-serde

Tvixbolt Demo

Evaluator: Current Work

Current goal: evaluate nixpkgs the same way as Nix does
Checked by comparing the calculated output paths, which checks correctness of all "parent" output paths too.
Required implementing a lot of Nix internals in nix-compat, and tvix-store (A-Term, hash modulo, NAR writer/hasher)

Note: Getting output hashing correct, and exposing this in a re-usable fashion took quite some iterations to get right.

Evaluator: Current Work (cont.)

🎉 Already correct for (and continously checked by CI on every commit):
stdenv, hello
pkgsCross.aarch64-multiplatform.stdenv, pkgsCross.aarch64-multiplatform.hello
Some work left for more complicated expressions
- infinite recursion when inheriting from a builtins.tryEval multiple times
- small details around file imports
Not too much performance finetuning until we're correct first.

Evaluator: Demo

Store: Design

Uses a very different underlying data model:
- Nix stores on a per- StorePath granularity
- tvix-store uses a Merkle DAG of directories, similar to git trees, but with BLAKE3 digests as pointers.
- Compat layer in front to still render/calculate NAR on demand where needed
- Substitution, caching, ... possible to describe via composition/layering

tvix-store graph

Store: Advantages

Less downloading of data that didn't change
Less duplicate data on disk/storage
Inherently content-addressed, so P2P substitution possible
Allows doing verified blob streaming (BAO, bao-tree)
Protocol has some "smarter" methods to avoid roundtrips, but could all be statically pre-rendered
Very little data that needs to be fetched from a trusted party (or be signed)

Note: Our way of addressing blobs by their raw blake3 digest is natively compatible with iroh, the IPFS Re-implementation in Rust

Store: Status

Whole Merkle-based store implementation (and Nix NAR compat layer) is there
- exercised by the output path CI tests, and a test suite.
- three backends (Sled, in-memory, gRPC client)
- more backends and more test suites planned.
FUSE filesystem to expose the store (to Tvix Builders, "appliances")

Note: backends: RocksDB, sqlite, s3. fuse: lazy fetching of build input files | think about a minimal initrd to bring up network and mount the store, then run any closure.

Store: Demo

Store: Status (cont.)

More work on store composition needed (necessary for substition and caching)
More work on more granular blob substititon needed.
More work on bridges with Nix needed
- Get Nix to talk to a tvix-store
- Expose existing binary caches to tvix-store

Builder: Design

Build requests/Build protocol is less Nix-specific
- allows reusing builders for other usages (non-sandboxed builds, other build systems, playing with other addressing mechanisms)
Distinction between a specific build attempt and the general build recipe
- allows keeping metadata about failed builds
- stats (memory/cpu consumption)
- comparing different produced binary outputs (r11y)

Builder: Design

Invididual builds can be run in your desired container/virt engine/scheduler, as long as it speaks the same Build API
Build API composition/proxying, similar to Store composition
allows "unattended building" (evaluate nixpkgs locally and send all build requests to a remote builder)
allows tailing logs from currently/already running builds

Builder: Status

Dummy Builder implementation in go-nix (using OCI)
Some scribble notes on the Build Protocol
Glue code to trigger builds from inside builtins.derivation needs to be written
Builder implementation (using systemd-nspwan or some container engine needs to be written.
Web interface to visualize store contents and build graphs/builds/logs

Contributing

Join the IRC channel (#tvl on hackint), bridged to Matrix and XMPP
Check our issue tracker
Try to use it and tell us how you broke it!
Add various Nix bits to nix-compat

Note: or if you like what you seeing and want to sponsor parts, that's also cool :-)

Thanks to...

all TVL contributors (some drive-by, some long-term contributors)
countless Nix community members for their input on the architecture and rubberducking
NLNET and others to sponsor parts of this

Questions?

Florian Klink / flokli.de

Tvix / tvix.dev