1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
|
---
title: "Tvix - Architecture & data flow"
numbersections: true
author:
- adisbladis
- flokli
- tazjin
email:
- adis@blad.is
- mail@tazj.in
lang: en-GB
classoption:
- twocolumn
header-includes:
- \usepackage{caption, graphicx, tikz, aeguill, pdflscape}
---
# Background
We intend for Tvix tooling to be more decoupled than the existing,
monolithic Nix implementation. In practice, we expect to gain several
benefits from this, such as:
- Ability to use different builders
- Ability to use different store implementations
- No monopolisation of the implementation, allowing users to replace
components that they are unhappy with (up to and including the
language evaluator)
- Less hidden intra-dependencies between tools due to explicit RPC/IPC
boundaries
Communication between different components of the system will use
gRPC. The rest of this document outlines the components.
# Components
## Coordinator
*Purpose:* The coordinator (in the simplest case, the Tvix CLI tool)
oversees the flow of a build process and delegates tasks to the right
subcomponents. For example, if a user runs the equivalent of
`nix-build` in a folder containing a `default.nix` file, the
coordinator will invoke the evaluator, pass the resulting derivations
to the builder and coordinate any necessary store interactions (for
substitution and other purposes).
While many users are likely to use the CLI tool as their primary
method of interacting with Tvix, it is not unlikely that alternative
coordinators (e.g. for a distributed, "Nix-native" CI system) would be
implemented. To facilitate this, we are considering implementing the
coordinator on top of a state-machine model that would make it
possible to reuse the FSM logic without tying it to any particular
kind of application.
## Evaluator
*Purpose:* Eval takes care of evaluating Nix code. In a typical build
flow it would be responsible for producing derivations. It can also be
used as a standalone tool, for example, in use-cases where Nix is used
to generate configuration without any build or store involvement.
*Requirements:* For now, it will run on the machine invoking the build
command itself. We give it filesystem access to handle things like
imports or `builtins.readFile`.
In the future, we might abstract away raw filesystem access by
allowing the evaluator to request files from the coordinator (which
will query the store for it). This might get messy, and the benefits
are questionable. We might be okay with running the evaluator with
filesystem access for now and can extend the interface if the need
arises.
## Builder
*Purpose:* A builder receives derivations from the coordinator and
builds them.
By making builder a standardised interface it's possible to make the
sandboxing mechanism used by the build process pluggable.
Nix is currently using a hard-coded
[libseccomp](https://github.com/seccomp/libseccomp) based sandboxing
mechanism and another one based on
[sandboxd](https://www.unix.com/man-page/mojave/8/sandboxd/) on macOS.
These are only separated by [compiler preprocessor
macros](https://gcc.gnu.org/onlinedocs/cpp/Ifdef.html) within the same
source files despite having very little in common with each other.
This makes experimentation with alternative backends difficult and
porting Nix to other platforms harder than it has to be. We want to
write a new Linux builder which uses
[OCI](https://github.com/opencontainers/runtime-spec), the current
dominant Linux containerisation technology, by default.
With a well-defined builder abstraction, it's also easy to imagine
other backends such as a Kubernetes-based one in the future.
## Store
*Purpose:* Store takes care of storing build results. It provides a
unified interface to get file paths and upload new ones.
Most likely, we will end up with multiple implementations of store, a
few possible ones that come to mind are:
- Local
- SSH
- GCP
- S3
- Ceph
# Figures
|
