chore(tvix/docs): move from doc r/2378

Most other docs folders in the repo are called `doc` too, let's make
this consistent.

Change-Id: Icd712429b51763076548c977321e370f2a77877e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/2741
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>

5 files changed, 241 insertions, 0 deletions

diff --git a/tvix/docs/.gitignore b/tvix/docs/.gitignore
new file mode 100644
index 0000000000..77699ee8a3
--- /dev/null
+++ b/tvix/docs/.gitignore
@@ -0,0 +1,2 @@
+*.svg
+*.html
diff --git a/tvix/docs/Makefile b/tvix/docs/Makefile
new file mode 100644
index 0000000000..ba9e2bdef6
--- /dev/null
+++ b/tvix/docs/Makefile
@@ -0,0 +1,12 @@
+all: build
+
+puml:
+	plantuml *.puml -tsvg
+
+html:
+	pandoc *.md -f markdown --self-contained -t html -s -o tvix.html --csl=${CSL}
+
+build: puml html
+
+clean:
+	rm -f *.tex *.pdf *.png *.svg
diff --git a/tvix/docs/component-flow.puml b/tvix/docs/component-flow.puml
new file mode 100644
index 0000000000..3bcddbe746
--- /dev/null
+++ b/tvix/docs/component-flow.puml
@@ -0,0 +1,74 @@
+@startuml
+
+title Tvix build flow
+
+actor User
+participant CLI
+participant "Coordinator" as Coord
+participant "Evaluator" as Eval
+database Store
+participant "Builder" as Build
+
+note over CLI,Eval
+    Typically runs locally on the invoking machine
+end note
+/ note over Store, Build
+    Can be either local or remote
+end note
+
+User-->CLI: User initiates build of `hello` (analogous to `nix-build -f '<nixpkgs>' -A hello`)
+
+CLI-->Coord: CLI invokes coordinator
+
+Coord-->Eval: Sends message to start evaluation of `<nixpkgs>` (path lookup) with attribute `hello`
+note right: The paths to the evaluator are local file system paths
+
+Coord<--Eval: Yields derivations to be built
+note right
+    Immediately starts streaming derivations as they are instantiated across
+    the dependency graph so they can be built while the evaluation is still running.
+
+    There are two types of build requests: One for regular "fire and forget" builds
+    and another for IFD (import from derivation).
+
+    These are distinct because IFD needs to be fed back into the evaluator for
+    further processing while a regular build does not.
+end note
+
+loop while has more derivations
+
+    Coord-->Store: Check if desired paths are in store
+    alt Store has path
+        Coord<--Store: Success response
+    else Store does not have path
+        Coord-->Build: Request derivation to be built
+        note left
+            The build request optionally includes a desired store.
+            If a builder is aware of how to push to the store it will do so
+            directly when the build is finished.
+
+            If the store is not known by the builder results will be streamed
+            back to the coordinator for store addition.
+        end note
+
+        alt Build failure
+            Coord<--Build: Fail response
+            note left: It's up to the coordinator whether to exit on build failure
+        else Build success
+            alt Known store
+                Build-->Store: Push outputs to store
+                Build<--Coord: Send success & pushed response
+            else Unknown store
+                Build<--Coord: Send success & not pushed response
+                Coord<--Build: Stream build outputs
+                Coord-->Store: Push outputs to store
+            end
+        end
+
+    end
+end
+
+CLI<--Coord: Respond success/fail
+User<--CLI: Exit success/fail
+
+@enduml
diff --git a/tvix/docs/components.md b/tvix/docs/components.md
new file mode 100644
index 0000000000..19e7baa3ec
--- /dev/null
+++ b/tvix/docs/components.md
@@ -0,0 +1,114 @@
+---
+title: "Tvix - Architecture & data flow"
+numbersections: true
+author:
+- adisbladis
+- flokli
+- tazjin
+email:
+- adis@blad.is
+- mail@tazj.in
+lang: en-GB
+classoption:
+- twocolumn
+header-includes:
+- \usepackage{caption, graphicx, tikz, aeguill, pdflscape}
+---
+
+# Background
+
+We intend for Tvix tooling to be more decoupled than the existing,
+monolithic Nix implementation. In practice, we expect to gain several
+benefits from this, such as:
+
+- Ability to use different builders
+- Ability to use different store implementations
+- No monopolisation of the implementation, allowing users to replace
+  components that they are unhappy with (up to and including the
+  language evaluator)
+- Less hidden intra-dependencies between tools due to explicit RPC/IPC
+  boundaries
+
+Communication between different components of the system will use
+gRPC. The rest of this document outlines the components.
+
+# Components
+
+## Coordinator
+
+*Purpose:* The coordinator (in the simplest case, the Tvix CLI tool)
+oversees the flow of a build process and delegates tasks to the right
+subcomponents. For example, if a user runs the equivalent of
+`nix-build` in a folder containing a `default.nix` file, the
+coordinator will invoke the evaluator, pass the resulting derivations
+to the builder and coordinate any necessary store interactions (for
+substitution and other purposes).
+
+While many users are likely to use the CLI tool as their primary
+method of interacting with Tvix, it is not unlikely that alternative
+coordinators (e.g. for a distributed, "Nix-native" CI system) would be
+implemented. To facilitate this, we are considering implementing the
+coordinator on top of a state-machine model that would make it
+possible to reuse the FSM logic without tying it to any particular
+kind of application.
+
+## Evaluator
+
+*Purpose:* Eval takes care of evaluating Nix code. In a typical build
+flow it would be responsible for producing derivations. It can also be
+used as a standalone tool, for example, in use-cases where Nix is used
+to generate configuration without any build or store involvement.
+
+*Requirements:* For now, it will run on the machine invoking the build
+command itself. We give it filesystem access to handle things like
+imports or `builtins.readFile`.
+
+In the future, we might abstract away raw filesystem access by
+allowing the evaluator to request files from the coordinator (which
+will query the store for it). This might get messy, and the benefits
+are questionable. We might be okay with running the evaluator with
+filesystem access for now and can extend the interface if the need
+arises.
+
+## Builder
+
+*Purpose:* A builder receives derivations from the coordinator and
+builds them.
+
+By making builder a standardised interface it's possible to make the
+sandboxing mechanism used by the build process pluggable.
+
+Nix is currently using a hard-coded
+[libseccomp](https://github.com/seccomp/libseccomp) based sandboxing
+mechanism and another one based on
+[sandboxd](https://www.unix.com/man-page/mojave/8/sandboxd/) on macOS.
+These are only separated by [compiler preprocessor
+macros](https://gcc.gnu.org/onlinedocs/cpp/Ifdef.html) within the same
+source files despite having very little in common with each other.
+
+This makes experimentation with alternative backends difficult and
+porting Nix to other platforms harder than it has to be. We want to
+write a new Linux builder which uses
+[OCI](https://github.com/opencontainers/runtime-spec), the current
+dominant Linux containerisation technology, by default.
+
+With a well-defined builder abstraction, it's also easy to imagine
+other backends such as a Kubernetes-based one in the future.
+
+## Store
+
+*Purpose:* Store takes care of storing build results. It provides a
+unified interface to get file paths and upload new ones.
+
+Most likely, we will end up with multiple implementations of store, a
+few possible ones that come to mind are:
+
+- Local
+- SSH
+- GCP
+- S3
+- Ceph
+
+# Figures
+
+![component flow](./component-flow.svg)
diff --git a/tvix/docs/default.nix b/tvix/docs/default.nix
new file mode 100644
index 0000000000..606027424a
--- /dev/null
+++ b/tvix/docs/default.nix
@@ -0,0 +1,39 @@
+{ pkgs, lib, ... }:
+
+let
+
+  tl = pkgs.texlive.combine {
+    inherit (pkgs.texlive) scheme-medium wrapfig ulem capt-of
+    titlesec preprint enumitem paralist ctex environ svg
+    beamer trimspaces zhnumber changepage framed pdfpages
+    fvextra minted upquote ifplatform xstring;
+  };
+
+  csl = pkgs.fetchurl {
+    name = "numeric.csl";
+    url = "https://gist.githubusercontent.com/bwiernik/8c6f39cf51ceb3a03107/raw/1d75c2d62113ffbba6ed03a47ad99bde86934f2b/APA%2520Numeric";
+    sha256 = "1yfhhnhbzvhrv93baz98frmgsx5y442nzhb0l956l4j35fb0cc3h";
+  };
+
+in pkgs.stdenv.mkDerivation {
+  pname = "tvix-doc";
+  version = "0.1";
+
+  src = lib.cleanSource ./.;
+
+  CSL = csl;
+
+  nativeBuildInputs = [
+    pkgs.pandoc
+    pkgs.plantuml
+    tl
+  ];
+
+  installPhase = ''
+    runHook preInstall
+    mkdir -p $out
+    cp -v *.html $out/
+    runHook postSubmit
+  '';
+
+}