feat(ops/pipelines): Dynamically generate CI pipeline from targets r/1747

Create the pipeline by outputting a file that contains nix-build
invocations for each target's *derivation path*.

Each invocation has a generated Nix expression passed to it with `-E`
which fetches the correct target from the tree while correctly
handling targets with strange characters (such as in Go-packages).

This makes it possible to run target-level granular pipelines. We're
getting somewhere!

Change-Id: Ia6946e389dafd1d4926130bb8891446d6e17133b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1855
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: lukegb <lukegb@tvl.fyi>

1 files changed, 8 insertions, 3 deletions

diff --git a/default.nix b/default.nix
index b0b5399c4bc2..151d8987ea3d 100644
--- a/default.nix
+++ b/default.nix
@@ -73,9 +73,14 @@ in fix(self: {
 
   # List of all buildable targets, for CI purposes.
   #
-  # Note: This *must* be a nested attribute, otherwise we will get
-  # infinite recursion and everything blows up.
-  ci.targets = gather self;
+  # Note: To prevent infinite recursion, this *must* be a nested
+  # attribute set (which does not have a __readTree attribute).
+  ci.targets = gather (self // {
+    # remove the pipelines themselves from the set over which to
+    # generate pipelines because that also leads to infinite
+    # recursion.
+    ops = self.ops // { pipelines = null; };
+  });
 }
 
 # Add local packages as structured by readTree