about summary refs log tree commit diff
path: root/ops/pipelines/static-pipeline.yaml
diff options
context:
space:
mode:
authorVincent Ambo <mail@tazj.in>2021-10-01T10·43+0300
committertazjin <tazjin@tvl.su>2022-01-17T10·26+0000
commit0a21da2bb4db308d8cf01f454e7b9c3a01b8947f (patch)
tree5b672426fbad1b4d0ca03c8449f872e83b54ec50 /ops/pipelines/static-pipeline.yaml
parent546251678a5206b211c64e5093d2df986f1afc84 (diff)
feat(ops/pipelines): Create drvmap structure for each commit r/3601
Always create a structure that maps all targets to derivations, and
persist it as a JSON file.

This relates to some of the ideas expressed in:

https://docs.google.com/document/d/16A0a5oUxH1VoiSM8hyFyLW0WiUYpNo2e2D6FTW4BlH8/edit

The file is always uploaded to Buildkite as an artifact. This allows
for retrieving it based on the commit ID in a Buildkite GraphQL query.

By default, Buildkite stores artefacts for 6 months. Storage location
can be overridden (with custom retention) through some environment
variables, but for now at TVL the Buildkite-managed storage is fine.
See also: https://buildkite.com/docs/pipelines/artifacts

In the subsequent filtering implementation, when diffing commits
across a time-range that exceeds artefact retention time, we should
simply default to building everything.

Change-Id: I6d808461cd1c1fdd6983ba8c8ef075736d42caa7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/3662
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
Diffstat (limited to 'ops/pipelines/static-pipeline.yaml')
-rw-r--r--ops/pipelines/static-pipeline.yaml3
1 files changed, 3 insertions, 0 deletions
diff --git a/ops/pipelines/static-pipeline.yaml b/ops/pipelines/static-pipeline.yaml
index 78b53d692bb7..7fcb716b25f8 100644
--- a/ops/pipelines/static-pipeline.yaml
+++ b/ops/pipelines/static-pipeline.yaml
@@ -6,6 +6,7 @@
 ---
 steps:
   - label: ":llama:"
+    key: "pipeline-gen"
     command: |
       set -ue
 
@@ -22,6 +23,8 @@ steps:
         buildkite-agent pipeline upload $$chunk
       done
 
+      buildkite-agent artifact upload pipeline/drvmap.json
+
   # Wait for all previous steps to complete.
   - wait: null
     continue_on_failure: true