about summary refs log tree commit diff
path: root/ops/pipelines (follow)
AgeCommit message (Collapse)AuthorFilesLines
2021-04-12 r/2492 feat(ops/pipelines): pass --show-trace to nix-buildsterni1-1/+1
--show-trace should make it easier to debug tricky evaluation errors without running nix-build -A ops.pipelines.depot locally again. Change-Id: Ice540562c3b389fc2a49ec1fc0adacb17db2a528 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2947 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-04-11 r/2480 fix(pipelines/depot): Buildkite refers to branches by full refVincent Ambo1-1/+1
This change is required to run the :anchor: step on canon builds. Change-Id: Ib3cebac67c9f5337b27a948f120b0a9ba834ef2a Reviewed-on: https://cl.tvl.fyi/c/depot/+/2932 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org> Reviewed-by: glittershark <grfn@gws.fyi>
2021-04-11 r/2477 feat(ops/pipelines): Add gcroots for depot builds on canonVincent Ambo1-2/+22
Adds a conditional build step that only runs on the canon branch, and only if :duck: (the status reporting step) succeeds, which creates a new Nix GC root for all depot targets named `depot-canon`. In practice this might be a bit racey, as canon builds are not guaranteed to succeed in order (though it is likely). This shouldn't matter much in practice: We only want to prevent rebuilds of the whole world. This fixes b/102 Change-Id: Id3d0bf4158bffcb1ed6929888a29d31609b6ece1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2904 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2021-01-30 r/2160 fix(ops/piplines/static-pipeline): add --show-trace to nix-buildProfpatsch1-1/+1
Change-Id: Ib0473f916b1436934844e620ce981f52d11e8512 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2467 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-11-17 r/1882 feat(ops/pipelines): Check in the static pipelineVincent Ambo1-0/+15
This file represents the static pipeline which is configured in the Buildkite web UI. Updates to this file should be applied in the admin interface. These steps are responsible for launching the dynamic pipeline evaluation, or falling back to the fallback pipeline if evaluation fails. Change-Id: I6d7dd623cde65e8c69faea729f737c9bba00c2fb Reviewed-on: https://cl.tvl.fyi/c/depot/+/2103 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-11-17 r/1881 feat(ops/pipelines): Add a fallback Buildkite configurationVincent Ambo1-0/+8
This adds a simple fallback Buildkite pipeline configuration which always fails the pipeline, but correctly reports back the failure status. Note that this also requires changes in the Buildkite configuration that is not in version-control. Relates to b/66. Change-Id: I6802a6f76448c3893798a06d514e6ccba0f50dd2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2102 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-31 r/1748 feat(ci): Add subtarget support for buildsVincent Ambo1-6/+13
We have naturally evolved a distinction between logical and physical targets. Physical targets are those which correspond directly to a tree location on disk and can be built with `-A path.to.files`, while logical targets are those that are exported from within an expression but do not have a corresponding file on disk. This change adds support for exporting logical targets from any tree location by adding a `meta.targets` attribute containing keys into itself, which will be consumed by the CI target gathering logic and included in the generated pipeline. Note that the labels for subtargets are syntactically different to emphasise that they do not correspond to a file location. For example, this change enables 'ops.nixos.whitbySystem' as a subtarget, which is labeled in CI as `ops/nixos:whitbySystem`. Change-Id: Ied09647a62c2ba98e3914548e3742ad422c63ecf Reviewed-on: https://cl.tvl.fyi/c/depot/+/1893 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-31 r/1747 feat(ops/pipelines): Dynamically generate CI pipeline from targetsVincent Ambo1-13/+66
Create the pipeline by outputting a file that contains nix-build invocations for each target's *derivation path*. Each invocation has a generated Nix expression passed to it with `-E` which fetches the correct target from the tree while correctly handling targets with strange characters (such as in Go-packages). This makes it possible to run target-level granular pipelines. We're getting somewhere! Change-Id: Ia6946e389dafd1d4926130bb8891446d6e17133b Reviewed-on: https://cl.tvl.fyi/c/depot/+/1855 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi> Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-08-26 r/1725 feat: Implement automatic CI target detection for the depotVincent Ambo1-1/+1
Automatically walk the entire depot tree and pick out things that are "buildable", then include them in the attribute `ci.targets` (which is now also the target for CI builds). A long time ago, in a land far away, we (well, I, at the time) had a prototype of this which ran into constant issues with infinite recursions while trying to walk the tree. In fact, this is why readTree originally gained the `__readTree`-attribute which marks things that were imported automatically. Based on some code edef whipped up earlier (with the breakthrough being that we also add the attribute to top-level folders, which suddenly resolves a whole bunch of problems), I've now implemented this actually working version. At the moment all builds still happen as one big bag of builds, but at some point we will granularise this. Change-Id: I86f12ce7f63dae98e7e5c6646a4e9d220de783f2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1854 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com> Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-17 r/1669 docs: Update README for the repository itselfVincent Ambo1-3/+5
Adds a lot more information about what is actually going on here, as well as links to relevant things such as our Monorepo document. I'm aware that I didn't make all `//...` links clickable, but I realised at some point that it might be easier to just update cheddar to support highlighting those natively :-) Change-Id: Icbb212a6c07a5cf38ab8e65d83a64bec783eb8d0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1768 Tested-by: BuildkiteCI Reviewed-by: isomer <isomer@tvl.fyi> Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-07-17 r/1366 feat(ci): run buf check lint in CIKane York1-0/+4
Breaking change detection will run but not enforce. Emoji of water buffalo was chosen by @pedge fiat in the bufbuild slack. Change-Id: Ie292f2bfddc0e3bc512e4a138c0b5d0fa2603bad Reviewed-on: https://cl.tvl.fyi/c/depot/+/1247 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: glittershark <grfn@gws.fyi>
2020-06-29 r/1117 feat(pipelines/depot): Run with --show-traceGriffin Smith1-1/+1
So if an evaluation fails we get a stacktrace Change-Id: I54cdc9e93c765ef7cf3a4d0cd79e6d067f4789d3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/733
2020-06-28 r/1106 feat(besadii): Enable automatic builds for CLsVincent Ambo1-1/+1
This expands builds to also be triggered for updates to CL refs. The message displayed on Buildkite will contain a link back to the CL (& patchset) from which the build was triggered. Change-Id: Ib36dee454aeb11d623b89c78b384359ee7ea3477 Reviewed-on: https://cl.tvl.fyi/c/depot/+/708 Reviewed-by: ericvolp12 <ericvolp12@gmail.com> Reviewed-by: isomer <isomer@tvl.fyi>
2020-06-27 r/1099 feat(ops/pipelines): Add Buildkite pipeline configurationVincent Ambo2-0/+24
This adds configuration which generates the structure expected for Buildkite pipelines, which can then be dynamically ingested by Buildkite when a pipeline is triggered. Change-Id: I61e3dc3affb19c1f2550ef827fa73b17f8d8ae47 Reviewed-on: https://cl.tvl.fyi/c/depot/+/627 Reviewed-by: ericvolp12 <ericvolp12@gmail.com> Reviewed-by: lukegb <lukegb@tvl.fyi>