about summary refs log tree commit diff
path: root/nix (follow)
AgeCommit message (Collapse)AuthorFilesLines
2022-07-25 r/4322 fix(ops/buildkite): do not set branch if emptyJean-François Roche1-5/+4
To run an extra step for all branches, user don't set the `branches` attribute. This change avoid setting `branches` to null in such a case. Change-Id: Iabf2f3d0411b037ece5584f30b29c7e65420b63f Reviewed-on: https://cl.tvl.fyi/c/depot/+/5975 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-07-25 r/4321 feat(ops/buildkite): Allow agents targetingJean-François Roche1-2/+4
Using the `agents` attribute, it lets the user target specific agents to run a step. Change-Id: Id6fc0981d4879b77598854e1f296dffffb33a405 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5974 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-07-25 r/4320 feat(ops/buildkite): Allow skip of individual stepsJean-François Roche1-2/+4
Using the `skip` attribute, it lets the user decide to temporarily skip a step. Change-Id: If63ce60a2b4148f041655912730acc4ae9e28534 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5971 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-06-16 r/4242 chore(nixery): use nix-1p from within the depotVincent Ambo1-2/+3
Since the source of nix-1p is checked in under //nix/nix-1p, we should use it from there if Nixery is being built inside of depot. Change-Id: Iddd54f7b93b398b2f909db6ee105366a9914a2ac Reviewed-on: https://cl.tvl.fyi/c/depot/+/5882 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su>
2022-06-06 r/4222 refactor(nix/buildkite): Optimise steps if builds are not enabledVincent Ambo1-1/+2
Generating a release-only pipeline skips a bigger chunk of eval this way (the step itself is never actually evaluated, which means we never actually compute the drv), which can be quite beneficial in terms of evaluation time. Change-Id: I2739026ddd1c6a86f82627ac26a046c5fe7359ea Reviewed-on: https://cl.tvl.fyi/c/depot/+/5830 Tested-by: BuildkiteCI Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
2022-06-06 r/4221 fix(nix/buildkite): Disable `depends_on` if build phase is not runVincent Ambo1-4/+15
Extra steps that use `depends_on` (e.g. if they need output from their parent) should not actually depend on their parents build step if the build phase is not active. This is required to actually decouple the phases. Change-Id: I398da9a8a53e97ca3c635342259fc722d54b8e4a Reviewed-on: https://cl.tvl.fyi/c/depot/+/5829 Tested-by: BuildkiteCI Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
2022-06-06 r/4220 feat(nix/buildkite): Allow toggling of individual phasesVincent Ambo1-6/+18
Using the `activePhases` attribute, the set of phases included in an evaluation can be modified. This lets users generate e.g. ONLY the release steps of a pipeline. Change-Id: Ib0c38826dd69666094d619f5f324d1baafce8134 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5828 Tested-by: BuildkiteCI Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
2022-06-03 r/4208 fix(nix/emptyDerivation): Use system from readTree parametersVincent Ambo2-5/+4
Change-Id: Ia7f9f4d0e7c06fa5433213a315c3354a83e94545 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5833 Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: Profpatsch <mail@profpatsch.de>
2022-06-03 r/4205 feat(nix/buildkite): Validate available phases in extra stepsVincent Ambo1-5/+13
This will avoid things like extra steps being accidentally ignored because of typos. Change-Id: Ic4fa5925e42a7a449f89b4cde1510e216e91da6a Reviewed-on: https://cl.tvl.fyi/c/depot/+/5827 Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com> Tested-by: BuildkiteCI
2022-06-03 r/4204 fix(nix/buildkite): Forbid 'prompt' in build phase stepsVincent Ambo1-5/+11
This would block CI on human-approval if people were allowed to do it, so they're just not. Change-Id: I8a9b657d5c91636a7b4de249b977e24fc0941a1c Reviewed-on: https://cl.tvl.fyi/c/depot/+/5826 Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com> Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-06-03 r/4203 refactor(nix/buildkite): Explicit support for build phasesVincent Ambo1-63/+108
Previously the extra steps were roughly divided into steps that run "at build time" (i.e. before we publish results to Gerrit), and "post-build" (i.e. later on). In practice, these are something like a build/release pairing, where steps running after the build results are returned are mostly run for side-effects (e.g. publishing git subtrees to external repos). This refactoring makes this distinction explicit in //nix/buildkite and changes the extraSteps API with an explicit `phases` attribute instead of the previous `postStep` attribute. In practice the previous API is still supported, but will throw evaluation warnings until an arbitrarily chosen cutoff date of 2022-10-01 at which point we will change using it into a hard error. This uncovered a few strange behaviours which we only accidentally avoided, most of which I have left TODOs about and will clean up in subsequent commits. The purpose of this commit is to allow for separate evaluations of only build or only release steps, for example if release steps are evaluated in a slightly different context (e.g. with overridden versioning that is not relevant to standard CI functionality). Change-Id: I0b0186e3824273c15a774260708702d4a5974dac Reviewed-on: https://cl.tvl.fyi/c/depot/+/5825 Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com> Tested-by: BuildkiteCI
2022-06-03 r/4202 refactor(nix/buildkite): Rename "post" steps to "release" stepsVincent Ambo1-1/+1
This is in preparation for a subsequent CL that will do much more significant changes in //nix/buildkite. Change-Id: I80a8d67d3a7d593854c8d711572483c2581e7881 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5824 Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com> Tested-by: BuildkiteCI
2022-06-02 r/4201 docs(nix/buildkite): Fix an out-of-date comment about chunk sizeVincent Ambo1-1/+1
Change-Id: Ic1f874f4ca83f9088355dc0512723ea962e0db52 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5823 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-06-01 r/4200 fix(nix/buildLisp): resolve eval problem in ccl codesterni1-2/+2
The isPowerPC predicate has been [removed], since it was misleadingly named (it just matches PowerPC, 32bit, little endian). This means the 64bit code path could now actually work. Not sure about endianess, the CCL docs don't really say much regarding that topic. [removed]: https://github.com/NixOS/nixpkgs/pull/168113 Change-Id: Icf4a8c6b1df95fa597ed87508f57aaa73e6185ed Reviewed-on: https://cl.tvl.fyi/c/depot/+/5796 Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: tazjin <tazjin@tvl.su>
2022-05-29 r/4189 feat(nix-1p): Export subtree to GitHubVincent Ambo1-0/+15
We needed a derivation for that, but this can also be used in the Nixery docs building process (which includes nix-1p). Change-Id: If97cf785a33d703af975da3b41de9b69566dfa81 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5789 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-05-29 r/4188 subtree(nix/nix-1p): Merge nix-1p into depotVincent Ambo1-0/+599
It's time to stop maintaining this on GitHub, and use the export-from-depot feature instead. Change-Id: Ic6b840bb0e8580c7214113467b4995ea3d2fae02
2022-05-28 r/4175 feat(nix/utils): add onlyDrvPath to get the drvPath w/o the outputssterni2-0/+27
I want to use this utility in a deploy script where the .drv is nix-copy-closure-d to a remote host and realized there. Consequently it doesn't make sense that the local deploy script depends on the derivation's outputs which drvPath does by default. This also came up when working on //nix/buildkite, although we didn't end up using it there. Change-Id: I952bbfd4d7e9de212569d5ee12182eb50d360f53 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5767 Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: tazjin <tazjin@tvl.su>
2022-05-28 r/4168 fix(nix/utils): remove predicates based on symlink heuristicsterni2-63/+6
Due to [nix#6579] the heuristic which allowed us to determine if a symlink points to a directory is not reliable – if restrict-eval is enabled it _will_ return wrong results. Until upstream resolves this (and we backport the patch) it is probably best to not expose this functionality at all. [nix#6579]: https://github.com/NixOS/nix/issues/6579 Change-Id: Id847c794bb279be909c5426953c4fe13c2493343 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5761 Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: tazjin <tazjin@tvl.su>
2022-05-27 r/4149 chore(3p/sources): Bump channels & overlayssterni1-1/+2
* //nix/buildLisp: disable CCL once again due to The Mysterious Runtime Bug™. * //users/tazjin/nixos: uninstall dmd which is broken in nixpkgs atm. Change-Id: I8dd2220af48a7e087584b6f50529fb8477e6a2fb Reviewed-on: https://cl.tvl.fyi/c/depot/+/5699 Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: tazjin <tazjin@tvl.su>
2022-05-26 r/4141 chore(nix/utils): Comment out tests for detection-ish of symlinksVincent Ambo1-4/+8
This is broken for (as of yet unclear reasons) with restricted evaluation mode. Change-Id: Idbc16e7e21dfb113995c045659fefe2c1a535741 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5691 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org> Autosubmit: tazjin <tazjin@tvl.su>
2022-05-04 r/4003 chore(3p/sources): Bump channels & overlayssterni1-2/+1
* //nix/buildLisp: ccl dumped images have fixed themselves… again * //3p/git: rebase patch on 2.36.0 * //3p/overlays/haskell: remove upstreamed workarounds * Disable everything depending on cgit temporarily, since it doesn't compile with git 2.36 yet. Change-Id: I9dc11c0846641341adbdcc7162cbf149a15fe0cb Reviewed-on: https://cl.tvl.fyi/c/depot/+/5519 Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-05-02 r/3996 feat(nix/lazy-deps): Add function to generate lazy binary dispatcherVincent Ambo1-0/+75
There is a reoccuring problem in readTree-type repositories that use nix-shell, where evaluation of the full set of dependencies that should be made available to users takes a noticeable amount of time, slowing down operations when `direnv` is involved. In depot, we have so far fixed this by maintaining a manual `//bin` directory which contains a set of symlinks to a central dispatch script that can dispatch to various tools in depot lazily. This script can instead be generated ad-hoc by Nix (pretty fast if we can make assumptions like `git` and `nix-build` existing on user's machines already) and added to $PATH. The function introduced in this commit implements the logic for that. The structure of the script is based on the existing `//bin/__dispatch`. This does not yet switch depot's envrc to use this new method of installing dependencies lazily. Change-Id: I92efcd9bb6aa51aa2709ad910a464e9dac97ee89 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5512 Tested-by: BuildkiteCI Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
2022-04-21 r/3986 chore(3p/sources): Bump channels and overlaysVincent Ambo1-1/+1
Changes: * updated keycloak configuration for new version * migrate to emacs28 outside of //users, re-add emacs27 but with a warning attached urging people to migrate Change-Id: I3e5765a63934541f72f6c4a8673d3b4671850c93 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5501 Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: wpcarro <wpcarro@gmail.com>
2022-04-11 r/3942 chore(3p/sources): Bump channels & overlayssterni1-1/+2
Use nixos-unstable-small which fixes CVE-2018-25032 (out of bounds write while compressing). * //users/grfn/xanthous: - Supporting random-fu 0.3 requires considerable changes and patching random-extras (https://github.com/aristidb/random-extras/pull/5). For now we downgrade random-fu and its dependency rvar to 0.2.*, forcing us to build xanthous with GHC 8.10.7, due to random-fu 0.2.* not supporting that version. Nix expressions for the downgraded packages are checked in to avoid the potential need to compile Haskell at pipeline eval time. - generic-arbitrary exposes a GenericArbitrary newtype now. This means we no longer have to implement it in xanthous downstream and patch generic-arbitrary to expose the GArbitrary type class. - Minor adjustments for lens 5.0: Xanthous.Game.Memo: clear needs to use ASetter' instead of Lens' Xanthous.Data.EntityMap: TraversableWithIndex no longer has an itraversed function. - Xanthous.Orphans: adjust for aeson's KeyMap, use KM.size explicitly instead of relying on MonoTraversable's length * //nix/buildLisp: the CCL issue has resurfaced, disabling the implementation once again. * //3p/arion: remove, as depot uses the nixpkgs package of it anyways. * //users/wpcarro: accomodate GHC 9.0.1's stricter parsing of operators. * //users/tazjin: disable rustfmt as it stopped respecting settings * //3p/overlays: upgrade home-manager until fix for serivce generation has landed upstream * //users/grfn/system: remove rr override, as the pinned commit is part of the 5.5.0 release shipped by nixpkgs. Change-Id: If229e7317ba48498f85170b57ee9053f6997ff8a Reviewed-on: https://cl.tvl.fyi/c/depot/+/5428 Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: grfn <grfn@gws.fyi> Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: wpcarro <wpcarro@gmail.com>
2022-03-30 r/3924 refactor(ops/pipelines): Configurable GraphQL token locationVincent Ambo1-1/+2
For external users of the pipeline construction, the token might be in a different path than `/run/agenix/buildkite-graphql-token`. It is made configurable through the BUILDKITE_TOKEN_PATH environment variable. This should be configured on the pipeline level to apply to all steps. Change-Id: I23c52e2d705e4134b8b013f8603f92e5533a6e44 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5424 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: asmundo <asmundo@gmail.com>
2022-03-19 r/3917 chore(3p/sources): Bump channels & overlayssterni1-2/+2
* Remove use of aliases that have been removed in nixpkgs commit a36f455905d55838a0d284656e096fbdb857cf3a: - ncat - x11 - nologin - dbus_libs - emacsPackagesGen - man-pages - pulseaudioLight Change-Id: Ide603bf48bc7f77e10e4aa715ba025aece3644fd Reviewed-on: https://cl.tvl.fyi/c/depot/+/5387 Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: grfn <grfn@gws.fyi> Reviewed-by: wpcarro <wpcarro@gmail.com>
2022-02-24 r/3866 fix(nix/buildkite): Do not set 'branches' attribute in step groupsVincent Ambo1-1/+1
This is no longer accepted by the Buildkite API and causes build failures. Functionality is unchanged since we also set the property on the step itself. Change-Id: Ib0e0908e4093ca4522711170a7179ce4bacafdc0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5324 Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com> Tested-by: BuildkiteCI
2022-02-19 r/3858 docs(yants): Update josh cloning instructionsVincent Ambo1-3/+3
Change-Id: I4e9e31488f91465df6b2c6800a3940118107524b Reviewed-on: https://cl.tvl.fyi/c/depot/+/5315 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: tazjin <tazjin@tvl.su>
2022-02-13 r/3812 chore(3p/sources): Bump channels & overlayssterni1-2/+1
* //nix/buildLisp: re-enable CCL, as the crash has been fixed upstream, although it is unclear what exactly caused / fixed it. * //ops/whitby: the kitty build broke upstream, so we can't install the terminfo on whitby for a bit. Change-Id: I5710acbe837fbc936e334b2e81f9cf00ed6ae280 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5274 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-02-08 r/3791 style(rust): Format all Rust code with rustfmtVincent Ambo1-31/+24
Change-Id: Iab7e00cc26a4f9727d3ab98691ef379921a33052 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5240 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com> Reviewed-by: Profpatsch <mail@profpatsch.de> Reviewed-by: grfn <grfn@gws.fyi> Reviewed-by: tazjin <tazjin@tvl.su>
2022-02-07 r/3785 chore(nix/buildLisp): meta.targets -> meta.ci.targetsVincent Ambo2-2/+4
Change-Id: Ic8effcc94f3fcfa283436be3e5c0477ef89e9e94 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5247 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org> Autosubmit: tazjin <tazjin@tvl.su>
2022-02-07 r/3775 refactor(readTree): deprecate meta.targets for meta.ci.targetsVincent Ambo1-10/+30
This means that we use the meta.ci attribute more consistently. The meta.targets attribute is still read, but prints a big, red warning telling people to migrate to the new one. Fixes b/176 Change-Id: Ifb4452f529cfc6bbd5018ad7374cac1c83b10045 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5238 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-02-07 r/3774 refactor(nix): Move `mkLabel` from buildkite to readTreeVincent Ambo2-9/+10
This function is more generically useful than just for pipeline construction. A subsequent commit will use it inside of readTree itself. Change-Id: I5eabd6f659726484667e060958865dddbc205762 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5237 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-02-04 r/3770 fix(nix/buildkite): read & print the store path of built derivationsVincent Ambo1-1/+1
This was useful to have in CI, e.g. when targeting a specific NixOS system. The actual result symlink which is printed is not useful. Alternative solution would be to change the wrapping of this so that we conditionally create the symlink for extra steps, but I think it's not worth the complexity of evaluating the step twice. Change-Id: Id86eb5114bec935c63a2907ec5f169fc5d41a6cc Reviewed-on: https://cl.tvl.fyi/c/depot/+/5227 Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: sterni <sternenseemann@systemli.org>
2022-02-04 r/3767 chore(3p): Bump NixOS channels and emacs overlay to 2022-02-03sterni1-1/+2
* //nix/buildLisp: This channel bump brought a bizarre regression in ccl, causing binaries to crash on thread clean up. This was likely caused by a glibc update in nixpkgs. We'll disable emitting CI targets for ccl until we can find out and fix what's going on. Change-Id: I37629f384fa99ec4ef96ce7127fa7569adecb687 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5207 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: sterni <sternenseemann@systemli.org>
2022-02-02 r/3745 fix(nix/buildLisp): make uiop:argv0 work also for ccl and sbclsterni2-1/+46
Detection is broken there, too, as UIOP relies on setting the variable before dumping the image in its portability wrapper dump-image which we don't use at all. Change-Id: If7bea5a8522a2e64707b1ee88d62d420bd00a952 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5112 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-02-02 r/3743 feat(nix/buildkite): Add parentOverride argument to extra stepsVincent Ambo1-9/+23
This can be used to override the parent derivation if its output is required, for example to inject versions which are only used during releases to avoid cache-busting. Change-Id: I2211496efa8f9bc98ea43b23e4f3f92c61a6da73 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5184 Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
2022-02-02 r/3742 fix(nix/readTree): Use overrideAttrs with derivationsVincent Ambo1-2/+13
Derivations that support overrideAttrs now have their readTree markers merged in using it, as passthru attributes. This makes the significant difference that overriding readTree targets using `overrideAttrs` keeps their readTree data intact. Change-Id: Ieef635f048781bf4782c1a28532b89a66d9ca24d Reviewed-on: https://cl.tvl.fyi/c/depot/+/5186 Tested-by: BuildkiteCI Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com> Autosubmit: tazjin <tazjin@tvl.su>
2022-02-02 r/3740 fix(nix/buildGo): Use overrideAttrs to add metadataVincent Ambo1-5/+7
Required for using overrideAttrs in readTree (cl/5186). Since this uses pkgs.runCommand we know that overrideAttrs is available. Change-Id: I18fdcc34cc79872834052caf4bf74555fdb766ce Reviewed-on: https://cl.tvl.fyi/c/depot/+/5187 Tested-by: BuildkiteCI Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com> Autosubmit: tazjin <tazjin@tvl.su>
2022-01-31 r/3723 style: format entire depot with nixpkgs-fmtVincent Ambo43-1010/+1326
This CL can be used to compare the style of nixpkgs-fmt against other formatters (nixpkgs, alejandra). Change-Id: I87c6abff6bcb546b02ead15ad0405f81e01b6d9e Reviewed-on: https://cl.tvl.fyi/c/depot/+/4397 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org> Reviewed-by: lukegb <lukegb@tvl.fyi> Reviewed-by: wpcarro <wpcarro@gmail.com> Reviewed-by: Profpatsch <mail@profpatsch.de> Reviewed-by: kanepyork <rikingcoding@gmail.com> Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: cynthia <cynthia@tvl.fyi> Reviewed-by: edef <edef@edef.eu> Reviewed-by: eta <tvl@eta.st> Reviewed-by: grfn <grfn@gws.fyi>
2022-01-29 r/3701 chore(nix/buildkite): reduce chunk size to 192Vincent Ambo1-1/+1
We've seen the famous 1 minute timeout on Buildkite again, probably due to something (keys in targets?) increasing the overall payload size of our chunks. This reduces the chunk size by 25%. Lets keep an eye on it with this value ... Change-Id: I6bf0e9e4ab0d5b8de22773e6cd5da8d0959cc448 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5105 Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: sterni <sternenseemann@systemli.org>
2022-01-28 r/3692 feat(3p/overlays/ecl-static): 21.2.1 -> 1c98924sterni1-4/+3
Seems like some issues to do with bytecode compilation have been fixed at HEAD. closer-mop compiles again and an ironclad failure with the next quicklisp/channel bump is avoided. In this change pathname handling in ECL also changed somehow, causing it to make the :directory part absolute by prefixing it with a slash which made ld.bfd unhappy while linking an output path that began with a double slash. This problem can be avoided by constructing the path as ANSI Common Lisp intended. The truename on the out path is important to make it recognize that it is indeed a directory. Change-Id: I5e744022b92502f99ac0b33411a6be443707e200 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5076 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-01-24 r/3668 feat(nix/buildkite): Support un-gated post-build extra stepsVincent Ambo1-2/+7
Allows users to define steps with `postBuild = true` which always run after :duck:, but do not require human approvals. This can be useful for things like unconditional release steps. Change-Id: Idbf6c48a9dedcfc6cc9b7f098423364e2fa72d2d Reviewed-on: https://cl.tvl.fyi/c/depot/+/5052 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org> Autosubmit: tazjin <tazjin@tvl.su>
2022-01-24 r/3667 refactor(nix/buildkite): Restrict step conditionals to refs onlyVincent Ambo1-11/+9
The previous `condition` abstraction which allowed the full set of Buildkite conditionals is way too leaky (it lets users to very Buildkite-specific things which we may not want to allow, and which are mostly not relevant to a pure evaluation). Supporting only the `branches` condition (native to Buildkite) should make it possible to port this to other future CI systems later. Change-Id: Ib8adcc41db4f1a3566cbeecf13a4228403105c1f Reviewed-on: https://cl.tvl.fyi/c/depot/+/5051 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org> Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com> Autosubmit: tazjin <tazjin@tvl.su>
2022-01-24 r/3666 feat(nix/buildkite): Add support for gated stepsVincent Ambo1-7/+69
Adds support for extra build steps that specify a `prompt`. These steps will be run at the end of the pipeline and will be gated by human approval. This mechanism can be used to, for example, stage releases of software released from depot that are subject to approval. Change-Id: I97bb505664a2ccf01142286f14e20a370afaa345 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5033 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org> Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com> Autosubmit: tazjin <tazjin@tvl.su>
2022-01-22 r/3657 refactor(ops/pipelines): Split build/post steps into separate chunksVincent Ambo1-16/+9
This will create `build-chunk-$n.json` files for steps that should run _before_ duck, and `post-chunk-$n.json` files for steps that should run after duck. The post steps are not yet uploaded to Buildkite, but we also don't have any right now. Change-Id: I7e1b59cf55a8bf1d97266f6e988aa496959077bf Reviewed-on: https://cl.tvl.fyi/c/depot/+/5047 Tested-by: BuildkiteCI Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com> Autosubmit: tazjin <tazjin@tvl.su>
2022-01-21 r/3652 feat(nix/buildkite): Add support for extra stepsVincent Ambo1-17/+85
This introduces a new feature to our CI system in which targets can declare extra steps in `meta.ci.extraSteps`. See the comment in //nix/buildkite/default.nix for an explanation of how these extra steps are defined. Change-Id: Icce2890c743286dd37f43024cd390dcebac8cdba Reviewed-on: https://cl.tvl.fyi/c/depot/+/5008 Tested-by: BuildkiteCI Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
2022-01-20 r/3651 chore(nix/buildkite): Drop `--fork-point` parameterVincent Ambo1-1/+1
The --fork-point parameter is dependent on reflog data which may get garbage collected. This can lead to flaky behaviour where it returns no results and fails if `git gc` recently ran (Buildkite will do this occasionally). Though the parameter is semantically closer to what we're looking for, the output is *usually* the same commit since we're not dealing with more than one thing to compare. Change-Id: Idc31e7a26fda2b7113edfa162d9d3811b1a01bf6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5032 Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: sterni <sternenseemann@systemli.org>
2022-01-20 r/3640 fix(nix/buildkite): prevent swallowing errors from git-merge-baseVincent Ambo1-5/+7
The `local` usage we had before would silently swallow non-zero exit statuses from the substituted git commands. For some (as of yet unknown) reason, `git merge-base` seems to sometimes silently fail and produce no output, which broke the rest of the script logic. This change will lead to an earlier error, but we don't know if it is a fix for the actual cause of the git-merge-base problem because the shape of that problem is unclear. Change-Id: I4555c8638da450263fa2fd2c274dfdb69f65578e Reviewed-on: https://cl.tvl.fyi/c/depot/+/5012 Tested-by: BuildkiteCI Reviewed-by: kn <klemens@posteo.de>
2022-01-19 r/3637 refactor(nix/buildkite): Move fetch-parent-targets script hereVincent Ambo1-0/+41
This is no longer TVL-specific and should live here with the other generalised stuff. Change-Id: I95a1b4c0321f34812162d6fd40568269abf639dd Reviewed-on: https://cl.tvl.fyi/c/depot/+/5006 Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>