about summary refs log tree commit diff
path: root/nix
AgeCommit message (Collapse)AuthorFilesLines
2023-06-01 r/6226 docs(nix/buildkite): escape "release" in warningFlorian Klink1-1/+1
When using the buildkite tooling with `postBuild` parameter set, the following error message is shown: ``` This replaces the previous boolean postBuild API in extra step definitions. Please remove the postBuild parameter from this step and instead set phase = release;. ``` However, `release` is no symbol, this should be a string. Update the message to reflect that. Change-Id: I341c81d862204db06946512428910cada28481ca Reviewed-on: https://cl.tvl.fyi/c/depot/+/8690 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-05-31 r/6224 chore: remove comments formerly meant for alignmentsterni2-2/+0
/**/ is a nice way to align if statements which doesn't work with nixpkgs-fmt, since it'll reflow the comment to the line preceding the if. Consequently, we can delete these comments now. Change-Id: Ifa5327f846a903e07607b21f8eedbc32fc36f758 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8689 Reviewed-by: sterni <sternenseemann@systemli.org> Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2023-04-24 r/6106 feat(nix/lazy-deps): add override pattern for depsVincent Ambo1-50/+52
Introduces a `.overrideDeps` attribute with which additional tools can be supplied. This works like `.override` in nixpkgs. Change-Id: I69a009b51f7f073a2d030eda5e3b5310e0f8e883 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8491 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-02-28 r/5862 feat(nix/readTree): implement .skip-tree marker for subtree ignoringVincent Ambo6-18/+76
With this change, readTree gains the ability to notice a `.skip-tree` marker in addition to the `.skip-subtree` marker. The behaviour of the new marker will completely ignore the folder that the marker is located in (i.e. no node will be present for it in the parent at all). To make this work, the recursive function in readTree had to be modified to return a sentinel value (noting that a tree has requested to be skipped) which is then filtered out when constructing the list of children. The actual `readTree` function is now a wrapper around this inner, sentinel-yielding implementation which unwraps the result set. For obvious reasons, `.skip-tree` is not allowed at the top-level and readTree will throw an error if it encounters it there. Fixes: b/244 Change-Id: Ica731bc1af356e881fd3d31c7109f62ffd2762ea Reviewed-on: https://cl.tvl.fyi/c/depot/+/8185 Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2022-12-27 r/5501 feat(nix/bufCheck): always run from repo rootFlorian Klink1-3/+3
This doesn't work when run from subdirectories, and going there manually before running `mg run //nix/bufCheck` is annoying. Change-Id: Icd30a7596ff0dfe2781f7cfa1b4085cbfdebd6ac Reviewed-on: https://cl.tvl.fyi/c/depot/+/7324 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2022-12-27 r/5500 feat(nix/bufCheck): ensure .pb.go is up to dateFlorian Klink1-2/+15
Change-Id: I03e1c3aced266966f6959dbce05c1f1fb1480b5a Reviewed-on: https://cl.tvl.fyi/c/depot/+/7323 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-12-26 r/5497 chore(nix/buildGo): drop thing.proto from exampleFlorian Klink2-18/+0
This was missed from cl/7536. Change-Id: I12c12bf91bad00c30bb63518d65c320a61cbf442 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7641 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-12-26 r/5491 chore(nix/buildGo): drop buildGo.proto and buildGo.grpcFlorian Klink3-134/+2
As described in https://b.tvl.fyi/issues/221#comment-344, buildGo.proto was a mistake and should be removed. Change-Id: Ic588a5e8eea58e83e3ec9a37ac681ce526028718 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7536 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-12-26 r/5488 feat(nix/lazy-deps): add passthru.devShellzimbatm1-0/+6
Allows to easily convert a lazy-deps into a nix-shell compatible derivation. Change-Id: I65a0609642f12e0d00658c69cc394a73dddccd16 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5855 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-12-26 r/5487 refactor(nix/lazy-deps): use runCommandzimbatm1-29/+37
writeTextFile is nice, but not flexible enough to allow the passthru argument needed for a follow-up change. Change-Id: I4f0cffd0f29b2c06b0155101d3806c9c5745c37a Reviewed-on: https://cl.tvl.fyi/c/depot/+/5854 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-11-09 r/5274 revert(nix/utils): onlyDrvPath to get the drvPath w/o the outputssterni2-27/+0
This reverts commit 6813598c17959862734e5878d745e7dd8a197717. Reason for revert: This function is a re-implementation of builtins.unsafeDiscardOutputDependency which I missed at the time. Change-Id: I5bb52bfd5e8d51defaf90ee795b0fe99be84f6db Reviewed-on: https://cl.tvl.fyi/c/depot/+/7265 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-11-03 r/5232 chore(3p/sources): Bump channels & overlays (OpenSSL edition)sterni1-1/+4
* //ops/machines/whitby: Disable grafana, since the grafana module was changed upstream in a way that our configuration no longer works. Since the OpenSSL security update is relatively pressing, adapting the grafana configuration beforehand is not a hard requirement. See https://github.com/NixOS/nixpkgs/pull/191768. * //tools/depotfmt: keep Go at version 1.18 to forgo a reformat of the tree. * //nix/buildGo: keep Go at version 1.18, as 1.19 changed the CLI interface (?) in a way that breaks buildGo. * //3p/overlays/tvl: drop upstreamed tdlib upgrade. * //3p/overlays/tvl: patch buf to work around breakage due to git 2.38.1 TODO items for Go are tracked in b/215. Change-Id: Ie08fef49cf3db12e6b5225a8b992a990ddc5b642 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7141 Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: grfn <grfn@gws.fyi> Reviewed-by: tazjin <tazjin@tvl.su>
2022-11-01 r/5227 feat(ops/buildkite): Support "soft failures"William Carroll1-0/+4
Example Usage: ```nix { pkgs, ... }: (pkgs.writeText "foo" "bar").overrideAttrs (_: { meta.ci.extraSteps.bar = { label = "testing soft_fail"; softFail = [ { exit_status = 253; } { exit_status = 251; } ]; command = pkgs.writeShellScript "fail" '' # Trigger a soft failure by exiting 253 or 251 exit 253 ''; }; }) ``` Buildkite Documentation: https://buildkite.com/changelog/56-command-steps-can-now-be-made-to-soft-fail Change-Id: Idfce15a8d9cde568aa24ebd03956ada399bba5d2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7076 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-21 r/5173 chore(buf): Use nixpkgs-provided bufFlorian Klink1-2/+2
The version of buf used is quite old. nixpkgs provides a more recent version, but it requires us to migrate config to the latest version. depot_scanner.proto doesn't honor some of the conventions, so we need allow_comment_ignores and drop a bunch of comments in there. Change-Id: Ic978fe92fb7c8471f58c137497528f18aad8f3ab Reviewed-on: https://cl.tvl.fyi/c/depot/+/7053 Reviewed-by: sterni <sternenseemann@systemli.org> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: tazjin <tazjin@tvl.su>
2022-10-20 r/5171 refactor(nix/dependency-analyzer): use unsafeDiscardOutputDependencysterni1-18/+2
Turns out that pathContextDrvPath already exists as a builtin which is very convenient. Actually somewhat embarassing that I missed this for so long. Change-Id: Ieb5e113d70dec548b3053911ff9dbe9ed48402be Reviewed-on: https://cl.tvl.fyi/c/depot/+/7050 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-20 r/5169 fix(nix/buildkite) don't fail if gated step has no branchesEvgeny Zemtsov1-1/+1
Change-Id: Id3365c18bc3904b42e8b129a513cff4480a2cb07 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7051 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-11 r/5108 fix(nix/dependency-analyzer): don't read from non-drv store pathssterni1-1/+7
Emitting dependencies on non-drv store paths from drv directDrvDeps is fine and actually correct, even though the Nix 2.3 version can't do it at the moment (but this would change when the placeholder implementation is replaced using a drv parser). However, we can't necessarily determine the dependencies of non-drv store paths because such store paths may be binary files that can't be read in by readFile due to NUL bytes. Change-Id: Ifbd101adaee4f32f10c010fa79e19b9b1127fc6a Reviewed-on: https://cl.tvl.fyi/c/depot/+/6945 Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-10-11 r/5106 fix(nix/buildGo): fix for Nix >= 2.6 readFile changessterni1-1/+4
The string context retained by readFile would leak into attribute keys in fromJSON which may not have string context in any Nix version. We don't need Nix >= 2.6 support, but buildGo may have external users and this change is simple enough. Change-Id: I593f1ef513502691119428d26d508a5f4d378543 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6946 Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-10-11 r/5104 fix(nix/buildkite): split extra steps based on all known phasesEvgeny Zemtsov1-1/+1
Fix a bug introduced by 2ca153141 (I merged the wrong patchset). Issue happens when pipeline is split by phases into independent evals (e.g. build/release/deploy). Splitting extra steps requires knowledge of all known phases, otherwise pipeline evaluation fails due to extra steps from inactive phases. Change-Id: Iab0f2dc3eadda281e483055e26f00a95442e15b0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6942 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-10 r/5079 feat(nix/readTree): Handle a builtins w/o scopedImportGriffin Smith1-1/+1
We (might) not want to implement scopedImport in tvix given it's considered a bit of a misfeature; this makes readTree work with a `builtins` set that doesn't have it (and if we decide we do want tvix to have scopedImport, we can revert this pretty easily). Change-Id: Ia3bbc847514672063a607d977ce167d489fa1131 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6915 Reviewed-by: sterni <sternenseemann@systemli.org> Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-10-10 r/5078 feat(nix/buildkite): allow custom phasesEvgeny Zemtsov1-17/+24
This change automatically extends the list of known phases as soon as they are added to active phase list. This is great when a user wants to design pipelines with multiple groups of dynamic steps. For example in Resoptima we want to design deployment pipeline where first only staging k8s namespaces are updated/tested and only after, we update production. Change-Id: Iab0f2dc3eadda281e483055e26f00a95442e15b9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6923 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-09 r/5071 fix(nix/tag): correct no match check in discrsterni2-1/+6
It uses discrDef internally, but passes `null` as the default tag name, causing Nix to drop the attribute and return an empty attribute set if the default case is hit. Consequently we need to check for the empty attribute set, not `null` to figure out if there was no match found. We can also test this behavior using `assertThrows` which was introduced after the tag library was originally written. Change-Id: I45adb2f9602762dfc867956323fb3f5ae4c8bd1d Reviewed-on: https://cl.tvl.fyi/c/depot/+/6904 Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: Profpatsch <mail@profpatsch.de> Tested-by: BuildkiteCI
2022-10-08 r/5062 refactor(nix/stateMonad): optimize aftersterni1-1/+1
This should save on one function application which can be a big deal for bigger for_ loops, I suspect. It's not really complicated, so why not. Change-Id: I2bfcd254e55f1bea366b09de294b2bef9f5b5dda Reviewed-on: https://cl.tvl.fyi/c/depot/+/6834 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-10-08 r/5061 feat(nix/dependency-analyzer): improved directDrvDeps for Nix >= 2.6sterni1-11/+25
This codepath will basically never be used in depot, but I want to add it as kind of a note to myself. It's kind of a neat feature, although I'm not quite sure it is going to stick around. Change-Id: If0e26ef47bdedc6dbf3d048ad4fc9a3a1fd6c5a2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6833 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-10-08 r/5060 feat(nix/dependency-analyzer): find deps among a list of known drvssterni4-0/+307
This was written with the same intention (and reuses a little of its code) as cl/5060 and cl/5063: We want to be able to emit dependencies between //nix/buildkite pipeline steps, so that no agent is occupied with waiting on locks for derivations built by a different agent. This dependency information is already available to the Nix store implementation (e.g. via `nix-store --query --references`) and can also be obtained in the Nix language which is important, since the pipeline is generated at evaluation time. (Note: For Nix 2.3, you either need a strong convention about how derivations expose their dependencies (which we don't) or rely on store implementation internals (drv files). For Nix 2.6 there is a better trick, but it also relies on the existence of drv files.) The actual task can be formulated as follows: Given a set of derivations, calculate the the closest derivations also in the input each derivation depends on. (We call these (next) known dependencies.) This is crucial because pipeline step often depend on each other only indirectly with any number of intermediate derivations. For cl/5064 I determined that 6 intermediate layers is quite common for dependencies that are perceived to be “direct”. This problem is solved as follows: 1. Calculate the dependency graph of the combined dependency closure of all input derivations. This is quite easy and fairly quick thanks to the C++ implementation of builtins.genericClosure. One weak point of the current implementation is that the function to determine the direct derivation dependencies for Nix < 2.6 is quite hacky. 2. Take the graph from 1. and calculate a dependency graph that only connects the known derivations of the input, but retains all connections between them (minus intermediate nodes). In practice the dependency graph is represented as an attribute set mapping derivation paths to a list of derivation paths it depends on. The second step is performed by adding a second list of known derivation paths it depends on. The main improvements over the previous concept (cl/5060 and cl/5063): * We only try to find the closest known dependencies in the dependency graph whereas we would traverse emit dependencies for the entire dependency closure. * We immediately store the calculation of the closest known dependency in the dependency graph, even for intermediate nodes. This avoids recalculating the connection (which was a big drawback of the previous approach) and makes the calculation itself cheaper. You can run `mg build //nix/dependency-analyzer:example` to build a visualization of the internal dependencies between `depot.ci.targets` as discovered by dependency-analyzer. Change-Id: If8c0cdfc8470d4b337336257d9818aaa0d51110f Reviewed-on: https://cl.tvl.fyi/c/depot/+/6832 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-05 r/5036 docs(nix/nix-1p): Update to use final and prevvpfeiffer1-5/+7
The convention has changed to use `final` instead of `self` and `prev` instead of `super`. This new convention is much easier to understand especially for users that are new to Nix and Nixpkgs. This change is notable in the Nixpkgs project code. I added a note mentioning that the official documentation hasn't been update with this information. The documentation is currently undergoing an overhaul and that is why I believe this hasn't been reflected in the nixpkgs documentation as of yet. Imported-From: https://github.com/tazjin/nix-1p/pull/9/commits Change-Id: I901df69a564969b1784b3d7ac629cfaf85093acb Reviewed-on: https://cl.tvl.fyi/c/depot/+/6859 Reviewed-by: Profpatsch <mail@profpatsch.de> Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-10-01 r/5009 feat(nix/stateMonad): simple Nix state monad implementationsterni2-0/+186
In the absence of do syntactic sugar relatively tedious to write, but useful to express certain types of algorithms. I found it useful to memoize intermediate results as they are calculated in order to avoid recomputing them later in a drv dependency analyzer I've written. Change-Id: I47cf3c644a96952c70276c9fa4cb3190b1c1e027 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6828 Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-09-30 r/5007 fix(nix/buildkite): follow parent skip behavior in extra stepsEvgeny Zemtsov1-1/+12
We found a bug after updating to latest tvl-kit which broke incremental releases. Bug was related to the fact that extra steps skip attribute had precedence over parent configuration. This is a desired behavior when extra step is explicitly set to `skip=true` but otherwise it must follow parent. Due to extra step normalization skip parameter is always set to false if not explicitly set. Along the way, I'm adding support for setting skip attribute on extra steps as string so that people can define skip reasons. The bug was introduced by commit: b9d79109d feat(ops/buildkite): Allow skip of individual steps Change-Id: I8a46d0926a749d2434412b309c661b749e9dbf37 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6827 Autosubmit: ezemtsov <eugene.zemtsov@gmail.com> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-09-28 r/4980 chore(3p/sources): Bump channels & overlayssterni4-9/+9
Upstream nixpkgs removed a lot of aliases this time, so we needed to do the following transformations. It's a real shame that aliases only really become discoverable easily when they are removed. * runCommandNoCC -> runCommand * gmailieer -> lieer We also need to work around the fact that home-manager hasn't catched on to this rename. * mysql -> mariadb * pkgconfig -> pkg-config This also affects our Nix fork which needs to be bumped. * prometheus_client -> prometheus-client * rxvt_unicode -> rxvt-unicode-unwrapped * nix-review -> nixpkgs-review * oauth2_proxy -> oauth2-proxy Additionally, some Go-related builders decided to drop support for passing the sha256 hash in directly, so we need to use the generic hash arguments. Change-Id: I84aaa225ef18962937f8616a9ff064822f0d5dc3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6792 Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi> Reviewed-by: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: wpcarro <wpcarro@gmail.com>
2022-09-20 r/4934 refactor(nix/buildLisp/tests/argv0): use derivation for testssterni1-35/+57
There is no need to use an extraStep, actually, and using derivations reduces noise on CI. Change-Id: I897c3c3f7e0acee8f051fcc01450ff57176726f8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6573 Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-19 r/4922 chore(gerrit): migrate OWNERS files to code-owners styleLuke Granger-Brown6-18/+6
Change-Id: Iacc521dfdd4b4a2d5cef3920cf8189bcce35a488
2022-09-01 r/4565 feat(buildkite/mkPipeline): allow fast failure for build stepsEvgeny Zemtsov1-3/+10
This is supposed to help with resource wasting in situations when multiple dynamic steps depend on a failing derivation. Cost of failure currently is `C = T * D` where: - T -> time to compile the failing derivation - D -> amount of CI targets depending on a failing derivation Switching to `Fail Fast` limits it to just T (time of a single failure). Which helps a lot, especially while upgrading nixpkgs. Fast fail at this moment is in preview, so to enable it: - Enable `Fail Fast` feature at pipeline or organization level - Set `cancelOnBuildFailing` parameter to true for `mkPipeline` Change-Id: I4373a46633522d21e94cfa8bac35243b4eeb0b9c Reviewed-on: https://cl.tvl.fyi/c/depot/+/6243 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-08-30 r/4546 feat(nix/buildLisp): re-enable CCLsterni1-2/+1
The problem went away once again, let's see how long it'll last this time. As it turns out, CCL has a Unicode Standard conforming string implementation that doesn't allow the use of (lone) surrogate code points, requiring us to disable a test in cl-json which tested the behavior of en- and decoding of such a (technically illegal) string. Change-Id: I8bfa482934bbf94f86cecdde02d5c3d4e77950a5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6204 Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: tazjin <tazjin@tvl.su>
2022-07-25 r/4322 fix(ops/buildkite): do not set branch if emptyJean-François Roche1-5/+4
To run an extra step for all branches, user don't set the `branches` attribute. This change avoid setting `branches` to null in such a case. Change-Id: Iabf2f3d0411b037ece5584f30b29c7e65420b63f Reviewed-on: https://cl.tvl.fyi/c/depot/+/5975 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-07-25 r/4321 feat(ops/buildkite): Allow agents targetingJean-François Roche1-2/+4
Using the `agents` attribute, it lets the user target specific agents to run a step. Change-Id: Id6fc0981d4879b77598854e1f296dffffb33a405 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5974 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-07-25 r/4320 feat(ops/buildkite): Allow skip of individual stepsJean-François Roche1-2/+4
Using the `skip` attribute, it lets the user decide to temporarily skip a step. Change-Id: If63ce60a2b4148f041655912730acc4ae9e28534 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5971 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-06-16 r/4242 chore(nixery): use nix-1p from within the depotVincent Ambo1-2/+3
Since the source of nix-1p is checked in under //nix/nix-1p, we should use it from there if Nixery is being built inside of depot. Change-Id: Iddd54f7b93b398b2f909db6ee105366a9914a2ac Reviewed-on: https://cl.tvl.fyi/c/depot/+/5882 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su>
2022-06-06 r/4222 refactor(nix/buildkite): Optimise steps if builds are not enabledVincent Ambo1-1/+2
Generating a release-only pipeline skips a bigger chunk of eval this way (the step itself is never actually evaluated, which means we never actually compute the drv), which can be quite beneficial in terms of evaluation time. Change-Id: I2739026ddd1c6a86f82627ac26a046c5fe7359ea Reviewed-on: https://cl.tvl.fyi/c/depot/+/5830 Tested-by: BuildkiteCI Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
2022-06-06 r/4221 fix(nix/buildkite): Disable `depends_on` if build phase is not runVincent Ambo1-4/+15
Extra steps that use `depends_on` (e.g. if they need output from their parent) should not actually depend on their parents build step if the build phase is not active. This is required to actually decouple the phases. Change-Id: I398da9a8a53e97ca3c635342259fc722d54b8e4a Reviewed-on: https://cl.tvl.fyi/c/depot/+/5829 Tested-by: BuildkiteCI Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
2022-06-06 r/4220 feat(nix/buildkite): Allow toggling of individual phasesVincent Ambo1-6/+18
Using the `activePhases` attribute, the set of phases included in an evaluation can be modified. This lets users generate e.g. ONLY the release steps of a pipeline. Change-Id: Ib0c38826dd69666094d619f5f324d1baafce8134 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5828 Tested-by: BuildkiteCI Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com>
2022-06-03 r/4208 fix(nix/emptyDerivation): Use system from readTree parametersVincent Ambo2-5/+4
Change-Id: Ia7f9f4d0e7c06fa5433213a315c3354a83e94545 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5833 Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: Profpatsch <mail@profpatsch.de>
2022-06-03 r/4205 feat(nix/buildkite): Validate available phases in extra stepsVincent Ambo1-5/+13
This will avoid things like extra steps being accidentally ignored because of typos. Change-Id: Ic4fa5925e42a7a449f89b4cde1510e216e91da6a Reviewed-on: https://cl.tvl.fyi/c/depot/+/5827 Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com> Tested-by: BuildkiteCI
2022-06-03 r/4204 fix(nix/buildkite): Forbid 'prompt' in build phase stepsVincent Ambo1-5/+11
This would block CI on human-approval if people were allowed to do it, so they're just not. Change-Id: I8a9b657d5c91636a7b4de249b977e24fc0941a1c Reviewed-on: https://cl.tvl.fyi/c/depot/+/5826 Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com> Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-06-03 r/4203 refactor(nix/buildkite): Explicit support for build phasesVincent Ambo1-63/+108
Previously the extra steps were roughly divided into steps that run "at build time" (i.e. before we publish results to Gerrit), and "post-build" (i.e. later on). In practice, these are something like a build/release pairing, where steps running after the build results are returned are mostly run for side-effects (e.g. publishing git subtrees to external repos). This refactoring makes this distinction explicit in //nix/buildkite and changes the extraSteps API with an explicit `phases` attribute instead of the previous `postStep` attribute. In practice the previous API is still supported, but will throw evaluation warnings until an arbitrarily chosen cutoff date of 2022-10-01 at which point we will change using it into a hard error. This uncovered a few strange behaviours which we only accidentally avoided, most of which I have left TODOs about and will clean up in subsequent commits. The purpose of this commit is to allow for separate evaluations of only build or only release steps, for example if release steps are evaluated in a slightly different context (e.g. with overridden versioning that is not relevant to standard CI functionality). Change-Id: I0b0186e3824273c15a774260708702d4a5974dac Reviewed-on: https://cl.tvl.fyi/c/depot/+/5825 Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com> Tested-by: BuildkiteCI
2022-06-03 r/4202 refactor(nix/buildkite): Rename "post" steps to "release" stepsVincent Ambo1-1/+1
This is in preparation for a subsequent CL that will do much more significant changes in //nix/buildkite. Change-Id: I80a8d67d3a7d593854c8d711572483c2581e7881 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5824 Reviewed-by: ezemtsov <eugene.zemtsov@gmail.com> Tested-by: BuildkiteCI
2022-06-02 r/4201 docs(nix/buildkite): Fix an out-of-date comment about chunk sizeVincent Ambo1-1/+1
Change-Id: Ic1f874f4ca83f9088355dc0512723ea962e0db52 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5823 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-06-01 r/4200 fix(nix/buildLisp): resolve eval problem in ccl codesterni1-2/+2
The isPowerPC predicate has been [removed], since it was misleadingly named (it just matches PowerPC, 32bit, little endian). This means the 64bit code path could now actually work. Not sure about endianess, the CCL docs don't really say much regarding that topic. [removed]: https://github.com/NixOS/nixpkgs/pull/168113 Change-Id: Icf4a8c6b1df95fa597ed87508f57aaa73e6185ed Reviewed-on: https://cl.tvl.fyi/c/depot/+/5796 Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: tazjin <tazjin@tvl.su>
2022-05-29 r/4189 feat(nix-1p): Export subtree to GitHubVincent Ambo1-0/+15
We needed a derivation for that, but this can also be used in the Nixery docs building process (which includes nix-1p). Change-Id: If97cf785a33d703af975da3b41de9b69566dfa81 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5789 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-05-29 r/4188 subtree(nix/nix-1p): Merge nix-1p into depotVincent Ambo1-0/+599
It's time to stop maintaining this on GitHub, and use the export-from-depot feature instead. Change-Id: Ic6b840bb0e8580c7214113467b4995ea3d2fae02
2022-05-28 r/4175 feat(nix/utils): add onlyDrvPath to get the drvPath w/o the outputssterni2-0/+27
I want to use this utility in a deploy script where the .drv is nix-copy-closure-d to a remote host and realized there. Consequently it doesn't make sense that the local deploy script depends on the derivation's outputs which drvPath does by default. This also came up when working on //nix/buildkite, although we didn't end up using it there. Change-Id: I952bbfd4d7e9de212569d5ee12182eb50d360f53 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5767 Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: tazjin <tazjin@tvl.su>