about summary refs log tree commit diff
path: root/ops
AgeCommit message (Collapse)AuthorFilesLines
2021-12-02 r/3133 refactor(ops/besadii): Move Sourcegraph config to a fileVincent Ambo1-20/+56
Initial step towards moving besadii away from hardcoded values and onto config files. This is required because I want to reuse besadii outside of the TVL context. Change-Id: Id4fa7a49c5d4f876a02b202f04a421ab5ba0dcc4
2021-12-02 r/3132 fix(ops/nixery): Temporarily stop serving depot packages in NixeryVincent Ambo1-1/+1
Change the Nixery configuration to use the plain nixpkgs package path instead of the depot path. AFAIK, nobody uses this to fetches depot packages at the moment - but plenty of people fetch non-depot packages. This means that Nixery is cache-busted less often (previously on every commit => every deploy). We'll figure out another way to have a depot Nixery later. Change-Id: Iba632333346181c3d2ce992fbab396ed0d9f86aa
2021-12-01 r/3131 fix(ops/www): Redirect tvl.fyi/blog -> tvl.fyiVincent Ambo1-0/+4
The blog index page is at the root and people may manually edit the URL. Change-Id: I6cdaaaee6223524a9e950584379cfac34f8be160
2021-12-01 r/3125 feat(besadii): Support invocation as different Gerrit hooksVincent Ambo2-68/+119
Removes besadii support for the previously used 'ref-updated' hook and instead introduces support for the 'change-merged' and 'patchset-created' hooks. These hooks more accurately capture the semantics of when besadii should trigger CI builds and using them will avoid problems such as skipping 'canon' builds if chains of CLs are submitted together. Change-Id: Ib90356c069780bf0c0250e56b927e46a5b31ce7f
2021-11-30 r/3118 fix(ops/www): Strip `.html` from TVL blog post URLsVincent Ambo1-0/+8
Change-Id: I4d1f9284ec004931c07c04d614b01f28eedea508
2021-11-29 r/3116 refactor(ops/pipelines): Query build status from Buildkite APIVincent Ambo2-8/+21
Instead of manually tracking the build status through Buildkite metadata, use the Buildkite GraphQL API in the `:duck:` build step (i.e. the one that determines the status of the entire pipeline to be reported back to Gerrit) to fetch the number of failed jobs. This way we have less manual state accounting in the pipeline. The downside is that the GraphQL query embedded here is a little hard to read. Notes: * This needs an access token for Buildkite. We already have one for besadii which is also run by the agents, so I've given it GraphQL permissions and reused it. * I almost introduced a very rare bug here: My initial intuition was to simply `exit $FAILED_JOBS` - in the extremely rare case where `$FAILED_JOBS % 256 = 0` this would mean we would ... fail to fail the build :) Change-Id: I61976b11b591d722494d3010a362b544efe2cb25
2021-11-29 r/3111 chore(ops/users): Update password hash for asmundoVincent Ambo1-1/+1
... some issue snuck in on the first one, as is tradition. Change-Id: I06ce4df82cde26231cd1ab3df500de02e981d9bc
2021-11-29 r/3106 feat(ops/users): Add user asmundoasmundo1-0/+5
Change-Id: Ie666b6556d91513babd884b2ed1140cd6c0ed2a9
2021-11-29 r/3105 refactor(besadii): Rename refUpdated -> buildTriggerVincent Ambo1-34/+40
We are changing the Gerrit hooks which invoke besadii, but this structure will be used for both kinds. Change-Id: Idb1cb0c640d2c42db8e7af39f3ab372a97bfef91
2021-11-28 r/3102 fix(ops/besadii): Trim whitespace of auth tokensVincent Ambo1-4/+7
This is causing failures when trying to update Sourcegraph at least, for good measure I've trimmed both. Change-Id: I40266ee83b4e266ffe50f16bb365eb2e51952513
2021-11-23 r/3088 refactor(readTree): Move 'drvTargets' into readTreeVincent Ambo1-5/+1
This function is also generally useful for readTree consumers that have the concept of subtargets. Change-Id: Ic7fc03380dec6953fb288763a28e50ab3624d233
2021-11-21 r/3078 fix(ops/restic): Move whitby's backup to GleSYS object storageVincent Ambo2-13/+17
Since GCP nuked us, the backups are now moving to GleSYS' S3-compatible object storage. This refactors the restic module to support S3-compatible storage instead of GCP, and switches to the appropriate new secret paths. The secrets were placed on whitby manually and I verified that the backups work. This fixes b/157 Change-Id: I6a9d2b0581967605ce736605a3befb44cdeae7e1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3883 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2021-11-06 r/3007 fix(ops/pipelines): Fix tagging of commit revisionsVincent Ambo1-5/+1
It seems that shell variables don't work as expected inside the Buildkite pipeline, so usage of variables has been removed. We also don't echo the revision anymore because of that, but it does still appear in the log of `git push`. Change-Id: I124e3b09af896da898f2a78715ed371651a1c5f8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3780 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2021-11-05 r/3005 refactor(ops/pipelines): Move revision tagging into static pipelineVincent Ambo2-14/+18
This makes the revision number available much earlier (before the rest of the pipeline runs, while Nix eval is happening) which should only be a few seconds after a commit to canon. It is also more readable in this shape. Change-Id: Iccbb17dfef6afe68f54fda41e8d10c4dc52b08c2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3775 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2021-11-04 r/2999 feat(ops/pipelines): Create revision numbers in CIVincent Ambo1-0/+14
This automatically pushes a new ref at refs/r/$revision to Gerrit whenever a CI run completes on canon. Revision numbers can be fetched from Gerrit with this command: git fetch gerrit "refs/r/*:refs/r/*" Note that this build step requires credentials to be provisioned on the CI runner machine. Change-Id: I37bb14346832f891240aa47bb55affaace3d5f21
2021-10-05 r/2952 fix(ops/users): correct password hash for smitopSmitty' via Issues & Patches1-1/+1
The previous hash had a weird salt length and a trailing newline. This fixes it. Change-Id: I1f03238181d0caad38e1f1dbc477356bc20fc32d Reviewed-on: https://cl.tvl.fyi/c/depot/+/3689 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2021-10-05 r/2951 feat(ops/users): add smitop to userssterni1-0/+5
Change-Id: I1fc67c0e33e1e1add8a4ea53c8c94e90e53d8bd5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3687 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi> Reviewed-by: tazjin <mail@tazj.in>
2021-10-01 r/2946 feat(whitby): serve static.tvl.{fyi|su} with max cache settingsVincent Ambo2-0/+43
The setup is explained in the comment, but TL;DR: Use the derivation hash of static files to create permanent URLs. Relates to b/151. Change-Id: Ib1ca3a1a00c90a47f4bf39c29a8b4bbf5b215e7d Reviewed-on: https://cl.tvl.fyi/c/depot/+/3664 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2021-10-01 r/2945 feat(ops/dns): add static.tvl.{fyi|su}Vincent Ambo2-3/+5
This hostname can be used for hosting static assets with aggressive caching for everything, or potentially CDNing stuff if we ever have large things here. Change-Id: I10afdad5eb08125d8d09108e9e099f5573362fe5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3663 Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2021-10-01 r/2941 feat(whitby): Serve //corp/website on tvl.suVincent Ambo2-0/+21
Change-Id: I21e1ddf9a32568cac8ad2595869ac8670867efa9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3658 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-09-24 r/2914 chore(ops/git-serving): Remove josh state from whitby backupsVincent Ambo1-2/+0
As cschilling explained on cl/3563, there isn't actually anything in this state that we *need* to persist. We're still keeping it in a persistent directory on disk as this serves as an optimisation after restarts of josh. Change-Id: Ia88886792a5acac34508b5b8a669bd519ca033de Reviewed-on: https://cl.tvl.fyi/c/depot/+/3631 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2021-09-18 r/2890 refactor(whitby): Move restic path configuration into modulesVincent Ambo4-7/+8
This lets each service declare their backup paths together with the configuration for the service, which is a lot more sensible than what we had before. Fixes b/147 Change-Id: If76fe62639f4cc0e6fbb63a2959d584479d8f0fb Reviewed-on: https://cl.tvl.fyi/c/depot/+/3583 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2021-09-18 r/2889 fix(ops/restic): types.string -> types.strVincent Ambo1-3/+3
I can never remember which is which. Change-Id: I69b8235862b8c5b49030a74bfca25aaa113273b7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3582 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2021-09-18 r/2888 feat(besadii): Link to started builds in CL commentsVincent Ambo1-5/+35
This makes it easier to click through to a build from Gerrit after submitting a CL. Change-Id: Ic5c6eeb81c87bc4ea23c5c5ca25704434b081fd0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3572 Tested-by: BuildkiteCI Reviewed-by: lukegb <lukegb@tvl.fyi>
2021-09-18 r/2887 refactor(besadii): Extract logic for posting review commentsVincent Ambo1-42/+50
Currently besadii only posts comments when builds succeed, but it might be very useful to also have a link to a build when the build is started. This just shuffles code around. The only functional change is that the `labels` field in the review input is marked as `omitempty`, as this will not be needed when posting the build start comment. Change-Id: Id4a43fad8817c9a15da02f01ab2b781d48b46978 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3571 Tested-by: BuildkiteCI Reviewed-by: lukegb <lukegb@tvl.fyi>
2021-09-18 r/2885 docs(kontemplate): Remove mention of kontemplate websiteVincent Ambo1-3/+2
This doesn't exist anymore. Change-Id: I4535e056acba3bbc7bbd1e764a0b3043639b0877 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3570 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2021-09-16 r/2879 docs(kontemplate): Update cloning docs in READMEVincent Ambo1-4/+4
Change-Id: I42bf2524650bf09104e48c1c1a54c97f3470b628 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3566 Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2021-09-16 r/2878 refactor(ops/restic): Move restic configuration into a new moduleVincent Ambo2-22/+75
Relates to b/147. First step towards giving depot modules the ability to declare their own backup directories by moving all restic configuration into a new module and adding a NixOS option for inclusion/exclusion paths for backups. This still keeps all backup paths within the whitby config. Change-Id: Ia96833668f1a3d02da892261153d8b02156b8ac0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3565 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2021-09-16 r/2877 feat(git-serving): Configure josh to serve the depot over HTTPVincent Ambo3-16/+72
Previously we served the dumb git HTTP protocol from code.tvl.fyi via cgit. This CL disables this feature and instead runs josh in the same location (by redirecting appropriately), but while also enabling partial cloning of all subtrees of the depot. For example, after this CL the following would result in an independent clone of //nix/readTree: git clone https://code.tvl.fyi/depot.git:/nix/readTree.git Note that there are no josh workspaces configured at all for now, these references are only for static depot subpaths. Please refer to the documentation for josh for more information on available kinds of josh filters. Josh state is kept in a systemd state directory in /var/lib/josh and backed up to Restic. Backing this up is necessary, as josh uses stateful information to do things like tracking merges and rewriting history per subtree appropriately to avoid cloned repositories ending up in peculiar states. Change-Id: I156f0298c2aa42e3bdbf5a0e86109070d640c56e Reviewed-on: https://cl.tvl.fyi/c/depot/+/3563 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2021-09-11 r/2848 fix(deploy-whitby): Add jq to script $PATHVincent Ambo1-1/+2
Change-Id: Ide669bce545394335b8643fa2896a242cac3df65 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3528 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2021-09-11 r/2847 fix(deploys.*): Folder for diffs is in /diff/Vincent Ambo1-1/+1
... this was missing before. Change-Id: I5b79cb78665f24fdb7cc6496e3782d3940dc77b6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3527 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2021-09-11 r/2846 feat(sourcegraph): Upgrade 3.30.4 -> 3.31.2Vincent Ambo1-1/+1
This one seems a little more involved: https://docs.sourcegraph.com/admin/migration/3_31 I believe we skip that corruption issue in the previous CL though, by simply never deploying a version with that weird broken image. See b/144 Change-Id: I3bbf1b719d00905e08a92011ace5485467f504ef Reviewed-on: https://cl.tvl.fyi/c/depot/+/3525 Tested-by: BuildkiteCI Reviewed-by: lukegb <lukegb@tvl.fyi>
2021-09-11 r/2845 feat(sourcegraph): Upgrade 3.29.1 -> 3.30.4Vincent Ambo1-1/+1
See b/144 Change-Id: Ied9490f3ce6fb3fda8cbb9983416b02ea451fb44 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3524 Tested-by: BuildkiteCI Reviewed-by: lukegb <lukegb@tvl.fyi>
2021-09-11 r/2844 feat(sourcegraph): Upgrade 3.28.0 -> 3.29.1Vincent Ambo1-1/+1
See b/144 Change-Id: Ia62d4cbf581caaefa0dba455376eec60b8c817d6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3523 Tested-by: BuildkiteCI Reviewed-by: lukegb <lukegb@tvl.fyi>
2021-09-11 r/2843 fix(sourcegraph): Temporarily comment out our syntax highlighterVincent Ambo1-1/+2
We changed away from the default sourcegraph one because it didn't support Nix, but it seems that there's been a change in the interaction protocol. Change-Id: I3a2691df6a87672cf83b819143f25d93d9cd6d13 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3531 Tested-by: BuildkiteCI Reviewed-by: eta <tvl@eta.st> Reviewed-by: sterni <sternenseemann@systemli.org>
2021-09-11 r/2842 feat(sourcegraph): Upgrade 3.27.5 -> 3.28.0Vincent Ambo1-1/+1
See b/144 Change-Id: Ia09ad2af6043dcac6681c549103d1e6f52b4e0a0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3522 Tested-by: BuildkiteCI Reviewed-by: lukegb <lukegb@tvl.fyi>
2021-09-11 r/2841 feat(sourcegraph): Upgrade 3.26.0 -> 3.27.5Vincent Ambo1-1/+1
See b/144 Change-Id: I50d417c51b05bafcd3fe7e285f30079db8be499a Reviewed-on: https://cl.tvl.fyi/c/depot/+/3521 Tested-by: BuildkiteCI Reviewed-by: lukegb <lukegb@tvl.fyi>
2021-09-11 r/2840 fix(deploy-whitby): Make diffs world-readableVincent Ambo1-0/+1
Change-Id: I1610a8d189f95908bab4cd00057cc080ae47a21a Reviewed-on: https://cl.tvl.fyi/c/depot/+/3530 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2021-09-11 r/2839 fix(deploy-whitby): Add .html suffix to diff filenamesVincent Ambo1-1/+1
This makes nginx' content-type recognition work correctly. Change-Id: I990b00f1e0f4ef311f53a8885718fa33d249c886 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3529 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2021-09-10 r/2838 feat(ops/deploy-whitby): Add the start of a script to deploy whitbyGriffin Smith2-0/+75
Add the beginnings of an auto-deploy script for whitby, intended to be (eventually) suitable for running automatically in a systemd timer. The current iteration of the script doesn't actually do any deploying, but instead takes as an argument a revision, creates a new git worktree in /tmp with that revision checked out, runs a nix-diff of whitby's system derivation in the running system and at that closure, puts an html-rendered version of that diff in the public directory used by deploy.tvl.fyi, and finally sends a message to IRC via irccat with a link to that HTML page. Refs: b/110 Change-Id: Id40525567f8845590c909568befd8d00c07a481c Reviewed-on: https://cl.tvl.fyi/c/depot/+/3145 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: kn <klemens@posteo.de>
2021-09-10 r/2837 feat(whitby): Serve static HTML dir for deploys.tvl.fyiGriffin Smith2-1/+22
Add a new domain and nginx virtual host at deploys.tvl.fyi, serving out of a static directory on whitby which is created by systemd-tmpfiles. This will be used to serve diffs rendered by nix-diff for pending deploys for whitby Since this contains stateful data, it is added to the restic backups on whitby. Refs: b/110 Change-Id: I5869d40800bbf5fb8fb39878a857f66ff5787830 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3144 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-08-31 r/2804 fix(ops/users): Another try at a working password hash for mdjnsnMike Johnson1-1/+1
Change-Id: I8b4aea53abb2004585241ad17c5fdfd9186c58f4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3481 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2021-08-31 r/2803 feat(ops/users): Add mdjnsn to usersMike Johnson1-0/+6
Change-Id: I94975d848287c32e11b1d3986986f2dbc6c220b9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3466 Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2021-08-29 r/2799 refactor(ops/pipelines): Move failure status zeroing to setupVincent Ambo2-13/+7
We changed the configured pipeline in Buildkite to upload `static-pipeline.yaml` instead of containing the steps of that pipeline itself. This makes it easier to test changes to builds and such, but adds another build step with scheduling overhead etc. However - we can work around this by killing one of the existing build steps. There's no reason the failure status zeroing (required for status reporting) shouldn't be part of the pipeline setup, so I've moved it there instead and nuked that step. This should mean that the pipeline is configurable from within the repo, but without slowing anything down. Change-Id: I206ecc02647de42a461e33c02879ab84daf5ed2b Reviewed-on: https://cl.tvl.fyi/c/depot/+/3461 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2021-08-26 r/2791 refactor(gs/system): Remove chupacabraGriffin Smith1-1/+0
This machine no longer exists Change-Id: I8e549b8397777a01404bd84c10c195e80f281744 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3431 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi> Reviewed-by: tazjin <mail@tazj.in>
2021-08-26 r/2787 fix(ops/pipelines/depot): Buildkite branches use full ref namesVincent Ambo1-1/+1
... otherwise the filtering also applies to canon. Change-Id: Ia1c67b99282fb8fd0e4d22e997535170f0326e33 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3432 Reviewed-by: sterni <sternenseemann@systemli.org> Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2021-08-26 r/2786 feat(pipelines/depot): Skip build steps if their out paths existVincent Ambo1-0/+12
Skip build steps if they have already been built, reducing pipelines to the things that actually changed between builds. On canon all targets are always built (we require this for anchoring). Note that this is not perfect, garbage collection and competing pipelines may affect each other. Also note that we have some impure targets that change on every commit. Change-Id: Ic6bae3b6c8e1e7fd2116ec252f5089f471854ab6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3427 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org> Reviewed-by: grfn <grfn@gws.fyi>
2021-08-26 r/2783 feat(ops/pipelines/depot): only evaluate once if possiblesterni1-3/+15
We currently evaluate every target twice -- once when the depot pipeline is built and once when actually running the build step in question. Nix evaluation is quite slow especially given heavy use of import from derivation in depot, so avoiding the second evaluation is desireable. Evaluating a derivation yields a `drv` file in the nix store which can be passed to `nix-store --realise` in order to build it eliminating the need to wait for evaluation. We can obtain the path to the `drv` file while building the pipeline via `target.drvPath` and remember it for the build later. However we need to work around a flaw (or oversight) in Nix's dependency tracking via string context: This is based on derivations, not output path (because this is what evaluation deals with, likely). This is no problem per se, but an issue is that Nix can't express a dependency on a `drv` file without any of its output paths. This means for us that we either have to build all output paths at evaluation time (which we don't want, obviously) or to deal with the fact that the `drv` file we need may be garbage collected at any moment after discarding the string context -- then nix is unable to track the reference from the pipeline to the `drv` file in the store. So to prevent a race condition between the pipeline and the garbage collector we fall back to the normal nix-build invocation as we did before. Change-Id: I9ef8bd233085dc6e30eba54f403ea03ac2d35748 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3426 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-08-24 r/2758 fix(tvl-sso): set memory limit to 512MLuke Granger-Brown1-0/+1
This is because I'm bored of CAS gradually consuming all the RAM on Whitby. Change-Id: Idcc14c19d99a6d3553739c5765be3faf2bdf9d84 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3233 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi> Reviewed-by: tazjin <mail@tazj.in>
2021-08-24 r/2757 feat(besadii): Tag gerrit comments as autogeneratedGriffin Smith1-7/+10
This is a bit of an under-documented feature, but if the "tag" field for a gerrit review starts with the string "autogenerated:<something>~<something-else>", only the last comment per instance of <something> will be shown by default on the CL page (with the rest viewable by toggling the "Show all entries" switch). The idea behind the "<something-else>" tag is to be used for the "type" of comment within a particular system - gerrit's documentation gives the example of one tag for "the build is running" and another for "the build has finished, here's the result". Change-Id: I9199a6ed97beca1b3a51ec5d6230c6c8358ba2b3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3374 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-11-24T09·12+0000