about summary refs log tree commit diff
path: root/ops
AgeCommit message (Collapse)AuthorFilesLines
2020-11-22 r/1907 feat(whitby): Move wigglydonke.rs to whitbyGriffin Smith2-0/+16
Mugwump is too unstable for such an important internet service Change-Id: Ic714200ce5ce51f366777f538b4a6f443f010960 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2124 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-11-22 r/1905 feat(ops/nixos): Give all nixoses a config.depotGriffin Smith1-1/+7
Add the depot.nix module and a depot config option to all nixos system derivations that're build through the `bin/rebuild-system` machinery. I can't imagine a scenario where we wouldn't want this level of integration. Change-Id: Ieeb98db2eee23919256adb4654bc45d540e055ec Reviewed-on: https://cl.tvl.fyi/c/depot/+/2128 Tested-by: BuildkiteCI Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-11-17 r/1882 feat(ops/pipelines): Check in the static pipelineVincent Ambo1-0/+15
This file represents the static pipeline which is configured in the Buildkite web UI. Updates to this file should be applied in the admin interface. These steps are responsible for launching the dynamic pipeline evaluation, or falling back to the fallback pipeline if evaluation fails. Change-Id: I6d7dd623cde65e8c69faea729f737c9bba00c2fb Reviewed-on: https://cl.tvl.fyi/c/depot/+/2103 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-11-17 r/1881 feat(ops/pipelines): Add a fallback Buildkite configurationVincent Ambo1-0/+8
This adds a simple fallback Buildkite pipeline configuration which always fails the pipeline, but correctly reports back the failure status. Note that this also requires changes in the Buildkite configuration that is not in version-control. Relates to b/66. Change-Id: I6802a6f76448c3893798a06d514e6ccba0f50dd2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2102 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-11-17 r/1880 feat(ops/panettone): Add configuration for irccatVincent Ambo2-0/+21
Adds configuration options for the (inconsistently named) environment variables that configure irccat integration with Panettone. The defaults match the irccat setup on whitby. Change-Id: I6857512a2e3f29f16777493eb981cc69ce3c045f Reviewed-on: https://cl.tvl.fyi/c/depot/+/2080 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-11-08 r/1876 feat(whitby): Enable irccat moduleVincent Ambo1-0/+19
Enables irccat, running as 'tvlbot' on ##tvl and ##tvl-dev and listening on TCP 4722. Change-Id: Ia1eb533d0aacb0c15d6b3fa1cfd854ffbce27d23 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2075 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi> Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-11-08 r/1875 feat(ops/irccat): Add a NixOS module for launching irccatVincent Ambo1-0/+49
This module configures irccat by creating a JSON configuration file from a user-supplied Nix struct (this is not checked for correctness), and merging it recursively with secrets from `/etc/secrets/irccat.json` at service launch time. This way we get the ability to configure (most) options declaratively via Nix, while providing the secrets outside of Nix. Side note: We need to figure out a secrets distribution mechanism. Tested: Wrote a dummy config in whitby/default.nix locally and checked that this builds, but I have not actually run the service yet. I expect that some minor tweaks will end up being necessary. Change-Id: I02a2e8dc40a7f8417fd77afcf8a12ac3df117988 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2074 Tested-by: BuildkiteCI Reviewed-by: lukegb <lukegb@tvl.fyi> Reviewed-by: glittershark <grfn@gws.fyi>
2020-11-05 r/1870 fix(whitby): Use new IRC bouncer location for clbotVincent Ambo1-1/+1
... I found this location in the logs, because the certs are now valid for this, but I'm not actually sure if it's right. Change-Id: I5ac88073e3bf6a95fead4c1d34515622c4416c6a Reviewed-on: https://cl.tvl.fyi/c/depot/+/2070 Tested-by: BuildkiteCI Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-11-05 r/1869 feat(ops/nixos/paroxysm): Set Restart = "always"Griffin Smith1-0/+1
Sometimes (like today) paroxysm crashes. We'd like it to restart if that happens. Change-Id: I98841096bcd6605c4279744ae5c65a9c92092a21 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2069 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-10-22 r/1854 feat(tvl-slapd): add etu to slapdElis Hirwing1-0/+5
Change-Id: I39ecf2167fd65f305853bf0e48c6208d94a5bf1f Reviewed-on: https://cl.tvl.fyi/c/depot/+/2055 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-10-18 r/1850 feat(tvl-slapd): add htbfhtbf1-0/+5
Change-Id: I6da03700708bcafc4f476b01c0a27d27fb85cc4a Reviewed-on: https://cl.tvl.fyi/c/depot/+/2050 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2020-10-05 r/1834 chore(tvl-slapd): use ARGON2 for cynthiaCynthia Revström1-1/+1
Change-Id: I81efffe384644cc2d2a625fc96ef5264dedd76ea Reviewed-on: https://cl.tvl.fyi/c/depot/+/2019 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2020-09-28 r/1828 feat(whitby): add firefly userJonas Höglund1-0/+6
Change-Id: Ib785577c173795d5cc6ccd7a3ee7e6a568439a0d Reviewed-on: https://cl.tvl.fyi/c/depot/+/2013 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-09-27 r/1825 feat(tvl-slapd): add fireflyJonas Höglund1-0/+5
Change-Id: I28b71a429f2093e2ff3d7148cfaa2425f322dfea Reviewed-on: https://cl.tvl.fyi/c/depot/+/2010 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-09-27 r/1823 feat(whitby): add cynthia owoCynthia Revström1-0/+6
Change-Id: Id9e06ce8645ec2dbe1167d2b0b023159d3e91487 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2008 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-09-27 r/1819 feat(tvl-slapd): add ben to slapdBen Cartwright-Cox1-0/+5
Change-Id: Iadec1d04b086f878c408a8867778f6bd75254dfc Reviewed-on: https://cl.tvl.fyi/c/depot/+/2005 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-09-06 r/1765 chore(whitby): Double the number of build agents againVincent Ambo1-1/+1
The main bottleneck of our builds right now is Nix evaluation, which means that most of the time is spent idling during builds. Since we're evaluating in parallel, lets give it a few more builders. I don't want to go all the way to 64 immediately to first see if we get any adverse effects from highly concurrent builds running concurrently (if we do we could group them into different "concurrency groups" in Buildkite). Change-Id: Ibc3f89fb59cb4ee471b152ff36887ffe2b39f8f8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1932 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-31 r/1752 fix(ops/monorepo-gerrit): Escape escape characters with more escape charactersGriffin Smith1-1/+1
Escape! Change-Id: Idec7d17feda13ccf1ec03c41106ec6ed581d9e47 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1901 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-08-31 r/1751 feat(whitby): Double the number of Buildkite agentsVincent Ambo1-1/+1
Allow 16 things to happen in parallel, which is useful now that the CI granularity is on a per-target level. Change-Id: Ie65dd119ea0666618fbb249613e70a68276db834 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1902 Reviewed-by: glittershark <grfn@gws.fyi> Tested-by: BuildkiteCI
2020-08-31 r/1748 feat(ci): Add subtarget support for buildsVincent Ambo2-6/+14
We have naturally evolved a distinction between logical and physical targets. Physical targets are those which correspond directly to a tree location on disk and can be built with `-A path.to.files`, while logical targets are those that are exported from within an expression but do not have a corresponding file on disk. This change adds support for exporting logical targets from any tree location by adding a `meta.targets` attribute containing keys into itself, which will be consumed by the CI target gathering logic and included in the generated pipeline. Note that the labels for subtargets are syntactically different to emphasise that they do not correspond to a file location. For example, this change enables 'ops.nixos.whitbySystem' as a subtarget, which is labeled in CI as `ops/nixos:whitbySystem`. Change-Id: Ied09647a62c2ba98e3914548e3742ad422c63ecf Reviewed-on: https://cl.tvl.fyi/c/depot/+/1893 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-31 r/1747 feat(ops/pipelines): Dynamically generate CI pipeline from targetsVincent Ambo2-14/+67
Create the pipeline by outputting a file that contains nix-build invocations for each target's *derivation path*. Each invocation has a generated Nix expression passed to it with `-E` which fetches the correct target from the tree while correctly handling targets with strange characters (such as in Go-packages). This makes it possible to run target-level granular pipelines. We're getting somewhere! Change-Id: Ia6946e389dafd1d4926130bb8891446d6e17133b Reviewed-on: https://cl.tvl.fyi/c/depot/+/1855 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi> Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-08-30 r/1741 feat(monorepo-gerrit): Auto-link panettone issuesGriffin Smith1-0/+6
Auto-link b/\d+ in gerrit comments + patch comments to panettone issues. This is a departure from the previous syntax of #\d+ that we were using historically, but has been requested by multiple people Fixes: b/40 Change-Id: I8e556eb6b2f55ae80d945b86b51041751ecb053b Reviewed-on: https://cl.tvl.fyi/c/depot/+/1888 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-08-26 r/1725 feat: Implement automatic CI target detection for the depotVincent Ambo1-1/+1
Automatically walk the entire depot tree and pick out things that are "buildable", then include them in the attribute `ci.targets` (which is now also the target for CI builds). A long time ago, in a land far away, we (well, I, at the time) had a prototype of this which ran into constant issues with infinite recursions while trying to walk the tree. In fact, this is why readTree originally gained the `__readTree`-attribute which marks things that were imported automatically. Based on some code edef whipped up earlier (with the breakthrough being that we also add the attribute to top-level folders, which suddenly resolves a whole bunch of problems), I've now implemented this actually working version. At the moment all builds still happen as one big bag of builds, but at some point we will granularise this. Change-Id: I86f12ce7f63dae98e7e5c6646a4e9d220de783f2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1854 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com> Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-26 r/1724 fix(besadii): Remove unused Nix code from the derivationVincent Ambo1-4/+0
This was used previously when build granularity was besadii's task, which it no longer is. Change-Id: I6df2db1ed4730a7953199b7b48aa9ad916418b22 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1853 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-26 r/1723 chore: Add __readTree attributes on systems that need to be builtVincent Ambo1-0/+6
This is a temporary state (TODO added) to be picked up by the new CI logic. Change-Id: Id4702740ffd18325088e2a8a0c6157a8cee7ccf7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1852 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-08-23 r/1705 feat(whitby): Enable log forwarding via journaldriverVincent Ambo1-0/+7
Change-Id: I474159acfe514f6f2eb7867e4eba854016590ab1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1836 Tested-by: BuildkiteCI Reviewed-by: isomer <isomer@tvl.fyi>
2020-08-21 r/1701 revert(whitby): Use Tvix as the system Nix on whitbytazjin1-1/+0
This reverts commit 8fa30870673fcdd5410c2593321153ab7cfbe36f. Reason for revert: This almost worked. We discovered two important issues: - The daemon startup does not correctly handle the socket passed in by systemd. - There is some issue with chunking of large calls, running a build for ci-builds resulted in: tazjin@whitby /depot (canon)> nix-build -A ciBuilds.__allTargets E20200821 01:42:22.846053 12601 shared.cc:306] error: Rpc call addTextToStore to unix:///nix/var/nix/daemon-socket/socket failed (RESOURCE_EXHAUSTED): Received message larger than max (10889961 vs. 4194304) Change-Id: Ic5ba4ef06a4953cf71a36b139fe25ea673cb6fee Reviewed-on: https://cl.tvl.fyi/c/depot/+/1802 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-21 r/1700 feat(whitby): Use Tvix as the system Nix on whitbyVincent Ambo1-0/+1
... this is going to break so much stuff. Lets have some fun. Change-Id: If0185e0323391c7055d47b797083bb5afde57cb5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1829 Reviewed-by: glittershark <grfn@gws.fyi> Tested-by: BuildkiteCI
2020-08-19 r/1680 chore(tvl-slapd): decapitalise V's username, use displayName insteadV1-1/+2
Change-Id: I59cf5e1c850960ae639c6a3ebeb273a4441c48bb Reviewed-on: https://cl.tvl.fyi/c/depot/+/1788 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com> Reviewed-by: tazjin <mail@tazj.in>
2020-08-19 r/1679 feat(whitby): add VV1-0/+6
Change-Id: I887760edd67135df4e2f58a874314b317838d2e8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1787 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-08-19 r/1678 feat(tvl-slapd): add VV1-0/+5
Change-Id: Id9253635b73b0eac7871a6baa4f0c7417d135cfe Reviewed-on: https://cl.tvl.fyi/c/depot/+/1786 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2020-08-17 r/1670 feat(whitby): add etaeta1-0/+6
Change-Id: I7aa2bd2cb2c001b48ebd25b20f28cdfb0883ba3f Reviewed-on: https://cl.tvl.fyi/c/depot/+/1782 Tested-by: BuildkiteCI Reviewed-by: isomer <isomer@tvl.fyi> Reviewed-by: tazjin <mail@tazj.in>
2020-08-17 r/1669 docs: Update README for the repository itselfVincent Ambo2-5/+7
Adds a lot more information about what is actually going on here, as well as links to relevant things such as our Monorepo document. I'm aware that I didn't make all `//...` links clickable, but I realised at some point that it might be easier to just update cheddar to support highlighting those natively :-) Change-Id: Icbb212a6c07a5cf38ab8e65d83a64bec783eb8d0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1768 Tested-by: BuildkiteCI Reviewed-by: isomer <isomer@tvl.fyi> Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-17 r/1668 chore(nixos/sourcegraph): Bump version to 3.18.0Vincent Ambo1-1/+1
Change-Id: I5cb8b2da7e40075c99fab6bd57295c8c1d770e86 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1781 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-17 r/1667 feat(nixos/clbot): Add ability to post in multiple channelsVincent Ambo2-13/+41
Adds the ability to post to multiple channels by simply running multiple instances of clbot. We should probably implement support for this in clbot itself, but right now I can't be bothered to write Go. Change-Id: I5cffd0dc10a7f6cc19c37c5834c5610166b4ae23 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1771 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com> Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-08-14 r/1651 feat(whitby): enable programs.mosh.multi1-0/+1
Change-Id: Ibc8df6f6382b5b64e272bedece6b65762f9693c9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1750 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-08-13 r/1643 fix(whitby): disable sshd(8) password authentication.multi1-1/+5
Change-Id: I44068c253840a34e3c21be2bd03b7569df1c3b98 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1718 Reviewed-by: glittershark <grfn@gws.fyi> Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2020-08-09 r/1629 feat(whitby): add multimulti1-0/+6
Change-Id: Ibfc2a5fcf73099b8414b8c46958007374d14fd0a Reviewed-on: https://cl.tvl.fyi/c/depot/+/1701 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: isomer <isomer@tvl.fyi>
2020-08-09 r/1625 feat(tvl-slapd): add multimulti1-0/+5
Change-Id: I2fec9b5dd92da0343426c4a129d882fa87d92e6a Reviewed-on: https://cl.tvl.fyi/c/depot/+/1699 Reviewed-by: cynthia <cynthia@tvl.fyi> Reviewed-by: eta <eta@theta.eu.org> Reviewed-by: edef <edef@edef.eu> Reviewed-by: lukegb <lukegb@tvl.fyi> Tested-by: BuildkiteCI
2020-08-06 r/1606 fix(whitby): Increase nrBuildUsers to 128Vincent Ambo1-0/+1
Change-Id: I3a444e163745d17d10f923c0be7565840937c53a Reviewed-on: https://cl.tvl.fyi/c/depot/+/1662 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-06 r/1605 fix(whitby): I'm a trusted user, owoVincent Ambo1-0/+1
Change-Id: I2666b3cf8bdefcb5d4caeddf191dc65f6a8cb05f Reviewed-on: https://cl.tvl.fyi/c/depot/+/1661 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-05 r/1602 chore(whitby): add edefedef1-0/+6
Change-Id: I7265259bc87594bd481c7bd455187c09b1effd1c Reviewed-on: https://cl.tvl.fyi/c/depot/+/1650 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-08-03 r/1562 chore(tvl-slapd): refresh edef's passwordedef1-1/+1
Change-Id: I5a7a913656bfb9dd6c9fb4e2b4a1212607c50dd3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1592 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-08-02 r/1537 fix(besadii): Don't add people back to attention set.Luke Granger-Brown1-3/+7
This uses the not very documented ignore_default_attention_set_rules review API. Change-Id: I650777bbbd24a1922f26967fbbd7da06d14b6782 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1513 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-08-01 r/1533 fix(whitby): Move Restic's cache into /var/backup/resticVincent Ambo1-0/+1
It tries to write this to ~/.cache otherwise, which worked for the git user but does not work for root (??) Change-Id: I02d04da7d8e2b8782ce70bc72bce0b90c3961aa0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1546 Reviewed-by: glittershark <grfn@gws.fyi> Tested-by: BuildkiteCI
2020-08-01 r/1532 fix(whitby): Make timer unit match the unit it should startVincent Ambo1-1/+1
Oversight in the previous CL. Change-Id: I8767322d7d860fc410796f8d63b7a6c38a8ab447 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1545 Reviewed-by: glittershark <grfn@gws.fyi> Tested-by: BuildkiteCI
2020-08-01 r/1531 feat(whitby): Include PostgreSQL dumps in Restic backupsVincent Ambo1-7/+6
Changes the restic backup service to run as root, rather than git, and include the PostgreSQL dumps in its scope. The on-machine credentials have already been placed in the right location in /var/backup/restic Fixes: 27 Change-Id: Iae76357442f07596a2297ce7b6d51aae392d2074 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1541 Reviewed-by: kanepyork <rikingcoding@gmail.com> Reviewed-by: glittershark <grfn@gws.fyi> Tested-by: BuildkiteCI
2020-08-01 r/1523 feat(whitby): Enable daily PostgreSQL backupsVincent Ambo1-0/+7
... daily is just the default cron pattern for this, but we might also want this to happen more frequently. Not sure yet. Change-Id: I4e433fefebd93488891e765b5842fdb6537e3c6d Reviewed-on: https://cl.tvl.fyi/c/depot/+/1518 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-01 r/1518 fix(ops/paroxysm): Ensure paroxysm is started on bootVincent Ambo1-0/+1
Change-Id: Iba6557cbf4e0001277bd996df59318b4308fc92e Reviewed-on: https://cl.tvl.fyi/c/depot/+/1510 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-07-31 r/1513 feat(ops/nixos): Add module for running paroxysm on whitbyVincent Ambo2-0/+29
Change-Id: I415e3b046d4e0fcd7e800ddab0c7f1aeb639c5e2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1502 Tested-by: BuildkiteCI Reviewed-by: eta <eta@theta.eu.org>
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-11-01T16·42+0000