about summary refs log tree commit diff
path: root/ops
AgeCommit message (Collapse)AuthorFilesLines
2021-12-27 r/3474 feat(ops/secrets): Import secrets for tf-glesysVincent Ambo3-0/+21
Adds the secrets and some instructions for deploying the GleSYS Terraform infrastructure. Change-Id: I1a10f9cee7648d406b3d27ef45fc74b6923cbc30 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4712 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2021-12-27 r/3473 feat(ops/keycloak): Import Buildkite OIDC clientVincent Ambo1-0/+21
This was previously configured in the UI. Change-Id: I68361b1489093b76736adab2e38ed7b474b10881 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4711 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2021-12-27 r/3472 feat(ops/keycloak): Import Gerrit OIDC clientVincent Ambo1-0/+21
This was previously configured in the UI. Change-Id: Ib15b8ecca96d7814dc85d62199865b22bdb63f95 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4710 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2021-12-27 r/3471 fix(ops/keycloak): Move Terraform state to GleSYS bucketVincent Ambo2-12/+24
This should never sit around locally the way it does now. Change-Id: Icfbdaf1949d6d948a796a0759282ea6144af3621 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4709 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2021-12-27 r/3470 feat(ops/secrets): Add tf-keycloak secrets fileVincent Ambo3-0/+32
This file can be sourced (somehow, depending on the user) while working with //ops/keycloak to get the relevant secrets. Change-Id: Ibb3051c4b019f64824964475451c1c3996db6421 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4708 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2021-12-27 r/3469 feat(ops/keycloak): Add OIDC client for GrafanaVincent Ambo1-0/+14
Completely forgot about Grafana, so it's currently broken. Oops! Change-Id: Ia4e6405428ad8e514d6e61635f9692c57f61defe Reviewed-on: https://cl.tvl.fyi/c/depot/+/4705 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi> Autosubmit: tazjin <mail@tazj.in>
2021-12-27 r/3468 fix(whitby): Point grafana at new auth providerVincent Ambo2-18/+18
Grafana was still pointing at the (now non-existent) CAS setup. This changes the endpoints to use Keycloak instead and updates the client secret. Change-Id: Ib25d38330aba2ef6d894e8c33d86852c884ab5be Reviewed-on: https://cl.tvl.fyi/c/depot/+/4706 Tested-by: BuildkiteCI Autosubmit: tazjin <mail@tazj.in> Reviewed-by: grfn <grfn@gws.fyi>
2021-12-27 r/3453 refactor(ops/secrets): generalize out a mkSecrets functionGriffin Smith2-21/+22
Generalize out a reusable mkSecrets function from the secrets-tree-building that's happening in //ops/secrets, so the same thing can happen in other places in the depot (I want to use it for my personal infrastructure). Change-Id: I059295c8c257d78ad7fa0802859f57c2c105f29b Reviewed-on: https://cl.tvl.fyi/c/depot/+/4679 Reviewed-by: grfn <grfn@gws.fyi> Reviewed-by: zseri <zseri.devel@ytrizja.de> Autosubmit: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2021-12-26 r/3447 feat(ops/machines/all-systems): Add grfn/mugwumpGriffin Smith1-0/+1
Change-Id: I7770b58c44a5700e86c80d1058e89e9fa65d719b Reviewed-on: https://cl.tvl.fyi/c/depot/+/4686 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi> Autosubmit: grfn <grfn@gws.fyi>
2021-12-26 r/3446 fix(auto-deploy): Add missing packages to pathGriffin Smith1-3/+5
Building nix derivations needs tar (provided by gnutar) and gzip on the PATH in order to extract .tar.gz archives. Change-Id: Ia2df7a3a770cfd342dfede58ad34e04805fbd1f8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4685 Tested-by: BuildkiteCI Autosubmit: grfn <grfn@gws.fyi> Reviewed-by: wpcarro <wpcarro@gmail.com>
2021-12-26 r/3428 fix(ops/keycloak): set up client for usage with oauth2_proxyVincent Ambo1-7/+7
This will be useful for things like panettone, pending a NixOS module for oauth2-proxy (the upstream one is too complicated and doesn't support what we need). Change-Id: I4ca193e10a94a29b1fb9003e945896ff8eb61116 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4662 Tested-by: BuildkiteCI Reviewed-by: Profpatsch <mail@profpatsch.de> Autosubmit: tazjin <mail@tazj.in>
2021-12-26 r/3427 fix(ops/keycloak): trust email addresses from LDAPVincent Ambo1-0/+1
Verified emails are required for some things, like e.g. oauth2_proxy Change-Id: Ifb124be40d6d2863cd1b7ed5fbdfcf4827e8808c Reviewed-on: https://cl.tvl.fyi/c/depot/+/4661 Tested-by: BuildkiteCI Autosubmit: tazjin <mail@tazj.in> Reviewed-by: Profpatsch <mail@profpatsch.de>
2021-12-26 r/3426 feat(ops/keycloak): Set up oauth2_proxy clientVincent Ambo1-0/+21
Change-Id: I996d9644ed7e870d6e5a42af117eafbf841da679 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4640 Tested-by: BuildkiteCI Autosubmit: tazjin <mail@tazj.in> Reviewed-by: Profpatsch <mail@profpatsch.de>
2021-12-26 r/3425 feat(ops/keycloak): Check in initial Keycloak configurationVincent Ambo3-0/+51
This is still missing most of the client configuration etc., in part due to bugs in the provider which are preventing resource imports. Change-Id: Ic224ffc001f8e1fe6dcd47b7d002580fdf7b0774 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4628 Tested-by: BuildkiteCI Autosubmit: tazjin <mail@tazj.in> Reviewed-by: Profpatsch <mail@profpatsch.de>
2021-12-26 r/3414 feat(ops/auto-deploy): Support auto-deployWilliam Carroll3-3/+98
Automatically rebuild the current system's NixOS config from the latest checkout of depot. Change-Id: I23aa7af50e16e985ac34df214e0905e770316e5e Reviewed-on: https://cl.tvl.fyi/c/depot/+/4390 Reviewed-by: wpcarro <wpcarro@gmail.com> Reviewed-by: zseri <zseri.devel@ytrizja.de> Reviewed-by: grfn <grfn@gws.fyi> Autosubmit: wpcarro <wpcarro@gmail.com> Tested-by: BuildkiteCI
2021-12-26 r/3411 chore: friendship ended with cas, now keycloak is our best friendVincent Ambo2-30/+1
Note that the login.tvl.fyi WWW configuration is still kind of hanging around until we've settled where Keycloak lives. Change-Id: Iaca4e394a7371cafa3716ca66ef09c4eca5b1520 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4626 Autosubmit: tazjin <mail@tazj.in> Tested-by: BuildkiteCI Reviewed-by: lukegb <lukegb@tvl.fyi>
2021-12-26 r/3410 feat(monorepo-gerrit): Configure for Keycloak compatibilityVincent Ambo2-5/+6
Change-Id: Ic3fce02b071c09cf03e652510f16bafb795a5a1d Reviewed-on: https://cl.tvl.fyi/c/depot/+/4614 Autosubmit: tazjin <mail@tazj.in> Tested-by: BuildkiteCI Reviewed-by: lukegb <lukegb@tvl.fyi> Reviewed-by: grfn <grfn@gws.fyi>
2021-12-25 r/3402 refactor(ops/whitby): Move Gerrit secrets into agenixVincent Ambo3-0/+23
Gerrit has OAuth2 and email related secrets which now live in agenix instead of a random file on disk. Change-Id: I6220fbb7a2e2ec0102a900b4bcf6150b8b4d32ef Reviewed-on: https://cl.tvl.fyi/c/depot/+/4612 Tested-by: BuildkiteCI Autosubmit: tazjin <mail@tazj.in> Reviewed-by: lukegb <lukegb@tvl.fyi>
2021-12-25 r/3401 feat(whitby): Configure initial Keycloak setupVincent Ambo4-1/+59
Trialing this as an alternative to CAS that is a little easier to configure and can help us delegate authentication to other OIDC services. Change-Id: Iad63724d349334910af8fed0b148e4ba428f796b Reviewed-on: https://cl.tvl.fyi/c/depot/+/4608 Tested-by: BuildkiteCI Autosubmit: tazjin <mail@tazj.in> Reviewed-by: lukegb <lukegb@tvl.fyi>
2021-12-24 r/3369 feat(ops/glesys): Provide tf-glesys wrapperVincent Ambo1-0/+8
This provides the right Terraform provider with a wrapper in $PATH. Change-Id: Idcb4fa89dff0161e8a73addfce81959e825c331e Reviewed-on: https://cl.tvl.fyi/c/depot/+/4562 Tested-by: BuildkiteCI Autosubmit: tazjin <mail@tazj.in> Reviewed-by: grfn <grfn@gws.fyi>
2021-12-24 r/3367 style(ops/glesys): apply terraform fmtVincent Ambo1-10/+10
Change-Id: Ibbba78aaecc3b3cba23961a1b10ce5a8eb8ff296 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4580 Tested-by: BuildkiteCI Autosubmit: tazjin <mail@tazj.in> Reviewed-by: grfn <grfn@gws.fyi>
2021-12-24 r/3366 feat(ops/glesys): Add gitignore for Terraform filesVincent Ambo1-0/+3
Change-Id: I67b971f875819fd9daa3b2e952604206b89ee216 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4578 Tested-by: BuildkiteCI Autosubmit: tazjin <mail@tazj.in> Reviewed-by: grfn <grfn@gws.fyi>
2021-12-24 r/3365 feat(ops/glesys): Create objectstorage key for litestreamVincent Ambo1-0/+5
Change-Id: I8b3e4f767440ae7763c1e6ce9fd97c557fe516ee Reviewed-on: https://cl.tvl.fyi/c/depot/+/4577 Tested-by: BuildkiteCI Autosubmit: tazjin <mail@tazj.in> Reviewed-by: grfn <grfn@gws.fyi>
2021-12-24 r/3364 feat(ops/glesys): Move Terraform state to GleSYS bucketVincent Ambo1-0/+11
Change-Id: Ib14fba9a5f06ecdb065dd14580c8088f98e9cb3a Reviewed-on: https://cl.tvl.fyi/c/depot/+/4576 Tested-by: BuildkiteCI Autosubmit: tazjin <mail@tazj.in> Reviewed-by: grfn <grfn@gws.fyi>
2021-12-24 r/3363 feat(ops/glesys): Create bucket & key for storing terraform stateVincent Ambo1-0/+14
Change-Id: I73cfaa614d46afb65ba312e767d1e924669fbae1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4575 Tested-by: BuildkiteCI Autosubmit: tazjin <mail@tazj.in> Reviewed-by: grfn <grfn@gws.fyi>
2021-12-24 r/3362 feat(ops/glesys): Import existing object storage instanceVincent Ambo1-0/+22
Change-Id: I5a5269ef0d385d864dd8f62eb2332e6ae2cb2672 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4574 Tested-by: BuildkiteCI Autosubmit: tazjin <mail@tazj.in> Reviewed-by: grfn <grfn@gws.fyi>
2021-12-21 r/3332 style(ops/besadii): run depotfmtsterni1-2/+2
Unclear if this reformat is caused by the channel update or if this file was ignored previously. Change-Id: I3498ab181c7fff1b132419783e33a96f7bebfe42 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4520 Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-12-19 r/3313 feat(ops/pipelines): annotate patchset builds with Gerrit URLsVincent Ambo1-0/+6
If available, provide a link back to Gerrit on the overview page of a build. Uses the default style (i.e. style unset), which makes it non-intrusive visually. Change-Id: I4271d589d548015b75762fd0584f3958bfcc53e5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4442 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2021-12-19 r/3311 style: format all Go codeVincent Ambo2-5/+5
The code in //users/wpcarro/tools/monzo_ynab/ynab/client.go was not valid Go and has been commented out. Change-Id: Icb4003607f30294dcbf60132eb7722702c7f0d84 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4400 Tested-by: BuildkiteCI Reviewed-by: wpcarro <wpcarro@gmail.com> Reviewed-by: Profpatsch <mail@profpatsch.de>
2021-12-19 r/3307 fix(ops/besadii): fix Gerrit URL format ... againVincent Ambo1-1/+1
got into some kind of race with different patchsets of this CL somehow, idk Change-Id: I3dcdb708f141829b866fbd786483710b43ea9824 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4481 Autosubmit: tazjin <mail@tazj.in> Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2021-12-19 r/3306 fix(ops/besadii): Only set branch to CL when building patchsetsVincent Ambo1-3/+5
If we set this for canon, then stuff starts to fail in non-obvious ways. Change-Id: I3bf38e29151c6066aaf4eba68ae387272d8a82c2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4463 Tested-by: BuildkiteCI Autosubmit: tazjin <mail@tazj.in> Reviewed-by: grfn <grfn@gws.fyi>
2021-12-19 r/3305 fix(ops/besadii): Stop path.Join from eating our URLVincent Ambo1-1/+1
apparently this chomps away at things inside of fragment strings Change-Id: Ie60d52d101dc4281b3a62c228af076791e1c7928 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4462 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2021-12-19 r/3304 feat(ops/besadii): Pass Gerrit link to builds as an envvarVincent Ambo1-1/+7
This makes it possible to annotate builds with a link back to Gerrit. Change-Id: If351785d3b631b96753d41f417ca94bc7a95ac54 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4441 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2021-12-19 r/3303 feat(ops/besadii): Make branch key cl/XXXXGriffin Smith1-1/+5
The branch key for buildkite builds isn't actually used to fetch if a commit is given - instead, it's just a visual grouping of multiple builds. This means we can just make the branch key cl/<cl number>, which is the convention we already use to refer to CLs and gets us a nice visual grouping of builds of successive patchsets of the same CL number, even though the ref we're providing isn't a real ref. Change-Id: Iaa9111297a88f965fda94cd8266240106f58a100 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4347 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in> Autosubmit: grfn <grfn@gws.fyi>
2021-12-19 r/3302 feat(whitby): Add buildkite agents to docker groupGriffin Smith1-1/+1
I'd like to be able to run extra CI steps that include running docker containers (to integration test things like webapps that connect to a database). To do this the buildkite agents themselves need permission to do docker things. Change-Id: I3c9a488708f0e12a508754ac41f04148ca7aedac Reviewed-on: https://cl.tvl.fyi/c/depot/+/4408 Tested-by: BuildkiteCI Autosubmit: grfn <grfn@gws.fyi> Reviewed-by: tazjin <mail@tazj.in>
2021-12-17 r/3288 fix(ops/diogenes): Ensure diogenes buildsWilliam Carroll1-0/+1
diogenes "passed" CI because the file was named configuration.nix (vestage from the NixOS default /etc/nixos/configuration). This CL fixes some issues I encountered after running depot/bin/rebuild-system. TL;DR: - rename configuration.nix -> default.nix to trigger CI - add diogenes to my systems - add public SSH key Change-Id: I24197b8936c201267db6f71f00099dce590eac1d Reviewed-on: https://cl.tvl.fyi/c/depot/+/4388 Tested-by: BuildkiteCI Reviewed-by: wpcarro <wpcarro@gmail.com> Reviewed-by: tazjin <mail@tazj.in> Autosubmit: wpcarro <wpcarro@gmail.com>
2021-12-16 r/3272 feat(wpcarro/marcus): Add marcus to the depotWilliam Carroll1-0/+4
me: marcus, meet depot. depot, meet marcus. Change-Id: Ic6a25ac85e4c7f6dfea2a42b46a4400f92df70a2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4351 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2021-12-16 r/3270 feat(ops/users): Add user zserizseri1-0/+5
Submitted via IRC, instead of the usual Google Groups due to email issues. Change-Id: I71a2bdfd10b02370df61bbba4dabc2f45b6c1009 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4384 Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: tazjin <mail@tazj.in>
2021-12-16 r/3266 feat(ops/modules): Provide some modules to all nixosesGriffin Smith2-1/+13
For modules that are gated behind a mkEnableOption, it's reasonable to just provide them to all Depot-built nixos systems without requiring people to explicitly import them. This defines a special module called `default-imports.nix` which imports these modules (currently just tvl-cache.nix and automatic-gc.nix, as I'm being rather conservative adding things here to avoid breaking anyone's system), then provides that module as one of the `modules` passed at the top-level nixos/eval-config invocation. Change-Id: I3be299ab10ae4c451ef11c514edb3c89318a2278 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4345 Tested-by: BuildkiteCI Autosubmit: grfn <grfn@gws.fyi> Reviewed-by: tazjin <mail@tazj.in>
2021-12-15 r/3261 feat(ops/modules): Add shared module for TVL cacheGriffin Smith1-0/+19
Add a shared nixos module for configuring whitby as a binary nix cache, and refactor tverskoy to use this module. This is enabled via an option to pave the way for including it as an import in all depot-generated nixos configs at some point in the future. Change-Id: I6dcc0e8eb48b1ac34457666dceebeedd5da6c526 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4344 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: wpcarro <wpcarro@gmail.com> Autosubmit: grfn <grfn@gws.fyi>
2021-12-15 r/3253 fix(ops/besadii): Don't send notifications for CI statusGriffin Smith1-0/+8
Don't notify reviewers ever on CI status changes, and only notify the owner if the build fails. Change-Id: If2cf63581b49e3de77181024ce8a4213031f4bd5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4337 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in> Autosubmit: grfn <grfn@gws.fyi>
2021-12-15 r/3250 fix(ops/pipelines): Allow steps to run immediately after uploadVincent Ambo1-0/+11
This fix was recommended by Buildkite and is explained in the comment. Change-Id: I3f1c1c07cba0b417857d69c021c8af4750d645c4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4334 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2021-12-15 r/3248 fix(ops/pipelines): Chunk build pipeline into multiple uploadsVincent Ambo2-10/+51
The number of jobs in the depot pipeline is reaching the limits of the Buildkite backend's ability for a single pipeline upload. Based on a conversation with their support my understanding is that this has to do with internal locking mechanisms at Buildkite. To work around this, we can instead chunk the pipeline into several smaller chunks that are uploaded serially. This commit introduces logic to chunk the pipeline accordingly. The chunk size chosen is 256 for now (a multiple of our number of agents, which is useful if we can get builds from the first chunk to start before the next ones are uploaded). Note that this chunk size is significantly below even the current number of targets (~460 as of this commit), but choosing a lower chunk size might alleviate problems we've been seeing with timeouts during pipeline uploads. Change-Id: I77030aaf8b874c330218b78c77d15216e13b9af7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4332 Tested-by: BuildkiteCI Reviewed-by: wpcarro <wpcarro@gmail.com> Autosubmit: tazjin <mail@tazj.in>
2021-12-14 r/3244 docs(ops/irccat): link to credentials RFEFlorian Klink1-0/+4
https://cl.tvl.fyi/c/depot/+/4264 did move merging config with secrets into ExecStart=, which is tracked in an RFE upstream: https://github.com/systemd/systemd/issues/19604#issuecomment-989279884 We didn't link to this so far, neither in the commit message, nor in a comment. Let's add a comment, so people know when we can undo this. Change-Id: I7bed370b671093bb876592b4dccd562f1c256cd2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4326 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: grfn <grfn@gws.fyi>
2021-12-14 r/3240 docs(ops/pipelines/depot): correct comment about fallback build cmdsterni1-4/+3
We can gcroot the derivation files and drop this step, but have elected not to do so for the moment, see cl/3436. Change-Id: I993a1f3921e9f21e18fa260e76d3dd15ffa556bd Reviewed-on: https://cl.tvl.fyi/c/depot/+/4327 Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: tazjin <mail@tazj.in>
2021-12-14 r/3237 feat(besadii): Make Gerrit label configurableÅsmund Østvold1-5/+11
By default besadii will set the `Verified` label in Gerrit. This adds a config option to set a different label instead if desired. Co-authored-by: Vincent Ambo <mail@tazj.in> Change-Id: I254159e46994e01182987ed5e5e26e27c57f46ce
2021-12-13 r/3220 feat(ops/users): Add wpcarroVincent Ambo1-0/+5
... this was overdue! Change-Id: I435768007db4a0f3663e1aa9376e8cae4d1d0381
2021-12-13 r/3219 chore(ops/users): Rotate password hash for asmundoVincent Ambo1-1/+1
New hash received via an authenticated channel, of course. Change-Id: Idca688d8a8bb2e943aef3937f54d292b48f79fad
2021-12-13 r/3218 feat(ops/whitby): install alacritty terminfosterni1-0/+1
alacritty is used by grfn atm. Change-Id: I10dacd301044f9c37790e22e955cb068fcbd2cfc
2021-12-13 r/3211 feat(ops/whitby): add terminfos for other terminals usedsterni1-0/+2
* foot (me) * kitty (lukegb) Change-Id: I65303e39c4adb05e362792a544134fc2884175bf
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-11-09T22·52+0000