about summary refs log tree commit diff
path: root/ops/machines/whitby
AgeCommit message (Collapse)AuthorFilesLines
2022-02-17 r/3838 refactor(ops/modules): Move user configuration into moduleVincent Ambo1-80/+1
Rather than defining all system users inline on whitby, move them into a module that can be imported on multiple machines. Configuration for terminfos that we've added follows along. Note that while doing this I've disabled logins for riking and isomer since they are currently inactive in TVL. Change-Id: Id18031d355afc34079c5e6e49dc6943e61809a8f Reviewed-on: https://cl.tvl.fyi/c/depot/+/5298 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org> Autosubmit: tazjin <tazjin@tvl.su>
2022-02-16 r/3837 refactor(ops/modules): Rename git-serving -> joshVincent Ambo1-2/+2
cgit has its own module now Change-Id: I9b4cc322374517b8bd3db43345831e2bf43c4bb1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5295 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-02-16 r/3836 refactor(ops/modules): Move cgit configuration into a moduleVincent Ambo1-0/+2
The ancient `//web/cgit-taz` path stems from the time I had code.tazj.in serving my initial version of the depot. I've been meaning to clean this up for forever, so here we go. Note that this leaves the git-serving module in a strange state where it only deals with josh. I'll rename it accordingly. Change-Id: I47ed1e9d90958299b5440a18a1b9075274754e33 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5294 Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: sterni <sternenseemann@systemli.org>
2022-02-13 r/3812 chore(3p/sources): Bump channels & overlayssterni1-1/+2
* //nix/buildLisp: re-enable CCL, as the crash has been fixed upstream, although it is unclear what exactly caused / fixed it. * //ops/whitby: the kitty build broke upstream, so we can't install the terminfo on whitby for a bit. Change-Id: I5710acbe837fbc936e334b2e81f9cf00ed6ae280 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5274 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-01-31 r/3723 style: format entire depot with nixpkgs-fmtVincent Ambo1-64/+77
This CL can be used to compare the style of nixpkgs-fmt against other formatters (nixpkgs, alejandra). Change-Id: I87c6abff6bcb546b02ead15ad0405f81e01b6d9e Reviewed-on: https://cl.tvl.fyi/c/depot/+/4397 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org> Reviewed-by: lukegb <lukegb@tvl.fyi> Reviewed-by: wpcarro <wpcarro@gmail.com> Reviewed-by: Profpatsch <mail@profpatsch.de> Reviewed-by: kanepyork <rikingcoding@gmail.com> Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: cynthia <cynthia@tvl.fyi> Reviewed-by: edef <edef@edef.eu> Reviewed-by: eta <tvl@eta.st> Reviewed-by: grfn <grfn@gws.fyi>
2022-01-29 r/3716 fix(ops/whitby): enable services.nginx.statusPageGriffin Smith1-0/+3
This is necessary for the nginx prometheus exporter to work Change-Id: I2343d6f5d3d6d6772777d5e14426a537aa1c8ef7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5127 Autosubmit: grfn <grfn@gws.fyi> Reviewed-by: lukegb <lukegb@tvl.fyi> Tested-by: BuildkiteCI
2022-01-29 r/3715 feat(whitby): Enable nginx prometheus exporterGriffin Smith1-7/+22
Might be nice to look at rates of requests etc. Change-Id: I4d12ab0c1a555793e803de4a9614e616951a94e5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/5125 Reviewed-by: lukegb <lukegb@tvl.fyi> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: grfn <grfn@gws.fyi>
2022-01-13 r/3590 feat(whitby): Install a handful of systemPackagesVincent Ambo1-5/+13
Adds more things I keep using via nix-shell, as well as the deploy-whitby script (which is independent of a particular depot checkout). Change-Id: I36f87de7645768a05268c90ba9b3ab833bacca05 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4881 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-01-12 r/3588 fix: resolve remaining security.acme.email warningssterni1-1/+1
These were missed in cl/4784. Change-Id: I01a5827900c1b3bdfdf9b1c36dcca8d6b59073a1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4866 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: wpcarro <wpcarro@gmail.com> Autosubmit: sterni <sternenseemann@systemli.org>
2022-01-04 r/3518 feat(ops): Add initial oauth2_proxy configurationVincent Ambo1-0/+5
The intent is to configure oauth2_proxy pointing at Keycloak to enable usage with nginx auth_request directives. I want to expose this as a function from within the module in which nginx server configuration blocks can be wrapped, but the function for that is currently a placeholder. Change-Id: I5ed7deb9bf1c62818f516e68c33e8c5b632fccfe Reviewed-on: https://cl.tvl.fyi/c/depot/+/4767 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-01-04 r/3517 chore(ops): Remove login.tvl.fyi moduleVincent Ambo1-1/+0
It looks like we won't need this for oauth2_proxy when combined with nginx auth_request setups. Change-Id: I2294aee6226b4f64a27bf6592c2d18092d0268cc Reviewed-on: https://cl.tvl.fyi/c/depot/+/4766 Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: grfn <grfn@gws.fyi>
2022-01-01 r/3509 refactor(modules/smtprelay): Load credentials via agenixVincent Ambo1-0/+1
Change-Id: I56f6887e1fd35551cfc83ad08cafebb611f4a341 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4760 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org> Reviewed-by: Profpatsch <mail@profpatsch.de> Autosubmit: tazjin <mail@tazj.in>
2021-12-27 r/3468 fix(whitby): Point grafana at new auth providerVincent Ambo1-4/+6
Grafana was still pointing at the (now non-existent) CAS setup. This changes the endpoints to use Keycloak instead and updates the client secret. Change-Id: Ib25d38330aba2ef6d894e8c33d86852c884ab5be Reviewed-on: https://cl.tvl.fyi/c/depot/+/4706 Tested-by: BuildkiteCI Autosubmit: tazjin <mail@tazj.in> Reviewed-by: grfn <grfn@gws.fyi>
2021-12-26 r/3411 chore: friendship ended with cas, now keycloak is our best friendVincent Ambo1-2/+1
Note that the login.tvl.fyi WWW configuration is still kind of hanging around until we've settled where Keycloak lives. Change-Id: Iaca4e394a7371cafa3716ca66ef09c4eca5b1520 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4626 Autosubmit: tazjin <mail@tazj.in> Tested-by: BuildkiteCI Reviewed-by: lukegb <lukegb@tvl.fyi>
2021-12-25 r/3402 refactor(ops/whitby): Move Gerrit secrets into agenixVincent Ambo1-0/+7
Gerrit has OAuth2 and email related secrets which now live in agenix instead of a random file on disk. Change-Id: I6220fbb7a2e2ec0102a900b4bcf6150b8b4d32ef Reviewed-on: https://cl.tvl.fyi/c/depot/+/4612 Tested-by: BuildkiteCI Autosubmit: tazjin <mail@tazj.in> Reviewed-by: lukegb <lukegb@tvl.fyi>
2021-12-25 r/3401 feat(whitby): Configure initial Keycloak setupVincent Ambo1-1/+34
Trialing this as an alternative to CAS that is a little easier to configure and can help us delegate authentication to other OIDC services. Change-Id: Iad63724d349334910af8fed0b148e4ba428f796b Reviewed-on: https://cl.tvl.fyi/c/depot/+/4608 Tested-by: BuildkiteCI Autosubmit: tazjin <mail@tazj.in> Reviewed-by: lukegb <lukegb@tvl.fyi>
2021-12-16 r/3266 feat(ops/modules): Provide some modules to all nixosesGriffin Smith1-1/+0
For modules that are gated behind a mkEnableOption, it's reasonable to just provide them to all Depot-built nixos systems without requiring people to explicitly import them. This defines a special module called `default-imports.nix` which imports these modules (currently just tvl-cache.nix and automatic-gc.nix, as I'm being rather conservative adding things here to avoid breaking anyone's system), then provides that module as one of the `modules` passed at the top-level nixos/eval-config invocation. Change-Id: I3be299ab10ae4c451ef11c514edb3c89318a2278 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4345 Tested-by: BuildkiteCI Autosubmit: grfn <grfn@gws.fyi> Reviewed-by: tazjin <mail@tazj.in>
2021-12-13 r/3218 feat(ops/whitby): install alacritty terminfosterni1-0/+1
alacritty is used by grfn atm. Change-Id: I10dacd301044f9c37790e22e955cb068fcbd2cfc
2021-12-13 r/3211 feat(ops/whitby): add terminfos for other terminals usedsterni1-0/+2
* foot (me) * kitty (lukegb) Change-Id: I65303e39c4adb05e362792a544134fc2884175bf
2021-12-13 r/3210 feat(whitby): Add some more useful programsVincent Ambo1-0/+3
I keep using these in nix-shell but really they should just be installed. Change-Id: Ic2c36bae8b582fef88029b288accdfd3c8bc0f1b
2021-12-12 r/3209 feat(ops/secrets): Make (encrypted) secrets part of the treeVincent Ambo1-1/+1
Currently in NixOS configuration using agenix secrets there is no build time validation of secret paths - things fail at runtime (system activation). To prevent that, this CL makes the secrets part of the tree based on the same configuration file used by agenix itself. This guards against: * agenix secrets.nix definition for a non-existent file * age.secrets value in a NixOS config for a non-existent secret Change-Id: I5b191dcbd5b2522566ff7c38f8a988bbf7679364
2021-12-10 r/3203 refactor(ops): Move panettone secrets to agenixVincent Ambo1-2/+2
Relates to b/161 Change-Id: I508e5a0eacab668f4bd39a2c888d894b96bed093
2021-12-10 r/3201 fix(whitby): Fix typo in buildkite-agents group nameVincent Ambo1-2/+2
... really would like some assertion helpers for this sort of stuff. Change-Id: I32d1de18ebfbbdfa5128a8fbdad2efcc511f8514
2021-12-10 r/3199 refactor(ops): Move Nix cache secret to agenixVincent Ambo1-2/+9
... and also the public key, just to keep the distribution mechanism the same. Change-Id: Ief14daf9344c0fb99eeb5789c1ec9bfb1f12bee0
2021-12-10 r/3198 refactor(ops): Use besadii configuration from agenixVincent Ambo1-0/+17
We already checked this in, but this commit adds the configuration for making use of it. There are two copies of besadii's JSON configuration with different permissions. Note that the buildkite-graphql-token path needs to be updated in static-pipeline.yml, but this needs to happen in a separate commit after deploy because the pipeline will break otherwise. Change-Id: I6fab4bf1a2e679df7cf76521e2b53bd9dadbac62
2021-12-10 r/3197 refactor(ops): Move grafana secret into agenixVincent Ambo1-1/+2
Change-Id: Id141758135c796881e91d20b950dae74c40d9ab3
2021-12-10 r/3191 fix(ops/irccat): Avoid permissions issue with LoadCredentials=Vincent Ambo1-6/+1
The DynamicUser + Group configuration does not work as planned, thus the systemd LoadCredentials feature is used instead which makes the file (which itself is only readable by root) available in a memory-backed location only readable by the service. The secret is only available to `ExecStart` commands, so units using this feature can not be used with pre/post units and the like if those commands need secrets. To accommodate this, the merge of configuration files has been moved into the service launch script, which is now the ExecStart= process. For details take a look at https://www.freedesktop.org/software/systemd/man/systemd.exec.html#LoadCredential=ID:PATH Change-Id: I693fe5677cc0d63c7aa485c2c7472457c5262166
2021-12-10 r/3184 refactor(ops): Move irccat secret into agenixVincent Ambo1-0/+6
The irccat module uses DynamicUser, so to grant permission to it a new group has been added for irccat. I have some vague memory of DynamicUser + Group not behaving as one would expect, but we'll see what happens. Change-Id: Iab9f6a3f1a53c4133b635458ce173250cc9a3fac
2021-12-10 r/3183 refactor(ops): Move clbot SSH key into agenixVincent Ambo1-2/+7
Change-Id: Iae03ead7dda0509689a76f0d76f9cfeb8434e967
2021-12-10 r/3180 chore(whitby): Remove shadowsocks serviceVincent Ambo1-6/+0
No longer required on whitby. Change-Id: I93951c6b708eae81ddb03df920a4068c1ccde9e7
2021-12-10 r/3176 refactor(ops): Move buildkite-agent-token into agenixVincent Ambo1-0/+6
Relates to b/161 Change-Id: I5d3a698d437928966d8b78ce9e0ba226c1437655
2021-12-10 r/3175 refactor(ops): Move owothia secret into agenixVincent Ambo1-0/+1
Relates to b/161 Change-Id: I25445281b0dd3c3f3660f8bb0d8337506a1e427b
2021-12-10 r/3174 refactor(ops): Move clbot secret into agenixVincent Ambo1-3/+7
Relates to b/161 Change-Id: I7badf22ff93bb4e8b06e4dd4a8bf880b0bd48f09
2021-12-10 r/3173 feat(ops/secrets): Configure secrets for gerrit-queueVincent Ambo1-0/+6
Adds a systemd EnvironmentFile secret that contains the Gerrit username & password for gerrit-queue. Change-Id: I25acf87764c26774045138402b8a417b6813ee8f
2021-12-10 r/3172 feat(ops/modules): Add module for running gerrit-queueVincent Ambo1-0/+4
This is not yet including the secret configuration for gerrit-queue, and just expects the secret (gerrit username & password) to be available in /etc/secrets. Change-Id: Ia465ef7f3f521c70d606d7fdeba9aa83c7e1b98b
2021-11-21 r/3078 fix(ops/restic): Move whitby's backup to GleSYS object storageVincent Ambo1-1/+1
Since GCP nuked us, the backups are now moving to GleSYS' S3-compatible object storage. This refactors the restic module to support S3-compatible storage instead of GCP, and switches to the appropriate new secret paths. The secrets were placed on whitby manually and I verified that the backups work. This fixes b/157 Change-Id: I6a9d2b0581967605ce736605a3befb44cdeae7e1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3883 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2021-10-01 r/2946 feat(whitby): serve static.tvl.{fyi|su} with max cache settingsVincent Ambo1-0/+1
The setup is explained in the comment, but TL;DR: Use the derivation hash of static files to create permanent URLs. Relates to b/151. Change-Id: Ib1ca3a1a00c90a47f4bf39c29a8b4bbf5b215e7d Reviewed-on: https://cl.tvl.fyi/c/depot/+/3664 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2021-10-01 r/2941 feat(whitby): Serve //corp/website on tvl.suVincent Ambo1-0/+1
Change-Id: I21e1ddf9a32568cac8ad2595869ac8670867efa9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3658 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-09-18 r/2890 refactor(whitby): Move restic path configuration into modulesVincent Ambo1-7/+0
This lets each service declare their backup paths together with the configuration for the service, which is a lot more sensible than what we had before. Fixes b/147 Change-Id: If76fe62639f4cc0e6fbb63a2959d584479d8f0fb Reviewed-on: https://cl.tvl.fyi/c/depot/+/3583 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2021-09-16 r/2878 refactor(ops/restic): Move restic configuration into a new moduleVincent Ambo1-22/+18
Relates to b/147. First step towards giving depot modules the ability to declare their own backup directories by moving all restic configuration into a new module and adding a NixOS option for inclusion/exclusion paths for backups. This still keeps all backup paths within the whitby config. Change-Id: Ia96833668f1a3d02da892261153d8b02156b8ac0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3565 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2021-09-16 r/2877 feat(git-serving): Configure josh to serve the depot over HTTPVincent Ambo1-14/+6
Previously we served the dumb git HTTP protocol from code.tvl.fyi via cgit. This CL disables this feature and instead runs josh in the same location (by redirecting appropriately), but while also enabling partial cloning of all subtrees of the depot. For example, after this CL the following would result in an independent clone of //nix/readTree: git clone https://code.tvl.fyi/depot.git:/nix/readTree.git Note that there are no josh workspaces configured at all for now, these references are only for static depot subpaths. Please refer to the documentation for josh for more information on available kinds of josh filters. Josh state is kept in a systemd state directory in /var/lib/josh and backed up to Restic. Backing this up is necessary, as josh uses stateful information to do things like tracking merges and rewriting history per subtree appropriately to avoid cloned repositories ending up in peculiar states. Change-Id: I156f0298c2aa42e3bdbf5a0e86109070d640c56e Reviewed-on: https://cl.tvl.fyi/c/depot/+/3563 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2021-09-10 r/2837 feat(whitby): Serve static HTML dir for deploys.tvl.fyiGriffin Smith1-1/+2
Add a new domain and nginx virtual host at deploys.tvl.fyi, serving out of a static directory on whitby which is created by systemd-tmpfiles. This will be used to serve diffs rendered by nix-diff for pending deploys for whitby Since this contains stateful data, it is added to the restic backups on whitby. Refs: b/110 Change-Id: I5869d40800bbf5fb8fb39878a857f66ff5787830 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3144 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-08-24 r/2751 feat(ops): Serve nixery.dev from whitbyVincent Ambo1-0/+1
Adds a new module for the nixery.dev domain and serves it from whitby. Note that the DNS records do *not* point to whitby yet, so deploying this will lead to a failed TLS provisioning unit - but this is intentional. Change-Id: I911f67a0aa24f8df3cb52d2cfc49a8b6132cf718 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3383 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2021-08-13 r/2726 feat(ops/www): Point images.tvl.* at NixeryVincent Ambo1-0/+1
Change-Id: I39f979c68e7b74f6da6a7da0f07aaa470886d451 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3346 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Reviewed-by: sterni <sternenseemann@systemli.org>
2021-08-12 r/2721 feat(ops/modules): Add module for running NixeryVincent Ambo1-0/+4
This sets up a very simple Nixery instance with some things lacking: * no support for garbage-collecting image fragments (yet) * no popularity setup The plan is to use this to get the ball rolling on a separate domain (e.g. images.tvl.fyi), iron things out and then look into flipping over nixery.dev Change-Id: Ic594809f9d487fec7a0f632d608752a3f9c61315 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3280 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Reviewed-by: sterni <sternenseemann@systemli.org>
2021-05-25 r/2630 chore(nixpkgs): Bump channels to 2021-05-25Vincent Ambo1-1/+1
* users/grfn/system/home/yeren: remove obsolete awscli2 overrides * ops: make new isSystemUser || isNormalUser assertion happy * users/grfn/system/system/mugwump: make buildkite agents system users * users/tazjin/nixos/camden: set isSystemUser = true for git * users/tazjin/emacs: Remove missing & broken packages * third_party/openldap: remove, as the argon2 module is now enabled upstream * third_party/gerrit_plugins: Pinned new unstable hashes * third_party/nix, third_party/grpc: Disabled CI as these are broken * third_party/overlays/emacs: Bumped version to stay in sync with channel * third_party/buzz: Update LIBCLANG_PATH to reference libclang.lib, since libclang's default output no longer contains libclang.so * users/grfn/system/home: Install julia-stable instead of julia (which aliases to julia-lts), as the latter depends on an insecure version of libgit Change-Id: Iff33b0ecb0ef07a82d1de35e23c40d2f4bf0f8ed Reviewed-on: https://cl.tvl.fyi/c/depot/+/3001 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org> Reviewed-by: grfn <grfn@gws.fyi>
2021-05-25 r/2628 chore(whitby): Add ZNC state to Restic backupsVincent Ambo1-1/+1
Until we have declarative ZNC config (which requires a solution for secrets handling in it), make sure we back this up as well. Change-Id: Idb186327da171eb6d3dbbd83801639f1f9321a40 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3159 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2021-05-23 r/2619 fix(whitby): Fix irccat configuration for incorrectly named optionVincent Ambo1-1/+3
irccat is passing the realname option as the ident of the user, which doesn't match what is in ZNC. It hasn't seen any upstream commits in a long time, so I'm just leaving this as is and fixing it locally in our config. Change-Id: I3bf865f37b8df9c1cd891a94245ca3fad376bbe1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3150 Reviewed-by: sterni <sternenseemann@systemli.org> Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2021-05-23 r/2617 feat(whitby): Let sterni bear the wheelVincent Ambo1-1/+1
Change-Id: Ib4f7dcbdc754d2fc271f501a9ea270e983a3645f Reviewed-on: https://cl.tvl.fyi/c/depot/+/3147 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2021-05-23 r/2610 feat(whitby): Enable fail2banGriffin Smith1-0/+2
I like running fail2ban on any machine that has stuff like ssh world-open, to limit the potential for password brute-force attacks etc. Change-Id: I0c60811ae5a2fddb44f04679fb455e646b8e39c5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3138 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-11-14T07·09+0000