about summary refs log tree commit diff
path: root/tools
AgeCommit message (Collapse)AuthorFilesLines
2021-12-24 feat: add /tmpJérôme Petazzoni1-5/+7
Examples of programs that fail when /tmp doesn't exist: - terraform - anything using mktemp and similar helpers
2021-12-24 docs: mention arm64 metapackageEthan Davidson1-2/+4
2021-12-24 chore: fix env var name in error messageJérôme Petazzoni1-1/+1
The error message shows the wrong variable name, which might be confusing for new users.
2021-12-24 docs: update installation instructionsJérôme Petazzoni1-12/+60
These instructions were not up-to-date (they didn't mention the different storage backends, and some variables were tagged as optional while they were mandatory). With this update, they should (hopefully) be more accurate! :) I also added instructions if someone wants to run Nixery outside of the container image (I found it convenient when working on Nixery's code).
2021-12-24 r/3344 fix(depot-scanner): Reenable CI runsVincent Ambo2-2/+2
(zseri:) It seems like relativePath should be replaced with relativeFlag, as no variable with the former name exists, a boolean is as far as I can tell expected, and a boolean with a similar name exists. Lets give it a try in the CI. Change-Id: I0e7e522a41a517a38222dcda3b66731344613c1e Reviewed-on: https://cl.tvl.fyi/c/depot/+/3581 Reviewed-by: tazjin <mail@tazj.in> Autosubmit: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2021-12-21 r/3329 fix(tvl.el): Change prompt for different commandsGriffin Smith1-4/+4
Prompting with "Checkout CL" for the gerrit cherry-pick command doesn't make any sense. Change-Id: I51495e7975202146fae2da0807e525596f2d490e Reviewed-on: https://cl.tvl.fyi/c/depot/+/4516 Autosubmit: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-12-18 r/3295 feat(tools/depotfmt): add a treefmt wrapper with depot configurationVincent Ambo1-0/+16
Right now the only included formatter is gofmt, but we will extend this over time. The version of treefmt is bumped to 0.3.0 (which supports custom config files) until this lands in nixpkgs. Change-Id: I1e1aafd05ec7427c616f90c90490c528ecb2615c Reviewed-on: https://cl.tvl.fyi/c/depot/+/4399 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2021-12-18 r/3294 fix(tools/magrathea): prevent trailing slash for // in pathsterni1-6/+7
All targets would have no trailing slash, so it was at least ugly that // had one as the only legal target. Change-Id: I1b60850ac86d8c550f262841694fb00c518413b8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4404 Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-12-18 r/3293 feat(tools/cheddar): shortlink depot revisionssterni2-0/+12
r/3000 will now be shortlinked to show the commit in cgit. Going via atward probably doesn't make sense at this point, since the depot refs are not available in sourcegraph at all (for reasons I can't repeat). Switching to atward might be interesting when/if we introduce support for shortlinking // paths. Fixes: b/163 Change-Id: I57c1a7d02d881e4f8b3ee1f71755dd7930925dc4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4402 Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: tazjin <mail@tazj.in>
2021-12-18 r/3292 feat(tools/magrathea): let users override the repository rootVincent Ambo1-3/+4
for global installations of magrathea, setting MG_ROOT can be a way to switch quickly between different repositories (esp. in combination with `cd (mg path)`). Change-Id: I4627fe78b7cc112b75ab57e7806ffd85c6d38aee Reviewed-on: https://cl.tvl.fyi/c/depot/+/4396 Tested-by: BuildkiteCI Reviewed-by: wpcarro <wpcarro@gmail.com>
2021-12-18 r/3291 feat(tools/magrathea): add a path commandVincent Ambo1-4/+19
this command prints the absolute directory for a given target. it can be combined with shell aliases to add quick navigation commands. unfortunately due to the nature of computers implementing something like `mg cd` directly is not possible. Change-Id: Icc88eb97384812c620c49fe2de8fa331f4d7153b Reviewed-on: https://cl.tvl.fyi/c/depot/+/4395 Tested-by: BuildkiteCI Reviewed-by: wpcarro <wpcarro@gmail.com>
2021-12-18 r/3290 feat(tools/magrathea): wire up parsed targets with commandsVincent Ambo1-24/+61
magrathea now does what it says on the tin - build and shell commands can be used with the targets specified on the command line. implementation notes: * string representation of target has been changed to look like the target spec format, this is now used in user-facing messages * errors returned by the target parser make the program exit with status 1 * normalisation could be done better (for example, maybe it makes sense to always do it) but it's good enough for now Change-Id: Ib85f389a5cec92b3c2f3b9c0b40764435bbcc68b Reviewed-on: https://cl.tvl.fyi/c/depot/+/4394 Tested-by: BuildkiteCI Reviewed-by: wpcarro <wpcarro@gmail.com>
2021-12-17 r/3287 feat(tools/magrathea): add weirdly overengineered target spec parserVincent Ambo1-1/+105
i'm not sure what happened here, but it works (yes, the fancy target printing is completely unnecessary, but oh well): #;152> (parse-target "foo") #target(foo) #;153> (parse-target "//foo") #target(//foo) #;154> (parse-target "//foo/bar") #target(//foo/bar) #;155> (parse-target "//foo/bar/") #target(//foo/bar) #;156> (parse-target "//foo/bar:baz") #target(//foo/bar:baz) #;157> (parse-target "//foo/bar/:baz") #target(//foo/bar:baz) #;158> (parse-target "//foo/bar:") (error . "unexpected end of input while parsing virtual target") #;159> (parse-target "//foo//") (error . "unexpected root-anchor while parsing normal target") the most notable thing is that trailing slashes are allowed in the physical targets, since people may be autocompleting these on the shell from folder names. Change-Id: I32975ad77fe2a327130dc9574011fe92cce49f84 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4393 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi> Reviewed-by: wpcarro <wpcarro@gmail.com>
2021-12-17 r/3286 feat(tools/magrathea): bootstrap a tool for working with depotVincent Ambo2-0/+133
this is going to be a serious version of //tools/depot-build. right now it doesn't support parsing any target specs yet, so only shells and builds for the physical project of the current folder work. Change-Id: I4308e29da940571622ff9e539fbb8ededd27aca7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4335 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: grfn <grfn@gws.fyi>
2021-12-15 r/3260 fix(tvl.el): Replace fifth with nth 4Griffin Smith1-1/+1
as before, fifth doesn't exist on all emacsen, but nth definitely does Change-Id: Ic0e4e3790402d960d1546d37187758a4d9ca33c2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4346 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-12-15 r/3252 chore(3p): bump NixOS channels to 2021-12-15sterni1-1/+1
* Switch from (unused) NixOS 21.05 to newly released NixOS 21.11 channel. * grfn/machines/yeren: Linux 5.14.x -> 5.15.x * 3p/gerrit, 3p/gerrit_plugin/oauth: update dependency bundle output hashes to what whitby comes up with now. Change-Id: Id3bfe3d07b9223f77ec628954783a6b0a48f4086 Reviewed-on: https://cl.tvl.fyi/c/depot/+/4310 Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: grfn <grfn@gws.fyi> Reviewed-by: tazjin <mail@tazj.in>
2021-12-13 r/3214 fix(passively): Load known terms on startupVincent Ambo1-0/+1
Change-Id: Ia32f0a1c158255a098e7d4017bb585b5c4e3c657
2021-12-10 r/3206 fix(tvl.el): Fix use of label command in refsVincent Ambo1-4/+4
The l= is part of the command, not of the shape of commands, and the previous command concatenation logic was wrong because of that. Fix is done in the most obvious way: Make the l= part of the command. Change-Id: Ia3c08c3da60fe5fc38f29a2d94adcd123e4f3052
2021-12-10 r/3196 feat(tvl.el): Add autosubmit feature to magit-gerrit-rubberstampVincent Ambo1-4/+6
This makes this function a true rubberstamp again, leading to rubberstamped CLs automatically being merged after CI passes. This is similar to the initial functionality we had last year, where this directly submitted changes, but with the addition of the CI checks. Change-Id: I946b074b968eb18a64c4edb0043f7a4af28759b4
2021-12-10 r/3195 feat(tvl.el): Add shortcut for push with Autosubmit+1Vincent Ambo1-0/+11
This almost makes for a sort of fire&forget button, except we don't have a way to automatically pick reviewers yet :) Change-Id: I6f446270f8aaf0409ccb6321bdbb5c349079cd19
2021-12-09 r/3163 feat(tvl.el): Add magit-gerrit-cherry-pickGriffin Smith1-0/+20
Bound to `A g`, this behaves similarly to `magit-gerrit-checkout` - it prompts for a CL number, then cherry-picks the latest patchset of that CL number Change-Id: Ieef970b99d96170e8c960cc7687ead9022948f8b
2021-12-07 r/3152 fix(tvl.el): s/fourth/cadddrGriffin Smith1-1/+1
Apparently some emacsen don't have functions like `fourth` etc. Change-Id: I3d8b698685ce3b1757b427b32d8e27938cc26661
2021-12-06 r/3142 feat(tools/passively): Implement automatic passive learning in EmacsVincent Ambo2-0/+128
Adds all the functionality described in the README in cl/4066. This code is very closely related to //users/tazjin/russian/russian.el Change-Id: I14f1052cebfbe4886e75e8efc730eacbf8773f29
2021-12-03 r/3138 feat(tools/passively): Bootstrap new Emacs package 'passively'Vincent Ambo2-0/+79
Passively is a tool to help people learn information via Emacs, designed for language learning. As of this CL, the actual implementation still lives in //users/tazjin/russian/russian.el but I am generalising it here. Change-Id: Iac5a8cfc78415496637a7ba5ddc4c2a1aa6bee26
2021-11-23 r/3088 refactor(readTree): Move 'drvTargets' into readTreeVincent Ambo1-1/+1
This function is also generally useful for readTree consumers that have the concept of subtargets. Change-Id: Ic7fc03380dec6953fb288763a28e50ab3624d233
2021-11-20 r/3075 feat(tvl.el): Add magit-gerrit-checkout commandGriffin Smith1-1/+57
Add a new magit-gerrit-checkout command, which prompts for a CL number then fetches and checks out the latest patchset of that CL with a detached HEAD. Change-Id: I88b8209d40017479d97ed40ecbd5fd1ccd7cf650 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3880 Reviewed-by: grfn <grfn@gws.fyi> Reviewed-by: Profpatsch <mail@profpatsch.de> Tested-by: BuildkiteCI
2021-10-29 chore: Bump all Go dependenciesVincent Ambo3-36/+169
Result of 'go get -u && go mod tidy'
2021-10-29 chore: Bump nixpkgs pin to nixos-unstable 2021-10-29Vincent Ambo1-2/+2
2021-10-29 chore(docs): Bump included nix-1p versionVincent Ambo1-2/+2
... basically never updated this, oops.
2021-10-29 docs: Remove note about unsupported Google projectsVincent Ambo1-2/+0
I no longer work at Google and the repo has moved, so this is no longer relevant.
2021-10-12 r/2967 refactor(tools/rust-crates-advisory): move advisory-db to 3psterni1-7/+1
Change-Id: Iaaed35de078292c0c99a7c83de9ca5fdf27b8135 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3711 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-10-11 r/2965 chore(tools/cheddar): update dependenciessterni1-350/+187
Change-Id: I64c305c60d2187633cb84e8642e4639f1402325c Reviewed-on: https://cl.tvl.fyi/c/depot/+/3725 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-10-08 revert: "feat(storage): Add generic support for content-types"Jérôme Petazzoni1-10/+0
This reverts commit 7db252f36a68d875429a25e06d88fbfc804d84fd. Superseded by the implementation in #127.
2021-09-15 r/2868 refactor(rust-crates-advisory): check type instead of blacklistingsterni1-3/+2
`our-crates` can just check if the attributes in question are derivation (i. e. have an `outPath`) instead of blacklisting the `__readTree` attribute specifically. Change-Id: I472692e89c0e9eff551372c72a73ab765b0b6599
2021-09-10 r/2835 refactor(cheddar): Support multiple shortlinks & custom link setsVincent Ambo1-10/+42
This makes it possible for users of cheddar as a library to supply their own shortlinks. In practice it is unlikely anyone will do this, but the change also allows us to (relatively) easily add additional shortlinks to the set used by TVL. Note that Cheddar is primarily intended for use by TVL and the default rendering function interfaces have not changed, and will default to using TVL shortlinks. A new public function `format_markdown_with_shortlinks` has been added with which users can use an alternative set of shortlinks. This function should not be used in TVL depot code. Change-Id: I4ddab28cbcf45d07c51323b7b730b96e62922816 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3083 Tested-by: BuildkiteCI Reviewed-by: lukegb <lukegb@tvl.fyi>
2021-08-25 feat(prepare-image): Ensure /usr/bin/env is always presentVincent Ambo1-0/+12
This is required by common patterns in shell scripts. There are some caveats around this. Adding logic to filter whether coreutils is included in an image would slow down the Nix evaluation, so the link is currently created even in cases where it doesn't point to anything. Fixes #109
2021-08-24 r/2764 fix(tvl.el): Fix compilation warningsVincent Ambo1-5/+8
This fixes all compilation warnings except the one about 'tvl lacking a parent group, which we can look into later (it doesn't matter that much). Change-Id: Iaff5e7f5f251f0670afb0a47031ccf197de69818 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3408 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2021-08-17 r/2746 feat(tvl.el): Make target branch configurableGriffin Smith1-4/+9
I'm using gerrit at work now, and would like to use tvl.el to interact with it via Emacs, but we use a different default branch than "canon". This makes it configurable, and also marks it as safe so I can configure it in .dir-locals.el Change-Id: I66d4c7ce94351f2df863ec49dbc3e1d1d6d1547a Reviewed-on: https://cl.tvl.fyi/c/depot/+/3369 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-08-06 chore(build): Allow passing in a specific commit hash when buildingVincent Ambo1-2/+3
Required for builds where the full repository isn't available (e.g. from a tarball).
2021-07-28 r/2697 feat(tools/depot-nixpkgs-update): tool to update our pinsterni1-0/+44
Very simplistic tool to aid with updating our nixpkgs channel pins: This tool prints the correct two `*Hashes` sets to stdout, so you can easily delete the appropriate lines in `third_party/nixpkgs/default.nix`, run ./bin/depot-nixpkgs-update | wl-copy and paste it into your editor. Doing this fully automatically would be possible, but would either a) require changing `default.nix`, so it is regex-able more easily, b) touching something like rnix-parser which I have no motivation to at the moment or c) searching for the old hashes and replacing them with the new ones. This may be a simple and worthwhile improvement in the future. Change-Id: I4df44e3827ce9ff6e4fe2d336c08016d799e21a7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3252 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2021-07-15 docs: Update build badge in READMEVincent Ambo1-1/+1
Moves the build badge to point at Github Actions, instead of the old (failing) Travis build
2021-06-26 feat(storage): Store blob content-type in extended attributesJérôme Petazzoni4-3/+20
After the discussion in #116, this stores the blob content types in extended attributes when using the filesystem backend. If the underlying filesystem doesn't support extended attributes, storing blobs won't work; also, if extended attributes get removed, blobs won't be served anymore. We can relax this behavior if needed (i.e. log errors but still accept to store or serve blobs). However, since the Docker Engine (and possibly other container engines) won't accept to pull images from a registry that doesn't use correct content types for manifest files, it could be argued that it's better to give a hard fail. (Otherwise, the container engine gives cryptic error messages like "missing signature key".) I can change that behavior (and log errors but still store/serve blobs to the filesystem) if you think it's better.
2021-06-20 feat(ci): don't mount /var/cache/nixery from tmpfs into docker containerFlorian Klink2-2/+13
With https://github.com/google/nixery/pull/127, nixery will use extended attributes to store metadata (when using local storage). Right now, our integration test mounts a tmpfs to /var/cache/nixery. However, *user* xattrs aren't supported with tmpfs [1], so setting xattrs would fail. To workaround this, use a folder in the current working directory and hope it's backed by something supporting user xattrs (which is the case for GitHub Actions). [1]: https://man7.org/linux/man-pages/man5/tmpfs.5.html#NOTES
2021-05-25 r/2630 chore(nixpkgs): Bump channels to 2021-05-25Vincent Ambo1-2/+2
* users/grfn/system/home/yeren: remove obsolete awscli2 overrides * ops: make new isSystemUser || isNormalUser assertion happy * users/grfn/system/system/mugwump: make buildkite agents system users * users/tazjin/nixos/camden: set isSystemUser = true for git * users/tazjin/emacs: Remove missing & broken packages * third_party/openldap: remove, as the argon2 module is now enabled upstream * third_party/gerrit_plugins: Pinned new unstable hashes * third_party/nix, third_party/grpc: Disabled CI as these are broken * third_party/overlays/emacs: Bumped version to stay in sync with channel * third_party/buzz: Update LIBCLANG_PATH to reference libclang.lib, since libclang's default output no longer contains libclang.so * users/grfn/system/home: Install julia-stable instead of julia (which aliases to julia-lts), as the latter depends on an insecure version of libgit Change-Id: Iff33b0ecb0ef07a82d1de35e23c40d2f4bf0f8ed Reviewed-on: https://cl.tvl.fyi/c/depot/+/3001 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org> Reviewed-by: grfn <grfn@gws.fyi>
2021-05-24 r/2622 chore(3p/overlays): Remove LLVM11 bump from overlayVincent Ambo1-4/+0
* This was mostly for //third_party/nix and its dependencies which now have been set to use llvmPackages_11 manually. * For //users/grfn/achilles we also manually select the newer LLVM version. * //tools/cheddar doesn't seem to need llvm anymore. * //third_party/buzz also compiles with clang 7.1.0 * replace clang-tools everywhere with new attribute clang-tools_11 For the future we may want to have something similar again, but it may not be necessary to invest too much time into it: nixpkgs is set to upgrade their default llvmPackages to LLVM 11 as well at some point in the near future. Co-Authored-By: sterni <sternenseemann@systemli.org> Change-Id: Id83868dbc476a6c776b59518b856c933f30ea79d Reviewed-on: https://cl.tvl.fyi/c/depot/+/3135 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: sterni <sternenseemann@systemli.org> Reviewed-by: grfn <grfn@gws.fyi>
2021-05-23 r/2612 fix(hash-password): Ensure OpenLDAP with correct module is usedVincent Ambo1-2/+2
This is fixed in upstream nixpkgs, but we're not yet at a commit where it's used, so it's important to use the OpenLDAP from //third_party Change-Id: I7c033cd23f45a95c4a4af864ffe561c496833a0d Reviewed-on: https://cl.tvl.fyi/c/depot/+/3143 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2021-05-17 r/2595 feat(tools): add rust-crates-advisoryProfpatsch4-0/+176
We have a bunch of crates in `third_party/rust-crates`; it would be great if we could check them for existing CVEs. This tool does that, it takes the rust security advisory database, parses the applicable CVEs, and cross-checks them against the actual crate versions we list in our package database. The dumb parser we wrote is tested against all entries in the database, so we will notice when upstream breaks their shit. Checking the semver stuff is easy enough with the semver crate. If an advisory matches, it prints the whole thing and fails the build. Change-Id: I9e912c43d37a685d9d7a4424defc467a171ea3c4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2818 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: sterni <sternenseemann@systemli.org>
2021-05-12 r/2591 docs(cheddar): fix typo in build.rs commentsterni1-1/+1
Change-Id: Id323b4a187b3837d60ac6b8917f3e8571954b8e6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3119 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2021-05-04 r/2565 feat(cheddar): Implement highlighting of CL and bug shortlinksVincent Ambo4-1/+51
Implements highlighting of shortlinks like cl/123, or b/123. Highlighting works by replacing the input Markdown using a simple regular expression replacement. We also considered parsing and replacing these links in the Markdown AST, but it would have been significantly more complex (due to arena allocation) for little upside and no obvious performance benefit. Change-Id: I53f03fb17491046d89d0b7f605929571c11ee9a8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3082 Tested-by: BuildkiteCI Reviewed-by: eta <eta@theta.eu.org>
2021-05-03 r/2557 feat(cheddar): Add a build script that injects BAT_SYNTAXESVincent Ambo1-0/+50
The BAT_SYNTAXES environment variable is injected during Nix builds and points to the build result of //third_party/bat_syntaxes, which is included in the cheddar binary. During development this is a bit annoying: Opening a Nix shell will set the right variable, but it will also mess with the Cargo environment in ways that make it annoying to use (*and* require a pre-build of all dependencies as required by naersk). For an easier development flow, this adds a build script that automatically runs Nix, fetches the path to the syntax set and sets the environment variable. It is a no-op if the variable is already set. This makes for a smoother development experience when working on cheddar in a depot checkout. Change-Id: Ib5435518897e3443247edd9f362821c0b9ff756b Reviewed-on: https://cl.tvl.fyi/c/depot/+/3081 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-12-11T22·08+0000