about summary refs log tree commit diff
path: root/tools (follow)
AgeCommit message (Collapse)AuthorFilesLines
2021-05-24 r/2622 chore(3p/overlays): Remove LLVM11 bump from overlayVincent Ambo1-4/+0
* This was mostly for //third_party/nix and its dependencies which now have been set to use llvmPackages_11 manually. * For //users/grfn/achilles we also manually select the newer LLVM version. * //tools/cheddar doesn't seem to need llvm anymore. * //third_party/buzz also compiles with clang 7.1.0 * replace clang-tools everywhere with new attribute clang-tools_11 For the future we may want to have something similar again, but it may not be necessary to invest too much time into it: nixpkgs is set to upgrade their default llvmPackages to LLVM 11 as well at some point in the near future. Co-Authored-By: sterni <sternenseemann@systemli.org> Change-Id: Id83868dbc476a6c776b59518b856c933f30ea79d Reviewed-on: https://cl.tvl.fyi/c/depot/+/3135 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: sterni <sternenseemann@systemli.org> Reviewed-by: grfn <grfn@gws.fyi>
2021-05-23 r/2612 fix(hash-password): Ensure OpenLDAP with correct module is usedVincent Ambo1-2/+2
This is fixed in upstream nixpkgs, but we're not yet at a commit where it's used, so it's important to use the OpenLDAP from //third_party Change-Id: I7c033cd23f45a95c4a4af864ffe561c496833a0d Reviewed-on: https://cl.tvl.fyi/c/depot/+/3143 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2021-05-17 r/2595 feat(tools): add rust-crates-advisoryProfpatsch4-0/+176
We have a bunch of crates in `third_party/rust-crates`; it would be great if we could check them for existing CVEs. This tool does that, it takes the rust security advisory database, parses the applicable CVEs, and cross-checks them against the actual crate versions we list in our package database. The dumb parser we wrote is tested against all entries in the database, so we will notice when upstream breaks their shit. Checking the semver stuff is easy enough with the semver crate. If an advisory matches, it prints the whole thing and fails the build. Change-Id: I9e912c43d37a685d9d7a4424defc467a171ea3c4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2818 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: sterni <sternenseemann@systemli.org>
2021-05-12 r/2591 docs(cheddar): fix typo in build.rs commentsterni1-1/+1
Change-Id: Id323b4a187b3837d60ac6b8917f3e8571954b8e6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3119 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2021-05-04 r/2565 feat(cheddar): Implement highlighting of CL and bug shortlinksVincent Ambo4-1/+51
Implements highlighting of shortlinks like cl/123, or b/123. Highlighting works by replacing the input Markdown using a simple regular expression replacement. We also considered parsing and replacing these links in the Markdown AST, but it would have been significantly more complex (due to arena allocation) for little upside and no obvious performance benefit. Change-Id: I53f03fb17491046d89d0b7f605929571c11ee9a8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/3082 Tested-by: BuildkiteCI Reviewed-by: eta <eta@theta.eu.org>
2021-05-03 r/2557 feat(cheddar): Add a build script that injects BAT_SYNTAXESVincent Ambo1-0/+50
The BAT_SYNTAXES environment variable is injected during Nix builds and points to the build result of //third_party/bat_syntaxes, which is included in the cheddar binary. During development this is a bit annoying: Opening a Nix shell will set the right variable, but it will also mess with the Cargo environment in ways that make it annoying to use (*and* require a pre-build of all dependencies as required by naersk). For an easier development flow, this adds a build script that automatically runs Nix, fetches the path to the syntax set and sets the environment variable. It is a no-op if the variable is already set. This makes for a smoother development experience when working on cheddar in a depot checkout. Change-Id: Ib5435518897e3443247edd9f362821c0b9ff756b Reviewed-on: https://cl.tvl.fyi/c/depot/+/3081 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2021-05-03 r/2556 refactor(cheddar): Split out a library with rendering logicVincent Ambo2-131/+148
Splits `main.rs` into `lib.rs` and `bin/cheddar.rs`, which enables reuse of cheddar's rendering logic in other Rust applications. Change-Id: Ifd1a44a8d1620c595550a0a497a25b0563e917ca Reviewed-on: https://cl.tvl.fyi/c/depot/+/3060 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2021-04-30 feat(build): Run `go vet` as a step in the GitHub Actions workflowVincent Ambo2-1/+3
2021-04-30 refactor(build): Pin dependencies using Go modulesVincent Ambo5-164/+553
Drops the go2nix configuration in favour of pkgs.buildGoModule. Note that the go.sum file is bloated by issues with cyclic dependencies in some Google projects, but this large number of dependencies is not actually built.
2021-04-30 chore(build): Use current git commit hash as build versionVincent Ambo1-6/+4
2021-04-30 chore: Update default NixOS channel to nixos-20.09Vincent Ambo6-8/+8
2021-04-30 chore(ci): Remove unnecessary commands from new CI setupVincent Ambo1-7/+2
* remove a step that was not supposed to be committed ("Do we have Docker?") * remove setup of old temporary storage directory (now done in integration script test instead) * skip creation of out-link for initial Nixery build (to avoid cache-busting on the second build)
2021-04-29 docs: document unset GOOGLE_APPLICATION_CREDENTIALSFlorian Klink2-0/+8
In case the `GOOGLE_APPLICATION_CREDENTIALS` environment variable is not set, a redirect to storage.googleapis.com is issued, which means the underlying bucket objects need to be publicly accessible. This wasn't really obvious until now, so further clarify it.
2021-04-29 feat(ci): add integration tests to GitHub Actions, remove .travis.yamlFlorian Klink3-78/+53
This copies the integration tests from `.travis.yaml` into a script, documents the assumptions it makes, and wires it into GitHub Actions. Contrary to the travis version, we don't use Nixery's GCS backend, as handing out access to the bucket used, especially for PRs, needs to be done carefully. Adding back GCS to the integration test can be done at a later point, either by using a mock server, or by only exposing the credentials for master builds (and have the test script decide on whether GOOGLE_APPLICATION_CREDENTIALS is set or not). The previous travis version had some complicated post-mortem log gathering - instead of doing this, we can just `docker run` nixery, but fork it into the background with the shell - causing it to still be able to log its output as it's running. An additional `--rm` is appended, so the container gets cleaned up on termination - this allows subsequent runs on non-CI infrastructure (like developer laptops), without having to manually clean up containers. Fixes #119.
2021-04-29 feat(ci): remove unneeded permissions: read-allFlorian Klink1-1/+0
We don't intend to label, authenticate or whatever with the GITHUB_TOKEN, so there's not really a reason to give any broader permissions than the defaults.
2021-04-28 feat(ci): Configure initial GitHub Actions setupVincent Ambo1-0/+29
Travis is being deprecated, and this might be the best option for now.
2021-04-27 feat(storage): Add generic support for content-typesJerome Petazzoni1-0/+10
When serving a manifest, it is important to set the content-type correctly (otherwise pulling an image is likely to give a cryptic error message, "Error response from daemon: missing signature key"). This makes sure that we set the content-type properly for both manifests and layers.
2021-04-14 chore(nix): update channel URLJerome Petazzoni1-1/+1
It looks like NixPkgs channels have moved. Fixing this URL allows using nixos-20.09, for instance.
2021-04-13 r/2503 feat(tools/emacs-pkgs): Add new builder for internal packagesVincent Ambo6-21/+51
Adds a new internal builder that makes it possible to override the `emacsPackages` passed to our Emacs packages, which in turn makes it possible to inject them into the emacsPackages fixpoint and use them with features like Emacs native compilation. Change-Id: I80dad57115c83cf5693ae6ba4e4cf3105d103d5e Reviewed-on: https://cl.tvl.fyi/c/depot/+/3003 Tested-by: BuildkiteCI Reviewed-by: adisbladis <adisbladis@gmail.com> Reviewed-by: grfn <grfn@gws.fyi>
2021-04-12 r/2485 refactor(users/glittershark): Rename to grfnGriffin Smith1-1/+1
Rename my //users directory and all places that refer to glittershark to grfn, including nix references and documentation. This may require some extra attention inside of gerrit's database after it lands to allow me to actually push things. Change-Id: I4728b7ec2c60024392c1c1fa6e0d4a59b3e266fa Reviewed-on: https://cl.tvl.fyi/c/depot/+/2933 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: lukegb <lukegb@tvl.fyi> Reviewed-by: glittershark <grfn@gws.fyi>
2021-04-10 r/2463 refactor(cheddar): Consistent use of depot.third_party vs. pkgsVincent Ambo1-3/+3
In preparation for the solution of b/108, we need to consistently use `depot.third_party` for packages that are only packed in the TVL depot and `pkgs` for things that come from nixpkgs. Change-Id: I49d82726b2f3bd7d4923effdd9a7e3f67ddc0659 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2916 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2021-04-10 r/2461 refactor(third_party): Consistent use of depot.third_party vs. pkgsVincent Ambo1-3/+4
In preparation for the solution of b/108, we need to consistently use `depot.third_party` for packages that are only packed in the TVL depot and `pkgs` for things that come from nixpkgs. This commit cleans up a huge chunk of these uses in //third_party Change-Id: Ic382c0cdea7330a84d5f0b7d109c824ddceb94e7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2912 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2021-03-31 r/2375 feat(cheddar): Set front_matter_delimiter = '---' extensionVincent Ambo1-0/+1
This chops off "front matter", which we have - for example - in Tvix documentation that is rendered through Pandoc. Change-Id: I65c7a4ad8171423a50aaf47c046dab8c774346d8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2717 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2021-03-31 r/2374 chore(cheddar): Bump crate dependenciesVincent Ambo3-515/+553
This required some minor Comrak-related refactoring. Change-Id: I5c5898eb895bd5d8743949458ee9406087fcff22 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2690 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi> Reviewed-by: lukegb <lukegb@tvl.fyi> Reviewed-by: flokli <flokli@flokli.de>
2021-03-31 r/2373 test(cheddar): Add simple Markdown rendering testsVincent Ambo3-1/+66
Covers some of the odd things we do, specifically callouts and code rendering. Change-Id: Ib8542373b434b53d277b0d8c9ddb78ac7c5176a5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2689 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2020-12-05 docs: Update README with a link to the NixCon talkVincent Ambo1-4/+10
2020-12-04 docs(config): Fix comment typoDave Nicponski1-1/+1
2020-11-29 r/1960 feat(gerrit-update): Add helper script for updating Gerrit schemaLuke Granger-Brown1-0/+34
I've been running a script similar to this after doing Gerrit version bumps to make sure the schema is up to date, but in the spirit of making sure someone other that myself can do this task I'm formalising it into the depot, where I should've put it in the first place. Change-Id: I50a198e798e2ff26989b01e4bdd0571d85ab62aa Reviewed-on: https://cl.tvl.fyi/c/depot/+/2203 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-10-29 feat(storage): Add support for content-types (GCS only)Vincent Ambo7-13/+34
Extends storage.Persist to accept a Content-Type argument, which in the GCS backend is persisted with the object to ensure that the object is served back with this content-type. This is not yet implemented for the filesystem backend, where the parameter is simply ignored. This should help in the case of clients which expect the returned objects to have content-types set when, for example, fetching layers by digest.
2020-10-27 docs: Add a note about a Nix-native builder to the roadmapVincent Ambo1-0/+6
... if I don't mention this somewhere I'll probably never do it!
2020-10-27 feat(main): Implement caching of manifests in CASVincent Ambo1-0/+35
To ensure that registry clients which attempt to pull manifests by their content hash can interact with Nixery, this change implements persisting image manifests in the CAS in the same way as image layers. In combination with the previous refactorings this means that Nixery's serving flow is now compatible with containerd. I have verified this locally, but CI currently only runs against Docker and not containerd, which is something I plan to address in a subsequent PR. This fixes #102
2020-10-27 feat(main): Implement serving of manifests by digestVincent Ambo1-11/+12
Modifies the layer serving endpoint to be a generic blob-serving endpoint that can handle both manifest and layer object "types". Note that this commit does not yet populate the CAS with any manifests.
2020-10-27 refactor(storage): Rename ServeLayer -> ServeVincent Ambo4-12/+12
This is going to be used for general content-addressed objects, and is not layer specific anymore.
2020-10-27 refactor(main): Split HTTP handlers into separate functionsVincent Ambo2-57/+62
There is a new handler coming up to fix #102 and I want to avoid falling into the classic Go trap of creating thousand-line functions.
2020-10-27 fix(build): Work around arbitrary new maxLayers restrictionVincent Ambo1-1/+1
2020-10-27 fix(build): Completely remove Cachix from build setupVincent Ambo1-2/+2
Installing Cachix started failing on ARM64.
2020-09-18 r/1804 fix(notable): Truncate first lines if they don't fit the windowVincent Ambo1-4/+8
Change-Id: I00746aa285d7854b8e35b6d00aa7c608ccf8ce77 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1988 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-09-18 r/1803 refactor(notable): Remove dependency on dashVincent Ambo2-12/+9
It was only used for #'-each, for which the built-in dolist is sufficient. Change-Id: I326d8b733b36b2edfebd4e9384e70b8a570fee0f Reviewed-on: https://cl.tvl.fyi/c/depot/+/1987 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-09-18 r/1802 feat(notable): Add note archival functionVincent Ambo1-0/+22
Archiving notes is done by just changing the filename to an `archive-` instead of `note-` prefix. Unarchiving is not yet implemented and should be done by moving the note to a *new note ID*. Archiving is bound to 'a' in the note list. Change-Id: I8c225a25bdac5147a26030f47f24edee497f69df Reviewed-on: https://cl.tvl.fyi/c/depot/+/1986 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-09-18 r/1801 feat(notable): Add a mode for displaying single notesVincent Ambo1-1/+48
This mode displays the note's timestamp in its header line, and the content in its body. Some rudimentary key bindings are set up in the list view to view notes in full (RET), and in both modes to quit the current buffer (q). Change-Id: Icb35baec220a9efcfa2b86617b16178669df996f Reviewed-on: https://cl.tvl.fyi/c/depot/+/1985 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-09-17 r/1798 feat(notable): Add note listing functionalityVincent Ambo2-2/+61
Adds a very simple initial display of all notes in chronologically descending order. Currently this is a read-only view with no available actions. Change-Id: I6d47363909f1a9ab8d1d18a1c44966883aa758ad Reviewed-on: https://cl.tvl.fyi/c/depot/+/1983 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2020-09-17 r/1797 feat(notable): Add a function for listing existing notesVincent Ambo1-5/+16
Change-Id: I23697b4798ee4d4e94d3f7c1a4e4e9abf5115345 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1982 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-09-17 r/1796 feat(emacs-pkgs/notable): Begin implementation by adding note-takingVincent Ambo3-0/+124
I'm writing a note taking system because I don't like any of the existing ones. This adds a note-taking function which prompts the user for note content and persists it in a user-configurable directory. Next steps are roughly: * listing notes * editing notes * archiving notes At which point I would consider this functional. The idea is to keep adding incremental improvements that fit my workflows after that, and if it ends up being useful for anyone else - great. Change-Id: I7ae0eb4015511ebdbaeff92d8d32502726eb57c4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1981 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-09-05 r/1763 fix(term-switcher.el): Explicitly fail if buffers are missingVincent Ambo1-4/+5
Since upgrading to Emacs 27 I have observed a strange behaviour where this terminal switcher sometimes fails to select a valid buffer, in which case it falls through to the case that just opens a new buffer instead. This is kind of annoying and to aid in debugging this change makes the creation of new buffers explicit and fails if no matching buffer is found. Note that this is likely not a fix for the issue itself, but it will help debug what is going on. Change-Id: I906869aba7d25156aaf92c090b169ce02785b85e Reviewed-on: https://cl.tvl.fyi/c/depot/+/1930 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-08-26 r/1721 chore: Add meta.ci=false to various broken projectsVincent Ambo1-1/+1
These projects, which are not currently included in CI runs, don't build at the moment. Upcoming logic changes would mean that we would start including them in CI, which is undesirable until they're fixed - but I'm not going to be doing that now. Change-Id: I7c337e098be8bff00db6d99fc7236a695f5a85f5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1850 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-26 r/1716 fix(tvlc): Do not call mkDerivation in default.nixVincent Ambo1-1/+1
This should actually just be an attribute set. Change-Id: Idea1a9f7cfbb2eecd7e6342c6b5aeb66d3f3441a Reviewed-on: https://cl.tvl.fyi/c/depot/+/1845 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-25 r/1711 feat(tools/tvlc): pass list of derivations to depot-scannerKane York4-7/+36
You can now provide a list of Nix derivations to tvlc to get a git worktree + sparse-checkout containing only the paths needed to build the specified derivations. Known bugs: even though //third_party is only passed to readdir(), git doesn't know this and includes all of //third_party/*. Change-Id: I9dccebd3fbff4bb04ebd568175cf0a7e37d71ab3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1826 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-08-20 r/1683 feat(tools/depot-scanner): init projectKane York5-0/+280
depot-scanner is a tool that runs Nix and parses the --trace-file-access output to deduce what files are necessary to evaluate a derivation. Take DEPOT_ROOT from the environment. If depotRoot doesn't exist, print an error early. Fix the build of the protobuf library. Switch to the GRPC build rule, as a service is in this proto file. Create the PathType enum and parse it from cmdline flags. Change-Id: I537b5c6bceecf76ca510f7ac04ab9dad7785feb1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1769 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-08-19 r/1681 feat(tools/tvlc): init projectKane York4-0/+170
tvlc is a tool for managing sparse git worktrees of the TVL depot. It is still in development; near-term tasks include a setup script, Nix dependency resolution, worktree removal, and the dispatch script. See cs.tvl.fyi/depot/docs/designs/SPARSE_CHECKOUTS.md for more info. Change-Id: Iad96656f0206178980fe7dcadd3dffe70d690f8f Reviewed-on: https://cl.tvl.fyi/c/depot/+/1760 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-08-12 r/1641 fix(nsfv-setup): Resample pipeline at 48kHz with pulseaudioVincent Ambo1-1/+1
This gives better overall results and may also prevent drift, which was an issue I've had previously. See also this upstream commit for NSFV: https://github.com/werman/noise-suppression-for-voice/commit/34003bd9ab1509708eab61ef458feaae23327495 Change-Id: I45534cb60130755949cdbfbc39aed2478a249c72 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1717 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
generated by cgit-pink 1.4.1 (git 2.44.0) at 2024-05-28T23·01+0000