about summary refs log tree commit diff
path: root/ci-builds.nix (unfollow)
AgeCommit message (Collapse)AuthorFilesLines
2020-08-26 r/1725 feat: Implement automatic CI target detection for the depotVincent Ambo1-119/+0
Automatically walk the entire depot tree and pick out things that are "buildable", then include them in the attribute `ci.targets` (which is now also the target for CI builds). A long time ago, in a land far away, we (well, I, at the time) had a prototype of this which ran into constant issues with infinite recursions while trying to walk the tree. In fact, this is why readTree originally gained the `__readTree`-attribute which marks things that were imported automatically. Based on some code edef whipped up earlier (with the breakthrough being that we also add the attribute to top-level folders, which suddenly resolves a whole bunch of problems), I've now implemented this actually working version. At the moment all builds still happen as one big bag of builds, but at some point we will granularise this. Change-Id: I86f12ce7f63dae98e7e5c6646a4e9d220de783f2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1854 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com> Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-26 r/1724 fix(besadii): Remove unused Nix code from the derivationVincent Ambo1-4/+0
This was used previously when build granularity was besadii's task, which it no longer is. Change-Id: I6df2db1ed4730a7953199b7b48aa9ad916418b22 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1853 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-26 r/1723 chore: Add __readTree attributes on systems that need to be builtVincent Ambo2-1/+7
This is a temporary state (TODO added) to be picked up by the new CI logic. Change-Id: Id4702740ffd18325088e2a8a0c6157a8cee7ccf7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1852 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-08-26 r/1722 fix(3p/haskell_overlay): Fix imports and remove unused fileVincent Ambo2-50/+1
Change-Id: I1973bc1b11442dc8150c3b07bd06a6f94c4c4380 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1851 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi> Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-26 r/1721 chore: Add meta.ci=false to various broken projectsVincent Ambo4-1/+8
These projects, which are not currently included in CI runs, don't build at the moment. Upcoming logic changes would mean that we would start including them in CI, which is undesirable until they're fixed - but I'm not going to be doing that now. Change-Id: I7c337e098be8bff00db6d99fc7236a695f5a85f5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1850 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-26 r/1720 fix(multi/home-manager): Prevent readTree failures on home-manager.nixVincent Ambo1-1/+1
This file is not a readTree-compatible Nix file, but rather a NixOS module. At some point it should be moved elsewhere and .skip-subtree'd to avoid this issue. Change-Id: If1b3f7cc80084af1f44036b8b9272f7b76438c2c Reviewed-on: https://cl.tvl.fyi/c/depot/+/1849 Tested-by: BuildkiteCI Reviewed-by: multi <depot@in-addr.xyz>
2020-08-26 r/1719 fix(readTree): Add __readTree attribute to foldersVincent Ambo2-2/+2
This is required to automatically walk the tree (see subsequent commits). Note: Lisp packages are removed from the CI builds in this commit because the attrValues of third_party.lisp will contain an element that is simply `true`, which causes a type error. These packages are re-added when CI refactoring is complete. Change-Id: I21e2b719e6c7161c23d2867a216f4daa1c6c8394 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1848 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-26 r/1718 chore(glittershark): Temporarily skip grfn's subtreeVincent Ambo2-9/+6
... explanation within. Change-Id: Ia357d30577fcc8b3d8ebc85288683e50cc318fea Reviewed-on: https://cl.tvl.fyi/c/depot/+/1847 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-26 r/1717 chore(third_party): Skip subtrees of large external projectsVincent Ambo3-0/+3
Change-Id: Icef3900f9d1e55fd15c5286a794293368c33492c Reviewed-on: https://cl.tvl.fyi/c/depot/+/1846 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-26 r/1716 fix(tvlc): Do not call mkDerivation in default.nixVincent Ambo1-1/+1
This should actually just be an attribute set. Change-Id: Idea1a9f7cfbb2eecd7e6342c6b5aeb66d3f3441a Reviewed-on: https://cl.tvl.fyi/c/depot/+/1845 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-26 r/1715 chore(tazjin/dt): Move project into user folderVincent Ambo4-1/+2
... missed this when going multi-user. Change-Id: If2c9f24555f82e944e3cebb45b2a13801cc1f35e Reviewed-on: https://cl.tvl.fyi/c/depot/+/1844 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-26 r/1714 fix: Remove ./presentations from top-levelVincent Ambo1-1/+0
This folder doesn't exist, it's part of my user folder now. We didn't notice because nothing is walking the tree. Change-Id: Idc6f20a8e4806a158c598fd63d381ab07934be1e Reviewed-on: https://cl.tvl.fyi/c/depot/+/1843 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-26 r/1713 chore(net): Remove mushroomVincent Ambo5-223/+0
We don't need to build this anymore. Change-Id: I0ddd4ec3db9eb4774553003e18c5503b0f431810 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1842 Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: lukegb <lukegb@tvl.fyi> Tested-by: BuildkiteCI
2020-08-26 r/1712 fix(tazjin/emacs): Override vterm's internal title generatorVincent Ambo1-8/+11
This used to be part of the public interface, but was removed and replaced with a (less useful) format string. Change-Id: I387557c20c2eddde16974c3fcad1712569db5325 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1841 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2020-08-25 r/1711 feat(tools/tvlc): pass list of derivations to depot-scannerKane York4-7/+36
You can now provide a list of Nix derivations to tvlc to get a git worktree + sparse-checkout containing only the paths needed to build the specified derivations. Known bugs: even though //third_party is only passed to readdir(), git doesn't know this and includes all of //third_party/*. Change-Id: I9dccebd3fbff4bb04ebd568175cf0a7e37d71ab3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1826 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-08-24 r/1710 feat(gs/emacs): Make cider test report popup biggerGriffin Smith1-0/+4
Change-Id: I72a7af7cbbcde3eee7ef1e61d1a431a2f7779c62 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1839 Reviewed-by: glittershark <grfn@gws.fyi> Tested-by: BuildkiteCI
2020-08-24 r/1709 fix(owothia): Use correct articlesGriffin Smith1-4/+16
I'm an owothia, owo Change-Id: I59d523268851e792067e10d440147d12409c60f5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1833 Tested-by: BuildkiteCI Reviewed-by: eta <eta@theta.eu.org>
2020-08-23 r/1708 feat(tazjin/emacs): Add magit-find-file-worktree functionVincent Ambo2-3/+10
This will look up a file in the current worktree of the git repository enclosing `default-directory'. In combination with project-find-file this lets me toggle between switching to a file within a project, and within the whole depot. Change-Id: Ie1011f10051fc2c4bd4279b0944a79c7edf92f3b Reviewed-on: https://cl.tvl.fyi/c/depot/+/1838 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-08-23 r/1707 feat(tazjin/emacs): Add depot-aware project.el extensionVincent Ambo1-2/+10
If this ends up working well I'll extract it to tvl.el Change-Id: I83722abf33a3346ccc7957c8d64d6381b15c6ee9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1837 Tested-by: BuildkiteCI Reviewed-by: isomer <isomer@tvl.fyi>
2020-08-23 r/1706 feat(nix/buildLisp): Expose an sbcl attributeGriffin Smith1-4/+6
Expose an `sbcl` attribute on packages and programs, to allow for easier development either with SLY or on a REPL. Change-Id: Ide4d087a5223561e1fe192ef32dc593c54b5a20e Reviewed-on: https://cl.tvl.fyi/c/depot/+/1834 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-08-23 r/1705 feat(whitby): Enable log forwarding via journaldriverVincent Ambo1-0/+7
Change-Id: I474159acfe514f6f2eb7867e4eba854016590ab1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1836 Tested-by: BuildkiteCI Reviewed-by: isomer <isomer@tvl.fyi>
2020-08-23 r/1704 refactor(tvix/libutil): Mark single-argument constructors explicitVincent Ambo34-274/+311
This is the clang-tidy lint 'google-explicit-constructor'. There's a whole bunch of breakage that was introduced by this, and we had to opt out a few types of this (esp. the string formatting crap). In some cases minor other changes have been done to keep the code working, instead of converting between types (e.g. an explicit comparison operator implementation for nix::Pid). Change-Id: I12e1ca51a6bc2c882dba81a2526b9729d26988e7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1832 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com> Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-21 r/1703 style(tvix): Add missing braces in expressionsVincent Ambo16-72/+132
The previous clang-tidy invocation missed some header files, which has now been rectified. Change-Id: I31547754fbf52f439dc7aeefb08ab90bd50c4156 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1831 Reviewed-by: glittershark <grfn@gws.fyi> Tested-by: BuildkiteCI
2020-08-21 r/1702 style(tvix): Replace remaining typedefs with using statementsVincent Ambo16-45/+48
Change (mostly) created via: fd . -e hh -e cc | parallel clang-tidy -p ~/builds/nix --fix "'--checks=-*,modernize-use-using'" Change-Id: I90ad6e64a5399f2005dfb9956074ec51c2c78cea Reviewed-on: https://cl.tvl.fyi/c/depot/+/1830 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-21 r/1701 revert(whitby): Use Tvix as the system Nix on whitbytazjin1-1/+0
This reverts commit 8fa30870673fcdd5410c2593321153ab7cfbe36f. Reason for revert: This almost worked. We discovered two important issues: - The daemon startup does not correctly handle the socket passed in by systemd. - There is some issue with chunking of large calls, running a build for ci-builds resulted in: tazjin@whitby /depot (canon)> nix-build -A ciBuilds.__allTargets E20200821 01:42:22.846053 12601 shared.cc:306] error: Rpc call addTextToStore to unix:///nix/var/nix/daemon-socket/socket failed (RESOURCE_EXHAUSTED): Received message larger than max (10889961 vs. 4194304) Change-Id: Ic5ba4ef06a4953cf71a36b139fe25ea673cb6fee Reviewed-on: https://cl.tvl.fyi/c/depot/+/1802 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-21 r/1700 feat(whitby): Use Tvix as the system Nix on whitbyVincent Ambo1-0/+1
... this is going to break so much stuff. Lets have some fun. Change-Id: If0185e0323391c7055d47b797083bb5afde57cb5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1829 Reviewed-by: glittershark <grfn@gws.fyi> Tested-by: BuildkiteCI
2020-08-21 r/1699 fix(tvix): Add newlines to build logsVincent Ambo1-2/+2
.. turns out producing an entire build log on a single line is not useful. This does not use `<< std::endl` because we have run into buffering issues with the implementation of the logs->gRPC sink, but intend to replace this in the future using a structured sink for BuildEvent protos rather than a raw stream. Change-Id: Ia9b05fa804391d389e2ef53ab4436c0ec5cc452e Reviewed-on: https://cl.tvl.fyi/c/depot/+/1828 Reviewed-by: glittershark <grfn@gws.fyi> Tested-by: BuildkiteCI
2020-08-21 r/1698 refactor(tvix): Replace several DiscardLogsSink() with std::cerrVincent Ambo7-24/+15
Introduces the actual log sink in several places where we actually want the build logs to thread through correctly. This should cover *most* build paths. Change-Id: I735dff8a79f7e35a5874eb89b4abb980f9703dc2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1827 Reviewed-by: glittershark <grfn@gws.fyi> Tested-by: BuildkiteCI
2020-08-21 r/1697 feat(tvix): Write build logs into the build log sinkVincent Ambo1-3/+4
This *should* wire up the builder's logs all the way back through the gRPC client, where they are then conveniently discarded. Change-Id: I65f22526d0b5a8b8d90f28665bc1b4bc7f7c802a Reviewed-on: https://cl.tvl.fyi/c/depot/+/1825 Reviewed-by: glittershark <grfn@gws.fyi> Tested-by: BuildkiteCI
2020-08-21 r/1696 chore(tvix): Thread std::ostream through builder goalsVincent Ambo1-42/+66
This passes an output stream for build logs to almost all relevant functions inside of build.cc by threading it through the `Goal`-abstraction. Store calls that create goals but don't have a sink available use the DiscardLogsSink(). Change-Id: I2c0cb1aec1f9150f33113f4752055cea518ede8b Reviewed-on: https://cl.tvl.fyi/c/depot/+/1824 Reviewed-by: glittershark <grfn@gws.fyi> Tested-by: BuildkiteCI
2020-08-21 r/1695 chore(tvix): Add TODO comment for std::ostream in buildDerivationVincent Ambo1-0/+1
Change-Id: I1e3b6abd4606d86f00ddaa8e2a1809f51e27126d Reviewed-on: https://cl.tvl.fyi/c/depot/+/1823 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-21 r/1694 style(tvix): Move a comment to fix a formatting issueVincent Ambo1-4/+3
Change-Id: Idd490b978c07d1b1113ee97b5d19719a27436fba Reviewed-on: https://cl.tvl.fyi/c/depot/+/1822 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-20 r/1693 feat(gs/emacs): Make forge popups 0.75 heightGriffin Smith1-0/+5
Change-Id: I019d9fc3846a66690b946fb3a0213881697d9caf Reviewed-on: https://cl.tvl.fyi/c/depot/+/1821 Reviewed-by: glittershark <grfn@gws.fyi> Tested-by: BuildkiteCI
2020-08-20 r/1692 refactor(gs/emacs): Make slack popups take up half the windowGriffin Smith1-1/+1
Only having a third has left me feeling like I need a little more space. Change-Id: I9424ec8bf2eeb55a62f21dd72b5ee6251670b0b4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1800 Reviewed-by: glittershark <grfn@gws.fyi> Tested-by: BuildkiteCI
2020-08-20 r/1691 feat(gs/emacs): Add capture template for windtunnelGriffin Smith1-0/+3
Change-Id: I6c9079318f13746d50d239dc0ca86a0f86a5920d Reviewed-on: https://cl.tvl.fyi/c/depot/+/1799 Reviewed-by: glittershark <grfn@gws.fyi> Tested-by: BuildkiteCI
2020-08-20 r/1690 feat(gs/emacs): Tweak slack configGriffin Smith2-15/+32
Disable the prompt for sending thread replies to the channel, tweak the slack popup buffer settings, add some bindings, and move all the slack config to its own file. Change-Id: I670394942bd9b7e4cfc22953227c21d50a315c22 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1798 Reviewed-by: glittershark <grfn@gws.fyi> Tested-by: BuildkiteCI
2020-08-20 r/1689 chore(tvix): Thread a std::ostream through Store::buildPathsVincent Ambo15-43/+63
This part of the store API needs to carry a handle to the log sink from now on, so that it can be passed in as appropriate from the gRPC handlers. In all places where there is no such handler available at the moment, the discarding log sink has been inserted. This can be used as a convenient grep target in the future. Change-Id: I26628e30b4c6437dccdf8f722ca2e8ed827dfc19 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1797 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com> Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-20 r/1688 style(tvix): typedef -> using in build.cc/store-api.hhVincent Ambo2-18/+17
this shuts up a few lints ... Change-Id: I51ee9418767e97247f98b170cdad137e5b12b44d Reviewed-on: https://cl.tvl.fyi/c/depot/+/1796 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com> Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-20 r/1687 refactor(tvix): Remove some default values from virtual methodsVincent Ambo2-6/+18
This is not actually legal code, but it kind of ... works. There are more of these around, these were just the ones Griffin stumbled upon while working on the build logs. Change-Id: Iff9821d8fe145dd426648a8ff4510a73f67c9b7d Reviewed-on: https://cl.tvl.fyi/c/depot/+/1795 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com> Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-20 r/1686 feat(tvix): Implement std::streambuf for a build log -> gRPC sinkGriffin Smith1-0/+18
Introduces a class which implements std::streambuf by sending build log lines to the provi