about summary refs log tree commit diff
AgeCommit message (Collapse)AuthorFilesLines
2020-08-05 r/1600 feat(3p/nix): Implement FindRoots, CollectGarbageGriffin Smith5-3/+117
Implement the RPC client and server handlers for the FindRoots and CollectGarbage RPC calls Change-Id: Ifa5d582c6a33bd1e7661ac2fc860505ef404dad0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1656 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-05 r/1599 fix(3p/nix): fix usage error of absl::Base64UnescapeKane York1-3/+4
The source and destination strings cannot be the same string - absl will write to the destination in a streaming manner, causing the source to become invalid. Change-Id: I3578cf1f8789a51d85e0950f7987c398f0a00953 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1659 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-05 r/1598 fix(3p/nix/store): fix race condition in downloaderKane York1-1/+2
In certain circumstances, the decompression thread could race ahead of the downloader thread and process the same chunk twice. Clear the data buffer while the lock is held to prevent this kind of incident. Change-Id: I19a84a0c5768d1228c6c18a7664a7b8893ef96de Reviewed-on: https://cl.tvl.fyi/c/depot/+/1658 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-05 r/1597 refactor(tvix): Use SANDBOX_SHELL from environment in CMakeVincent Ambo2-3/+4
This makes the configuration work out-of-the-box using CMake in a Nix shell, rather than having to pass the additional variable to CMake on the command line. Change-Id: I04d3cc4f2e5ecf47bf2ee459d5e48588b84ae4dd Reviewed-on: https://cl.tvl.fyi/c/depot/+/1643 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com> Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-05 r/1596 feat(3p/nix): add tests for BinaryCacheStoreKane York2-0/+106
These tests are in preparation for factoring the Store away from libutil's Callback to absl::StatusOr. They use the newly added MockBinaryCacheStore. Updates: #25 Change-Id: I30c207589ec38254806ebc9a983f35668e353ae9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1595 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-05 r/1595 feat(3p/nix): add MockBinaryCacheStoreKane York3-1/+170
This adds an implementation of BinaryCacheStore to be used by tests exercising both the logic inherent to BinaryCacheStore and more general Store tests. A new library target, nixstoremock, is created to indicate that this file is intended only for use in tests and not in user-facing code. Change-Id: Ib68f777238843a4f3a2303db8a69735fbc22d161 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1645 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-05 r/1594 feat(3p/nix): Implement AddToStore rpc clientGriffin Smith4-2/+69
Implement the AddToStore RPC client method, which uses an AddToStorePathWriterSink (the dual of the AddToStorePathReaderSource on the server side) to hook into the dumpPath machinery (which we should refactor not to use sinks or sources, but not yet) and write dumped paths as binary data over gRPC. With this commit and sandboxing disabled, the following derivation builds cleanly: derivation { name = "test"; builder = ./build.sh; system = "x86_64-linux"; } where build.sh has chmod +x and contains: #!/bin/sh echo 1 > $out Change-Id: I94cab86f0825a3a9993262a9807130645c13bf44 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1638 Reviewed-by: kanepyork <rikingcoding@gmail.com> Tested-by: BuildkiteCI
2020-08-05 r/1593 fix(3p/nix): fix clang-tidy for int-string appendsKane York1-3/+6
Change-Id: I276de7a5fd1c705c87d35dd616e5980c747190aa Reviewed-on: https://cl.tvl.fyi/c/depot/+/1597 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-05 r/1592 fix(edef/keys): add readTree headeredef1-0/+2
Change-Id: I5e1e0f58377201b591730a49f44678a33243b1d5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1653 Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI
2020-08-05 r/1591 chore(edef/users/depot-scan): move to a subdiredef3-3/+1
Change-Id: If202b466aa0874048e89e0dc270ff14883510131 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1652 Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI
2020-08-05 r/1590 fix(users/edef/depot-scan): properly trace readFile/readDiredef3-2/+19
Change-Id: I493e7d20504d34f83a53db50b69e86668874756e Reviewed-on: https://cl.tvl.fyi/c/depot/+/1651 Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI
2020-08-05 r/1589 fix(users/edef/depot-scan): properly depend on pl scriptKane York2-2/+2
Change-Id: I83c75ab75dae9c904e127ad6160c7641d4523e40 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1647 Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI
2020-08-05 r/1588 feat(edef): add SSH CAedef1-0/+5
Change-Id: If6839735d2730fa49363fa46e6fa059dd19bfaf7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1649 Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI
2020-08-05 r/1587 feat(edef/depot-scan): initedef2-0/+23
A small script that parses `nix-instantiate -vv` output to track the files read while instantiating a depot build. Change-Id: I5acf31d55f39c1d1acf9cdead03d33e2c8abab6a Reviewed-on: https://cl.tvl.fyi/c/depot/+/1646 Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI
2020-08-04 r/1586 feat(nix/readTree): allow arguments to depend on tree locationedef1-3/+3
This accepts a function for args, and passes it an attrset containing the locatedAt parameter. Change-Id: I3c0f8ca00605e02a787fda88f32b06f5ef9998f2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1639 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-08-04 r/1585 chore(nix/fetchGoModule): don't move downloaded fileedef1-3/+1
unzip(1) doesn't care about the extension, unlike the unpackFile hook. Change-Id: Ia9d0e17c4b343ee5d32350329eb6267b9dc35605 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1641 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com> Reviewed-by: tazjin <mail@tazj.in>
2020-08-04 r/1584 chore(nix/fetchGoModule): remove overrideAttrs hackedef1-5/+3
This was based off fetchzip, but unlike fetchzip we don't need to support arbitrary archive formats, so we don't actually need the unpack hooks. Change-Id: I5655c031f420422f13e72bb79f4b4b95f6f1204e Reviewed-on: https://cl.tvl.fyi/c/depot/+/1640 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com> Reviewed-by: tazjin <mail@tazj.in>
2020-08-04 r/1583 feat(tvix): Implement AddIndirectRoot, SyncWithGCGriffin Smith2-2/+32
Implement RPC handlers and client methods for the AddWithRoot and SyncWithGC calls. With this and sandboxing disabled, the following derivation is now building successfully: derivation { name = "test"; builder = "/bin/sh"; args = [ "-c" "echo 1 > $out" ]; system = "x86_64-linux"; } Co-authored-by: Vincent Ambo <mail@tazj.in> Change-Id: I245dca0dfca882e2eae5d93364287a8b79fead71 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1637 Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: kanepyork <rikingcoding@gmail.com> Tested-by: BuildkiteCI
2020-08-04 r/1582 chore(tvix): Make not implemented errors nix::UnsupportedGriffin Smith1-22/+24
Without this, these errors print as "uncaught exception of type std::string", but with this they print using nix's built-in error printer, which will be nice to have as we bang out the remaining store API methods. Co-authored-by: Vincent Ambo <mail@tazj.in> Change-Id: I9bd4f77f65b27c8f97b727b98ef62e2061999e71 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1636 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2020-08-04 r/1581 fix(tvix): Partially revert "eliminate exposed global variable"Griffin Smith3-21/+39
This reverts part of commit 976a36c2e482f416acd79a624e6d96cce2564b5b, which was causing scanForReferences to hang indefinitely. I'm not sure what the original intent of the commit was since there's not really much of a commit message on it, but with this I am able to fully realise derivations. Change-Id: I620e892e2fe2244f066512286b086b3dd8f8f6c0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1635 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-04 r/1580 chore: simplify buildGo expressionsedef13-15/+15
This removes redundant `map (p: p.gopkg)` expressions. Change-Id: If25d603c27d63ae011f396caf5d26cdfdb8c28d5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1625 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2020-08-04 r/1579 feat(nix/buildGo): use gopkg attribute of depsedef2-3/+3
This removes the need for `map (x: x.gopkg)` in dependency lists. Change-Id: Ia19e51f14110bf22f5cdbb64fa7ce182e1b34241 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1624 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-08-04 r/1578 feat(nix/buildGo): expose gopkg attribute on buildGo.packageedef1-7/+13
This makes the derivations more uniform, since both third-party packages generated by buildGo.external and native buildGo.package expose their libraries as gopkg attributes now. Change-Id: I547f9860082f36e3300139bf67613eb4fc600d24 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1623 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-08-04 r/1577 fix(3p/gopkgs/cloud.google.com/go): add missing `depot` argedef1-1/+1
Change-Id: If08a43e18a41abd3e2ae7c08ceb6f48844d20df1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1626 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-08-04 r/1576 fix(3p/nix): Don't assert on argc count for build-remoteGriffin Smith1-4/+0
This executable doesn't appear to ever actually look at argv, so throwing an error if argc is wrong seems kinda silly. Change-Id: Iabc9dad785d4c99be9e8fd4ecd954d38b876cecd Reviewed-on: https://cl.tvl.fyi/c/depot/+/1633 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-08-04 r/1575 fix(3p/nix): Properly configure SANDBOX_SHELLGriffin Smith4-2/+23
point the SANDBOX_SHELL macro at the actual path to busybox on the build machine, or allow it to be configured at build-time with a cmake option. Change-Id: I044a1315ba9baa3bc9ceddf29f36d14f9f9ccd96 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1632 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-08-04 r/1574 fix(3p/nix): pass SkipEmpty to StrSplit("", ...)Griffin Smith1-4/+8
When tokenizeString was changed to absl::StrSplit, there was a behavior change because tokenizeString on an empty string returned an empty vector - which the derivation builder (and likely a bunch of other stuff) was depending on. The canonical way of fixing this is by passing absl::SkipEmpty() to the function - there may be other places we need to fix this as well. This commit also includes some opportunistic absl::StrFormats and StrCats, because I was here anyway, but those have no semantic difference. Change-Id: Ibf9bb602284f793fa55728481f63b838fb7a41db Reviewed-on: https://cl.tvl.fyi/c/depot/+/1631 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2020-08-04 r/1573 feat(3p/nix/daemon): catch-all explicit Error-Status conversionKane York2-233/+413
We wrap every server-side proto handler with a macro that catches exceptions and turns them into proper grpc error codes. For the time being, most exceptions map to INTERNAL, the existing mapping. Change-Id: Id6ed6a279b198ad185d32562f39000ccc15eadbf Reviewed-on: https://cl.tvl.fyi/c/depot/+/1599 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-03 r/1572 fix(3p/nix): Initialise glog when starting nix-daemonVincent Ambo1-0/+3
Change-Id: Ic9b5bd2f66fb54675498d8328edecfd31a03997a Reviewed-on: https://cl.tvl.fyi/c/depot/+/1630 Reviewed-by: glittershark <grfn@gws.fyi> Tested-by: BuildkiteCI
2020-08-03 r/1571 fix(3p/nix): Fix parseDrvPathWithOutputsGriffin Smith2-3/+33
At some point the behavior of this function got changed as part of our cleanup - this fixes it to behave the way the rest of the codebase expects (and how it is documented in the header) and covers it with a few tests. Change-Id: Id4c91232968e73489cd866fb4a2a84bcf20d875e Reviewed-on: https://cl.tvl.fyi/c/depot/+/1629 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-08-03 r/1570 chore(3p/nix/daemon): replace all assertStorePath with a macroKane York1-14/+20
Two exceptions: IsValidPath needs to return success for invalid paths, and QueryAllValidPaths shouldn't need to check the paths it gets from itself. Change-Id: I4d9d4125d34e8de42f30070aec607f8a902eded7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1598 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-03 r/1569 refactor(3p/nix): Build nix-daemon as a separate binaryVincent Ambo4-34/+46
Skips over all the monobinary stuff and moves to a separate binary for nix-daemon. This also replaces the flag parsing logic with absl::flags. This causes a behaviour change for --help, which no longer tries to display a man page but instead shows the actual command-line help. Note: This binary no longer links to the Boehm GC. Change-Id: Ib852e994b82f2d56e91262878c10650e656427a9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1622 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-03 r/1568 chore(3p/nix): Move legacy Nix daemon implementationVincent Ambo1-0/+6
The filename this is squatting is too good. Change-Id: I694132b94d24786df84b6b5a698679bdce818b2e Reviewed-on: https://cl.tvl.fyi/c/depot/+/1621 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-03 r/1567 feat(3p/nix): Implement two more RPC callsGriffin Smith4-6/+76
Implement AddTextToStore and BuildPaths both on the client and the server Refs: #29 Change-Id: I45294c3e1c1a7489e42099d36425b7acc04e0427 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1560 Reviewed-by: kanepyork <rikingcoding@gmail.com> Tested-by: BuildkiteCI
2020-08-03 r/1566 feat(3p/nix/build-shell): add run_clang_tidy scriptKane York1-1/+8
This makes it easy to quickly run clang-tidy on tvix without seeing errors from the generated files. Change-Id: I0e25089c5626aebdb5c016629a68da9ccd26c124 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1556 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-03 r/1565 feat(owothia): further owo nounsProfpatsch1-1/+5
Change-Id: I11876d617b6efed443b35b8193cc7a42cf0f0201 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1600 Reviewed-by: Profpatsch <mail@profpatsch.de>
2020-08-03 r/1564 fix(web/panettone): Fix more calls to (id nil) in new issue formGriffin Smith1-32/+35
Change-Id: Id514d80e00005b7b7eb2f305e92af33ca70d6964 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1594 Tested-by: BuildkiteCI Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-08-03 r/1563 fix(web/panettone): Fix new issue formGriffin Smith1-1/+2
Passing a nil issue to this was breaking because you can't get the id of nil. I am too used to clojure. Change-Id: Icf76cbb23d902ec59fa97c21b134936fa40eb43e Reviewed-on: https://cl.tvl.fyi/c/depot/+/1593 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-08-03 r/1562 chore(tvl-slapd): refresh edef's passwordedef1-1/+1
Change-Id: I5a7a913656bfb9dd6c9fb4e2b4a1212607c50dd3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1592 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-08-03 r/1561 fix(nix/fetchGoModule): correctly case-fold mixed-case package pathsedef1-2/+14
Change-Id: I5a93462a23bf3c9f22fc3de0b173763a9bc8d526 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1591 Tested-by: BuildkiteCI Reviewed-by: edef <edef@edef.eu>
2020-08-03 r/1560 chore(3p/loxy): Move //users/edef/loxy to //third_partyedef2-0/+1
Change-Id: Ia451d190d4e415ef875c51ebeca12c7c5dc72342 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1590 Tested-by: BuildkiteCI Reviewed-by: lukegb <lukegb@tvl.fyi> Reviewed-by: tazjin <mail@tazj.in>
2020-08-03 r/1559 chore(nix/fetchGoModule): Move //users/edef/fetchGoModule to //nixedef3-1/+3
Change-Id: Ie7f52367ef91c0c8017a36c2a6c7edb06f10e57d Reviewed-on: https://cl.tvl.fyi/c/depot/+/1589 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-08-03 r/1558 fix(3p/cgit): transpose html_option parametersAlyssa Ross1-1/+1
I got these the wrong way round, fixed them in the patch in my Nix configuration, and then neglected to fix them in the version for depot. Fixes: be98295dc24492f6607cce04ba586e3897a83543 Change-Id: Ib7d4e433887f474ae9675e9948ab38f640181095 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1383 Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: kanepyork <rikingcoding@gmail.com> Tested-by: BuildkiteCI
2020-08-03 r/1557 refactor(3p/nix): Only initialise garbage-collector where neededVincent Ambo6-70/+72
Only libexpr depends on the garbage collector, specifically only instantiations of EvalState actually require the GC to be initialised. Rather than always starting it for the whole program, even if it is not needed, this change moves the GC initialisation into libexpr, guarded by absl::call_once. This should make it possible to run the nix daemon without the garbage collector interfering, granted that things are correcty separated and the daemon does not actually invoke the evaluator. Based on my investigation so far, the daemon logic itself does not require libexpr to be present at all - so I think it is safe - but the current monobinary might have some tricks up its sleeve that will cause problems for us. We can deal with those if they arise. Relates to https://b.tvl.fyi/issues/30 Change-Id: I61c745f96420c02e089bd3c362ac3ccb117d3073 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1584 Tested-by: BuildkiteCI Reviewed-by: edef <edef@edef.eu> Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-03 r/1556 feat(edef/loxy): initedef1-0/+31
loxy(8) is a logging IRC proxy. Change-Id: I78fd798ec32636cf38f877c364abc87ef9f14836 Co-authored-by: V <v@anomalous.eu> Reviewed-on: https://cl.tvl.fyi/c/depot/+/1587 Tested-by: BuildkiteCI Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-08-03 r/1555 feat(edef/fetchGoModule): initedef1-0/+24
Change-Id: I160a4264b9ade35d7cfcc1609ab395eab164d4d8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1586 Tested-by: BuildkiteCI Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-08-03 r/1554 feat(edef): take ownership of //users/edefedef1-0/+3
Change-Id: Ic3bd9bf4ef354688bdd06961a348eb127dab97bd Reviewed-on: https://cl.tvl.fyi/c/depot/+/1588 Tested-by: BuildkiteCI Reviewed-by: ericvolp12 <ericvolp12@gmail.com> Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-08-03 r/1553 feat(3p): Whitelist unzipedef1-0/+1
Change-Id: Iae7e661dc45f8bc8668c1aa4fe82b88e8868c22f Reviewed-on: https://cl.tvl.fyi/c/depot/+/1585 Tested-by: BuildkiteCI Reviewed-by: ericvolp12 <ericvolp12@gmail.com> Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-08-02 r/1552 style(cheddar): Apply rustfmtVincent Ambo1-10/+4
Change-Id: I378b5ae618b01244edc090ad1495fbd56857d7c0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1562 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-02 r/1551 style(paroxysm): Apply rustfmtVincent Ambo4-18/+18
Change-Id: I959b6a1008f1b0ef632c0194adc0f680e7c27d62 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1561 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-11-08T14·59+0000