about summary refs log tree commit diff
AgeCommit message (Collapse)AuthorFilesLines
2020-08-10 r/1633 fix(3p/nix): inherit Expr from gc, make parser state traceableKane York4-40/+56
The parser contained vectors, and the primary parser state, that were not participating in GC tracing. Change-Id: Ie198592cd7acffd390e3e2ae9595138b56416838 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1706 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-10 r/1632 style(tvix): Sort out minor formatting errorsVincent Ambo6-16/+16
Invocations of the MakeError macro that were not followed by a semicolon messed up indentation in the next lines. Change-Id: I03d7d1443f062a38af2c7da3da8928e0ed05e274 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1708 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-10 r/1631 fix(tvix): Print invalid store path when encounteredVincent Ambo1-3/+2
... instead of silently swallowing it. Change-Id: I084c51d633d632bd4e3f1ca42987dd6068fbb2a9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1707 Reviewed-by: Profpatsch <mail@profpatsch.de> Tested-by: BuildkiteCI
2020-08-10 r/1630 feat(users/multi): bootstrap home-manager configuration for whitbymulti15-0/+3677
Change-Id: Iad3024a5a640d33377cfae436134fda9f358397b Reviewed-on: https://cl.tvl.fyi/c/depot/+/1703 Tested-by: BuildkiteCI Reviewed-by: multi <depot@in-addr.xyz>
2020-08-09 r/1629 feat(whitby): add multimulti2-2/+17
Change-Id: Ibfc2a5fcf73099b8414b8c46958007374d14fd0a Reviewed-on: https://cl.tvl.fyi/c/depot/+/1701 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: isomer <isomer@tvl.fyi>
2020-08-09 r/1628 fix(web/panettone): revert "implement shorthand issue URLs"glittershark2-16/+6
This reverts commit 311511385455f680aedf78aab761fcebf3ca7731. Reason for revert: this is causing all issues to return a 404 - reverting until we can get it working. Change-Id: I5f3c5ec3b24f245a1f7ef12645200d16ed0f1b35 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1721 Tested-by: BuildkiteCI Reviewed-by: edef <edef@edef.eu>
2020-08-09 r/1627 test(tvix): Cover scanForReferences in a testGriffin Smith6-5/+98
Aded a few test cases covering the scanForReferences function, which had been accidentally broken in 976a36c (which is now partially-reverted). As part of this, since the test needed to generate hashes for store paths, the logic in MakeStorePath to compress a sha256 hash down to 20 bytes and convert it to base32 has been extracted to a member function on the Hash class. Fixes: #34 Change-Id: Ie2d914688a80f42d0234d351a7cc0714fd15709e Reviewed-on: https://cl.tvl.fyi/c/depot/+/1698 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-09 r/1626 feat(users/multi): init, add SSH keysmulti2-0/+8
Change-Id: I93d6733595c96e0d4e67281e4781e2e0ff424a63 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1700 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-09 r/1625 feat(tvl-slapd): add multimulti1-0/+5
Change-Id: I2fec9b5dd92da0343426c4a129d882fa87d92e6a Reviewed-on: https://cl.tvl.fyi/c/depot/+/1699 Reviewed-by: cynthia <cynthia@tvl.fyi> Reviewed-by: eta <eta@theta.eu.org> Reviewed-by: edef <edef@edef.eu> Reviewed-by: lukegb <lukegb@tvl.fyi> Tested-by: BuildkiteCI
2020-08-09 r/1624 fix(tvix): Make nix value structs inherit from GCGriffin Smith1-5/+5
All of the miscellanious structs that make up a nix Value should inherit from gc, as they contain pointers to GC'ed things as members Fixes: #42 Change-Id: I057d8c9f7dafbee7de7644ff152c9cba1aa6bc03 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1696 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-09 r/1623 refactor(tvix): Remove UDSRemoteStoreGriffin Smith2-76/+0
Now that we've fully implemented the RPC-based store client, we can get rid of the UDSRemoteStore, whose only use was connecting to the locally running nix daemon. The RemoteStore still needs to be around to connect to remote upstream nix stores over SSH. Change-Id: I0699819803cbfe966b9a46786f2c927d8e4bf1a2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1693 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-09 r/1622 feat(tvix): Implement all remaining RPC callsGriffin Smith8-49/+158
Implement all remaining RPC calls on the RpcSstore client, remove a few stub methods we had added that weren't actually present in the old RemoteStore implementation, and add one more RPC call for getBuildLog that is present in the store API, but that we hadn't added as a stub *or* to the proto. Fixes: #29 Change-Id: Id827f51a393ece4bc7bbecaf38aee9eb4b329770 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1692 Reviewed-by: kanepyork <rikingcoding@gmail.com> Tested-by: BuildkiteCI
2020-08-09 r/1621 refactor(tvix): Factor-out proto utilitiesGriffin Smith3-136/+162
Factor out the shared utilities for interacting with protobufs/grpc from libstore to a proto.hh header in libproto. Change-Id: I1cb8d94867d5d4b63a9994be0b53f8f612eb8e3a Reviewed-on: https://cl.tvl.fyi/c/depot/+/1691 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-09 r/1620 feat(tvix): Pass function name to SuccessOrThrowGriffin Smith2-18/+32
To aid in debugging RPC call failures, pass the name of the function being called to the SuccessOrThrow util funcion in the RpcStore Change-Id: I523dacfab896b85a3dbe6050c07ee6bd5906fa44 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1690 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-08 r/1619 chore(3p/nix): apply google-readability-castingKane York40-125/+161
Command run: jq <compile_commands.json -r 'map(.file)|.[]' | grep -v '/generated/' | parallel clang-tidy -p compile_commands.json -checks=-*,google-readability-casting --fix Manual fixes applied in src/nix-env/nix-env.cc, src/libstore/store-api.cc Change-Id: I406b4be9368c557ca59329bf6f7002704e955f8d Reviewed-on: https://cl.tvl.fyi/c/depot/+/1557 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi> Reviewed-by: tazjin <mail@tazj.in>
2020-08-08 r/1618 fix(tvix): Wrap remaining RPCs in HandleExceptionsGriffin Smith1-21/+29
Wrap the BuildPaths and AddTextToStore RPC handlers in HandleExceptions. These were missed in the original pass due to a merge. Change-Id: Ie5be45e6098fba7a2b6b1c1be81578cb742c2880 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1689 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-08 r/1617 feat(web/panettone): implement shorthand issue URLsedef2-6/+16
Fix #32 Change-Id: I6ccec959201673850b4b56a44734a2874aad5856 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1648 Reviewed-by: glittershark <grfn@gws.fyi> Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI
2020-08-08 r/1616 chore(3p/nix): don't optional-wrap EvalState::file_access_trace_fn furtheredef2-4/+3
std::function has a natural null we can't eliminate anyway, so this was effectively std::optional<std::optional<non_nullable_function>>. Change-Id: If99f139146021edb25d133dad7f0f6e125ef53df Reviewed-on: https://cl.tvl.fyi/c/depot/+/1688 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com> Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-08 r/1615 feat(tvix): Implement AddToStoreNarGriffin Smith4-33/+129
Implement both the client and server sides of AddToStoreNar, using a templated generalization of the sources and sinks we were using for AddToStore on both ends. Change-Id: I73d0ed34118c711b125851dff99a7518ced4af35 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1686 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-08 r/1614 docs(sparse): Add RFC design for depot worktree toolKane York1-0/+37
This is meant to be a high-velocity alternative to writing an entire FUSE filesystem to facilitate smaller checkouts of the depot. The script mostly works today, except that remote dependency detection is not set up. Change-Id: Idf9048df4f7ee39eaaedcdafcd9621f3d7479947 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1644 Tested-by: BuildkiteCI Reviewed-by: edef <edef@edef.eu> Reviewed-by: tazjin <mail@tazj.in>
2020-08-06 r/1613 fix(3p/nix): Use SkipEmpty in all calls to absl::StrSplitKane York28-55/+80
The behavior to return a list containing a single empty string when provided an empty string is a behavior that absl inherited from legacy code. However, the behavior expected by legacy code in Nix is the behavior provided by the SkipEmpty option. Switch all calls to use SkipEmpty, except for the call already using SkipWhitespace. See also commit 26a59482d2427f640893517f1b24dd650a5bd5da, with the partly-prophetic message: "there may be other places we need to fix this as well." Change-Id: I6e94856a12cfb1b7e4a3b4e221769ed446648861 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1687 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-06 r/1612 refactor(tvix): Use absl::btree_map for BasicDerivation's envVincent Ambo2-3/+4
Change-Id: I111a9a268debea322f23fdced3bed9ff3e8ed3b3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1685 Reviewed-by: glittershark <grfn@gws.fyi> Tested-by: BuildkiteCI
2020-08-06 r/1611 refactor(tvix): Use absl::btree_map for DerivationInputsVincent Ambo2-4/+2
Change-Id: If160ab1b09161969d9080d5d0d6f82e44a953e3c Reviewed-on: https://cl.tvl.fyi/c/depot/+/1684 Reviewed-by: glittershark <grfn@gws.fyi> Tested-by: BuildkiteCI
2020-08-06 r/1610 refactor(tvix): Use absl::btree_map for DerivationOutputsVincent Ambo4-7/+22
This container implementation is much faster than std::map. We have stuck to an ordered container because it's unclear whether the accesses of this field (of which there are *many*) are actually ordering dependent. Also includes an Arbitrary implementation for absl::btree_map (for any K, V that are also Arbitrary). Change-Id: I04f58ca0ce32b9ae1759313b01508b0e44bae793 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1683 Reviewed-by: glittershark <grfn@gws.fyi> Tested-by: BuildkiteCI
2020-08-06 r/1609 chore(tvix): Disable sandboxing by default until its time comesVincent Ambo1-15/+8
Change-Id: Ifcc776841924abcb43fb115f3de569fe500756c8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1682 Reviewed-by: glittershark <grfn@gws.fyi> Tested-by: BuildkiteCI
2020-08-06 r/1608 feat(tazjin/frog): Install sourcetrailVincent Ambo1-0/+1
Change-Id: I4858a1e374162ea1cf35aa906e0d723fe8c2c392 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1681 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2020-08-06 r/1607 docs: Add the RULEGriffin Smith1-0/+1
Change-Id: I90eddabb7e5e917524b300d5dfe86ddab4271432 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1660 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2020-08-06 r/1606 fix(whitby): Increase nrBuildUsers to 128Vincent Ambo1-0/+1
Change-Id: I3a444e163745d17d10f923c0be7565840937c53a Reviewed-on: https://cl.tvl.fyi/c/depot/+/1662 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-06 r/1605 fix(whitby): I'm a trusted user, owoVincent Ambo1-0/+1
Change-Id: I2666b3cf8bdefcb5d4caeddf191dc65f6a8cb05f Reviewed-on: https://cl.tvl.fyi/c/depot/+/1661 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-06 r/1604 chore(3p): Bump nixpkgs channelsVincent Ambo10-21/+91
Bumps both nixos-unstable and nixos-20.03 to today's versions, as per status.nixos.org Contains minor fixes to things that broke because of the update: * tazjin/frog: hardware.u2f is a deprecated setting * glittershark/system: modSha256 in Go modules is now vendorSha256 * glittershark/owothia: removed version constraint on relude Change-Id: Ib3e9612b1b06ed547b90e4f8b0ffe5ed7fe0a5c4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1642 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-05 r/1603 feat(3p/nix): add --trace-file-access to nix-instantiateKane York4-0/+49
This builds on edef's work with depot-scan by adding a dedicated flag to the command. We piggyback on upstream's restricted-mode implementation, the checkSourcePath function. Change-Id: I52bb613549f40dbca1e8caa036635910c1a3d6d0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1654 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-05 r/1602 chore(whitby): add edefedef1-0/+6
Change-Id: I7265259bc87594bd481c7bd455187c09b1effd1c Reviewed-on: https://cl.tvl.fyi/c/depot/+/1650 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: lukegb <lukegb@tvl.fyi>
2020-08-05 r/1601 fix(tools/depot-build): handle multiple levels of subdirectoriesedef1-1/+1
Change-Id: I99511c17d635b0222e1c933432007e6f8ea8cf85 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1655 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-08-05 r/1600 feat(3p/nix): Implement FindRoots, CollectGarbageGriffin Smith5-3/+117
Implement the RPC client and server handlers for the FindRoots and CollectGarbage RPC calls Change-Id: Ifa5d582c6a33bd1e7661ac2fc860505ef404dad0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1656 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-05 r/1599 fix(3p/nix): fix usage error of absl::Base64UnescapeKane York1-3/+4
The source and destination strings cannot be the same string - absl will write to the destination in a streaming manner, causing the source to become invalid. Change-Id: I3578cf1f8789a51d85e0950f7987c398f0a00953 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1659 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-05 r/1598 fix(3p/nix/store): fix race condition in downloaderKane York1-1/+2
In certain circumstances, the decompression thread could race ahead of the downloader thread and process the same chunk twice. Clear the data buffer while the lock is held to prevent this kind of incident. Change-Id: I19a84a0c5768d1228c6c18a7664a7b8893ef96de Reviewed-on: https://cl.tvl.fyi/c/depot/+/1658 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-05 r/1597 refactor(tvix): Use SANDBOX_SHELL from environment in CMakeVincent Ambo2-3/+4
This makes the configuration work out-of-the-box using CMake in a Nix shell, rather than having to pass the additional variable to CMake on the command line. Change-Id: I04d3cc4f2e5ecf47bf2ee459d5e48588b84ae4dd Reviewed-on: https://cl.tvl.fyi/c/depot/+/1643 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com> Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-05 r/1596 feat(3p/nix): add tests for BinaryCacheStoreKane York2-0/+106
These tests are in preparation for factoring the Store away from libutil's Callback to absl::StatusOr. They use the newly added MockBinaryCacheStore. Updates: #25 Change-Id: I30c207589ec38254806ebc9a983f35668e353ae9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1595 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-05 r/1595 feat(3p/nix): add MockBinaryCacheStoreKane York3-1/+170
This adds an implementation of BinaryCacheStore to be used by tests exercising both the logic inherent to BinaryCacheStore and more general Store tests. A new library target, nixstoremock, is created to indicate that this file is intended only for use in tests and not in user-facing code. Change-Id: Ib68f777238843a4f3a2303db8a69735fbc22d161 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1645 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-05 r/1594 feat(3p/nix): Implement AddToStore rpc clientGriffin Smith4-2/+69
Implement the AddToStore RPC client method, which uses an AddToStorePathWriterSink (the dual of the AddToStorePathReaderSource on the server side) to hook into the dumpPath machinery (which we should refactor not to use sinks or sources, but not yet) and write dumped paths as binary data over gRPC. With this commit and sandboxing disabled, the following derivation builds cleanly: derivation { name = "test"; builder = ./build.sh; system = "x86_64-linux"; } where build.sh has chmod +x and contains: #!/bin/sh echo 1 > $out Change-Id: I94cab86f0825a3a9993262a9807130645c13bf44 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1638 Reviewed-by: kanepyork <rikingcoding@gmail.com> Tested-by: BuildkiteCI
2020-08-05 r/1593 fix(3p/nix): fix clang-tidy for int-string appendsKane York1-3/+6
Change-Id: I276de7a5fd1c705c87d35dd616e5980c747190aa Reviewed-on: https://cl.tvl.fyi/c/depot/+/1597 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-05 r/1592 fix(edef/keys): add readTree headeredef1-0/+2
Change-Id: I5e1e0f58377201b591730a49f44678a33243b1d5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1653 Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI
2020-08-05 r/1591 chore(edef/users/depot-scan): move to a subdiredef3-3/+1
Change-Id: If202b466aa0874048e89e0dc270ff14883510131 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1652 Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI
2020-08-05 r/1590 fix(users/edef/depot-scan): properly trace readFile/readDiredef3-2/+19
Change-Id: I493e7d20504d34f83a53db50b69e86668874756e Reviewed-on: https://cl.tvl.fyi/c/depot/+/1651 Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI
2020-08-05 r/1589 fix(users/edef/depot-scan): properly depend on pl scriptKane York2-2/+2
Change-Id: I83c75ab75dae9c904e127ad6160c7641d4523e40 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1647 Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI
2020-08-05 r/1588 feat(edef): add SSH CAedef1-0/+5
Change-Id: If6839735d2730fa49363fa46e6fa059dd19bfaf7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1649 Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI
2020-08-05 r/1587 feat(edef/depot-scan): initedef2-0/+23
A small script that parses `nix-instantiate -vv` output to track the files read while instantiating a depot build. Change-Id: I5acf31d55f39c1d1acf9cdead03d33e2c8abab6a Reviewed-on: https://cl.tvl.fyi/c/depot/+/1646 Reviewed-by: edef <edef@edef.eu> Tested-by: BuildkiteCI
2020-08-04 r/1586 feat(nix/readTree): allow arguments to depend on tree locationedef1-3/+3
This accepts a function for args, and passes it an attrset containing the locatedAt parameter. Change-Id: I3c0f8ca00605e02a787fda88f32b06f5ef9998f2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1639 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-08-04 r/1585 chore(nix/fetchGoModule): don't move downloaded fileedef1-3/+1
unzip(1) doesn't care about the extension, unlike the unpackFile hook. Change-Id: Ia9d0e17c4b343ee5d32350329eb6267b9dc35605 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1641 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com> Reviewed-by: tazjin <mail@tazj.in>
2020-08-04 r/1584 chore(nix/fetchGoModule): remove overrideAttrs hackedef1-5/+3
This was based off fetchzip, but unlike fetchzip we don't need to support arbitrary archive formats, so we don't actually need the unpack hooks. Change-Id: I5655c031f420422f13e72bb79f4b4b95f6f1204e Reviewed-on: https://cl.tvl.fyi/c/depot/+/1640 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com> Reviewed-by: tazjin <mail@tazj.in>
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-11-09T12·30+0000