about summary refs log tree commit diff
path: root/third_party/nix/src/libstore
AgeCommit message (Collapse)AuthorFilesLines
2020-11-22 r/1909 fix(tvix): Add endlines to build log messagesGriffin Smith1-16/+19
Change-Id: Id6e871a2b7fbb24fb5c8e84e1638bfe94e363b57 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2135 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-11-22 r/1908 feat(tvix): Convert some DLOGs to VLOGsGriffin Smith2-4/+4
It seems like the amount of logging is causing things to move a little slower - even if that's not really the case, it gets in the way of debugging things. Refs: b/76 Change-Id: I9ea99a3b16e3307a0b0371bad22d03b0e2175af6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2134 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-11-21 r/1894 chore(3p): Bump nixpkgs to nixos-unstable from 2020-11-21Vincent Ambo5-29/+26
Included fixes for random breakage: * 3p/awscli: pick from the stable channel; it is broken on unstable * 3p/googletest: bumped version & removed patches that nixpkgs applies * 3p/lisp/cffi: bumped library version for SBCL compat * 3p/nix: fix libsystemd attribute * 3p/nix: reformatted (clang-format handling of ternaries changed) * glittershark/home: Use home-manager from nixkpgs * glittershark/kernel: bumped linux-ck patch hash * glittershark/kernel: removed "patch patch" * multi/whitby: Use home-manager from nixpkgs * tazjin/frog: drop Sourcetrail (it doesn't build currently) Note that in addition to these changes, some previous CLs updated the versions of git and cgit which was necessary for this channel bump, but which could not be done in the same commit due to the nature of the subtree merges. Change-Id: If2563e8a68e2750c4b913a976ff7b93b42e8b7f3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2110 Tested-by: BuildkiteCI Reviewed-by: multi <depot@in-addr.xyz> Reviewed-by: glittershark <grfn@gws.fyi>
2020-11-19 r/1886 fix(tvix): Use copy constructor to add strings to protosGriffin Smith1-1/+1
Passing a string directly to add_paths like this causes the proto class to take ownership over the string, meaning when it is destructed it will *explicitly* free the string. When the string's actual owner (the derivation struct) then goes out of scope it'll get freed again, causing a double-free. This fixes that to instead use the copy constructor to assign to a pointer to a new path, and covers the whole to_proto method with a rapidcheck test. Fixes: b/64 Change-Id: I84235bed9104ff430a0acf686d4a96f1e2e9a897 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2106 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
2020-11-19 r/1885 fix(tvix): Use mutable field for assigning proto fieldsGriffin Smith1-2/+2
This was accidentally using the proto arena API to assign the derivation field of a BuildDerivationRequest. We *thought* this was causing a double free, but even with this change that's still happening. That said, this change is probably still a good idea since it's using the proto API as intended. References: b/64 Change-Id: I950a4eafb214e9113639ea54d2dfd4659b7be931 Reviewed-on: https://cl.tvl.fyi/c/depot/+/2104 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-09-16 r/1795 fix(tvix): Convert INVALID_ARGUMENT to InvalidPathGriffin Smith1-2/+6
The code that calls queryPathInfoUncached explicitly catches the InvalidPath exception and translates it into a null result - but the RPC code was throwing a regular old Error for invalid paths. At some point we should get rid of all the exception-driven control flow in this whole thing, but in the meantime this gets us back to functional. Change-Id: I2a38790ee0c691ab0c8394c7738d7693fa42aa10 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1980 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-09-14 r/1791 refactor(tvix): Make static strings constexpr string_viewsGriffin Smith6-32/+38
Make all static std::strings constexpr std::string_views, and replace concatenation with absl::StrCat where necessary. Technically all of these are constant, so they really don't need to be top-level statics - and since I'm trying to get rid of as much global state as possible in preparation for making the nix daemon properly multithreaded I figured I'd knock these out while I was at it. Change-Id: Ibd3ad9ef68f0a0eacb135541b39fdb13dae042e1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1939 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-09-10 r/1783 refactor(tvix): Convert refLength to constexprGriffin Smith1-9/+9
This is constant, so let's make it a constexpr. Change-Id: I6a9eb9f740979740f730ebe142d2cbf23dbcfd70 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1961 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-09-10 r/1782 fix(tvix): Make working flag thread_localGriffin Smith1-5/+4
Since the daemon is running in threads now rather than forking a process per connection (thanks to grpc) this static flag to prevent accidentally initializing a new worker during a build is getting stepped on by multiple threads. This converts it to thread-local, and also adds an actual message to the check so that if it hits in the future we know what's going on. Fixes: b/58 Change-Id: I07a2f1582e56709c104f79935e5405fa24888f59 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1940 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com> Reviewed-by: V <v@anomalous.eu>
2020-09-06 r/1768 revert(tvix): "Disable sandboxing by default until its time comes"Griffin Smith1-8/+15
This reverts commit a41c3dedb18201aa689206079e203f41c9bef389. Reason for revert: I believe its time has come. Change-Id: I4133dc8a3b57b40b280646d1622ed7a0409c5246 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1941 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-09-06 r/1767 feat(tvix): Write even more build logs to clientGriffin Smith1-89/+84
Write *all* build logs (essentially everything that wasn't a DLOG), not just the stdout of the builder, to the client-bound log_sink_ that's threaded through build.cc. These logs are significantly more useful on the client, both when debugging tvix itself and (more importantly) when building things with it. To make this all work, this commit also refactors the passing-around of the log sink in build.cc slightly - the main reference is now owned by the Worker class, which already stores pointers to itself on the Goals it creates, so those Goals can then load the log sink from the Worker. This change was originally made to allow for logging to the log_sink from instance methods of the worker, but actually feels significantly more natural than storing the sink on the goal anyway. Change-Id: I461685bc7f5c0a326054940ac5971b568dd43f8e Reviewed-on: https://cl.tvl.fyi/c/depot/+/1925 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in>
2020-09-01 r/1758 refactor(tvix): Remove default buildMode arguments from libstoreGriffin Smith1-4/+4
These were not used anyway, and *may* have been causing a weird series of problems leading to things sometimes aborting. Or maybe it has nothing to do with it - the only correlation we've found is from disassembling the output of a core dump. Regardless, this feels stylistically like a good idea. Change-Id: I288b50945e74ea90c118a21767fa4387c9f47f7d Reviewed-on: https://cl.tvl.fyi/c/depot/+/1921 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-09-01 r/1756 refactor(tvix): getEnv(): Return std::optionalEelco Dolstra6-19/+24
This allows distinguishing between an empty value and no value. Patch ported from upstream at https://github.com/NixOS/nix/commit/ba87b08f8529e4d9f8c58d8c625152058ceadb75 Change-Id: I061cc8e16b1a7a0341adfc3b0edca1c0c51d5c97 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1884 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-29 r/1736 fix(tvix): Chunk the AddTextToStore requestGriffin Smith1-6/+27
Rather than sending the entire AddTextToStore request along in a single message, send it in a stream of chunks using the same metadata-first approach we've been using for the other store gRPC requests. This fixes a bug where certain builds could send more data than the maximum gRPC request size (4194304 bytes, it would appear), resulting in a RESOURCE_EXHAUSTED error. The initial chunk size, which is currently constant but should be made dynamic at some point in the future, has been chosen based on the IPC bandwidth delay product for tazjin's desktop, rounded up. Change-Id: I6f0232cdbc98653484816b39855126873fc59a03 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1835 Tested-by: BuildkiteCI Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-23 r/1704 refactor(tvix/libutil): Mark single-argument constructors explicitVincent Ambo11-200/+221
This is the clang-tidy lint 'google-explicit-constructor'. There's a whole bunch of breakage that was introduced by this, and we had to opt out a few types of this (esp. the string formatting crap). In some cases minor other changes have been done to keep the code working, instead of converting between types (e.g. an explicit comparison operator implementation for nix::Pid). Change-Id: I12e1ca51a6bc2c882dba81a2526b9729d26988e7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1832 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com> Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-21 r/1703 style(tvix): Add missing braces in expressionsVincent Ambo2-17/+32
The previous clang-tidy invocation missed some header files, which has now been rectified. Change-Id: I31547754fbf52f439dc7aeefb08ab90bd50c4156 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1831 Reviewed-by: glittershark <grfn@gws.fyi> Tested-by: BuildkiteCI
2020-08-21 r/1702 style(tvix): Replace remaining typedefs with using statementsVincent Ambo4-9/+9
Change (mostly) created via: fd . -e hh -e cc | parallel clang-tidy -p ~/builds/nix --fix "'--checks=-*,modernize-use-using'" Change-Id: I90ad6e64a5399f2005dfb9956074ec51c2c78cea Reviewed-on: https://cl.tvl.fyi/c/depot/+/1830 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-21 r/1699 fix(tvix): Add newlines to build logsVincent Ambo1-2/+2
.. turns out producing an entire build log on a single line is not useful. This does not use `<< std::endl` because we have run into buffering issues with the implementation of the logs->gRPC sink, but intend to replace this in the future using a structured sink for BuildEvent protos rather than a raw stream. Change-Id: Ia9b05fa804391d389e2ef53ab4436c0ec5cc452e Reviewed-on: https://cl.tvl.fyi/c/depot/+/1828 Reviewed-by: glittershark <grfn@gws.fyi> Tested-by: BuildkiteCI
2020-08-21 r/1697 feat(tvix): Write build logs into the build log sinkVincent Ambo1-3/+4
This *should* wire up the builder's logs all the way back through the gRPC client, where they are then conveniently discarded. Change-Id: I65f22526d0b5a8b8d90f28665bc1b4bc7f7c802a Reviewed-on: https://cl.tvl.fyi/c/depot/+/1825 Reviewed-by: glittershark <grfn@gws.fyi> Tested-by: BuildkiteCI
2020-08-21 r/1696 chore(tvix): Thread std::ostream through builder goalsVincent Ambo1-42/+66
This passes an output stream for build logs to almost all relevant functions inside of build.cc by threading it through the `Goal`-abstraction. Store calls that create goals but don't have a sink available use the DiscardLogsSink(). Change-Id: I2c0cb1aec1f9150f33113f4752055cea518ede8b Reviewed-on: https://cl.tvl.fyi/c/depot/+/1824 Reviewed-by: glittershark <grfn@gws.fyi> Tested-by: BuildkiteCI
2020-08-21 r/1695 chore(tvix): Add TODO comment for std::ostream in buildDerivationVincent Ambo1-0/+1
Change-Id: I1e3b6abd4606d86f00ddaa8e2a1809f51e27126d Reviewed-on: https://cl.tvl.fyi/c/depot/+/1823 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-21 r/1694 style(tvix): Move a comment to fix a formatting issueVincent Ambo1-4/+3
Change-Id: Idd490b978c07d1b1113ee97b5d19719a27436fba Reviewed-on: https://cl.tvl.fyi/c/depot/+/1822 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-20 r/1689 chore(tvix): Thread a std::ostream through Store::buildPathsVincent Ambo8-30/+33
This part of the store API needs to carry a handle to the log sink from now on, so that it can be passed in as appropriate from the gRPC handlers. In all places where there is no such handler available at the moment, the discarding log sink has been inserted. This can be used as a convenient grep target in the future. Change-Id: I26628e30b4c6437dccdf8f722ca2e8ed827dfc19 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1797 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com> Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-20 r/1688 style(tvix): typedef -> using in build.cc/store-api.hhVincent Ambo2-18/+17
this shuts up a few lints ... Change-Id: I51ee9418767e97247f98b170cdad137e5b12b44d Reviewed-on: https://cl.tvl.fyi/c/depot/+/1796 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com> Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-20 r/1687 refactor(tvix): Remove some default values from virtual methodsVincent Ambo2-6/+18
This is not actually legal code, but it kind of ... works. There are more of these around, these were just the ones Griffin stumbled upon while working on the build logs. Change-Id: Iff9821d8fe145dd426648a8ff4510a73f67c9b7d Reviewed-on: https://cl.tvl.fyi/c/depot/+/1795 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com> Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-20 r/1685 feat(tvix): Introduce build event streams in worker protocolVincent Ambo3-10/+49
Introduces a new `BuildEvent` proto type which is streamed in response to calls that trigger builds of derivations. This type can currently supply build statuses, log lines and information about builds starting. This is in preparation for threading build logs through the processes. Since we have nowhere to send the logs (yet), a null sink is used instead. Co-authored-by: Griffin Smith <grfn@gws.fyi> Change-Id: If7332337b89506c7e404cd20174acdaa1a3be4e8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1793 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi> Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-20 r/1684 feat(tvix): Add a no-op stream buffer for discarding build logsVincent Ambo2-0/+19
In some cases we don't have anywhere for the build logs to go. Until we understand those cases fully and can get rid of them, this null sink implementation can be used. Change-Id: Ib93c43caf268e2c01c43d59737a829e8c43d223e Reviewed-on: https://cl.tvl.fyi/c/depot/+/1792 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com> Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-18 r/1674 fix(tvix/config): properly handle nonexistent config filesKane York1-2/+7
If the global nix config, or any available user nix config location, does not exist, then the loadConfFile() function will throw and not finish initalizing the Nix configuration. Change-Id: I6db83bca54d9b66699356d107720603476e32c23 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1657 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-14 r/1649 refactor(tvix): Make Store::buildPaths return a StatusGriffin Smith8-14/+34
Make Store::buildPaths return a Status with [[nodiscard]] rather than throwing exceptions to signal failure. This is the beginning of a long road to refactor the entire store API to be status/statusor based instead of using exceptions. Change-Id: I2e32371c95a25b87ad129987c217d49c6d6e0c85 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1745 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-10 r/1632 style(tvix): Sort out minor formatting errorsVincent Ambo1-2/+2
Invocations of the MakeError macro that were not followed by a semicolon messed up indentation in the next lines. Change-Id: I03d7d1443f062a38af2c7da3da8928e0ed05e274 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1708 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-09 r/1627 test(tvix): Cover scanForReferences in a testGriffin Smith1-3/+2
Aded a few test cases covering the scanForReferences function, which had been accidentally broken in 976a36c (which is now partially-reverted). As part of this, since the test needed to generate hashes for store paths, the logic in MakeStorePath to compress a sha256 hash down to 20 bytes and convert it to base32 has been extracted to a member function on the Hash class. Fixes: #34 Change-Id: Ie2d914688a80f42d0234d351a7cc0714fd15709e Reviewed-on: https://cl.tvl.fyi/c/depot/+/1698 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-09 r/1623 refactor(tvix): Remove UDSRemoteStoreGriffin Smith2-76/+0
Now that we've fully implemented the RPC-based store client, we can get rid of the UDSRemoteStore, whose only use was connecting to the locally running nix daemon. The RemoteStore still needs to be around to connect to remote upstream nix stores over SSH. Change-Id: I0699819803cbfe966b9a46786f2c927d8e4bf1a2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1693 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-09 r/1622 feat(tvix): Implement all remaining RPC callsGriffin Smith6-49/+138
Implement all remaining RPC calls on the RpcSstore client, remove a few stub methods we had added that weren't actually present in the old RemoteStore implementation, and add one more RPC call for getBuildLog that is present in the store API, but that we hadn't added as a stub *or* to the proto. Fixes: #29 Change-Id: Id827f51a393ece4bc7bbecaf38aee9eb4b329770 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1692 Reviewed-by: kanepyork <rikingcoding@gmail.com> Tested-by: BuildkiteCI
2020-08-09 r/1621 refactor(tvix): Factor-out proto utilitiesGriffin Smith1-136/+47
Factor out the shared utilities for interacting with protobufs/grpc from libstore to a proto.hh header in libproto. Change-Id: I1cb8d94867d5d4b63a9994be0b53f8f612eb8e3a Reviewed-on: https://cl.tvl.fyi/c/depot/+/1691 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-09 r/1620 feat(tvix): Pass function name to SuccessOrThrowGriffin Smith2-18/+32
To aid in debugging RPC call failures, pass the name of the function being called to the SuccessOrThrow util funcion in the RpcStore Change-Id: I523dacfab896b85a3dbe6050c07ee6bd5906fa44 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1690 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-08 r/1619 chore(3p/nix): apply google-readability-castingKane York15-57/+78
Command run: jq <compile_commands.json -r 'map(.file)|.[]' | grep -v '/generated/' | parallel clang-tidy -p compile_commands.json -checks=-*,google-readability-casting --fix Manual fixes applied in src/nix-env/nix-env.cc, src/libstore/store-api.cc Change-Id: I406b4be9368c557ca59329bf6f7002704e955f8d Reviewed-on: https://cl.tvl.fyi/c/depot/+/1557 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi> Reviewed-by: tazjin <mail@tazj.in>
2020-08-08 r/1615 feat(tvix): Implement AddToStoreNarGriffin Smith2-24/+42
Implement both the client and server sides of AddToStoreNar, using a templated generalization of the sources and sinks we were using for AddToStore on both ends. Change-Id: I73d0ed34118c711b125851dff99a7518ced4af35 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1686 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-06 r/1613 fix(3p/nix): Use SkipEmpty in all calls to absl::StrSplitKane York15-35/+48
The behavior to return a list containing a single empty string when provided an empty string is a behavior that absl inherited from legacy code. However, the behavior expected by legacy code in Nix is the behavior provided by the SkipEmpty option. Switch all calls to use SkipEmpty, except for the call already using SkipWhitespace. See also commit 26a59482d2427f640893517f1b24dd650a5bd5da, with the partly-prophetic message: "there may be other places we need to fix this as well." Change-Id: I6e94856a12cfb1b7e4a3b4e221769ed446648861 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1687 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-06 r/1612 refactor(tvix): Use absl::btree_map for BasicDerivation's envVincent Ambo2-3/+4
Change-Id: I111a9a268debea322f23fdced3bed9ff3e8ed3b3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1685 Reviewed-by: glittershark <grfn@gws.fyi> Tested-by: BuildkiteCI
2020-08-06 r/1611 refactor(tvix): Use absl::btree_map for DerivationInputsVincent Ambo2-4/+2
Change-Id: If160ab1b09161969d9080d5d0d6f82e44a953e3c Reviewed-on: https://cl.tvl.fyi/c/depot/+/1684 Reviewed-by: glittershark <grfn@gws.fyi> Tested-by: BuildkiteCI
2020-08-06 r/1610 refactor(tvix): Use absl::btree_map for DerivationOutputsVincent Ambo2-7/+10
This container implementation is much faster than std::map. We have stuck to an ordered container because it's unclear whether the accesses of this field (of which there are *many*) are actually ordering dependent. Also includes an Arbitrary implementation for absl::btree_map (for any K, V that are also Arbitrary). Change-Id: I04f58ca0ce32b9ae1759313b01508b0e44bae793 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1683 Reviewed-by: glittershark <grfn@gws.fyi> Tested-by: BuildkiteCI
2020-08-06 r/1609 chore(tvix): Disable sandboxing by default until its time comesVincent Ambo1-15/+8
Change-Id: Ifcc776841924abcb43fb115f3de569fe500756c8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1682 Reviewed-by: glittershark <grfn@gws.fyi> Tested-by: BuildkiteCI
2020-08-05 r/1600 feat(3p/nix): Implement FindRoots, CollectGarbageGriffin Smith3-2/+61
Implement the RPC client and server handlers for the FindRoots and CollectGarbage RPC calls Change-Id: Ifa5d582c6a33bd1e7661ac2fc860505ef404dad0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1656 Tested-by: BuildkiteCI Reviewed-by: kanepyork <rikingcoding@gmail.com>
2020-08-05 r/1599 fix(3p/nix): fix usage error of absl::Base64UnescapeKane York1-3/+4
The source and destination strings cannot be the same string - absl will write to the destination in a streaming manner, causing the source to become invalid. Change-Id: I3578cf1f8789a51d85e0950f7987c398f0a00953 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1659 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-05 r/1598 fix(3p/nix/store): fix race condition in downloaderKane York1-1/+2
In certain circumstances, the decompression thread could race ahead of the downloader thread and process the same chunk twice. Clear the data buffer while the lock is held to prevent this kind of incident. Change-Id: I19a84a0c5768d1228c6c18a7664a7b8893ef96de Reviewed-on: https://cl.tvl.fyi/c/depot/+/1658 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-05 r/1595 feat(3p/nix): add MockBinaryCacheStoreKane York3-1/+170
This adds an implementation of BinaryCacheStore to be used by tests exercising both the logic inherent to BinaryCacheStore and more general Store tests. A new library target, nixstoremock, is created to indicate that this file is intended only for use in tests and not in user-facing code. Change-Id: Ib68f777238843a4f3a2303db8a69735fbc22d161 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1645 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-05 r/1594 feat(3p/nix): Implement AddToStore rpc clientGriffin Smith1-1/+51
Implement the AddToStore RPC client method, which uses an AddToStorePathWriterSink (the dual of the AddToStorePathReaderSource on the server side) to hook into the dumpPath machinery (which we should refactor not to use sinks or sources, but not yet) and write dumped paths as binary data over gRPC. With this commit and sandboxing disabled, the following derivation builds cleanly: derivation { name = "test"; builder = ./build.sh; system = "x86_64-linux"; } where build.sh has chmod +x and contains: #!/bin/sh echo 1 > $out Change-Id: I94cab86f0825a3a9993262a9807130645c13bf44 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1638 Reviewed-by: kanepyork <rikingcoding@gmail.com> Tested-by: BuildkiteCI
2020-08-05 r/1593 fix(3p/nix): fix clang-tidy for int-string appendsKane York1-3/+6
Change-Id: I276de7a5fd1c705c87d35dd616e5980c747190aa Reviewed-on: https://cl.tvl.fyi/c/depot/+/1597 Tested-by: BuildkiteCI Reviewed-by: glittershark <grfn@gws.fyi>
2020-08-04 r/1583 feat(tvix): Implement AddIndirectRoot, SyncWithGCGriffin Smith1-2/+7
Implement RPC handlers and client methods for the AddWithRoot and SyncWithGC calls. With this and sandboxing disabled, the following derivation is now building successfully: derivation { name = "test"; builder = "/bin/sh"; args = [ "-c" "echo 1 > $out" ]; system = "x86_64-linux"; } Co-authored-by: Vincent Ambo <mail@tazj.in> Change-Id: I245dca0dfca882e2eae5d93364287a8b79fead71 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1637 Reviewed-by: tazjin <mail@tazj.in> Reviewed-by: kanepyork <rikingcoding@gmail.com> Tested-by: BuildkiteCI
2020-08-04 r/1582 chore(tvix): Make not implemented errors nix::UnsupportedGriffin Smith1-22/+24
Without this, these errors print as "uncaught exception of type std::string", but with this they print using nix's built-in error printer, which will be nice to have as we bang out the remaining store API methods. Co-authored-by: Vincent Ambo <mail@tazj.in> Change-Id: I9bd4f77f65b27c8f97b727b98ef62e2061999e71 Reviewed-on: https://cl.tvl.fyi/c/depot/+/1636 Reviewed-by: tazjin <mail@tazj.in> Tested-by: BuildkiteCI
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-12-20T18·44+0000