| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
If the global nix config, or any available user nix config location, does not exist, then the loadConfFile() function will throw and not finish initalizing the Nix configuration.
Change-Id: I6db83bca54d9b66699356d107720603476e32c23
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1657
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
|
|
Make Store::buildPaths return a Status with [[nodiscard]] rather than
throwing exceptions to signal failure. This is the beginning of a long
road to refactor the entire store API to be status/statusor based
instead of using exceptions.
Change-Id: I2e32371c95a25b87ad129987c217d49c6d6e0c85
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1745
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
Invocations of the MakeError macro that were not followed by a
semicolon messed up indentation in the next lines.
Change-Id: I03d7d1443f062a38af2c7da3da8928e0ed05e274
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1708
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
Aded a few test cases covering the scanForReferences function, which had
been accidentally broken in 976a36c (which is now partially-reverted).
As part of this, since the test needed to generate hashes for store
paths, the logic in MakeStorePath to compress a sha256 hash down to 20
bytes and convert it to base32 has been extracted to a member function
on the Hash class.
Fixes: #34
Change-Id: Ie2d914688a80f42d0234d351a7cc0714fd15709e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1698
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
Now that we've fully implemented the RPC-based store client, we can get
rid of the UDSRemoteStore, whose only use was connecting to the locally
running nix daemon. The RemoteStore still needs to be around to connect
to remote upstream nix stores over SSH.
Change-Id: I0699819803cbfe966b9a46786f2c927d8e4bf1a2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1693
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
Implement all remaining RPC calls on the RpcSstore client, remove a few
stub methods we had added that weren't actually present in the old
RemoteStore implementation, and add one more RPC call for getBuildLog
that is present in the store API, but that we hadn't added as a
stub *or* to the proto.
Fixes: #29
Change-Id: Id827f51a393ece4bc7bbecaf38aee9eb4b329770
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1692
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Tested-by: BuildkiteCI
|
|
Factor out the shared utilities for interacting with protobufs/grpc from
libstore to a proto.hh header in libproto.
Change-Id: I1cb8d94867d5d4b63a9994be0b53f8f612eb8e3a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1691
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
To aid in debugging RPC call failures, pass the name of the function
being called to the SuccessOrThrow util funcion in the RpcStore
Change-Id: I523dacfab896b85a3dbe6050c07ee6bd5906fa44
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1690
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
Command run: jq <compile_commands.json -r 'map(.file)|.[]' | grep -v '/generated/' | parallel clang-tidy -p compile_commands.json -checks=-*,google-readability-casting --fix
Manual fixes applied in src/nix-env/nix-env.cc, src/libstore/store-api.cc
Change-Id: I406b4be9368c557ca59329bf6f7002704e955f8d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1557
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: tazjin <mail@tazj.in>
|
|
Implement both the client and server sides of AddToStoreNar, using a
templated generalization of the sources and sinks we were using for
AddToStore on both ends.
Change-Id: I73d0ed34118c711b125851dff99a7518ced4af35
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1686
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
The behavior to return a list containing a single empty string when
provided an empty string is a behavior that absl inherited from legacy
code. However, the behavior expected by legacy code in Nix is the
behavior provided by the SkipEmpty option. Switch all calls to use
SkipEmpty, except for the call already using SkipWhitespace.
See also commit 26a59482d2427f640893517f1b24dd650a5bd5da, with the
partly-prophetic message: "there may be other places we need to
fix this as well."
Change-Id: I6e94856a12cfb1b7e4a3b4e221769ed446648861
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1687
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
|
|
Change-Id: I111a9a268debea322f23fdced3bed9ff3e8ed3b3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1685
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
|
|
Change-Id: If160ab1b09161969d9080d5d0d6f82e44a953e3c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1684
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
|
|
This container implementation is much faster than std::map. We have
stuck to an ordered container because it's unclear whether the
accesses of this field (of which there are *many*) are actually
ordering dependent.
Also includes an Arbitrary implementation for absl::btree_map (for any
K, V that are also Arbitrary).
Change-Id: I04f58ca0ce32b9ae1759313b01508b0e44bae793
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1683
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
|
|
Change-Id: Ifcc776841924abcb43fb115f3de569fe500756c8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1682
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
|
|
Implement the RPC client and server handlers for the FindRoots and
CollectGarbage RPC calls
Change-Id: Ifa5d582c6a33bd1e7661ac2fc860505ef404dad0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1656
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
The source and destination strings cannot be the same string - absl
will write to the destination in a streaming manner, causing the
source to become invalid.
Change-Id: I3578cf1f8789a51d85e0950f7987c398f0a00953
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1659
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
|
|
In certain circumstances, the decompression thread could race ahead of the downloader thread and process the same chunk twice. Clear the data buffer while the lock is held to prevent this kind of incident.
Change-Id: I19a84a0c5768d1228c6c18a7664a7b8893ef96de
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1658
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
|
|
This adds an implementation of BinaryCacheStore to be used by tests exercising both the logic inherent to BinaryCacheStore and more general Store tests.
A new library target, nixstoremock, is created to indicate that this file is intended only for use in tests and not in user-facing code.
Change-Id: Ib68f777238843a4f3a2303db8a69735fbc22d161
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1645
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
|
|
Implement the AddToStore RPC client method, which uses an
AddToStorePathWriterSink (the dual of the AddToStorePathReaderSource on
the server side) to hook into the dumpPath machinery (which we should
refactor not to use sinks or sources, but not yet) and write dumped
paths as binary data over gRPC. With this commit and sandboxing
disabled, the following derivation builds cleanly:
derivation {
name = "test";
builder = ./build.sh;
system = "x86_64-linux";
}
where build.sh has chmod +x and contains:
#!/bin/sh
echo 1 > $out
Change-Id: I94cab86f0825a3a9993262a9807130645c13bf44
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1638
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Tested-by: BuildkiteCI
|
|
Change-Id: I276de7a5fd1c705c87d35dd616e5980c747190aa
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1597
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
|
|
Implement RPC handlers and client methods for the AddWithRoot and
SyncWithGC calls. With this and sandboxing disabled, the following
derivation is now building successfully:
derivation {
name = "test";
builder = "/bin/sh";
args = [ "-c" "echo 1 > $out" ];
system = "x86_64-linux";
}
Co-authored-by: Vincent Ambo <mail@tazj.in>
Change-Id: I245dca0dfca882e2eae5d93364287a8b79fead71
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1637
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Tested-by: BuildkiteCI
|
|
Without this, these errors print as "uncaught exception of type
std::string", but with this they print using nix's built-in error
printer, which will be nice to have as we bang out the remaining store
API methods.
Co-authored-by: Vincent Ambo <mail@tazj.in>
Change-Id: I9bd4f77f65b27c8f97b727b98ef62e2061999e71
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1636
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
|
|
This reverts part of commit 976a36c2e482f416acd79a624e6d96cce2564b5b,
which was causing scanForReferences to hang indefinitely. I'm not sure
what the original intent of the commit was since there's not really much
of a commit message on it, but with this I am able to fully realise
derivations.
Change-Id: I620e892e2fe2244f066512286b086b3dd8f8f6c0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1635
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
When tokenizeString was changed to absl::StrSplit, there was a behavior
change because tokenizeString on an empty string returned an empty
vector - which the derivation builder (and likely a bunch of other
stuff) was depending on. The canonical way of fixing this is by passing
absl::SkipEmpty() to the function - there may be other places we need to
fix this as well.
This commit also includes some opportunistic absl::StrFormats and
StrCats, because I was here anyway, but those have no semantic
difference.
Change-Id: Ibf9bb602284f793fa55728481f63b838fb7a41db
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1631
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
|
|
We wrap every server-side proto handler with a macro that catches
exceptions and turns them into proper grpc error codes. For the
time being, most exceptions map to INTERNAL, the existing mapping.
Change-Id: Id6ed6a279b198ad185d32562f39000ccc15eadbf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1599
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
|
|
At some point the behavior of this function got changed as part of our
cleanup - this fixes it to behave the way the rest of the codebase
expects (and how it is documented in the header) and covers it with a
few tests.
Change-Id: Id4c91232968e73489cd866fb4a2a84bcf20d875e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1629
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
|
|
Implement AddTextToStore and BuildPaths both on the client and the
server
Refs: #29
Change-Id: I45294c3e1c1a7489e42099d36425b7acc04e0427
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1560
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Tested-by: BuildkiteCI
|
|
Add a rapidcheck test covering roundtrip parse and serialize for Nix
derivations. This covers a bug we discovered in ef54f5d which broke this
roundtrip.
Change-Id: I72d140334b5f24f79e82e34f98609c695dbfbf93
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1582
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
It turns out these aren't supposed to be reused after the first request
- reusing it was causing things to blow up.
Change-Id: I935f1ee82ed4e7f26528ae105c8fa2501fa81b97
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1559
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Tested-by: BuildkiteCI
|
|
Prepend the unix:// URI scheme to the daemon socket so that grpc knows
we want to connect to a unix socket rather than another type of URI. As
part of debugging this I made the failure message for the RPCStore
include the URI, which I'm leaving in since it'll be nice to have.
Change-Id: I6e70596895117b9a0d53fe2a61d8542ceb64c940
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1543
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Tested-by: BuildkiteCI
|
|
This reverts commit ef54f5da9fa30b5c302f2a49595ee5d041f9706a.
Resolved conflicts:
third_party/nix/src/libexpr/eval.cc
third_party/nix/src/libstore/builtins/fetchurl.cc
third_party/nix/src/libstore/references.cc
third_party/nix/src/libutil/hash.cc
third_party/nix/src/nix-daemon/nix-daemon.cc
Change-Id: Ib9cf6e96a79a23bde3983579ced3f92e530cb011
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1547
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
|
|
Implement the RPC client calls for QueryPathFromHashPart,
QuerySubstitutablePaths, and QuerySubstitutablePathInfos, and the
handler for QuerySubstitutablePathInfos.
Refs: #29
Change-Id: Idf383b771f159f267d8f65367bc4af3d239e32b7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1515
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
Implement the proto handler for AddToStore, which adds a nix path to the
store. This is implemented by adding a new (probably
soon-to-be-generalized) Source concretion that wraps a grpc ServerReader
for the stream of data we're receiving from the client - this is less
than ideal, as it's perpetuating the source/sink thing that's going on
and storing entire nars in memory, but is at the very worst an
incremental step towards a functioning nix that we can refactor in the
future.
Paired-With: Perry Lorier <isomer@tvl.fyi>
Paired-With: Vincent Ambo <mail@tazj.in>
Change-Id: I48db734e7460a47aee4a85dd5137b690980859e3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1441
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Reviewed-by: tazjin <mail@tazj.in>
|
|
The use of `unwrap_throw` can be used as a later grep target.
Change-Id: I8c54ed90c4289f07aecb8a1393dd10204c8bce4e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1493
Reviewed-by: glittershark <grfn@gws.fyi>
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
|
|
Change-Id: I3b34e3e17a751e225831ae599c6c6bb782a25679
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1486
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
|
|
Change-Id: I265e763393422ee1881653527c91024458060825
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1432
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
|
|
This fixes a clang-tidy DeadStore warning by turning debug asserts into production checks.
Updates: #11
Change-Id: Ia6e5a4cb1b56594c9844c6bbd3d152f84b426d09
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1409
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: glittershark <grfn@gws.fyi>
|
|
Add a stub class for wrapping a gRPC client to the new, proto-backed nix
store protocol, along with several methods implemented but several left
throwing a not implemented exception.
Paired-With: Vincent Ambo <mail@tazj.in>
Paired-With: Perry Lorier <isomer@tvl.fyi>
Change-Id: Id943d4f6d75084b8498786d580e6c9f7c92c104d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1436
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
Implement the main function for the new, proto-based nix daemon. This
mostly replicates the behavior of the previous nix daemon - it starts a
grpc server listening on the same nix socket path and supports the same
set of options - with the exception of --stdio, which has been renamed
to --pipe and documented in the man page.
Change-Id: Ib729283c1d5d35c0e1b0a968bc1f052f5527f2d7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1356
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
Replace the custom, rather questionable base64 implementation with
absl::Base64{Une,E}scape. To make sure that the custom implementation
was doing the same thing I've also added a test covering
nix::Hash::to_string, which was one function that used it - the test
passed prior to the replacement, and continued to pass afterwards.
The previous base64Decode function threw an exception on failure - to
avoid going too far down the rabbit hole I've replicated that
functionality at all call sites, but this should be replaced with more
sensible error handling such as StatusOr eventually.
Also, before this change:
❯ nix eval -f . users.tazjin.emacs.outPath
"/nix/store/g6ri2q8nra96ix20bcsc734r1yyaylb1-tazjins-emacs"
And after:
❯ ./result/bin/nix eval -f . users.tazjin.emacs.outPath
"/nix/store/g6ri2q8nra96ix20bcsc734r1yyaylb1-tazjins-emacs"
Change-Id: Id292ffbb82fe808f3f1b34670afbe7b8c13ad615
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1385
Reviewed-by: tazjin <mail@tazj.in>
Tested-by: BuildkiteCI
|
|
The use of vfork() in Nix is entirely illegal. Quote:
If the process created by vfork() returns from the function in which vfork() was
called, or calls any other function before successfully calling _exit() or
one of the exec*() family of functions, the behavior is undefined.
-- Linux man-pages, release 5.05
Add a TODO to use the higher-performance variants of clone() on Linux when it
is available.
Change-Id: I42370e1568ad6e2d00d70d0b66c8aded8f1288bb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1418
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: Alyssa Ross <hi@alyssa.is>
|
|
Change-Id: I1e8a4852ffa450c552a4cdfc709554799854934c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1416
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
|
|
nixutil depends on bzip2, lzma, boost::context, brotli{enc,dec}, and
openssl, but wasn't directly linking to them. This was causing linker
errors in a test that only depended on nixutil.
Change-Id: I60e77ea7b18b08e2946fcf9176ae0f355cd71844
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1384
Tested-by: BuildkiteCI
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
Change-Id: I86125609f433469a8722c780fd758234211d677e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1381
Tested-by: BuildkiteCI
Reviewed-by: Alyssa Ross <hi@alyssa.is>
Reviewed-by: glittershark <grfn@gws.fyi>
|
|
The SQL schemas are included as string constants which are
concatenated into a header file. In the previous Makefiles, this was
done with envsubst or something - we moved it to CMake.
There was a missing quote around the string to be interpolated, which
meant that CMake interpreted the semicolons as part of its language
syntax and did not emit them.
Change-Id: Ibb4512788b26b53f297db3535094dc0194614446
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1342
Reviewed-by: kanepyork <rikingcoding@gmail.com>
Reviewed-by: glittershark <grfn@gws.fyi>
Tested-by: BuildkiteCI
|
|
Plugins seem to not really be used anywhere (I can find one plugin
that's actually defined, and it doesn't seem very useful, especially
since we got rid of builtins.exec) and their presence is adding
additional complexity and potential sources of bugs to an already
unsteady refactor. At some point we may want to bring back
something *like* plugins, but their design will likely be different and
it will definitely be after we have a functioning Nix again.
Change-Id: I3bc40e55917f70bf260fbc208c1705e2e6a7c626
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1291
Tested-by: BuildkiteCI
Reviewed-by: Alyssa Ross <hi@alyssa.is>
Reviewed-by: isomer <isomer@tvl.fyi>
|
|
This compiles under `-Wall -Werror`.
The largest chunk of this change is `final` qualifiers for the various
Nix CLI command structs, which inherit from a Command class that has
more virtual functions than are implemented by each command.
Change-Id: I0925e6e1a39013f026773db5816e4a77d50f3b4a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1294
Tested-by: BuildkiteCI
Reviewed-by: isomer <isomer@tvl.fyi>
Reviewed-by: kanepyork <rikingcoding@gmail.com>
|
|
Removes the verbosity enum, which is no longer actively used anywhere
other than a daemon protocol implementation bit that doesn't actually
work.
Since the verbosity was marked deprecated, this removes one of the
last remaining warnings.
Change-Id: Iaee9d1d6c14b30daac83bb44bcacff32a0e07fb0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1289
Tested-by: BuildkiteCI
Reviewed-by: glittershark <grfn@gws.fyi>
|
|
Implement the proto handler on the server side for
Worker::BuildDerivation. This includes several additions to the proto
which I had missed on the first pass, including the actual proto
definition for the Derivation itself and a few sequence number
reorderings which are fine because this is all provisional and not
deployed yet.
A couple things to note
- I implemented a couple constructors for nix classes that initialize
themselves based on their proto variants, which felt nice and didn't
end up causing any issues.
- I've made the conversions between the enum types in nix and in proto
explicit via switch statements rather than using a static_cast, out of
an abundance of caution that the error would get mismatched in the
future and we'd convert the wrong thing to the wrong thing - this is
verbose, but exceptionally future proof.
Change-Id: Iecf6b88e76bc37e49efa05fd65d6cd0cb0deffed
Reviewed-on: https://cl.tvl.fyi/c/depot/+/1249
Tested-by: BuildkiteCI
Reviewed-by: tazjin <mail@tazj.in>
Reviewed-by: Kane York <rikingcoding@gmail.com>
|
