Age | Commit message (Collapse) | Author | Files | Lines |
|
Nix builds also have some other inputs, outside of inputs_dir, used in
derivations setting `passAsFile` and `__structuredAttrs`.
This extends our `BuildRequest` structure to accomodate these usecases.
Change-Id: I8de6516eb467fa48a4961d88933ebd548f131049
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10519
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
|
|
And include a test to ensure we show the warning.
Change-Id: Ib6a436dbba2592b398b54e44f15a48d1aa345099
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10470
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
|
|
In the past, we had `emit_warning` be no-op and we used `push_warnings` exclusively
but as we have consumers of this function, we need it to work somewhat.
Change-Id: I78a5ece199a473dec9ef5ea1fae60b36e35137b8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10477
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
|
|
Nix is quite tolerant when it comes to parsing SRI hashes and their
padding (and only for SRI hashes, it funnily is strict about that in the
non-SRI-hash case).
Nix essentially accepts any number of padding characters, no matter if
it's too much or too little. So we do the only sane thing - simply strip
all padding characters, and parse it with BASE64_NOPAD and the length
the algo uses.
Change-Id: I6a721aa289b06cc36741589792b9dd4c4f930b86
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10468
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
|
|
This is behind the otlp feature flag (currently enabled by default).
By default, this will try to push traces to a OTLP collector running at
https://localhost:4317.
You can make one available by running:
```
docker run -d --name jaeger \
-e COLLECTOR_ZIPKIN_HOST_PORT=:9411 \
-e COLLECTOR_OTLP_ENABLED=true \
-p 6831:6831/udp \
-p 6832:6832/udp \
-p 5778:5778 \
-p 16686:16686 \
-p 4317:4317 \
-p 4318:4318 \
-p 14250:14250 \
-p 14268:14268 \
-p 14269:14269 \
-p 9411:9411 --rm \
jaegertracing/all-in-one:1.42
```
Started like that, jaeger brings a web interface at
http://localhost:16686/search
As documented in
https://docs.rs/opentelemetry-otlp/latest/opentelemetry_otlp/, you can
point this to another location by setting `OTEL_EXPORTER_OTLP_ENDPOINT`.
Change-Id: Id1dca367d70027b2ea98bb70bcf99a68363ec2be
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8194
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: aaqaishtyaq <aaqaishtyaq@gmail.com>
|
|
Only convert to and reuse an Arc<…> where needed.
Change-Id: I2c1bc69cca5a4a3ebd3bdb33d6e28e1f5fb86cb9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10514
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
|
|
Change-Id: If9a536949f36f428abea1a893f937fe7063e2f41
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10517
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
|
|
Change-Id: I9655f5588c7dc98427de6af47d74b4ab7ce22071
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10516
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
|
|
We can also drop the Clone requirement. Because the trait is async since
some time, there's no need to clone before moving into an async closure,
allowing us to simplify the code a bit.
Change-Id: I9b0a0e10077d8c548d218207b908bfd92c5b8de0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10515
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
|
|
This works on Box<dyn DirectoryService> too.
Change-Id: Ib869f0f4d963ef4dbaeab22db03ff6afb71ede04
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10513
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
|
|
This is doing the exact same thing.
Change-Id: Iadc5e13dd192efc91cc3d36b2bdf4b8b99a312b2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10512
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
|
|
While at it, make it a bit more generic.
Change-Id: Ic4caefda93aca3ffb656a09f8b4d648b41415532
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10511
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
|
|
We don't actually care if it's an Arc<dyn BlobService>, or something
else, as long as we can Deref to a BlobService and clone.
Change-Id: I0852aaf723f51c5e6b820be8db1199d17309ab08
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10510
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
Change-Id: Ia74ee870f38b7966501458bace541092256c3213
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10509
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
We already have that data in output_path.
Change-Id: Iadf615551f0b44d539098446ff5926994deb606d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10508
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
|
|
Instead, return an error, and move the entire check before starting to
ingest the data underneath.
Change-Id: Idcfba115cb7d599f5fc72a156aaad9d4d4714fcf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10507
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
|
|
We can use cli.json.then_some(…) to create a Some(…), allowing us to
omit the else { None } lines.
Change-Id: I6c8142a08d8cb88d6c8302e5ca7570698fcf2aa3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10505
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
|
|
This takes three URLs, and constructs Arc'ed
{Blob,Directory,PathInfo}Service, allowing to remove some of the
boilerplate.
Change-Id: I40e7c2b551442ef2acdc543dfc87ab97e7c742bb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10484
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
|
|
While we currently mostly use it in an Arc, as we need to clone it
inside PathInfoService, there might be other usecases not requiring it
to be Clone.
Change-Id: Ia05bb370340792a048e2036be30e285ef1e63870
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10483
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
|
|
While we currently mostly use it in an Arc, as we need to clone it
inside PathInfoService, there might be other usecases not requiring it
to be Clone.
Change-Id: I7bd337cd2e4c2d4154b385461eefa62c9b78345d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10482
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
|
|
At some point, tvix-cli needs to talk to the outside world to persist
things into a real store.
Introduce the same CLI options to configure
{Blob,Directory,PathInfo}Service URLs.
We need to be a bit careful with how we set up stores, and make
this separate from setting up TvixStoreIO, as it's holding a
Rc<RefCell<KnonPath>> which not Send.
At some point, we might make this a Arc<RwLock<_>> later anyways, and
then this can be simplified a bit, but for now, this is sufficient.
Change-Id: I87d84ca3a10ce947e194ff985073791469773f35
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10474
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
|
|
We're also happy to consume strings, or other owned stringy types.
Change-Id: I5bead4407976134815d8f879f9f70468e6af1dc4
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10476
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
|
|
Instead, it's passed in the evaluate/compile_only functions, which feels
more naturally. It lets us set up the Evaluation struct long before
we actually feed it with data to evaluate.
Now that Evaluation::new() would be accepting an empty list of
arguments, we can simply implement Default, making things a bit more
idiomatic.
Change-Id: I4369658634909a0c504fdffa18242a130daa0239
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10475
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
|
|
There's no need to require builds to also put their outputs in store_dir.
So rename store_dir to inputs_dir and make outputs consume paths, not
basenames.
Also document the possibility for builds to write to inputs_dir. Let's
see if we want to make this a constraint later.
Change-Id: Ib4df20afcdde2d771c269c422f04c7e95587cd0f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10447
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
|
|
By default, we don't want contextful strings and we almost always want contextless strings.
To this end, we make taking a contextful string a very explicit operation under `to_contextful_str`
and we implement manually the `to_str` cast which requires a `if !s.has_context()` guard that
the macro cannot cover.
Change-Id: I7aae8e57a7d73e547e62b1edb0b1cc7e8c0c69b6
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10425
Autosubmit: raitobezarius <tvl@lahfa.xyz>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
|
|
`hasContext` is now functional.
Change-Id: I23b128afc9150b833bc0d9b042d31fee35badadb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10422
Tested-by: BuildkiteCI
Autosubmit: raitobezarius <tvl@lahfa.xyz>
Reviewed-by: tazjin <tazjin@tvl.su>
|
|
This prepares the data structures to implement string contexts
in Nix.
Change-Id: Idd913c9c881daeb8d446907f4b940e462e730978
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10420
Autosubmit: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
|
|
Explain why we had to take the bullet on context strings now
and move the historical approach in a historical section.
Change-Id: Ie3bcc2213b391c6ba06547cc05c850891a41d06b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10446
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
|
|
Change-Id: Id14e39543239272aed041998fd9a78465c9cb8b2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10359
Autosubmit: Adam Joseph <adam@westernsemico.com>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
|
|
Change-Id: I4ada8cf10611e98519cb1b1da11bfd06815f2932
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10358
Autosubmit: Adam Joseph <adam@westernsemico.com>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
|
|
Change-Id: I13d7ce0c7328a8e6fbc6d2c4ff5c4fe6095b96ea
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10357
Autosubmit: Adam Joseph <adam@westernsemico.com>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
|
|
Change-Id: I165ff77764e272cc94d18cb03ad6cbc9a8ebefde
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10348
Autosubmit: Adam Joseph <adam@westernsemico.com>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
|
|
Change-Id: I14c9e625c91369e10d0c00380dca992811ae9059
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10346
Reviewed-by: sterni <sternenseemann@systemli.org>
Autosubmit: Adam Joseph <adam@westernsemico.com>
Tested-by: BuildkiteCI
|
|
Change-Id: I92d58ef216d7e0766af70f019b3dcd445284a95d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10344
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
|
|
I'm still trying to work out the exact stack invariants for tvix.
We really should add assertions for them; getting the stack messed
up is no fun. This commit adds one simple assertion. It also adds
a missing stack-push (my mistake) in one place, which was uncovered
by the assertion.
Change-Id: I9d8b4bd1702d954e325832c5935b0d7e3eb68422
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10369
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
|
|
We only do things with the reference, so we don't need to locally borrow
it.
Change-Id: I6073f7ec7aff717ae3069e28a00b1cb408a50ceb
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10455
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
|
|
This is a bit more useful than evans when sending over the same message
over and over again.
Change-Id: I600b6b9f591c0c963c5d270005aa1cc84d2a0770
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10453
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: edef <edef@edef.eu>
Tested-by: BuildkiteCI
|
|
Make it more clear this is not tokio::io.
Change-Id: Ic2fa56f0baf1c200b6631098d556388a19629a45
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10452
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
|
|
Change-Id: I696b7ab6b4c08004db147c0fda7312bbebaa0eec
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10451
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
|
|
Change-Id: If4748cb800276fc42779ba8973d1eef72df0503e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10450
Reviewed-by: edef <edef@edef.eu>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
- directory in which the castore input nodes are mounted
- working directory for the build command
- scratch paths
- network access y/n
- whether a (static) /bin/sh should be provided
Populate these fields appropriately, and extend the tests in tvix-glue
with a FOD example.
Change-Id: I4f9de1483d6696d74694a09784910c407acb0be0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10412
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
|
|
r/7176 introduced an incorrect assumption was the benefit of the
nonrecursive coercion algorithm, namely that a coercion operation always
returns a non empty string. This allows to detect whether we are
coercing a list or not by checking if the intermediate result is empty
or not. Unfortunately, coercing null and false yields an empty string,
so we need to explicitly track whether we are coercing a list.
Updated the test case to hopefully catch similar bugs in the future. I'm
not a hundred percent certain I have not introduced a new edge case with
this, so it may be interesting to add a prop test case for this to
nix_oracle down the line. At least lists are the only nested data
structures that can be serialized as nested data structures, so the
problem is kind of limited.
Change-Id: Ia41e904356f1c41a9d35e4e65ec02f2fe5a4100e
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10418
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
|
|
The bugs have been fixed,
https://github.com/rust-lang/futures-rs/pull/2801 and
https://github.com/rust-lang/futures-rs/pull/2812 were merged and ended
up in that release.
Change-Id: I301c0ffc951f04a5b3b7267e922771c837a3f5a9
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10415
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
|
|
The bugs have been fixed,
https://github.com/rust-lang/futures-rs/pull/2801 and
https://github.com/rust-lang/futures-rs/pull/2812 were merged and ended
up in that release.
Change-Id: Iefd990d2d1719b884504093343e54e9c5258e2e2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10414
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
There's no need to pass in an unused directory service into the
populate_blob_* method, and considering we have one or two invocation of
each of these, we don't really gain much from having all these functions
follow the same structure, at least for now.
Also, update some function names to better describe what they're doing.
Change-Id: I92f680745c157fb0a602b07342f8838bfad23ecd
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10411
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
|
|
cl/10378 did already move store/fs to castore/fs, but we kept the tests
in tvix-store, as they were populating a PathInfoService to make nodes
appear in the mount root.
Update these tests to now just insert root nodes into a BTreeMap, and
ensure we can use that as a RootNodes too.
Change-Id: Iad7d1ee4f9423eb6e3a1da33f433842c9ae0de1f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10410
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
|
|
Change-Id: If4456c565abf32385b1239467b96e7398190ea4b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10383
Reviewed-by: flokli <flokli@flokli.de>
Reviewed-by: sterni <sternenseemann@systemli.org>
Tested-by: BuildkiteCI
|
|
With the recent introduction of the RootNodes trait, there's nothing in
the fs module pulling in tvix-store dependencies, so it can live in
tvix-castore.
This allows other crates to make use of TvixStoreFS, without having to
pull in tvix-store.
For example, a tvix-build using a fuse mountpoint at /nix/store doesn't
need a PathInfoService to hold the root nodes that should be present,
but just a list.
tvix-store now has a pathinfoservice/fs module, which contains the
necessary glue logic to implement the RootNodes trait for a
PathInfoService.
To satisfy Rust orphan rules for trait implementations, we had to add a
small wrapper struct. It's mostly hidden away by the make_fs helper
function returning a TvixStoreFs.
It can't be entirely private, as its still leaking into the concrete
type of TvixStoreFS.
tvix-store still has `fuse` and `virtiofs` features, but they now simply
enable these features in the `tvix-castore` crate they depend on.
The tests for the fuse functionality stay in tvix-store for now, as
they populate the root nodes through a PathInfoService.
Once above mentioned "list of root nodes" implementation exists, we
might want to shuffle this around one more time.
Fixes b/341.
Change-Id: I989f664827a5a361b23b34368d242d10c157c756
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10378
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
|
|
This makes PathInfoService::from_addr return a Box<dyn PathInfoService>,
rather than an Arc<dyn …>, and leaves it up to the consumers to rewrap
it into an Arc where needed.
This allows us to drop the Arc for the tvix-store daemon subcommand.
Change-Id: Ic83aa2ade6c51912281bd17c7eef7252e152b2d1
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10409
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
|
|
Change-Id: I7a2b951d9c9251a053a0de40f31836bda03a922d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/10408
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: sterni <sternenseemann@systemli.org>
|