Age | Commit message (Collapse) | Author | Files | Lines |
|
We already have these hashes accessible in the Cargo.nix file created
by cargo2nix, so there's no need to also manually maintain them here.
It removes one potential footgun I ran into while updating wu-manber to
a different rev, without updating it here.
Change-Id: I93932ac8ba55f83746ee38571b7740af2d49bbdf
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9090
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
|
|
Change-Id: I46660da84065fd6938f581e14d67e231dca3c3ea
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9112
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
|
|
There was a NixHash::new() before, which didn't perform any validation
of the digest length. We had some length validation when parsing nix
hashes or SRI hashes, but some places didn't perform validation and/or
constructed the struct directly.
Replace NixHash::new() with a
`impl TryFrom<(HashAlgo, Vec<u8>)> for NixHash`, which does do this
validation, and update constructing code to use that, rather than
populating structs directly. In some rare cases where we're sure the
digest length is correct we still populate the struct manually.
Fixes b/291.
Change-Id: I7a323c5b18d94de0ec15e391b3e7586df42f4229
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9109
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
Change-Id: I1c1608a6e75e451940fe1c61dc5ace5f0e7d7752
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9111
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
|
|
This question popped up every once in a while. While already explained
quite well at
https://inbox.tvl.su/depot/20230316120039.j4fkp3puzrtbjcpi@tp/T/#t,
it's not easily accessible.
Lift it from there into tvix/docs for better visibility.
Change-Id: I5f2d4aff31ab4adc421e06a7d36c871f45e09100
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9080
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
|
|
Change-Id: I0b2fdb418c2e36280d5c551a81634e1742193903
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9105
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
|
|
HashMap already is on the heap.
Change-Id: I53763e17469359e85862f297b5c2e7c0d8c3a980
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9104
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
|
|
… instead of new(). Suggested by clippy.
Change-Id: Iac7be733392afefc2b4ff2e38386eee95f3bce94
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9103
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
|
|
Change-Id: I11d7d55f31ddd27952f201a0abbed7445013f434
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9102
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
|
|
This passes a unit value to the function.
Change-Id: I4df3ad8fb0f35c0f110cee3349971ae28ce2878c
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9101
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
|
|
Change-Id: Ifec2a4fc93c482e41ff5993183643b5126a7728b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9100
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
|
|
Change-Id: I9d931ffcc03c6df7c0392dbc1c9a4ae0e3804213
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9099
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
|
|
Change-Id: I6b1b902ccbc12bf2acdb0fdf406d6ef336ff0b2f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9098
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
|
|
Change-Id: Iaeb9ed7024c2ce85373f8aec0d223f52e1a3a539
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9097
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
|
|
Change-Id: I3a80560d036e7ed08036b5e9f0974080d1a30ded
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9096
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
|
|
Change-Id: Ib48063660432bea9de2fc9aff5914ef76db3ad64
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9095
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
Change-Id: I723bc9b29733f5ee4e3094b92696fbdeb4b15a33
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9094
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
Change-Id: I3c86225040a04c9cb5702b226b4a5008ae219c42
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9093
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
|
|
We already have the parsed output_hash from above, no need to construct
it again.
Change-Id: Ie6d924ab446137c25c29fbeaf671aa7e5418262d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9110
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
|
|
Yes:
```
$ nix-build -E 'derivation { name = ""; builder = "/bin/sh"; system = "x86_64-linux"; }'
error: store path 'nr7i5pf18hw2zg487vkdyrbasdqylfcj-' has an empty name
```
Change-Id: I552f9ed1c1fe3bfceca18ca9b8e13d4b06dc6ff7
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9108
Reviewed-by: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
Previously, Output deserialization would silence validation errors and
provide `None` for `hash_with_mode` as soon as a validation error would
happen inside of the `NixHashWithMode` deserialization, e.g. invalid
hash length would not provide an validation error but a silent `None`
value.
This is problematic, we workaround a serde limitation here by writing
our own Deserializer.
As you can see, we write some boilerplate code unfortunately, as, for
example:
- `#[serde(fail_if_unparsed_as="Option::is_none")]` is not a thing,
otherwise, we could have been able to just bubble up errors in case of
"not fully parsed" (and not missing) values.
- `From<&serde_json::Value> for serde::de::Unexpected` is not a thing,
otherwise, we could just map invalid type errors and reuse the
existing types instead of doing extremely bizarre things with
`serde::de::Unexpected::Other`, note: this is a not problem for
expected, we know what we expect, we don't know what we received in
practice.
I decided to write a `NixHashWithMode::from_map` which will eat a map
deserialized via `serde_json`, so our serde magic is not totally "data
model" agnostic.
I wanted to go for data model agnosticity and enable maximal
performance, e.g. building the structure as the map values are streamed
in the Visitor, this is needlessly painful because `Output` and
`NixHashWithMode` are in different files and this really makes sense
only if we write the full implementation in one file, indeed, otherwise,
you end up duplicating the work or having strange coupling.
So, for now, we will allocate a full map of the fields inside the
`Output`, i.e. if any "unknown field" is in that map, we will
deserialize it for no reason.
Doing it properly, as I repeat it in the code and to flokli at C3Camp
2023, requires to patch serde upstream IMHO.
Change-Id: I46fe6ccb8c390c48d6934fd3e3f02a0dfe59557b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9107
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
|
|
We don't need rust-analyzer to run cargo doc.
Change-Id: I5e2fd559e4045cadeab24b438c28d6df7f1d5d5f
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9092
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
|
|
cl/8306 fixed building all docs, but we forgot to update the comment.
Change-Id: I17829612f13e7357bd0efe8223cc28ed0f6cdea2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9091
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
|
|
This `.into_iter()` call is equivalent to `.iter()` and will not consume
the `BTreeMap`.
Change-Id: Ie26637ebecb0bea5b09c447cc45ed207f8b50913
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9088
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
|
|
Change-Id: Ie95ac265f8707e138c6a7b529760650d211f259d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9087
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
|
|
Change-Id: I1e52b752408a70fdb27e3e5c6f3bef5417b3e922
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9085
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
|
|
Don't import thiserror::Error directly, so rustdoc knows what
`crate::store_path::Error` we're talking about.
Change-Id: I755c9377521a6833e9a77cb1a41b48edafb31fe0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9086
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
|
|
We only need bstr::ByteSlice to be able to use replace, it doesn't need
to return a BString.
Change-Id: I811948436fb89652e880970c2c05356183f3e439
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9084
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
|
|
Change-Id: Icb91b102208fea512e04383ce9d65b0681af18ab
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9079
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
|
|
Change-Id: Ic58653254b36694f1991a18db9ceea0d532c2e31
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9068
Autosubmit: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
Reviewed-by: flokli <flokli@flokli.de>
|
|
builtins.div ought to truncate towards zero so that
-(builtins.div a b) == builtins.div (-a) b
-(builtins.div a b) == builtins.div a (-b)
Change-Id: I8b7c08cd7f4fa8a1363c786d42c8d484f6cd133d
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9006
Autosubmit: sterni <sternenseemann@systemli.org>
Reviewed-by: tazjin <tazjin@tvl.su>
Tested-by: BuildkiteCI
|
|
This allows pinning the name of the sparse tree derivation, which
stops the continous rebuilding of tvix-store-proto dependents.
I've opted to let the function take an attribute set instead and
refactored the call sites appropriately.
Change-Id: I3e57785094b1adbfffa24caf9f1c3384844fa200
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8965
Reviewed-by: grfn <grfn@gws.fyi>
Tested-by: BuildkiteCI
Reviewed-by: sterni <sternenseemann@systemli.org>
|
|
Change-Id: I764232378f79f16c781870d467469db91a2f14d2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9001
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
|
|
Change-Id: I43cc1d5b3419255f815490b2e10f05db19e3bffc
Reviewed-on: https://cl.tvl.fyi/c/depot/+/9000
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
|
|
Make it more obvious if these are bytes pointing to JSON or ATerm, so we
don't get confused.
Change-Id: I2402c687b7ba9c05aac20ed63b0df54e4e96a9d8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8998
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
|
|
Change-Id: I85e44e24e9ec3fe4284f02623a1ac7ca9935e554
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8999
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
|
|
This is more concise than a io::copy of a Cursor to bytes, and we have
everything to be written in memory.
Change-Id: I81f34666aa61aef4e16b33423ce4a69c3781efc3
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8997
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
|
|
PR'ed at https://github.com/tvlfyi/wu-manber/pull/1, and now merged.
Change-Id: I8c71e359196396a1d42a3ea2ab7ac15b137b2db0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8992
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
|
|
write_input_derivations shouldn't need to write a comma to separate it
from the previous output from write_outputs.
This is better placed in the function calling all of these helper
functions.
Change-Id: I9ccc440e4665b52369ef39e75151b9a29469ce48
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8995
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
We're happy with any &[S], as long as <S: AsRef<[u8]>.
This allows passing both strings and &[u8].
Change-Id: If2a80d9b1ee33ba328c9cdab4fa83ca7b98a71e2
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8994
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
|
|
Let the escape function only take care of string escaping, not quoting.
Let write_array_elements always quote and escape strings it consumes.
Move the business of writing additional wrapping characters around it to
the caller.
Change-Id: Ib8dea69c409561b49862c531ba5a3fe6c2f061f8
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8993
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
Derivations can have non-unicode strings in their env values, so the
ATerm representations are not necessarily String anymore, but Vec<u8>.
Change-Id: Ic23839471eb7f68d9c3c30667c878830946b6607
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8990
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
|
|
This allows sorting Store Paths. We delegate the sorting business to the
PartialOrd, Ord impls for our digest fields only, as two StorePaths with
the same digest, but different names can't exist.
Change-Id: I5f81631e5f5063893b316c63a240c5266b7e5bad
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8988
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
|
|
The comment still mentions Aho-Corasick, which isn't correct.
Change-Id: I846a2ca9ea7075c2456ca6ef04a132ff6161227a
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8991
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
|
|
HashAlgo implements Copy, no need to clone here.
Change-Id: Ief11d2cfbd4fd0cd44224c7ddd575f518edbbd55
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8989
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
|
|
Makes use of https://github.com/tokio-rs/prost/pull/341, which makes our
bytes field cheaper to clone.
It's a bit annoying to configure due to
https://github.com/hyperium/tonic/issues/908, but the workaround does
get the job done.
Change-Id: I25714600b041bb5432d3adf5859b151e72b12778
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8975
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Reviewed-by: tazjin <tazjin@tvl.su>
Autosubmit: flokli <flokli@flokli.de>
|
|
Add the position in the string where the name is problematic.
Change-Id: If6fd8be6100b718f8d68568eafc77ebb3cfb82d0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8979
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
|
|
This will save us some copies, because a clone will simply create an
additional pointer to the same data.
Change-Id: I017a5d6b4c85a861b5541ebad2858ad4fbf8e8fa
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8978
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Autosubmit: flokli <flokli@flokli.de>
Tested-by: BuildkiteCI
|
|
Some paths might use names that are not valid UTF-8. We should be able
to represent them.
We don't actually need to touch the PathInfo structures, as they need to
represent StorePaths, which come with their own harder restrictions,
which can't encode non-UTF8 data.
While this doesn't change any of the wire format of the gRPC messages,
it does however change the interface of tvix_eval::EvalIO - its
read_dir() method does now return a list of Vec<u8>, rather than
SmolStr. Maybe this should be OsString instead?
Change-Id: I821016d9a58ec441ee081b0b9f01c9240723af0b
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8974
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
Tested-by: BuildkiteCI
|
|
This "reverts" commit 9f600de22671ee1f88e6fb9e53a5a385b434871b (the
initial revert of f5e291cf8328096d790f5416cf1968cb9164220a).
Now with BlobService returning a BlobReader that implements io::Seek, we
can actually just call blob_reader.seek(io::SeekFrom::Start(offset as
u64)).
This means, we currently will fail to seek backwards inside a file.
Change-Id: I9c19448df6831a3537252f99210374f2126ecfc0
Reviewed-on: https://cl.tvl.fyi/c/depot/+/8886
Tested-by: BuildkiteCI
Autosubmit: flokli <flokli@flokli.de>
Reviewed-by: raitobezarius <tvl@lahfa.xyz>
|