about summary refs log tree commit diff
path: root/tvix/serde
AgeCommit message (Collapse)AuthorFilesLines
2024-11-04 r/8890 refactor(tvix/eval): remove Value::Json and related functionalityBob van der Linden1-1/+0
Currently Value::Json is used in combination with VMRequest::ToJson to recursively convert tvix Value to serde_json::Value. This functionality is used in builtins.toJSON as well as derivation __structuredAttrs. Both Value::Json and VMRequest::ToJson were removed in this commit. Related functionality in vm.rs is also removed: vm.rs does not know about JSON anymore. Recursively converting to serde_json now happens without going through the VM. Thrown errors that are part of the value of toJSON are now directly propagated as ErrorKind, were-as previously there was a split between CatchableErrorKind and ErrorKind, where eventually CatchableErrorKind would be converted to ErrorKind::Catchable. Change-Id: I066f064926c491e4c087a984f07af43d19124cfe Reviewed-on: https://cl.tvl.fyi/c/depot/+/12732 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-10-12 r/8796 refactor(tvix/eval): Make `strict` an EvalMode enumAspen Smith1-2/+2
Refactor the `strict` boolean passed into evaluation at the top-level to be a (two-variant, so far) EvalMode enum of Lazy and Strict. This is more explicit than a boolean, and if we ever add more EvalModes it's a simple extension of the enum. Change-Id: I3de50e74ec971011664f6cd0999d08b792118410 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12186 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: aspen <root@gws.fyi>
2024-08-29 r/8610 chore(tvix): Migrate members to inherit deps from workspaceIlan Joselevich1-2/+2
From now on we will add the dependencies and their version in the root Cargo.toml and in order to enable the dependency for a workspace member we set `workspace = true` in the member's Cargo.toml. Change-Id: I9738c1cf99810b7ace87ca712c3ea965ba846e25 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12389 Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-08-29 r/8609 chore(tvix): Bump versions of all compatible deps via cargo upgradeIlan Joselevich1-1/+1
This updates all the dependencies and their "minimum" versions in Cargo.{lock,toml} to the latest compatible version using `cargo-edit`'s `cargo upgrade` command that will eventually be merged into `cargo update`. Change-Id: Iccb2aa4a1c84a0465222244a0bd0cafe2a82e781 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12388 Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com> Tested-by: BuildkiteCI
2024-07-18 r/8368 refactor(tvix): use composition & registry for from_addrYureka1-4/+3
Change-Id: I3c94ecb5958294b5973c6fcdf5ee9c0d37fa54ad Reviewed-on: https://cl.tvl.fyi/c/depot/+/11976 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: yuka <yuka@yuka.dev>
2024-07-06 r/8351 refactor(tvix/eval): Builderize EvaluationAspen Smith3-12/+10
Make constructing of a new Evaluation use the builder pattern rather than setting public mutable fields. This is currently a pure refactor (no functionality has changed) but has a few advantages: - We've encapsulated the internals of the fields in Evaluation, meaning we can change them without too much breakage of clients - We have type safety that prevents us from ever changing the fields of an Evaluation after it's built (which matters more in a world where we reuse Evaluations). More importantly, this paves the road for doing different things with the construction of an Evaluation - notably, sharing certain things like the GlobalsMap across subsequent evaluations in eg the REPL. Fixes: b/262 Change-Id: I4a27116faac14cdd144fc7c992d14ae095a1aca4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11956 Tested-by: BuildkiteCI Autosubmit: aspen <root@gws.fyi> Reviewed-by: flokli <flokli@flokli.de>
2024-04-13 r/7899 feat(tvix/eval): contextful JSON operationsRyan Lahfa1-1/+1
`toJSON` transform a Nix structure into a JSON string. For each context in that Nix structure, the JSON string must possess it. Thus, it is necessary to take the union of all contexts and attach it to the final structure. Unfortunately, the return type of `into_json` is a serde's JSON object, not a string. Therefore, it is not possible to reuse `NixString` machinery. Context tests are reinforced as Nix does not test those behaviors. Fixes b/393. Change-Id: I5afdbc4e18dd70469192c1aa657d1049ba330149 Signed-off-by: Ryan Lahfa <tvl@lahfa.xyz> Reviewed-on: https://cl.tvl.fyi/c/depot/+/11266 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-02-20 r/7575 feat(tvix/serde): add an example using nixpkgs/libVincent Ambo1-0/+34
This displays how users can configure an impure evaluation for tvix-serde, which makes it possible to use e.g. `nixpkgs/lib`. We might want to add an example showing how the full Nix-glue compatibility stuff can be added here, too. Change-Id: I2224a3fc66e739969d4c723c3d9d8127a046b6fd Reviewed-on: https://cl.tvl.fyi/c/depot/+/10994 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: tazjin <tazjin@tvl.su>
2024-02-20 r/7572 refactor(tvix/eval): use internal SourceCode field in error printersVincent Ambo2-8/+3
Makes use of the SourceCode field now being stored directly in errors (see parent CL). With this change, the default `Display` implementation can now format errors correctly, and there is no need to keep a `SourceCode` around just for error formatting. Updates dependent crates (CLI, serde, tvixbolt) to use this correctly. Change-Id: Iddc5d7a6b4bab391f30a999e4c68aca34304c059 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10987 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-02-02 r/7467 refactor(tvix/eval): Box Value::StringAspen Smith2-2/+2
NixString is *quite* large - like 80 bytes - because of the extra capacity value for BString and because of the context. We want to keep Value small since we're passing it around a lot, so let's box the NixString inside Value::String to save on some memory, and make cloning ostensibly a little cheaper Change-Id: I343c8b4e7f61dc3dcbbaba4382efb3b3e5bbabb2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10729 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2024-01-31 r/7460 fix(tvix): Represent strings as byte arraysAspen Smith3-9/+25
C++ nix uses C-style zero-terminated char pointers to represent strings internally - however, up to this point, tvix has used Rust `String` and `str` for string values. Since those are required to be valid utf-8, we haven't been able to properly represent all the string values that Nix supports. To fix that, this change converts the internal representation of the NixString struct from `Box<str>` to `BString`, from the `bstr` crate - this is a wrapper around a `Vec<u8>` with extra functions for treating that byte vector as a "morally string-like" value, which is basically exactly what we need. Since this changes a pretty fundamental assumption about a pretty core type, there are a *lot* of changes in a lot of places to make this work, but I've tried to keep the general philosophy and intent of most of the code in most places intact. Most notably, there's nothing that's been done to make the derivation stuff in //tvix/glue work with non-utf8 strings everywhere, instead opting to just convert to String/str when passing things into that - there *might* be something to be done there, but I don't know what the rules should be and I don't want to figure them out in this change. To deal with OS-native paths in a way that also works in WASM for tvixbolt, this also adds a dependency on the "os_str_bytes" crate. Fixes: b/189 Fixes: b/337 Change-Id: I5e6eb29c62f47dd91af954f5e12bfc3d186f5526 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10200 Reviewed-by: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de> Reviewed-by: sterni <sternenseemann@systemli.org> Autosubmit: aspen <root@gws.fyi> Tested-by: BuildkiteCI
2024-01-18 r/7409 refactor(tvix/eval): move Evaluation::{default,new_pure}() againFlorian Klink1-1/+1
Have a Evaluation::new() function that's used to set up the Evaluation struct initially - which is also used by both new_pure and new_impure internally. It's generic over the exact type of IO, making it easier to instantiate Evaluation with non-tvix-eval EvalIO implementations, that might not be in a Box. Change-Id: Ibf728da24aca59639c5b6df58d00ae98c99a63f5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10640 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2024-01-18 r/7407 refactor(tvix/eval): generalize EvalIO containerFlorian Klink1-2/+2
Don't restrict to a Box<dyn EvalIO>. There's still one or two places where we do restrict, this will be solved by b/262. Change-Id: Ic8d927d6ea81fa12d90b1e4352f35ffaafbd1adf Reviewed-on: https://cl.tvl.fyi/c/depot/+/10639 Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2023-12-31 r/7289 refactor(tvix/eval): remove code and location from structFlorian Klink1-2/+2
Instead, it's passed in the evaluate/compile_only functions, which feels more naturally. It lets us set up the Evaluation struct long before we actually feed it with data to evaluate. Now that Evaluation::new() would be accepting an empty list of arguments, we can simply implement Default, making things a bit more idiomatic. Change-Id: I4369658634909a0c504fdffa18242a130daa0239 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10475 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de>
2023-09-24 r/6650 fix(tvix/eval): fix b/281 by adding Value::CatchableAdam Joseph1-0/+1
This commit makes catchable errors a variant of Value. The main downside of this approach is that we lose the ability to use Rust's `?` syntax for propagating catchable errors. Change-Id: Ibe89438d8a70dcec29e016df692b5bf88a5cad13 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9289 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: Adam Joseph <adam@westernsemico.com> Tested-by: BuildkiteCI
2023-08-20 r/6497 refactor(tvix/serde): cargo clippyFlorian Klink3-14/+13
Change-Id: I3c86225040a04c9cb5702b226b4a5008ae219c42 Reviewed-on: https://cl.tvl.fyi/c/depot/+/9093 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-06-22 r/6345 fix(tvix/serde): remove unnecessary dependency on genawaiterEvgeny Zemtsov2-6/+2
Change-Id: I52f13c6b508793603bc726353e9bf7d5f12fddf4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8850 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-06-22 r/6344 feat(tvix/eval): allow extending builtins outside of tvix_evalEvgeny Zemtsov2-0/+37
The change allows applications that use tvix_serde for parsing nix-based configuration to extend the language with domain-specific set of features. Change-Id: Ia86612308a167c456ecf03e93fe0fbae55b876a6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8848 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-06-22 r/6342 feat(tvix/serde): make from_str_with_config publicEvgeny Zemtsov1-0/+1
Change-Id: I61962297101d35ed02673edf1bca0584b7be37cf Reviewed-on: https://cl.tvl.fyi/c/depot/+/8845 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2023-06-20 r/6336 fix(tvix/eval): only finalise formal arguments if defaultingsterni1-1/+2
When dealing with a formal argument in a function argument pattern that has a default expression, there are two different things that can happen at runtime: Either we select its value from the passed attribute successfully or we need to use the default expression. Both of these may be thunks and both of these may need finalisers. However, in the former case this is taken care of elsewhere, the value will always be finalised already if necessary. In the latter case we may need to finalise the thunk resulting from the default expression. However, the thunk corresponding to the expression may never end up in the local's stack slot. Since finalisation goes by stack slot (and not constants), we need to prevent a case where we don't fall back to the default expression, but finalise anyways. Previously, we worked around this by making `OpFinalise` ignore non-thunks. Since finalisation of already evaluated thunks still crashed, the faulty compilation of function pattern arguments could still cause a crash. As a new approach, we reinstate the old behavior of `OpFinalise` to crash whenever encountering something that is either not a thunk or doesn't need finalisation. This can also help catching (similar) miscompilations in the future. To then prevent the crash, we need to track whether we have fallen back or not at runtime. This is done using an additional phantom on the stack that holds a new `FinaliseRequest` value. When it comes to finalisation we check this value and conditionally execute `OpFinalise` based on its value. Resolves b/261 and b/265 (partially). Change-Id: Ic04fb80ec671a2ba11fa645090769c335fb7f58b Reviewed-on: https://cl.tvl.fyi/c/depot/+/8705 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org>
2023-06-16 r/6325 test(tvix/serde): add a test for from_str_with_configVincent Ambo1-1/+12
Change-Id: I451851f4e01cb6dd0005e55bb852823b618f2ee6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8809 Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-06-16 r/6324 feat(tvix/serde): add a function to with custom configurationVincent Ambo1-1/+16
This adds a `from_str_with_config` function which takes a user-supplied closure that sets additional settings on the `tvix_eval::Evaluation`. Note that users can not set `strict = false`, but other settings are not restricted. This solves b/262. Change-Id: Ice184400b843cfbcaa5b6fe251ced12b6815e085 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8808 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2023-03-22 r/6037 feat(tvix/eval): add Evaluation::strict to toggle top-level deepseqVincent Ambo1-1/+2
This makes it possible for callers to control whether they can receive partially evaluated values from an evaluation or not. We're actually flipping the default behaviour to non-strict top-level evaluation, which means that callers have to set `strict = true` on the Evaluation to get the previous behaviour. Change-Id: Ic048e9ba09c88866d4c3177d5fa07db11c4eb20e Reviewed-on: https://cl.tvl.fyi/c/depot/+/8325 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2023-03-13 r/5979 fix(tvix/eval): implement cppnix JSON-serialisation semanticsVincent Ambo1-1/+2
This drops the usage of serde::Serialize, as the trait can not be used to implement the correct semantics (function colouring!). Instead, a manual JSON serialisation function is written which correctly handles toString, outPath and other similar weirdnesses. Unexpectedly, the eval-okay-tojson test from the C++ Nix test suite now passes, too. This fixes an issue where serialising data structures containing derivations to JSON would fail. Change-Id: I5c39e3d8356ee93a07eda481410f88610f6dd9f8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8209 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-01-31 r/5797 refactor(tvix/serde): allow dead_code in structFlorian Klink1-0/+1
This is an example, which uses the debug trait to print all field values. Silence the compiler warning about unused fields. Change-Id: I5f1216c77819003302e83ba1af1ff13c924f3b38 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7971 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2023-01-31 r/5785 feat(tvix/serde): add an example application to the projectVincent Ambo3-0/+57
This shows how people can use tvix_serde to deserialise configuration structs for their programs from Nix code. Change-Id: I71bf4e03dce19dddafe67dd729b4e4b10719a739 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7945 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-01-04 r/5585 feat(tvix/serde): implement enum deserialisationVincent Ambo3-6/+188
Implements externally tagged enum deserialisation. Other serialisation methods are handled by serde internally using the existing methods. See the tests for examples. Change-Id: Ic4a9da3b5a32ddbb5918b1512e70c3ac5ce64f04 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7721 Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de>
2023-01-04 r/5584 feat(tvix/serde): add newtype & tuple deserialisationVincent Ambo2-10/+27
Only missing enums at this point, but they're a bit of a beast. Change-Id: I4ad47c034851f9a8794c81f39a5149a8ac1826e8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7716 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: tazjin <tazjin@tvl.su>
2023-01-03 r/5576 test(tvix/serde): add initial set of deserialisation testsVincent Ambo3-1/+86
Change-Id: I0a9779edf0296c25d37fb5f75f8fc5852fe00121 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7715 Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de>
2023-01-03 r/5575 feat(tvix/serde): handle nested data structuresVincent Ambo1-12/+55
Change-Id: I543fc05d31bbb9ad2edb887bce4510e9a1cdb102 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7714 Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: flokli <flokli@flokli.de>
2023-01-02 r/5566 feat(tvix/serde): initial Nix->serde::Deserialize implVincent Ambo5-0/+444
This will make it possible fairly easily use Nix to represent arbitrary data structures, e.g. for using Nix as a config language. Only pure Nix (i.e. no `import` etc.) is supported for now. Not all types, specifically no struct traversal, are implemented in this commit. Change-Id: I9ac91a229a0d12bf818e6e3249f3e5a691599a2c Reviewed-on: https://cl.tvl.fyi/c/depot/+/7712 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-12-11T16·27+0000