about summary refs log tree commit diff
path: root/tvix/eval/src/value/string.rs
AgeCommit message (Collapse)AuthorFilesLines
2023-09-15 r/6589 fix(tvix/eval): update identifier quoting to match cppnix 2.17Adam Joseph1-1/+12
In cppnix 2.17, commit b72bc4a972fe568744d98b89d63adcd504cb586c, the libexpr pretty-printing routine was fixed so that it would no longer pretty-print attrsets with keywords in their attrnames incorrectly. This commit implements the corresponding fix for tvix, fixes our tests to work with cppnix>=2.17 oracles, and expands our test cases to cover all the keywords. Change-Id: I4b51389cd3a9c44babc8ab2a84b383b7b0b116ca Reviewed-on: https://cl.tvl.fyi/c/depot/+/9283 Autosubmit: Adam Joseph <adam@westernsemico.com> Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2023-07-11 r/6404 fix(tvix/eval): use byte, not codepoint index for slicing in escapesterni1-1/+5
This fixes a subtle issue which would occasionally lead to a crash (e.g. when evaluating (pkgs.systemd.outPath with --trace-runtime): With each character in the string that has a multi byte representation in UTF-8, the actual byte position and what tvix thought it was would get out of sync. This could either lead to * Tvix swallowing characters or jumbling characters if multi byte characters would cause the tracked index to become out of sync with the byte position before the first character to be escaped, or * Tvix crashing if (in the same situation) the out of sync index would be within a UTF-8 byte sequence. Luckily, std's `char_indices()` iterator implements exactly what `nix_escape_char()`'s original author had in mind with `.chars().enumerate()`. Using `i + 1` for continuing is safe, since all characters that need (in fact, can) to be escaped in Nix are represented as a single byte in UTF-8. Change-Id: I1c836f70cde3d72db1c644e9112852f0d824715e Reviewed-on: https://cl.tvl.fyi/c/depot/+/8952 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: sterni <sternenseemann@systemli.org>
2023-03-13 r/5968 chore(tvix/eval): remove `From<SmolStr> for NixString` instanceVincent Ambo1-4/+3
No longer needed, and in some cases caused some extra work. Change-Id: I64e8e7292573bdc92a9c7a8e470e33f8c526f311 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8152 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Tested-by: BuildkiteCI
2023-03-13 r/5967 refactor(tvix/eval): simplify NixString representation(s)Vincent Ambo1-36/+13
Instead of the two different representations (which we don't really use much), use a `Box<str>` (which potentially shaves another 8 bytes off `Value`). NixString values themselves are immutable anyways (which was a guarantee we already had with `SmolStr`), so this doesn't change anything else. Change-Id: I1d8454c056c21ecb0aebc473cfb3ae06cd70dbb6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/8151 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2023-01-12 r/5652 feat(tvix/eval): implement builtins.toJSONVincent Ambo1-3/+4
Implements `Serialize` for `tvix_eval::Value`. Special care is taken with serialisation of attribute sets, and forcing of thunks. The tests should cover both cases well. Change-Id: I9bb135bacf6f87bc6bd0bd88cef0a42308e6c335 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7803 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su>
2023-01-10 r/5640 feat(tvix/eval): implement serde::Deserialize for ValueRyan Lahfa1-0/+36
Co-Authored-By: Vincent Ambo <tazjin@tvl.su> Change-Id: Ib6f7d1f4f4faac36b44f5f75cccc57bf912cf606 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7626 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2023-01-04 r/5582 fix(tvix/eval): ' is allowed in nonfirst position in Nix identifierssterni1-1/+1
With this is_valid_nix_identifier should line up with the upstream lexer definition: ID [a-zA-Z\_][a-zA-Z0-9\_\'\-]* While we're working on this, add a simple test checking the various formatting rules. Interestingly, it would not be suitable as an identity test, since you have to write { "assert" = null; } in order to avoid an evaluation error, but C++ Nix is happy to print this as { assert = null; } – maybe should be considered to be a bug. Change-Id: I0a4e1ccb5033a80f3767fb8d1c4bba08d303c5d8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7744 Autosubmit: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-12-25 r/5486 fix(tvix/eval): fix current clippy warningsVincent Ambo1-1/+1
It's been a while since the last time, so quite a lot of stuff has accumulated here. Change-Id: I0762827c197b30a917ff470fd8ae8f220f6ba247 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7597 Reviewed-by: grfn <grfn@gws.fyi> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-11-04 r/5241 fix(tvix/eval): quote keys which are not valid identifiersAdam Joseph1-1/+24
The impl Display for NixAttrs needs to wrap double quotes around any keys which are not valid Nix identifiers. This commit does that, and adds a test (which fails prior to this commit and passes after this commit). Change-Id: Ie31ce91e8637cb27073f23f115db81feefdc6424 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7084 Autosubmit: Adam Joseph <adam@westernsemico.com> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-10-17 r/5153 feat(tvix/eval): Record formals on lambdaGriffin Smith1-0/+7
In preparation for both implementing the `functionArgs` builtin and adding support for validating closed formals, record information about the formal arguments to a function *on the Lambda itself*. This may seem a little odd for the purposes of just closed formal checking, but is something we have to have anyway for builtins.functionArgs so I figured I'd do it this way to kill both birds with one stone. Change-Id: Ie3770a607bf352a1eb395c79ca29bb25d5978cd8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/7001 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-10-10 r/5095 feat(tvix/eval): Implement builtins.readFileGriffin Smith1-0/+7
Change-Id: If3fd0b087009a2bfbad8bb7aca0aa20de906eb12 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6921 Tested-by: BuildkiteCI Reviewed-by: Adam Joseph <adam@westernsemico.com> Reviewed-by: kanepyork <rikingcoding@gmail.com> Autosubmit: grfn <grfn@gws.fyi> Reviewed-by: tazjin <tazjin@tvl.su>
2022-10-10 r/5087 feat(tvix/eval): Initial resolution of `<...>` pathsGriffin Smith1-0/+7
This commit implements (lazy) resolution of `<...>` paths via either the NIX_PATH environment variable, or the -I command-line flag - both handled via EvalOptions. As a result, EvalOptions can no longer derive Copy, meaning we have to clone it at each line of the repl - this is probably not a huge deal as repl performance is not exactly an inner loop and we're not cloning very much. Internally, this works by creating a thunk which pushes a constant containing the string inside the brackets to the stack, then a new opcode to resolve that path via the `NixPath`. To get that opcode to work, we now have to pass in the NixPath when constructing the VM. This (intentionally) leaves out proper implementation of path resolution via `findFile` (cppnix just calls whatever identifier called findFile is in scope!!!) as that's widely considered a bit of a misfeature, but if we do decide to implement that down the road it likely wouldn't be more than a few extra ops within the thunk introduced here. Change-Id: Ibc979b7e425b65cbe88599940520239a4a10cee2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6918 Autosubmit: grfn <grfn@gws.fyi> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-10-10 r/5084 feat(tvix/eval): Implement builtins.concatStringsSepGriffin Smith1-0/+15
Change-Id: I6e46bcdbf3b5258b60edb017709fee577eb8ec74 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6907 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-09-17 r/4894 test(tvix/eval): Add proptests covering trait impls for StringGriffin Smith1-0/+11
Add a suite of proptests covering the laws of the handwritten stdlib trait impls (Eq, Ord, and Hash) for String, generated from a new set of macros for generating those tests which can be applied to other types. Change-Id: Ib3276c9e96fca497aece094e5612707d3dc77ccd Reviewed-on: https://cl.tvl.fyi/c/depot/+/6626 Autosubmit: grfn <grfn@gws.fyi> Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-17 r/4893 test(tvix/eval): Test StringRepr::Smol as wellGriffin Smith1-1/+8
The From<String> impl for NixString only generates StringRepr::Heap strings, but we want to make sure we're testing StringRepr::Smol too Change-Id: I6d04b9cf12ef8462fe2788e0c6414b165f40311d Reviewed-on: https://cl.tvl.fyi/c/depot/+/6629 Autosubmit: grfn <grfn@gws.fyi> Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-17 r/4892 test(tvix/eval): impl Arbitrary for NixStringGriffin Smith1-0/+17
Change-Id: I3fe2d410c789429493a1278d571ca8fe74c2a69d Reviewed-on: https://cl.tvl.fyi/c/depot/+/6625 Autosubmit: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2022-09-15 r/4865 feat(tvix/eval): implement Value::coerce_to_path()sterni1-1/+8
This function is necessary for all builtins that expect some form of path as an argument. It is merely a wrapper around coerce_to_string that can shortcut if we already have a path. The absolute path check is done in the same way as in C++ Nix for compatibility, although it should probably be revised in the long term (think about Windows, for example). Since coercing to a path is not an operation possible in the language directly, this function can live in the builtins module as the only place it is required. Change-Id: I69ed5455c00d193fea88b8fa83e28907a761cab5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6574 Autosubmit: sterni <sternenseemann@systemli.org> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2022-09-10 r/4786 fix(tvix/eval): fix doc comment syntax where applicableVincent Ambo1-10/+10
As pointed out by grfn on cl/6091 Change-Id: I28308577b7cf99dffb4a4fd3cc8783eb9ab4d0d6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6460 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-07 r/4701 chore(tvix/eval): provide 'static references to "name"/"value"Vincent Ambo1-0/+3
These static strings show up a bunch when dealing with the internals of attribute sets, and having them available as static references is required. Due to the way const expressions are evaluated, taking a reference to the existing NixString::NAME / NixString::VALUE items does not work and the references themselves need to be const-evaluated. Change-Id: If6e75847af978118a3b266fe6a3242321722434d Reviewed-on: https://cl.tvl.fyi/c/depot/+/6366 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-04 r/4635 feat(tvix/eval): implement upvalue resolution in `with` scopesVincent Ambo1-0/+6
These need to be handled specially by the runtime if the compiler determines that a given local must be resolved via `with`. Note that this implementation has a bug: It currently allows `with` inside of nested lambdas to shadow statically known identifiers. This will be cleaned up in the next commit. Change-Id: If196b99cbd1a0f2dbb4a40a0e88cdb09a009c6b9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6299 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-09-03 r/4607 fix(tvix/eval): correctly escape `${` in stringsVincent Ambo1-11/+15
Without this escape, it is possible for Nix to produce escaped representations which are not literal Nix values again. This was fixed in upstream Nix in https://github.com/NixOS/nix/pull/4012 (though only for eval, not in the REPL) and the updated test is picked from upstream after that commit. Because we run the C++ Nix tests against our test suite as well, this also bumps our custom Nix 2.3 to a commit that includes the cherry-picked fix from the PR above. Change-Id: I478547ade65f655c606ec46f7143932064192283 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6271 Reviewed-by: grfn <grfn@gws.fyi> Reviewed-by: sterni <sternenseemann@systemli.org> Tested-by: BuildkiteCI
2022-09-01 r/4568 fix(tvix/eval): fix several string escapingsVincent Ambo1-2/+4
These were missing an additional level of escaping, silly oversight caught by an upstream test. Change-Id: I0312084475e4b88c83945614e9aa5b34c6bc3ec2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6232 Reviewed-by: sterni <sternenseemann@systemli.org> Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-08-30 r/4540 docs(tvix/eval): Use correct syntax for module doc commentsVincent Ambo1-3/+2
Change-Id: I35741856f34b86a538f226a8eaf8806edede60ec Reviewed-on: https://cl.tvl.fyi/c/depot/+/6207 Reviewed-by: grfn <grfn@gws.fyi> Tested-by: BuildkiteCI
2022-08-27 r/4512 chore(tvix/eval): explicitly set #[repr(transparent)] on wrappersVincent Ambo1-0/+1
For representation wrappers that are used to control the visibility of type internals, this ensures that the wrapper does not increase the size of the type. In practice, the optimiser likely does this anyways but it is good to guarantee it. Change-Id: Ic6df7d668fe6006dfbd5b6cfcfc2088afa95b810 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6178 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-08-26 r/4498 refactor(tvix/value): replace static representation with SmolStrVincent Ambo1-7/+8
The only uses of the static variant were for `"name"` and `"value"`, which are both small enough to fit into a SmolStr. The size of NixString accomodates `String` anyways, so we may as well inline them. Additionally smol_str is already in the dependency graph because rnix uses it, and using it for representations of identifiers is sensible. Change-Id: I9969312256d1657d69128e54c47dc7294a18ce58 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6165 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org> Reviewed-by: grfn <grfn@gws.fyi>
2022-08-25 r/4482 refactor(tvix/value): hide internal string representationVincent Ambo1-9/+12
Wraps the string representation in an additional newtype struct with a private field in order to hide the representation from other modules. This is done in order to avoid accidental leakage of the internals outside of value::string. In fact, this caught a mistake in the compiler module which was directly constructing an internal variant. Change-Id: If4b627d3cff7ab9cd50ca1a3ac73245d4dcf7aef Reviewed-on: https://cl.tvl.fyi/c/depot/+/6147 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-25 r/4474 fix(tvix/value): explicitly delegate `Ord` to &str representationVincent Ambo1-1/+9
I assumed that `Ord` is a marker trait like `Eq`, but it actually has a member. Without this ordering was incoherent. Change-Id: Id37cbdf333daf748d29b85243046c7e061b1ce29 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6139 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-25 r/4473 fix(tvix/value): ensure only string values of NixString are hashedVincent Ambo1-1/+8
Change-Id: I1b97cf5a9e4e235fac72a507de49a8df508bcaa1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6138 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-25 r/4472 feat(tvix/eval): implement string concatenationVincent Ambo1-0/+6
Change-Id: If61066e59232b2bad42b5cb5f0f2d9b9c416be8b Reviewed-on: https://cl.tvl.fyi/c/depot/+/6137 Tested-by: BuildkiteCI Reviewed-by: sterni <sternenseemann@systemli.org>
2022-08-24 r/4462 fix(tvix/value): add ident_str representation of stringsVincent Ambo1-7/+24
When printing strings as identifiers (in attribute sets), the string should only be quoted and escaped if it contains escape characters. Change-Id: If2bcfa1e93dc8f00be4d7a57ec1d82fc679103c3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6127 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi> Autosubmit: tazjin <tazjin@tvl.su>
2022-08-24 r/4459 fix(tvix/value): add escaping logic for Nix stringsVincent Ambo1-10/+46
Nix strings displayed to users must be escaped the same way as they are in C++ Nix. This adds the scaffolding for escapes, but is most likely not yet complete. Change-Id: Icfdcb2ac98d292c567ba894a92b6529a53e0cc17 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6124 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-08-24 r/4458 fix(tvix/value): implement PartialOrd/PartialEq for stringsVincent Ambo1-1/+13
Instead of comparing the enum variants (which does not yield useful behaviour), compare &str representations of the string instead. Change-Id: I5e94b5f6c91b4561e1bc7c36d586f3d23c243764 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6112 Tested-by: BuildkiteCI Reviewed-by: eta <tvl@eta.st>
2022-08-24 r/4457 feat(tvix/value): introduce string representation with &'static strVincent Ambo1-5/+29
For cases where the strings are statically known (such as the oft-occuring name/value), this can be a useful optimisation. It's also much more convenient in tests. Change-Id: Ie462b684805bd4986ea5e85ca4bff663bc2d3c3c Reviewed-on: https://cl.tvl.fyi/c/depot/+/6111 Tested-by: BuildkiteCI Reviewed-by: eta <tvl@eta.st>
2022-08-24 r/4456 test(tvix/value): add simple attrset construction testsVincent Ambo1-0/+6
These do not yet test nested attribute sets; we need to add some more inspection primitives first. Change-Id: Icfc99bf17c73ebefc0d882a84f0ca73ec688a54d Reviewed-on: https://cl.tvl.fyi/c/depot/+/6110 Reviewed-by: eta <tvl@eta.st> Tested-by: BuildkiteCI
2022-08-13 r/4425 feat(tvix/value): add some necessary helpers for stringsVincent Ambo1-2/+2
Deriving Ord/Eq is required for the ordered BTreeMaps. Once interning is implemented this will require some extra magic for the sort order, but that's fine. Change-Id: I0c654648eb3609a4a01d84868c25f43a4d35bc2e Reviewed-on: https://cl.tvl.fyi/c/depot/+/6089 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-08-13 r/4423 feat(tvix/eval): add Value variants for strings & attrsetsVincent Ambo1-1/+1
Change-Id: Idebf663ab7fde3955aae50f635320f7eb6c353e8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6087 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
2022-08-13 r/4421 feat(tvix/eval): add module for string type implementationVincent Ambo1-0/+13
Change-Id: I5e4465acc4a676c10d7374b14f7a09240202b466 Reviewed-on: https://cl.tvl.fyi/c/depot/+/6085 Tested-by: BuildkiteCI Reviewed-by: grfn <grfn@gws.fyi>
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-10-31T10·43+0000