about summary refs log tree commit diff
AgeCommit message (Collapse)AuthorFilesLines
2024-08-23 r/8566 feat(web/tvixbolt): Add crate2nix-check and re-generate Cargo.nixIlan Joselevich2-0/+87
Change-Id: Ib9881efd6a78ba28e283759e0ed5125fb175b89e Reviewed-on: https://cl.tvl.fyi/c/depot/+/12221 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-08-23 r/8565 feat(tvix/utils): Add mkCrate2nixCheckIlan Joselevich2-56/+36
This adds a function which can be used across the monorepo to create a an extra CI step that checks whether the Cargo.nix file is up-to-date. Change-Id: Idb8298b29ddc2ca5dff1facb1b9ed86a236ee66d Reviewed-on: https://cl.tvl.fyi/c/depot/+/12227 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-08-23 r/8564 feat(ops/users): add mrflosVincent Ambo1-0/+5
Patch submitted via public inbox. Change-Id: I5dc2d86aefd909216e8a16f428fc2cf818a125c2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12296 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2024-08-23 r/8563 chore(tools/depotfmt): remove terraform fmtVincent Ambo1-10/+0
This adds almost a gigabyte of closure size for formatting files that almost never change. I'll look into building just the formatter somehow, but it's not a very high priority task. Change-Id: Ib0f841e1a98133381c5ae154e2a57df8af52dc1f Reviewed-on: https://cl.tvl.fyi/c/depot/+/12293 Tested-by: BuildkiteCI Reviewed-by: emery <emery@dmz.rs> Autosubmit: tazjin <tazjin@tvl.su>
2024-08-23 r/8562 feat(users/emery): add custom Cyrillic keyboard layoutEmery Hemingway2-0/+121
Change-Id: I37f95c904e8a05cf8d8e437c41b9710c2ebf0300 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12294 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2024-08-23 r/8561 feat(tvix/nar-bridge): send content-type headersFlorian Klink3-9/+18
This prevents browsers from treating NARInfo and nix-cache-info paths as a separate "Download", but just show it in plaintext. Change-Id: If99abe20ef1d24e4fa86c055160861ca47aa81ce Reviewed-on: https://cl.tvl.fyi/c/depot/+/12267 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-08-23 r/8560 feat(nix-compat): add constants for mime typesFlorian Klink1-0/+7
This adds the MIME types Nix uses when uploading files to a S3 bucket / HTTP endpoint. It ignores them when retrieving. Change-Id: I75fa96d5e53349f0470bbe8cbba93f961fbd5f05 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12266 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-08-23 r/8559 feat(ops/users): add nikivVincent Ambo1-0/+5
Patch submitted to me IRL. Change-Id: Idd8aa75313ba73d5c1e92b98d390e43e7108c6b6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12292 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su>
2024-08-23 r/8558 docs: remove mention of Sourcegraph from depot READMEVincent Ambo1-4/+3
Change-Id: I721d08d2862256e5e973792b8fc7fac73b0326fc Reviewed-on: https://cl.tvl.fyi/c/depot/+/12288 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-08-23 r/8557 chore(ops/besadii): remove sourcegraph index update supportVincent Ambo2-38/+1
Change-Id: I4ee9a5a69c90e2050c60b2ef8483431d691b499f Reviewed-on: https://cl.tvl.fyi/c/depot/+/12287 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-08-23 r/8556 chore(whitby): remove Sourcegraph instanceVincent Ambo2-64/+0
Change-Id: I4d03f98e79de5e3a9c8c4a33682d5c78e3e0f028 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12286 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-08-23 r/8555 feat(ops/users): add azizVincent Ambo1-0/+5
Patch submitted to me IRL. Change-Id: I43805e3932ccbe383fb5ec7780a29fae187f64db Reviewed-on: https://cl.tvl.fyi/c/depot/+/12290 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2024-08-23 r/8554 feat(ops/users): add emeryVincent Ambo1-0/+5
Patch submitted to me IRL. Change-Id: I4cbfb138f616adf8635ca84f25cb77f8b8af7959 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12289 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2024-08-23 r/8553 feat(ops/www): redirect cs.tvl.fyi to livegrep & cgitVincent Ambo1-7/+42
Goodbye, Sourcegraph. Relates to b/290. Change-Id: Ic1cf3c1cf52ae17cdcc18c675b4c01d477644a3c Reviewed-on: https://cl.tvl.fyi/c/depot/+/12285 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-08-23 r/8552 chore(web/atward): remove sourcegraph supportVincent Ambo3-173/+5
We're moving away from sourcegraph to livegrep and so on, as Sourcegraph has gone fully proprietary. This removes support for redirecting to Sourcegraph. Relates to b/290 Change-Id: I04ccf8dfef72113cd49d444151cb0c3eb834845d Reviewed-on: https://cl.tvl.fyi/c/depot/+/12268 Reviewed-by: Profpatsch <mail@profpatsch.de> Tested-by: BuildkiteCI
2024-08-22 r/8551 feat(tazjin/german-string): add constructor that leaks a Vec<u8>Vincent Ambo1-2/+29
Change-Id: Id0b6b4794128573ddb62e82fd77d080e564978be Reviewed-on: https://cl.tvl.fyi/c/depot/+/12264 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: tazjin <tazjin@tvl.su>
2024-08-22 r/8550 feat(tazjin/german-string): add persistent construction from staticsVincent Ambo1-9/+56
Makes it possible to construct persistent German Strings from data with 'static lifetime. This doesn't leak any additional data. Change-Id: Ifb3a36ff72b106724883302e4c2195ac54acd352 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12263 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: tazjin <tazjin@tvl.su>
2024-08-22 r/8549 refactor(tazjin/german-string): add type for encoding storage classVincent Ambo1-6/+24
Adds a StorageClassPtr type that is set up to be able to steal bits from an aligned pointer to encode the storage class of a German String. Change-Id: I64591174eac1ebcb73e624a59bd107ba1e02c69d Reviewed-on: https://cl.tvl.fyi/c/depot/+/12262 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2024-08-21 r/8548 fix(tvix/castore): u32 -> u64 in DirectoryError::SizeOverflow messageBen Webb1-1/+1
Fix a discrepancy in the error message for DirectoryError::SizeOverflow. The message indicates that the SizeOverflow error occurs when total size exceeds u32::MAX, but that's not true. All size fields within the castore's internal Directory ADT are u64, and the SizeOverflow error is only returned after a call to the checked_add implementation on u64. See tvix/castore/nodes/directory.rs +111 and tvix/castore/nodes/directory.rs +88 as of this commit. Change-Id: I74d161ea8927362e1cb601ba163489aa96fb91b1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12259 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-08-21 r/8547 feat(nix-compat/nix_http): init parse_nar[info]_strFlorian Klink15-178/+634
This moves the URL component parsing code we had in nar-bridge to nix-compat. We change the function signature to return an Option, not a Result<_, StatusCode>. This allows returning more appropriate error codes, as we can ok_or(…) at the callsite, which we now do: on an upload to an invalid path, we now return "unauthorized", while on a GET/HEAD, we return "not found". This also adds support to parse compression suffixes. While not supported in nar-bridge, other users of nix-compat might very well want to parse these paths. Also fix the error message when parsing NAR urls, it mentioned 32, not 52, which is a copypasta error from the narinfo URL parsing code. Change-Id: Id1be9a8044814b54ce68b125c52dfe933c9c4f74 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12260 Reviewed-by: raitobezarius <tvl@lahfa.xyz> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-08-20 r/8546 feat(kontemplate): defaults can now have nested valuesArmin Schlegel2-1/+23
See https://b.tvl.fyi/issues/409 for details. Change-Id: Ibb54fab7a78e0e5f708c2a7dc8bb26ac0b2b4689 Signed-off-by: Armin Schlegel <a.schlegel@gridx.de> Reviewed-on: https://cl.tvl.fyi/c/depot/+/11972 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2024-08-20 r/8545 refactor(nix-compat/store_path): make StorePath generic on SFlorian Klink15-294/+280
Similar to how cl/12253 already did this for `Signature`, we apply the same logic to `StorePath`. `StorePathRef<'a>'` is now a `StorePath<&'a str>`, and there's less redundant code for the two different implementation. `.as_ref()` returns a `StorePathRef<'_>`, `.to_owned()` gives a `StorePath<String>` (for now). I briefly thought about only publicly exporting `StorePath<String>` as `StorePath`, but the diff is not too large and this will make it easier to gradually introduce more flexibility in which store paths to accept. Also, remove some silliness in `StorePath::from_absolute_path_full`, which now doesn't allocate anymore. Change-Id: Ife8843857a1a0a3a99177ca997649fd45b8198e6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12258 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-08-20 r/8544 refactor(nix-compat/aterm): update function names a bitFlorian Klink3-26/+26
Don't call functions bstr or str when they return BString or String, it's confusing. Rename them to `string` and `bytes`. We might be able to generalize over this being BString or Vec<u8> later. Change-Id: I8198551ed3ba1cfc479bf7e3cbbc13a426faf4c0 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12257 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-08-20 r/8543 feat(tools/depotfmt): Get gofmt from pkgs.go instead of buildGoIlan Joselevich1-2/+2
This allows importing just //tools/depotfmt.nix when using a josh workspace. Change-Id: Idf7eb438be1cc1e60445aace982fdca1f4367595 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12254 Tested-by: BuildkiteCI Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com> Reviewed-by: flokli <flokli@flokli.de>
2024-08-20 r/8542 chore(aspen/system): Turn off disableWhileTyping for libinputAspen Smith1-6/+1
I... actually really hate this, it turns out. Plus it breaks video games, lol. Change-Id: I4b86e38ee78f5f3b6ffec5e7cc4f1dc054d91c56 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12256 Reviewed-by: aspen <root@gws.fyi> Autosubmit: aspen <root@gws.fyi> Tested-by: BuildkiteCI
2024-08-19 r/8541 feat(tvix/nix-compat): entry locator in listing structuresRyan Lahfa2-1/+113
This implements a simple DFS locator in listing structures. It is interoperable with the Rust standard library paths, we also build our own errors to restrict path values to reasonable secure defaults, e.g. relative paths with no `..` component. Tests are added for this new feature for a positive and a negative check. In addition, a path validation test was added. The Windows-style prefix is gated on the Windows platform as UNIX does not parse `C:\\` as a `Component::Prefix(_)` but as a `Component::Normal(_)`. Change-Id: Iae2a80bebd8138e41af94aa7d09f2842c3c5a786 Signed-off-by: Ryan Lahfa <tvl@lahfa.xyz> Reviewed-on: https://cl.tvl.fyi/c/depot/+/12255 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-08-19 r/8540 feat(nix-compat/narinfo/signature): generalize name fieldFlorian Klink9-45/+117
Requiring `name` to be a `&str` means it'll get annoying to pass around `Signature`, but being able to pass them around in an owned fashion is kinda a requirement for a stronger typed `PathInfo` struct, where we want to have full ownership. Rework the `Signature` struct to become generic over the type of the `name` field. This means, it becomes possible to have owned versions of it. We don't want to impose `String` or `SmolStr` for example, but want to leave it up to the nix-compat user to decide. Provide a type alias for the existing `&str` variant (`SignatureRef`), and use it where we previously used the non-generic `Signature` one. Add some tests to ensure it's possible to *use* `Signature` with both `String` and `SmolStr` (but only pull in `smol_str` as dev dependency for the tests). Also, add some more docstrings, these were a bit sparse. Change-Id: I3f75691498c6bda9cd072d2d9dac83c4f6c57287 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12253 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-08-19 r/8539 feat(tvix/nix-compat): add a basic listing deserializerRyan Lahfa4-0/+77
.ls files are useful to seek in a NAR without parsing it entirely. The responsibility of validating the files is on the caller. Change-Id: I5d1da28b5479c38f20ca5babe60e362a2217c9ea Signed-off-by: Ryan Lahfa <tvl@lahfa.xyz> Reviewed-on: https://cl.tvl.fyi/c/depot/+/12196 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-08-19 r/8538 feat(tvix): drop usage of sparseTree in favor of lib.sourceByRegexIlan Joselevich6-41/+25
We can avoid depending on things outside //tvix by just using a similar util from nixpkgs. Change-Id: I9ea3e1f0a8a059ea10caaec173569ba9f316aec6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12247 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-08-19 r/8537 feat(tvix/boot/tests): use pkgs.nixos instead of nixosForIlan Joselevich1-4/+5
Using pkgs.nixos directly allows us to create a smaller nixos closure for the tests and also not depend on things in depot.ops which can be beneficial for extending the tvix josh workspace. Change-Id: Ic6ad2122733418114b43aa692d6e42ac1e308eeb Reviewed-on: https://cl.tvl.fyi/c/depot/+/12251 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
2024-08-19 r/8536 refactor(nix-compat/narinfo/signature): use ed25519::SignatureBytesFlorian Klink1-4/+5
It's a `[u8; SIGNATURE_LENGTH]` type alias, and conveys what we're accepting or returning a bit nicer. Change-Id: I974cd97d56d383e51417eb0f26e1431a05711922 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12252 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: raitobezarius <tvl@lahfa.xyz>
2024-08-19 r/8535 feat(tazjin/german-string): add constructor method from owned dataVincent Ambo1-12/+40
Makes it possible to construct a GermanString from an owned byte vector, without having to clone the data. This is done by "disowning" the vector using ManuallyDrop to access its internal pointer. For transient strings, this memory is then owned (and freed) by the GermanString instance. Small strings are copied out of the heap and stored inline as before, to avoid any dereferencing operations. Change-Id: I754736099f71d646d430aed73e558a5a7626c394 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12249 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2024-08-19 r/8534 feat(tazjin/german-string): add pointer equality check in EqVincent Ambo1-2/+3
This can short-circuit two large string comparisons. Change-Id: If45e7cf33921fe571482dc710c27ef8cc7c70885 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12245 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2024-08-19 r/8533 chore(tazjin/german-string): add Nix buildVincent Ambo1-0/+5
Change-Id: I1e8c500c9bb4ba365ba32f53f60bcf657f62ed62 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12243 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2024-08-19 r/8532 feat(tazjin/german-string): add [Partial]Ord implementationVincent Ambo1-0/+62
Change-Id: I8002ec63be45a15160387c21ed1fa8721a7c3eb4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12242 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: tazjin <tazjin@tvl.su>
2024-08-19 r/8531 test(tazjin/german-string): add roundtrip proptestsVincent Ambo1-1/+21
Change-Id: Iee392368252d5c6e96d879b213aee34a301d13d6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12241 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2024-08-19 r/8530 feat(tazjin/german-string): add Eq impl & corresponding proptestsVincent Ambo3-1/+428
Change-Id: I66a258fad5d4e249268b9d2743d46b30c5ac1dac Reviewed-on: https://cl.tvl.fyi/c/depot/+/12240 Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su>
2024-08-19 r/8529 feat(tazjin/german-string): add Debug implementationVincent Ambo1-0/+7
Change-Id: I9a4a8a4815ac210c402cfc2feb1fe0606affb327 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12239 Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2024-08-19 r/8528 feat(tazjin/german-string): PartialEq implementation for GermanStringVincent Ambo1-0/+16
This is where one of the advantages of this string representation starts to shine: For small strings there's no derefencing any heap memory at all, and for the long representation we can compare the prefix before moving on. Change-Id: Iac333a52e8d7c9dd09e33dbcf51754e321c880e6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12238 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2024-08-19 r/8527 feat(tvix/boot): Use buildGoModule instead of buildGo for tvix-initIlan Joselevich1-5/+8
//tvix depending less on other parts of depot is prefered as it will help with extending the josh workspace of tvix. Change-Id: Ifcac3af1782dfd82e7543cb4c3ae57fbd186edff Reviewed-on: https://cl.tvl.fyi/c/depot/+/12250 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-08-19 r/8526 feat(third_party/gitignoreSource): Get gitignore-nix from sources (niv)Ilan Joselevich2-10/+14
It's easier to implement readTree/depot polyfills for gitignoreSource when it's imported from third_party.sources, rather than in a file at //third_party.gitignoreSource. Change-Id: I1323f932bd0feeb2c50ccc76397a80e035842992 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12248 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2024-08-19 r/8525 fix(tazjin/german-string): add maximum length checkVincent Ambo1-0/+4
Change-Id: I4fd9b2a31749d65632b662807c4e703a9df9ec9e Reviewed-on: https://cl.tvl.fyi/c/depot/+/12237 Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2024-08-19 r/8524 feat(tazjin/german-string): add data accessors for &str and &[u8]Vincent Ambo1-1/+67
Change-Id: I992e625861f79ef6d9993e8caee4e02d3fc5557e Reviewed-on: https://cl.tvl.fyi/c/depot/+/12236 Tested-by: BuildkiteCI Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: tazjin <tazjin@tvl.su>
2024-08-19 r/8523 feat(tazjin/german-string): add Drop impl for transient stringsVincent Ambo1-0/+17
All of these strings are currently transient (the storage class is not yet represented anywhere), and the ones that are heap allocated need to be deallocated when the transient string dies. Change-Id: Iba0ca926df5db7594f304c5d5318db9d69c6f42c Reviewed-on: https://cl.tvl.fyi/c/depot/+/12235 Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2024-08-19 r/8522 feat(tazjin/german-string): init initial representationVincent Ambo4-0/+82
This adds an initial implementation of the so-called "German Strings" in Rust. https://cedardb.com/blog/german_strings/ This implementation is *far from* complete, the only thing that can be done right now is construct a string, and even that I'm not fully happy with. Change-Id: I2697932a0ef373be76ffd14d59677493a5783b58 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12234 Autosubmit: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>
2024-08-19 r/8521 refactor(tvix/eval): remove use of imbl::OrdMapVincent Ambo8-451/+95
Removes imbl::OrdMap in favour of an Rc over the standard library's BTreeMap, which allows us to drop the imbl dependency completely. In my local tests this is actually slightly faster for `hello` and `firefox`. Change-Id: Ic9597ead4e98bf9530f290c6a94a3c5c3efd0acc Reviewed-on: https://cl.tvl.fyi/c/depot/+/12201 Reviewed-by: aspen <root@gws.fyi> Tested-by: BuildkiteCI
2024-08-19 r/8520 refactor(tvix/eval): remove use of imbl::VectorVincent Ambo4-47/+46
This vector type has served us well for now, but it contains internal refcounts which are incompatible with upcoming changes related to garbage collection. The performance impact of this change within all benchmarks I ran was within the margin of error: [nix-shell:/tmp/perf]$ hyperfine "./before -E '(import <nixpkgs> {}).firefox.outPath' --log-level ERROR --no-warnings" Benchmark 1: ./u64 -E '(import <nixpkgs> {}).firefox.outPath' --log-level ERROR --no-warnings Time (mean ± σ): 7.528 s ± 0.272 s [User: 6.578 s, System: 0.631 s] Range (min … max): 7.160 s … 8.012 s 10 runs nix-shell:/tmp/perf]$ hyperfine "./std-vec -E '(import <nixpkgs> {}).firefox.outPath' --log-level ERROR --no-warnings" Benchmark 1: ./std-vec -E '(import <nixpkgs> {}).firefox.outPath' --log-level ERROR --no-warnings Time (mean ± σ): 7.515 s ± 0.178 s [User: 6.508 s, System: 0.652 s] Range (min … max): 7.276 s … 7.861 s 10 runs Change-Id: Ib95f871956e336a1e5771f6293583854b1efb276 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12197 Reviewed-by: aspen <root@gws.fyi> Tested-by: BuildkiteCI
2024-08-19 r/8519 refactor(tvix/eval): ensure VM operations fit in a single byteVincent Ambo14-611/+821
This replaces the OpCode enum with a new Op enum which is guaranteed to fit in a single byte. Instead of carrying enum variants with data, every variant that has runtime data encodes it into the `Vec<u8>` that a `Chunk` now carries. This has several advantages: * Less stack space is required at runtime, and fewer allocations are required while compiling. * The OpCode doesn't need to carry "weird" special-cased data variants anymore. * It is faster (albeit, not by much). On my laptop, results consistently look approximately like this: Benchmark 1: ./before -E '(import <nixpkgs> {}).firefox.outPath' --log-level ERROR --no-warnings Time (mean ± σ): 8.224 s ± 0.272 s [User: 7.149 s, System: 0.688 s] Range (min … max): 7.759 s … 8.583 s 10 runs Benchmark 2: ./after -E '(import <nixpkgs> {}).firefox.outPath' --log-level ERROR --no-warnings Time (mean ± σ): 8.000 s ± 0.198 s [User: 7.036 s, System: 0.633 s] Range (min … max): 7.718 s … 8.334 s 10 runs See notes below for why the performance impact might be less than expected. * It is faster while at the same time dropping some optimisations we previously performed. This has several disadvantages: * The code is closer to how one would write it in C or Go. * Bit shifting! * There is (for now) slightly more code than before. On performance I have the following thoughts at the moment: In order to prepare for adding GC, there's a couple of places in Tvix where I'd like to fence off certain kinds of complexity (such as mutating bytecode, which, for various reaons, also has to be part of data that is subject to GC). With this change, we can drop optimisations like retroactively modifying existing bytecode and *still* achieve better performance than before. I believe that this is currently worth it to pave the way for changes that are more significant for performance. In general this also opens other avenues of optimisation: For example, we can profile which argument sizes actually exist and remove the copy overhead of varint decoding (which does show up in profiles) by using more adequately sized types for, e.g., constant indices. Known regressions: * Op::Constant is no longer printing its values in disassembly (this can be fixed, I just didn't get around to it, will do separately). Change-Id: Id9b3a4254623a45de03069dbdb70b8349e976743 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12191 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-08-19 r/8518 docs(tvix/glue/tvix_io): update doc-commentsFlorian Klink1-4/+2
Since we switched from reference scanning to string context, this only handles the `__corepkgs__` hack. Update the docstrings. Change-Id: Ie857c8c99ae1cdb4697323ec738f88be0580df3e Reviewed-on: https://cl.tvl.fyi/c/depot/+/12246 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2024-08-19 r/8517 feat(ops/pipelines): support buildkite retriesFlorian Klink1-4/+6
cl/12228 did enable automatic retries for some flaky tests, which generally did work, as can be seen in https://buildkite.com/tvl/depot/builds/35893 However, ":duck:" still reports as failing, because we check the number of steps to be nonzero, which is not the case if retries have happened. We cannot check for the overall status of the build, as it's still "RUNNING", but instead of counting all failed steps so far, we can query all failed jobs and then filter out the ones that were already retried. Change-Id: Ib9d27587c8a8ba7970850812c4302fecdc4482e7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12233 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su>