about summary refs log tree commit diff
AgeCommit message (Collapse)AuthorFilesLines
2024-06-20 r/8301 docs(tvix): drop done TODOs in the O11Y sectionFlorian Klink1-8/+0
OTLP shutdown behaviour was fixed in cl/11803, and tracy landed in cl/10952 and cl/11815. Change-Id: I310dfcc41ba62ce5b006e75a35feb839327b0e7c Reviewed-on: https://cl.tvl.fyi/c/depot/+/11866 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Simon Hauser <simon.hauser@helsinki-systems.de> Tested-by: BuildkiteCI
2024-06-20 r/8300 refactor(tvix/*store): remove some trait boundsFlorian Klink3-24/+6
We don't need to require these things for these impl blocks yet. Change-Id: I3cec958a637a4f900bdd38abd00e9133bf75ce46 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11865 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Simon Hauser <simon.hauser@helsinki-systems.de> Tested-by: BuildkiteCI
2024-06-20 r/8299 feat(tvix/tracing): gRPC trace context propagationSimon Hauser18-48/+399
This introduces optional helper function in tvix/tracing for trace propagation and uses these helper in the `tvix-store`. The GRPCBlobService, GRPCDirectoryService and GRPCPathInfoService now accept a generic client, meaning the client can be generated with either `::new` or `::with_interceptor`. This was tested and validated by starting a `tvix-store daemon` and `tvix-store import`. Change-Id: I4b194483bf09266820104b4b56e4a135dca2b77a Reviewed-on: https://cl.tvl.fyi/c/depot/+/11863 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-06-20 r/8298 feat(tvix/castore): instrument tokio task with current spanSimon Hauser3-18/+27
By default tokio::spawn does not instrument the spawned task with the current spawn (https://github.com/tokio-rs/tokio/discussions/6008), do this manually for all tokio::spawn functions in functions that are instrumented. Change-Id: I83dd8145b3a62421454aff57d34180cebbee8304 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11864 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Simon Hauser <simon.hauser@helsinki-systems.de>
2024-06-20 r/8297 feat(tvix/tracing): optional progressbarSimon Hauser4-6/+19
Disable the progressbar on default and provide a interface for optionally enabling the progressbar. Change-Id: I0e31b1957e80cf64a8dcf65c6ceb3713975b8220 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11861 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Simon Hauser <simon.hauser@helsinki-systems.de>
2024-06-20 r/8296 docs(tvix/docs): document nar renderer optimization potentialFlorian Klink1-0/+12
Change-Id: I747ab5c8ba3424828b1c88e0353170ad0efab3a6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11849 Tested-by: BuildkiteCI Reviewed-by: Simon Hauser <simon.hauser@helsinki-systems.de>
2024-06-20 r/8295 feat(tvix/store/utils): detect gRPC NAR calculation serviceFlorian Klink1-5/+26
We were currently always using SimpleRenderer, which would mean the client would download every blob locally to calculate the checksum, which of course is very slow. Detect this special case and create a second instance (and client) for now. Change-Id: If39a862a5311e71c8073ac4e663f6c5dd437072e Reviewed-on: https://cl.tvl.fyi/c/depot/+/11848 Reviewed-by: Simon Hauser <simon.hauser@helsinki-systems.de> Tested-by: BuildkiteCI
2024-06-18 r/8294 chore(tvix): patch crate2nix to drop darwin dontStripIlan Joselevich3-2/+24
This patch has already been applied on upstream but there hasn't been an official release yet so we patch it ourselves. Upstreamed patch: https://github.com/nix-community/crate2nix/commit/0209f258cda8a9972a785e26d92fb477ce4d1b0e Follow-up of: https://cl.tvl.fyi/c/depot/+/11856 Change-Id: If56071ce5753fd26e2b4c203cd831bbe5d329009 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11858 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-06-17 r/8293 chore(tvix): move filterRustCrateSrc to utils.nixIlan Joselevich2-29/+29
Change-Id: Ib6bae1ea0457d3309a6ec1e08b9e4b320523c161 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11857 Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-06-17 r/8292 chore(tvix): Remove no longer needed libiconv addition for darwinIlan Joselevich1-7/+1
Since https://github.com/NixOS/nixpkgs/pull/220212 libiconv is now automatically added for Darwin. Change-Id: I2aaa266bcbc4d5a8256131aed36b4c2d040a8758 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11856 Tested-by: BuildkiteCI Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com> Reviewed-by: flokli <flokli@flokli.de>
2024-06-17 r/8291 chore(tvix): Use callPackage instead of import on Cargo.nixIlan Joselevich1-4/+1
Since crate2nix v0.11 the new preferred way of importing Cargo.nix is with callPackage instead of import. Change-Id: Ifcc5c21e6c101cb9d0b3f1e8025b5ddb8cf99f20 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11855 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
2024-06-17 r/8290 chore(tvix): Remove no longer needed rlib stripping fix for darwinIlan Joselevich1-12/+0
This issue was fixed in https://github.com/NixOS/nixpkgs/pull/255900 Change-Id: I4f569099e62c4284bdea18a512387735f139e702 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11854 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Ilan Joselevich <personal@ilanjoselevich.com>
2024-06-17 r/8289 feat(tvix/tvix-store): improve progress barsFlorian Klink8-40/+52
Don't show an empty spinner for daemon commands. Move the bar to the right, so the text is better aligned between spinner progress and bar progress styles. Generally, push progress bars a bit more down to the place where we can track progress. This includes adding one in the upload_blob span. Introduce another progress style template for transfers, which interprets the counter as bytes (not just a plain integer), and also a data rate. Use it for here and in the fetching code, and also make the progress bar itself a bit less wide. Change-Id: I15c2ea3d2b24b5186cec19cd3dbd706638497f40 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11845 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Simon Hauser <simon.hauser@helsinki-systems.de>
2024-06-17 r/8288 feat(tvix/store): display progress for NAR calculationFlorian Klink2-2/+14
This is currently still taking a noticeable amount of time, so make sure we show it is happening. Change-Id: I13d18785fbf41ae4479e1ea58d61ece1d7485719 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11847 Reviewed-by: Simon Hauser <simon.hauser@helsinki-systems.de> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-06-17 r/8287 docs(tvix/store/import): update import_path_as_nar_ca docstrFlorian Klink1-4/+5
Change-Id: I697172220efe73b2a215185027df09f815d32627 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11846 Tested-by: BuildkiteCI Reviewed-by: Simon Hauser <simon.hauser@helsinki-systems.de>
2024-06-16 r/8286 refactor(tvix/store): drop FUSE multithread workaround for DarwinFlorian Klink3-8/+27
The underlying issue in macFUSE has been fixed in https://github.com/osxfuse/osxfuse/issues/974. Bump our `macfuse` in nixpkgs to a version containing the fix. This can be removed while our nixpkgs pin is bumped past https://github.com/NixOS/nixpkgs/pull/320197. Change-Id: Ia0e644fb13198e45018b0a218647ef211acf4df1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11824 Tested-by: BuildkiteCI Reviewed-by: Brian Olsen <me@griff.name>
2024-06-16 r/8285 fix(tvix/store/bin): fix shutdown behaviour for FUSEFlorian Klink6-56/+113
Both umounts happening from another process, as well as tvix-store itself calling umount() on FuseDaemon will cause the FUSE worker threads to terminate. So far there was no nice way to wait on these threads to be terminated from multiple places, causing the `tvix-store mount` command to only be terminated if interrupted via ctrl-c, not via an external umount. Update FuseDaemon to use a ThreadPool, which gives us a join primitive over all threads, that can also be called from multiple places. Await on a join() from there to end the program, not the ctrl-c signal handler as it was before. Using FuseDaemon from multiple tasks requires Arc<>-ing both the ThreadPool as well as the inner FuseSession (which also needs to be inside a Mutex if we want to unmount), but now we can clone FuseDaemon around and use it in two places. We could probably also have used an Option and drop the FuseSession after the first umount, but this looks cleaner. Change-Id: Id635ef59b560c111db52ad0b3ca3d12bc7ae28ca Reviewed-on: https://cl.tvl.fyi/c/depot/+/11825 Reviewed-by: Brian Olsen <me@griff.name> Tested-by: BuildkiteCI
2024-06-16 r/8284 feat(castore/directory): add cache combinatorYureka2-0/+144
Change-Id: Ie8850a40e378f6cc0637a85e526fe5b5ed09fcd7 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11710 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: yuka <yuka@yuka.dev>
2024-06-16 r/8283 refactor(castore/directory/objectstore): use new order validation logicYureka1-14/+10
Use the new helper introduced in CL 11708 instead of rolling our own. Change-Id: I292a9bc8baf73a6c75efe784031bcda1835bb645 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11709 Tested-by: BuildkiteCI Autosubmit: yuka <yuka@yuka.dev> Reviewed-by: flokli <flokli@flokli.de>
2024-06-16 r/8282 refactor(castore/directory): separate order logic from ClosureValidatorYureka8-328/+640
ClosureValidator was previously only suitable for a very narrow use case: Validating incoming uploads, which are in leaves-to-root order. This is because the ordering validation was hard-wired into the add() function. This - Re-name ClosureValidator to DirectoryGraph, which is more suitable since it actually stores the Directory structs and is drained in the end. - Move the ordering-related logic to a separate OrderValidator, which can be used independently. - re-write DirectoryGraph to be a general purpose validator which can accept the input in both orders and can be drained in both orders as well. This means the DirectoryGraph and OrderValidator can now serve multiple new purposes: - Validating the incoming closure on the client while downloading. - Validating the incoming closure downloaded in a caching layer from the `far` cache, and re-order it for insertion into the `near` cache. Change-Id: I2b4b226348416912d7a31935bec050e53d911b70 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11708 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: yuka <yuka@yuka.dev>
2024-06-15 r/8281 docs: move Registration to the top; minor rewordtoastal1-17/+24
When trying to follow Review documentation, I got tripped up on the “Gerrit setup” section that was asking about SSH keys since I hadn’t yet gotten an account setup but the registration setion was near the bottom of the document despite being a requirement for setting up Gerrit. Included words specifying SSO, specifies that the TVL account is an alternative method, & commit→patch since you send a patch with the commit. Change-Id: I465c8a79979b368ab01bcebdc7e5ef16e65dd341 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11823 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-06-15 r/8280 feat(tvix/cli,store): add tracy featureFlorian Klink3-2/+11
If compiled with this features, this emits packets compatible with the [Tracy](https://github.com/wolfpld/tracy) format. Change-Id: I330f5d85ab290abe51f2df38dc55464f3ccfc6cd Reviewed-on: https://cl.tvl.fyi/c/depot/+/11815 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-06-15 r/8279 docs(tvix/TODO): dibs on crate2nix WASMIlan Joselevich1-1/+1
Change-Id: Idde9aaa31ecaa9570559ffe3271bc23b7a71133c Reviewed-on: https://cl.tvl.fyi/c/depot/+/11820 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
2024-06-15 r/8278 docs(tvix/TODO): add builtins.path roundtrip for flatFlorian Klink1-0/+9
Change-Id: I8291d2fdfa2ea31ec24c82dcfb447fa9eaa3a961 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11819 Reviewed-by: Connor Brewster <cbrewster@hey.com> Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-06-15 r/8277 docs(tvix/*store): add some internal linksFlorian Klink4-5/+5
We can now cross-link between pages, and it actually works. Change-Id: Id8bd85111672c687118db9adae7f5066f22441f5 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11817 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com>
2024-06-15 r/8276 docs(tvix/TODO): drop builtins:fetchurl itemFlorian Klink1-4/+0
This has been implemented in cl/11786, cl/11790 and cl/10975. This includes support for the `executable` and `unpack` variants of it. Change-Id: I5bcad5b8e86b9fd341219a59ba86660b692e66e8 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11818 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com> Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-06-14 r/8275 feat(tvix/tracing): add tracing-tracy supportFlorian Klink5-69/+1175
This introduces another feature flag, "tracy" to the `tvix-tracing` crate. If enabled (not enabled by default), it'll add an additional layer emitting packets in a format that https://github.com/wolfpld/tracy can display. I had to be a bit tricky with the combinatorial complexity when adding this, but the resulting code still seems manageable. Change-Id: Ica824496728fa276ceae3f7a9754be0166e6558f Reviewed-on: https://cl.tvl.fyi/c/depot/+/10952 Tested-by: BuildkiteCI Reviewed-by: Simon Hauser <simon.hauser@helsinki-systems.de> Reviewed-by: flokli <flokli@flokli.de>
2024-06-14 r/8274 refactor(tvix/tracing): move otlp setup into helper functionFlorian Klink1-67/+78
Having all this in the main control flow makes it a bit hard to read. Moving it into a helper function makes it a bit cleaner. Change-Id: Ibdb739dbd1e013b4f8c4aaf9b036a6bd556a1871 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11814 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Simon Hauser <simon.hauser@helsinki-systems.de> Tested-by: BuildkiteCI
2024-06-14 r/8273 fix(tvix/tracing): reduce the error logs of otlp if collector is offlineSimon Hauser2-12/+15
The problem is that opentelemetry_otlp tonic batch exporter tries to exports if either the `scheduled_delay` or if the `max_export_batch_size` is reached. Per default the `max_export_batch_size` is set to 512 spans, which means that we try to export these spans once that counter is reached. Each export will then try to connect to the exporter (if that not already happening) and will result in a `tcp connect error`. Increasing the max_export_batch_size to 4096 will then ensure that the export only happens if the `scheduled_delay` is met after the 10 seconds. `max_queue_size` is also increased, because `max_export_batch_size` should not be greater than `max_queue_size`, so similar to the default config its set to `max_export_batch_size * 4`. This will reduce the amount of tries to otlp if the collector is not available and otlp enabled. Change-Id: Ic3430006e8a104fa3b34d274678cae55b3620ce9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11791 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Simon Hauser <simon.hauser@helsinki-systems.de>
2024-06-14 r/8272 feat(tvix/tracing): apply EnvFilter to all layersSimon Hauser1-10/+12
Currently we apply the EnvFilter only to the stderr output writer. This didn't affect any other layer, like the otlp layer, causing spans from `h2`, `tokio_util` or other third party crate dependencies to be always sent out via OTLP. This changes that behaviour, applying EnvFilter to all exports, leading to a lot less spans being exported. Change-Id: I9f3a7233e9d0aeaa81fe08914579f0b3c80d134e Reviewed-on: https://cl.tvl.fyi/c/depot/+/11813 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Simon Hauser <simon.hauser@helsinki-systems.de>
2024-06-14 r/8271 feat(tvix/tracing): correctly close otlp on exitSimon Hauser7-65/+242
Provide a new interface for forcing a flush of otlp traces and use this interface to shutdown otlp prior to exiting tvix-store, either if the tool was stopped with a SIGTERM or ended regularly. This also fixes an issue where traces were not even exported if for example we just imported 10 paths and never even emitted more than 256 traces. The implementation uses a mpsc channel so a flush can be done without having to wait for it to complete. If you want to wait for a flush to complete you can provide a oneshot channel which will receive a message once flushing is complete. Because of a otlp bug `force_flush` as well as `shutdown_tracer_provider` need to be executed using `spawn_blocking` otherwise the function will deadlock. See https://github.com/open-telemetry/opentelemetry-rust/issues/1395#issuecomment-1953280335 Change-Id: I0a828391adfb1f72dc8305f62ced8cba0515847c Reviewed-on: https://cl.tvl.fyi/c/depot/+/11803 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: Simon Hauser <simon.hauser@helsinki-systems.de>
2024-06-14 r/8270 chore(tvix/eval): move eval docs to tvix/docsFlorian Klink12-0/+16
Change-Id: I75b33c43456389de6e521b4f0ad46d68bc9e98f6 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11809 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2024-06-14 r/8269 chore(tvix/docs): move [ca]store docs to tvix/docsFlorian Klink7-1/+8
Change-Id: Idd78ffae34b6ea7b93d13de73b98c61a348869fb Reviewed-on: https://cl.tvl.fyi/c/depot/+/11808 Tested-by: BuildkiteCI Reviewed-by: tazjin <tazjin@tvl.su> Autosubmit: flokli <flokli@flokli.de>
2024-06-14 r/8268 feat(tvix/docs): add d2 supportFlorian Klink3-0/+8
This provides a nicer syntax to draw some diagrams. Change-Id: I2ab580e61399a45f01ed422ee29382e860753ffa Reviewed-on: https://cl.tvl.fyi/c/depot/+/11807 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: yuka <yuka@yuka.dev>
2024-06-14 r/8267 feat(tvix/docs): add mdbook-admonishFlorian Klink5-0/+369
This provides a plugin for callouts. It needs to have additional CSS file added to `book.toml`, which can be (re-)generated by `mdbook-admonish install`. See https://github.com/tommilligan/mdbook-admonish/issues/171#issuecomment-2009166079 for more context. Use it by adding a warning one to the architecture document. Change-Id: I75c9a33d00acb603c6da10d3f9ce3485731c1672 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11805 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: yuka <yuka@yuka.dev>
2024-06-13 r/8266 fix(tvix-glue/fetchers/decompression): propagate error with ?Florian Klink1-3/+1
Change-Id: I5e9d4f067f09307cb6f990ed6eca48345e2fb342 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11810 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Brian Olsen <me@griff.name>
2024-06-13 r/8265 fix(tvix/glue/tvix_store_io): also populate input sourcesFlorian Klink1-2/+23
These also need to be present in the input nodes of the BuildRequest. Change-Id: Ie9b957805e42f766002581adc6182a6543c5333b Reviewed-on: https://cl.tvl.fyi/c/depot/+/11802 Reviewed-by: Brian Olsen <me@griff.name> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-06-13 r/8264 chore(tvix/docs): move daemon protocol to its own chapterFlorian Klink2-4/+20
… and add an introductionary page at the root. Change-Id: I89ce69a4221c332b7c0d99ac26b4e26bfd248341 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11806 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Brian Olsen <me@griff.name>
2024-06-13 r/8263 fix(tvix/glue/tvix_store_io): disable concurrent fetches for nowFlorian Klink1-1/+3
We need some shared queue, preventing the same fetches/builds from getting triggered multiple times unnecessarily. Change-Id: I7c4a3c66db558f5cccd66865b170242b758e3e02 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11800 Reviewed-by: aspen <root@gws.fyi> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-06-13 r/8262 feat(tvix/tracing): set release_max_level_debug for tracingFlorian Klink2-3/+3
This allows explicitly opting in to get DEBUG-level log lines, by setting RUST_LOG. It currently also causes traces to be emitted in all cases, so we might do some runtime filtering there too, as discussed in cl/11791. Change-Id: I2865bb06a62465836d63196422f5f734f7165386 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11801 Tested-by: BuildkiteCI Reviewed-by: aspen <root@gws.fyi> Autosubmit: flokli <flokli@flokli.de>
2024-06-13 r/8261 feat(tvix/cli): add toplevel progress spanFlorian Klink4-1/+15
This gives some better feedback something is going on, gives a (named) root for all the progress children we're drawing, and also counts the time we are in eval. Change-Id: Ibe81dcebf0a2b59bb0680da62e206eb5270f9d3c Reviewed-on: https://cl.tvl.fyi/c/depot/+/11798 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: aspen <root@gws.fyi>
2024-06-13 r/8260 fix(tvix/glue/tvix_store_io): distinguish waiting and buildingFlorian Klink1-1/+3
We immediately reported "Building", even though then populated necessary inputs, which looked a bit odd. Make it clear we're still waiting, and update the spinner message once we have all inputs we were waiting for. In the future, we might want to have separate spans for this, so the timer gets reset, but that's something for later. Change-Id: Ic22c9a906d0e7e7179c5ee328162401261efc224 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11799 Reviewed-by: aspen <root@gws.fyi> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-06-13 r/8259 feat(tvix/glue): report progress on all fetches, use progress barsFlorian Klink3-14/+41
This should also report progress on fetches which we couldn't delay until actually having to IO into them, like `builtins.fetchurl` calls without a upfront-provided hash. While at it, upgrade the progress spinners to progress bars, which increment if we know the size of the fetch. Change-Id: Ic3f332286d8bc2177f3d994ba25b165728d4b702 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11797 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: aspen <root@gws.fyi>
2024-06-13 r/8258 fix(tvix/glue/tvix_store_io): use same case for progress messagesFlorian Klink1-1/+1
"Fetching" was uppercase, "building" was lowercase. Let's make this consistent. Change-Id: I11c16f1a7d2057ada4d057e553a4ceaa59597f26 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11796 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-06-12 r/8257 fix(tvix-glue/fetchers): remove unused allow dead codeFlorian Klink1-2/+0
This isn't unused anymore, we can remove the `#![allow(dead_code)]`. Change-Id: Ia2048394a60161fbfd4ac2a2b64d03350526faa4 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11795 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com>
2024-06-12 r/8256 chore(tvix/castore): drop unused indicatif dependencyFlorian Klink3-6/+0
Change-Id: I9fbd32596fbffe09a590a50044a4f1d074a424c3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11794 Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de>
2024-06-12 r/8255 feat(tvix/glue/tvix_store_io): show progress infoFlorian Klink4-5/+20
In `store_path_to_node`, in case we need to build or fetch something, render a progress bar, using the spinner for now. We can upgrade this to a progress *bar* later. Change-Id: I4a7cf5ef8f639076f176af9b39d276be3f37c8ff Reviewed-on: https://cl.tvl.fyi/c/depot/+/11793 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-06-12 r/8254 feat(tvix/glue): support builtin:fetchurlFlorian Klink5-15/+197
nixpkgs calls <nix/fetchurl.nix> during nixpkgs bootstrap. This produces a fake derivation with system = builtin and builder = builtin:fetchurl, and needs to download files from the internet. At the end of the Derivation construction, if we have such a derivation, also synthesize a `Fetch` struct, which we add to the known fetch paths. This will then cause these fetches to be picked up like all other fetches in TvixStoreIO. Change-Id: I72cbca4f85da106b25eda97693a6a6e59911cd57 Reviewed-on: https://cl.tvl.fyi/c/depot/+/10975 Reviewed-by: Connor Brewster <cbrewster@hey.com> Tested-by: BuildkiteCI
2024-06-12 r/8253 feat(ops/users): Add toastal to userstoastal1-0/+5
Change-Id: Ie25e2f1f0b7557be01b6f78142f1a40952988e53 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11792 Autosubmit: tazjin <tazjin@tvl.su> Reviewed-by: tazjin <tazjin@tvl.su> Tested-by: BuildkiteCI
2024-06-12 r/8252 feat(tvix/glue/fetchers): add NAR fetching infrastructureFlorian Klink1-5/+134
The magic <nix/fetchurl.nix> derivation can cause two other types of fetch to happen, one that unpacks NAR files, and another one that puts a file as an executable at the store path root. This adds the necessary enum type and path calculation logic for it to the fetcher code. It also adds code to do the actual NAR fetching. The executable case is still stubbed out. Change-Id: I79103fd58c7e22ad7fde34efa5e2d89cad7d5a0e Reviewed-on: https://cl.tvl.fyi/c/depot/+/11790 Reviewed-by: Connor Brewster <cbrewster@hey.com> Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-11-23T02·48+0000