about summary refs log tree commit diff
path: root/tvix/tracing/src
AgeCommit message (Collapse)AuthorFilesLines
2024-11-07 r/8895 feat(tvix/tracing): Allow configuring additional layersVova Kryachko1-49/+61
This will be used by tvix-daemon to write tracing data into the active client's connection socket. Change-Id: I8889dd0a638e004ee2c8cb312946b029c9779313 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12734 Tested-by: BuildkiteCI Autosubmit: Vladimir Kryachko <v.kryachko@gmail.com> Reviewed-by: flokli <flokli@flokli.de>
2024-11-01 r/8879 fix(tvix/tracing): Enable tracing in tvix-build.Vova Kryachko1-0/+1
Change-Id: Ica9734d7774da4d5e2a5cacab6b9feb27becc3e2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12720 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-10-30 r/8877 refactor(tvix/tracing): drop log_level methodFlorian Klink1-24/+4
There's no more consumers left. Change-Id: I0585abbdbe0ffcf35cd20ac58077ade67cbf5c75 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12715 Reviewed-by: Ilan Joselevich <personal@ilanjoselevich.com> Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de>
2024-10-27 r/8863 feat(tvix/tracing): filter indicatif.pb_show fields in all casesFlorian Klink1-1/+7
Change-Id: I688cec79cefeae340872945c35743fae892cd053 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12688 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: yuka <yuka@yuka.dev> Reviewed-by: flokli <flokli@flokli.de>
2024-10-13 r/8805 refactor(tvix/tracing): remove use of lazy_staticFlorian Klink1-11/+15
This is now supported in the standard library via std::sync::LazyLock, but requires some manual shuffling around of code. Change-Id: I14bee4068dc73c948321481b5a4e1fc922a89a27 Reviewed-on: https://cl.tvl.fyi/c/depot/+/12611 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: tazjin <tazjin@tvl.su>
2024-07-21 r/8385 chore(tvix): upgrade to tonic 0.12 / hyper 1.0Yureka1-6/+7
Change-Id: Idd8ce48869ddd869d51a10959b920f1290a8a9b3 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11991 Autosubmit: yuka <yuka@yuka.dev> Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-07-21 r/8384 feat(tvix/tracing): http propagation for axumSimon Hauser2-3/+50
It introduces a new accept_trace function for axum0.7 which can be used to accept a header trace from a received request. This function can be used for tonic 0.12 once that version is released, and the specific `accept_trace` function within `tvix_tracing::propagate::tonic` can then be removed. This also integrates http propagation into the nar_bridge crate. Change-Id: I46dcc797d494bb3977c2633753e7060d88d29129 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11925 Reviewed-by: Brian Olsen <me@griff.name> Tested-by: BuildkiteCI Reviewed-by: Simon Hauser <simon.hauser@helsinki-systems.de> Reviewed-by: flokli <flokli@flokli.de>
2024-07-02 r/8339 feat(tvix/tracing): http trace propagationSimon Hauser3-3/+15
Introduces a helper function within tvix-tracing that returns a reqwest tracing middleware that will ingest the traceparent if otlp is enabled. It is feature flagged in tvix-tracing so not every consumer of that library automatically has reqwest in its dependencies. Tested using netcat to verify that the `traceparent` header is there if otlp is enabled and missing if otlp feature is disabled. Change-Id: I5abccae777b725f5ff7382e3686165383c477a39 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11886 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-07-01 r/8333 feat(tvix/tracing): expose stdout_writer and stderr_writerSimon Hauser1-3/+34
Using std::io::{Stdout,StdErr} directly will clobber the output by an active progress bar. To resolve this issue the exposed writers should be prefered over `println!` and `eprintln!`. Change-Id: Ic79465cd4e8b9dad5a138f6b08c5f0de9dcf54a1 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11860 Autosubmit: Simon Hauser <simon.hauser@helsinki-systems.de> Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de>
2024-06-26 r/8310 fix(tvix/tracing): make cargo check and clippy happyFlorian Klink1-10/+8
In case the otlp feature is not enabled, these generate warnings during `cargo check`. Fix by moving some imports into their functions, or using the fully-qualified name (and one #[allow(unused_mut)]) Change-Id: I5afd89dcd4c772b6002cebdd5d0469932eacfdac Reviewed-on: https://cl.tvl.fyi/c/depot/+/11873 Autosubmit: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Reviewed-by: Connor Brewster <cbrewster@hey.com> Reviewed-by: Simon Hauser <simon.hauser@helsinki-systems.de>
2024-06-20 r/8299 feat(tvix/tracing): gRPC trace context propagationSimon Hauser3-0/+75
This introduces optional helper function in tvix/tracing for trace propagation and uses these helper in the `tvix-store`. The GRPCBlobService, GRPCDirectoryService and GRPCPathInfoService now accept a generic client, meaning the client can be generated with either `::new` or `::with_interceptor`. This was tested and validated by starting a `tvix-store daemon` and `tvix-store import`. Change-Id: I4b194483bf09266820104b4b56e4a135dca2b77a Reviewed-on: https://cl.tvl.fyi/c/depot/+/11863 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI
2024-06-20 r/8297 feat(tvix/tracing): optional progressbarSimon Hauser1-4/+14
Disable the progressbar on default and provide a interface for optionally enabling the progressbar. Change-Id: I0e31b1957e80cf64a8dcf65c6ceb3713975b8220 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11861 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Simon Hauser <simon.hauser@helsinki-systems.de>
2024-06-17 r/8289 feat(tvix/tvix-store): improve progress barsFlorian Klink1-2/+6
Don't show an empty spinner for daemon commands. Move the bar to the right, so the text is better aligned between spinner progress and bar progress styles. Generally, push progress bars a bit more down to the place where we can track progress. This includes adding one in the upload_blob span. Introduce another progress style template for transfers, which interprets the counter as bytes (not just a plain integer), and also a data rate. Use it for here and in the fetching code, and also make the progress bar itself a bit less wide. Change-Id: I15c2ea3d2b24b5186cec19cd3dbd706638497f40 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11845 Tested-by: BuildkiteCI Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Simon Hauser <simon.hauser@helsinki-systems.de>
2024-06-14 r/8275 feat(tvix/tracing): add tracing-tracy supportFlorian Klink1-2/+23
This introduces another feature flag, "tracy" to the `tvix-tracing` crate. If enabled (not enabled by default), it'll add an additional layer emitting packets in a format that https://github.com/wolfpld/tracy can display. I had to be a bit tricky with the combinatorial complexity when adding this, but the resulting code still seems manageable. Change-Id: Ica824496728fa276ceae3f7a9754be0166e6558f Reviewed-on: https://cl.tvl.fyi/c/depot/+/10952 Tested-by: BuildkiteCI Reviewed-by: Simon Hauser <simon.hauser@helsinki-systems.de> Reviewed-by: flokli <flokli@flokli.de>
2024-06-14 r/8274 refactor(tvix/tracing): move otlp setup into helper functionFlorian Klink1-67/+78
Having all this in the main control flow makes it a bit hard to read. Moving it into a helper function makes it a bit cleaner. Change-Id: Ibdb739dbd1e013b4f8c4aaf9b036a6bd556a1871 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11814 Autosubmit: flokli <flokli@flokli.de> Reviewed-by: Simon Hauser <simon.hauser@helsinki-systems.de> Tested-by: BuildkiteCI
2024-06-14 r/8273 fix(tvix/tracing): reduce the error logs of otlp if collector is offlineSimon Hauser1-2/+15
The problem is that opentelemetry_otlp tonic batch exporter tries to exports if either the `scheduled_delay` or if the `max_export_batch_size` is reached. Per default the `max_export_batch_size` is set to 512 spans, which means that we try to export these spans once that counter is reached. Each export will then try to connect to the exporter (if that not already happening) and will result in a `tcp connect error`. Increasing the max_export_batch_size to 4096 will then ensure that the export only happens if the `scheduled_delay` is met after the 10 seconds. `max_queue_size` is also increased, because `max_export_batch_size` should not be greater than `max_queue_size`, so similar to the default config its set to `max_export_batch_size * 4`. This will reduce the amount of tries to otlp if the collector is not available and otlp enabled. Change-Id: Ic3430006e8a104fa3b34d274678cae55b3620ce9 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11791 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Simon Hauser <simon.hauser@helsinki-systems.de>
2024-06-14 r/8272 feat(tvix/tracing): apply EnvFilter to all layersSimon Hauser1-10/+12
Currently we apply the EnvFilter only to the stderr output writer. This didn't affect any other layer, like the otlp layer, causing spans from `h2`, `tokio_util` or other third party crate dependencies to be always sent out via OTLP. This changes that behaviour, applying EnvFilter to all exports, leading to a lot less spans being exported. Change-Id: I9f3a7233e9d0aeaa81fe08914579f0b3c80d134e Reviewed-on: https://cl.tvl.fyi/c/depot/+/11813 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Simon Hauser <simon.hauser@helsinki-systems.de>
2024-06-14 r/8271 feat(tvix/tracing): correctly close otlp on exitSimon Hauser1-44/+188
Provide a new interface for forcing a flush of otlp traces and use this interface to shutdown otlp prior to exiting tvix-store, either if the tool was stopped with a SIGTERM or ended regularly. This also fixes an issue where traces were not even exported if for example we just imported 10 paths and never even emitted more than 256 traces. The implementation uses a mpsc channel so a flush can be done without having to wait for it to complete. If you want to wait for a flush to complete you can provide a oneshot channel which will receive a message once flushing is complete. Because of a otlp bug `force_flush` as well as `shutdown_tracer_provider` need to be executed using `spawn_blocking` otherwise the function will deadlock. See https://github.com/open-telemetry/opentelemetry-rust/issues/1395#issuecomment-1953280335 Change-Id: I0a828391adfb1f72dc8305f62ced8cba0515847c Reviewed-on: https://cl.tvl.fyi/c/depot/+/11803 Reviewed-by: flokli <flokli@flokli.de> Tested-by: BuildkiteCI Autosubmit: Simon Hauser <simon.hauser@helsinki-systems.de>
2024-06-10 r/8242 feat(tvix/tracing): introduce common tvix-tracing crateSimon Hauser1-0/+90
Introduce a new common crate that contains tracing boilerplate which then can be used in the cli, tvix-store and tvix-build crates. It has otlp as an optional feature, which is currently only used by tvix-store. Change-Id: I41468ac4d9c65174515d721513b96fea463d6ed2 Reviewed-on: https://cl.tvl.fyi/c/depot/+/11758 Tested-by: BuildkiteCI Reviewed-by: flokli <flokli@flokli.de> Autosubmit: Simon Hauser <simon.hauser@helsinki-systems.de>
generated by cgit-pink 1.4.1 (git 2.44.2) at 2024-12-26T03·16+0000