-
Notifications
You must be signed in to change notification settings - Fork 494
Description
Discussed in #2993
Originally posted by xjanin July 8, 2024
Hi,
I've instrumented my application with opentelemetry-cpp (metrics only), and I use tags in my metrics. When testing with a low rate of metrics recording, I see my metrics with the expected tags in the prometheus export on the opentelemetry collector. However, when testing the application with a high rate of requests, and so a high rate of metrics recording, I get metrics aggregated in time series with no tags except "otel_metrics_overflow".
My understanding is that this should happens if the sdk collector has to collect metrics with a tag cardinality superior to 2000 in a collection cycle (by default).
Edit: https://opentelemetry.io/docs/specs/otel/metrics/sdk/#cardinality-limits
However, my tag cardinality doesn't increase with the number of request processed, and in the opentelemetry collector that my application uses, I don't see 2000 time series in the prometheus export.
So my questions are :
- Do I have a wrong understanding of what "otel_metrics_overflow" means ?
- Is there a way of monitoring the cardinality of the metrics produced by my application ?
Thank you and best regards,
Xavier