-
Notifications
You must be signed in to change notification settings - Fork 874
Add prometheus metric for in flight requests #17212
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Consider adding a separate gauge for active websocket connections |
AFAICT there's already code for doing both: coder/coderd/httpmw/prometheus.go Line 25 in d6c034d
requestsConcurrent := factory.NewGauge(prometheus.GaugeOpts{
Namespace: "coderd",
Subsystem: "api",
Name: "concurrent_requests",
Help: "The number of concurrent API requests.",
})
websocketsConcurrent := factory.NewGauge(prometheus.GaugeOpts{
Namespace: "coderd",
Subsystem: "api",
Name: "concurrent_websockets",
Help: "The total number of concurrent API websockets.",
}) used in: coder/coderd/httpmw/prometheus.go Line 77 in d6c034d
// We want to count WebSockets separately.
if httpapi.IsWebsocketUpgrade(r) {
websocketsConcurrent.Inc()
defer websocketsConcurrent.Dec()
dist = websocketsDist
} else {
requestsConcurrent.Inc()
defer requestsConcurrent.Dec()
dist = requestsDist
distOpts = []string{method}
} |
the labeling part seems to be missing, if we think the base is OK I can work on extending this with a path label |
Add a prometheus gauge metric that tracks the amount of in flight requests. Using a middleware, we would increment the gauge, and decrement it with a deferred function.
This metric will be useful to detect strange behavior (like websocket spam) or something, which will be super useful to alert on especially once we have #16904.
If it's easy, this should also label by route pattern e.g.
route="/api/v2/workspaceagents/:id
or something. Don't include IDs in the metrics to avoid ballooning labels.The text was updated successfully, but these errors were encountered: