How does this work with multiple Coderd replicas?

Similar to other collectors like in prometheusmetrics.go. Technically, you can't sum up this data.

Do you see any workaround or a better solution we can adopt, @spikecurtis?

Technically, you can't sum up this data.

It's probably fine that you can't sum as long as other aggregations give sensible results like max or avg. Whether this is true depends on how the metrics are computed.

For example, if a metric is computed entirely from "deployment"-wide state, like a database query, such that one can reasonably expect that all coderds will compute and expose the same value, then that's probably fine.

However, if a metric is computed from state that is local to each coderd, then we need to understand how the different values interact such that there is no under- or over-counting.

For example, if a metric is computed entirely from "deployment"-wide state, like a database query, such that one can reasonably expect that all coderds will compute and expose the same value, then that's probably fine.

This is the only use case we have. Query a database, transform results, and expose them as metrics. One can argue that the same query is executed N times, where N is the number of replicas.

Is prometheus the right tool for user session data?

Is prometheus the right tool for user session data?

What do you mean by data? Personal data like emails or IDs? I think we're already exposing them through the Prometheus channel.

BTW I copied this idea from the Sprint theme in the notion doc. Isn't that what you suggested? Alternatively, we can focus on generic stats first like "total active users per template", "used parameters with values", etc.

Wait, are these metrics per user, or are they aggregates like DAUs?

I wouldn't expect someone to go to prometheus to answer a question like, "how much has Marcin used Coder this last month"?

I will go with the "safe" metrics first, and leave user-related for later 👍

Battle plan:

• feat: show user limit on active users chart #10101 Show license seat limit on dashboard
• feat: expose user seat limits as Prometheus metrics #10169 Global metrics:
  • now: License utilisation + license limit
• Per Template - "last 5 minutes"
  • feat: expose template insights as Prometheus metrics #10325 Active Users counter
  • feat: expose app insights as Prometheus metrics #10346 App IDE usage - app name, total time
  • feat: expose parameter insights as Prometheus metrics #10574 Parameter usage - parameter name, parameter value, count