Background

Our template insights page shows active users and latency per-developer. Latency is a useful metric for template admins as they can identify users who are having a sub-optimal experience. Once identified, the admin could deploy additional clusters, a workspace proxy, or work with the user to improve their networking setup.

While the latency metric is important, a far more common question we get from our users (and customers) is: "how can I tell if people are doing serious work with this template, versus just trying it one?"

In Coder v1, we exposed the session length per-developer in our Metrics UI and API. We already calculate this information in v2 for auto-stop (disconnect when there are no active sessions) as well as metrics UI (we show session length by IDE/app).

This issue refers to "session length" as "time in IDE," meaning the total duration the developer spent connecting over SSH, web terminal, and coder_apps (open websockets). These are all things that keep the workspace alive and prevent an auto-stop.

Product Requirements

Must Have

• A template admin can use Coder's REST API to see session length per developer over a configurable time period. See how this works in our v1 API.

Should Have

• We replace the default sidebar to show the template's top users instead of top user latency. Here's one idea that doesn't show each developers' session time:
  

• The template admin can still switch to see latency per user, or see it further down on the insights page

Nice to Have

• Open to ideas here! cc @BrunoQuaresma @matifali @sharkymark

A note on accuracy & feature vision

While this metric isn't perfect (an engineer can keep an IDE open while not actively using it), this rough approximation will still be miles ahead of what our product provides now in terms of usage metrics. Additionally, in large enterprises Coder is often used with endpoint protection tools (e.g. browser security tools or VDI clients) that prevent windows from being open for a long time for security reasons. Down the road, this data could be augmented by IDE plugins that are used to better determine activity, both for metrics and autostop.

We do not expect this information to be used to micro-manage (the platform admin/template admin is typically not the end user's manager). Instead, this information helps justify that Coder is being actively used, and for serious work, to the rest of the organization, including leadership/stakeholders. Since we can expect that Coder power-users will already use our API and Prometheus metrics for fine-grained data, we can only show a basic "at glance" visualization in the dashboard that demonstrates Coder's top users. I'm totally open to various charts/lists we show in the dashboard