Skip to content

Insights: Show weekly active users instead of daily #9495

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
5 of 6 tasks
bpmct opened this issue Sep 3, 2023 · 18 comments · Fixed by #9997 or #10036
Closed
5 of 6 tasks

Insights: Show weekly active users instead of daily #9495

bpmct opened this issue Sep 3, 2023 · 18 comments · Fixed by #9997 or #10036
Assignees

Comments

@bpmct
Copy link
Member

bpmct commented Sep 3, 2023

Background

Coder has a "template insights" page and a "deployment insights" page, both of which show the number of daily active users. For the deployment, this is calculated by users who perform any API request (e.g. visiting the dashboard, connecting to workspace, viewing the users list, etc). For the template, this is calculated by any users who establish a connection to their workspace (e.g. SSH, opening code-server, connecting with VS Code Remote).

Both views show DAU (daily active users), which is a good metric, but not the best metric for measuring how many people are actively using a development platform. Here's why:

  • Most people will not connect during weekends, adding large fluctuations to the graph
  • Many developers may not use their workspace every weekday, especially if they have lots of meetings, are managers, or only use Coder for specific projects. However, they may log in multiple times per week or even have a very busy day on a specific day. This is still an activity that is important to admins.
  • It can be easier to measure growth by looking at week-over-week usage versus daily/

Product Requirements

Must have

  • By default, the template insights page shows weekly active users
  • By default, the deployment insights page shows weekly active users

Should have

  • The user can switch between a daily or weekly active users chart on the template insights page
  • The admin can switch between a daily or weekly active users chart on the template insights page

Nice to have

@cdr-bot cdr-bot bot added the feature label Sep 3, 2023
@matifali
Copy link
Member

matifali commented Sep 3, 2023

I agree. WAU is a better indication of usage. If we go with this, our default views will change. The X-axis becomes weeks and we should show at least previous ten weeks. Currently, we are only showing data from the last 30 days.

@BrunoQuaresma
Copy link
Collaborator

In the design and FE part, this is not very challenging but it can be for the BE. cc.: @mafredri

@mafredri
Copy link
Member

mafredri commented Sep 5, 2023

We currently support interval=day, I think this should be quite easily implemented by adding a new interval=week parameter. The query just needs to switch from 1 day to 7 day intervals when this parameter changes. The dashboard can then decide which type of request to make.

I would suggest making the interval selection automatic based on the selected time range, not template creation. There is, however, a question of how to represent/request the data when the user has selected 2 weeks + 2 days, this may be an inherent problem with "intelligent" selection of interval.

I think when the interval is set to week, a user should only be allowed to select increments of 7 days.

@mtojek mtojek self-assigned this Sep 13, 2023
@mtojek
Copy link
Member

mtojek commented Sep 14, 2023

I'm working on this now 👍

I think when the interval is set to week, a user should only be allowed to select increments of 7 days.

I agree, it could be caught as a simple precondition at request validation.

@mtojek
Copy link
Member

mtojek commented Sep 19, 2023

In the design and FE part, this is not very challenging but it can be for the BE. cc.: @mafredri

@BrunoQuaresma interval=week got merged now. Would you mind adjusting the FE side?

@BrunoQuaresma
Copy link
Collaborator

To accommodate this I see the following tasks that can take quite long:

  • Since this weekly control is only related to the active users, we have to move the "date range control" to be specific to each chart.
  • Do we want to keep the "date range picker"? If the user selects an interval of 30 days for example, what should we do? I have some ideas but any of them look very good:
    • Change the interval to daily but it can be unexpected to the user
    • If the number of days is multiple of 7 we can change the interval to weekly but again, it can be unpredictable to the user
  • How do we want to display the labels? Right now we display the day number but there is no standard for weeks. Maybe we could use "1 - 8" but I think it can consume too much horizontal space.
  • What is the default number of weeks?
  • What are going to be the presets for weekly values? Eg. Last 4 weeks, Last 2 weeks, etc.

@matifali
Copy link
Member

matifali commented Sep 25, 2023

@BrunoQuaresma

  1. I suggest we remove the date range picker and provide a default view of the last 10 weeks
  2. We show options to see
    • Last 10 weeks (default)
    • Last 3 months (12 weeks, so close to 10 weeks)
    • Last 6 months
  3. For labels show the start date of each week (it could be Monday or just the start day of our aggregated data of that week). For example, if a user is seeing this on a Thursday, all dates shown on the x-axis will be of previous Thursdays.

@BrunoQuaresma
Copy link
Collaborator

@matifali

For labels show the start date of each week (it could be Monday or just the start day of our aggregated data of that week). For example, if a user is seeing this on a Thursday, all dates shown on the x-axis will be of previous Thursdays.

If it is not the previous Thursday, what should be displayed?

@matifali
Copy link
Member

@BrunoQuaresma, I do not understand. It may also depend on how the stats are aggregated.

  1. If they are being fetched from db already aggregated for each week starting Monday, then we should display the date of each Monday.
  2. If they are being aggregated at runtime as the user navigates the page and aggregation is performed relative to today, then we should display the start date of each aggregation period (each week)

@BrunoQuaresma
Copy link
Collaborator

If they are being fetched from db already aggregated for each week starting Monday, then we should display the date of each Monday.

wondering if it would not make the user think it is about that day specifically

@mafredri
Copy link
Member

We could show Monday for the axis and Monday - Sunday on hover?

@BrunoQuaresma
Copy link
Collaborator

Idk, having multiple Mondays on the x-axis sounds... confusing. Something like "1 - 8" would not be better? Wondering which other apps aggregate data like this so we could use them as reference.

@matifali
Copy link
Member

@BrunoQuaresma We are not showing the date we show the date on that day
So in case we show the date of each Monday
or I suggested before the date of each Thursdays

@BrunoQuaresma
Copy link
Collaborator

I'm very confused about this haha. Can you please mock something on Figma or any other image editor to show me how you are visualizing this? I think it will be just easier this way

@mafredri
Copy link
Member

mafredri commented Sep 26, 2023

To be honest, for me the most logical thing would be to display the start date on the graph axis. E.g. if it's last weeks Monday to this Monday, we'd show 18.9 on the axis. And on hover we can optionally show weekdays (Monday - Sunday), but we'd at least show 18.9 - 24.9, or 18.9 00:00 - 25.9 00:00 if we want to be precise (in the short form we count the end date as inclusive, hence 24 vs 25).

@matifali
Copy link
Member

@BrunoQuaresma something like
image

@mtojek
Copy link
Member

mtojek commented Oct 3, 2023

@BrunoQuaresma posted this on slack:

PS: When adding weekly interval to the insights page, it is taking around 50 seconds ~70 seconds to display the data.

I'm going to take a look at this today.

@mtojek
Copy link
Member

mtojek commented Oct 3, 2023

@BrunoQuaresma I did a bit of QA work around this implementation, and found following problems:

  1. When I select 48 weeks, Coder UI dies.
Screenshot 2023-10-03 at 20 45 53
  1. There is no point in showing the period > 6 months. We prune stats older than 6 months. Look at the horizontal line showing zero usage until August. Note: I know Oct - Aug = 4 months, but the cleanup deadline was 3 months before we changed it.
Screenshot 2023-10-03 at 20 46 41

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
5 participants