From the plan:

Goal

• To observe how Coder handles a 9am autostart rush, at scale.

Infrastructure:

• One Kubernetes cluster in one region.

Implementation:

• Tools:
  • scaletest create-workspaces
    • Requires the ability to set the autostart time of those created workspaces.

Scenarios:

1. Without builds queued.
   • Where the number of provisioners is equal to the number of concurrent builds
2. With builds queued.
   • Where the number of provisioners does exceed the number of concurrent builds

Run sequence:

1. Create users & workspaces.
2. Stop workspaces.
3. Configure a single autostart time in the near future for all workspaces.
4. Wait for all workspaces to have been started.

Key Results:

• How quickly the jobs get created.
• How quickly they get queued.
• How long it takes for all workspaces to be built.
• How long it takes for a workspace to be built (p50/p95/p99).
• Any discrepancies in these results between scenarios.

Other results:

• Coder deployment + database resource utilization (incl. pprofs, Grafana Pyroscope).
• Workspace creation success rate.