Skip to content

batch insert workspace agent stat updates #8063

@johnstcn

Description

@johnstcn

Problem

Each running workspace agent reports stats every 30 seconds by default via POST to /api/v2/workspaceagents/me/report-stats. This handler synchronously persists this statistic to the database.

With large numbers of agents, this can potentially cause large numbers of transactions. With 10,000 workspace agents, this would equate to roughly 333.4 requests per second, each of which would kick off a database transaction.

Proposed Solution

We can reduce this overhead by batching updates to workspace agent stats.

  • Alter InsertWorkspaceAgentStat to persist multiple rows
  • Store a batch of workspace agent stats with a limited size (e.g. 10)
  • Insert the current batch every batch interval (e.g. 1s) or once the current batch reaches capacity, whichever happens first.
  • It may be prudent to add a limited retry on batch insertion in order to avoid dropping stats.

Metadata

Metadata

Assignees

Labels

scaletestIssues related to scale testing.

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions