Skip to content

batch coder agent metadata #9782

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
johnstcn opened this issue Sep 19, 2023 · 2 comments · Fixed by #10225
Closed

batch coder agent metadata #9782

johnstcn opened this issue Sep 19, 2023 · 2 comments · Fixed by #10225
Assignees
@mafredri
Labels
api Area: HTTP API s4 Internal bugs (e.g. test flakes), extreme edge cases, and bug risks

Comments

@johnstcn
Copy link
Member

We currently do not batch agent metadata. This causes a fair number of requests and DB load. Our documentation even has a formula for how much DB load a number of metadata would generate.

You can expect (10 * 6 * 2) / 4 or 30 writes per second.

There are a couple of ways we could optimize this:

  • Batch metadata on the agent side and only send them when there is new data
  • Batch metadata on the coderd side (similar to what we do for agent stats)
@johnstcn johnstcn added api Area: HTTP API bug labels Sep 19, 2023
@johnstcn
Copy link
Member Author

This is what 400 workspaces submitting 2 pieces of agent metadata every 10 seconds looks like:

image

This is curently enough to saturate a db-custom-1-3840 instance.

@johnstcn
Copy link
Member Author

Also noticing we end up duplicating a GetWorkspaceByAgentID query for some reason.

image

@matifali matifali added the s4 Internal bugs (e.g. test flakes), extreme edge cases, and bug risks label Oct 2, 2023
This was referenced Oct 11, 2023
Merged
Merged
Merged
mafredri added a commit that referenced this issue Oct 13, 2023
@mafredri
feat(coderd): add support for sending batched agent metadata (#10223)
7eeba15 
Part of #9782
mafredri added a commit that referenced this issue Oct 13, 2023
@mafredri
feat(codersdk/agentsdk): use new agent metadata batch endpoint (#10224)
4857d4b 
Part of #9782
mafredri added a commit that referenced this issue Oct 13, 2023
@mafredri
fix(agent): send metadata in batches (#10225)
76c65b1 
Fixes #9782

---

I recommend reviewing with ignore whitespace.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mafredri mafredri

Labels
api Area: HTTP API s4 Internal bugs (e.g. test flakes), extreme edge cases, and bug risks
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(agent): send metadata in batches coder/coder
3 participants
@mafredri @johnstcn @matifali