Skip to content

feat(cli): add trafficgen command for load testing #7307

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 26 commits into from
May 5, 2023

Conversation

johnstcn
Copy link
Member

@johnstcn johnstcn commented Apr 27, 2023

This PR adds a hidden scaletest trafficgen command for load testing.

  • Initially developed as a standalone command and later integrated into the existing scaletest harness.
  • Opens a ReconnectingPTY connection to each scaletest workspace, and concurrently writes and reads random data to/from the PTY.
  • Payloads are of the form #${RANDOM_ALPHANUMERIC_STRING}, which essentially drops garbage comments in the remote shell, and should not result in any commands being executed.

Future work:

  • Add a Prometheus handler to count bytes read/writen over time.

@johnstcn johnstcn force-pushed the cj/scaletest-trafficgen branch from 5163ed5 to 7fe2e74 Compare May 2, 2023 12:29
Comment on lines +168 to +170
Data string `json:"data,omitempty"`
Height uint16 `json:"height,omitempty"`
Width uint16 `json:"width,omitempty"`
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

review: Added omitempty here to reduce the size of the payload.

r.vscodeSSH(),
r.workspaceAgent(),
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

review: I'm assuming all of this wants to be sorted alphabetically.

Comment on lines -387 to -413
var (
pageNumber = 0
limit = 100
workspaces []codersdk.Workspace
)
for {
page, err := client.Workspaces(ctx, codersdk.WorkspaceFilter{
Name: "scaletest-",
Offset: pageNumber * limit,
Limit: limit,
})
if err != nil {
return xerrors.Errorf("fetch scaletest workspaces page %d: %w", pageNumber, err)
}

pageNumber++
if len(page.Workspaces) == 0 {
break
}

pageWorkspaces := make([]codersdk.Workspace, 0, len(page.Workspaces))
for _, w := range page.Workspaces {
if isScaleTestWorkspace(w) {
pageWorkspaces = append(pageWorkspaces, w)
}
}
workspaces = append(workspaces, pageWorkspaces...)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

review: extracted to function

Comment on lines -444 to -470
pageNumber = 0
limit = 100
var users []codersdk.User
for {
page, err := client.Users(ctx, codersdk.UsersRequest{
Search: "scaletest-",
Pagination: codersdk.Pagination{
Offset: pageNumber * limit,
Limit: limit,
},
})
if err != nil {
return xerrors.Errorf("fetch scaletest users page %d: %w", pageNumber, err)
}

pageNumber++
if len(page.Users) == 0 {
break
}

pageUsers := make([]codersdk.User, 0, len(page.Users))
for _, u := range page.Users {
if isScaleTestUser(u) {
pageUsers = append(pageUsers, u)
}
}
users = append(users, pageUsers...)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

review: extracted to function

@johnstcn johnstcn force-pushed the cj/scaletest-trafficgen branch from 89e9f95 to 4f165be Compare May 3, 2023 12:02
@johnstcn johnstcn requested review from mafredri and spikecurtis May 3, 2023 12:15
@johnstcn johnstcn marked this pull request as ready for review May 3, 2023 12:15
Copy link
Member

@mafredri mafredri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a bit unclear to me why we need the context based read/writes and why we're doing so one character at a time, but other than that the PR looks good. 👍🏻

codersdk.BypassRatelimitHeader: {"true"},
},
},
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: Since header transport isn't implementing close idler, client.HTTPClient.CloseIdleConnections() won't work. This might be a good addition to avoid leaks in tests:

(Or alternatively, implementing type closeIdler interface { CloseIdleConnections() } on headerTransport and doing the client variant.)

defer http.DefaultTransport.(*http.Transport).CloseIdleConnections()

cli/root.go Outdated
r.gitssh(),
r.scaletest(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

scaletest is no longer hidden, even if your trafficgen subcommand is

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Un-hidden!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I meant that coder scaletest is not hidden, so this shouldn't be in the // Hidden section of the list.

Copy link
Contributor

@spikecurtis spikecurtis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

found an outdated comment, but I don't need to review again before merging.

// Duration is the total duration for which to send traffic to the agent.
Duration time.Duration `json:"duration"`

// TicksInterval specifies how many times per second we send traffic.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this comment is outdated; this is now the interval/period, not the frequency.

@johnstcn johnstcn merged commit 08fb9a6 into main May 5, 2023
@johnstcn johnstcn deleted the cj/scaletest-trafficgen branch May 5, 2023 09:35
@github-actions github-actions bot locked and limited conversation to collaborators May 5, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants