Skip to content

feat(cli): add trafficgen command for load testing #7307

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 26 commits into from
May 5, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
220edbf
feat(cli): add trafficgen command for load testing
johnstcn Apr 18, 2023
737b475
skip test for now
johnstcn Apr 27, 2023
9b26587
make fmt
johnstcn Apr 27, 2023
c56d84e
lint
johnstcn Apr 27, 2023
e548892
swap order of waiting for read and write
johnstcn Apr 27, 2023
31ef743
close connection, add output formatting
johnstcn May 2, 2023
fafca95
do what the comment says
johnstcn May 2, 2023
65c6d88
move back under scaletest cmd
johnstcn May 2, 2023
0bfa9f6
integrate with scaletest harness
johnstcn May 2, 2023
da935a2
drain connection async
johnstcn May 3, 2023
5daa526
fix cancellation
johnstcn May 3, 2023
4f165be
handle deadline exceeded in drain
johnstcn May 3, 2023
31fa8be
address PR comments
johnstcn May 3, 2023
0817204
fixup! address PR comments
johnstcn May 3, 2023
a6d7870
ACTUALLY limit traffic instead of just blasting the firehose
johnstcn May 3, 2023
935dcbd
log config
johnstcn May 3, 2023
e2efeff
lint
johnstcn May 3, 2023
b105e67
chore(cli): scaletest: move logic for flushing traces into tracing pr…
johnstcn May 4, 2023
731b4db
remove unnecessary context-based I/O
johnstcn May 4, 2023
9dc28a2
refactor bytes per second to bytes per tick and tick interval
johnstcn May 4, 2023
7b98b35
rename trafficgen -> workspace-traffic
johnstcn May 4, 2023
b9c845f
make gen
johnstcn May 4, 2023
2574a00
use strategy.timeout instead of duration
johnstcn May 4, 2023
516ffa1
rm ctx from countReadWriter
johnstcn May 4, 2023
655d95a
fixup
johnstcn May 5, 2023
ca8b212
Merge remote-tracking branch 'origin/main' into cj/scaletest-trafficgen
johnstcn May 5, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
make gen
  • Loading branch information
johnstcn committed May 4, 2023
commit b9c845f2f0957386a21358b47a259b8851c7af2f
1 change: 1 addition & 0 deletions cli/testdata/coder_scaletest_--help.golden
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ Run a scale test against the Coder API
online. Optionally runs a command inside each
workspace, and connects to the workspace over
WireGuard.
workspace-traffic Generate traffic to scaletest workspaces through coderd

---
Run `coder --help` for a list of global options.
58 changes: 58 additions & 0 deletions cli/testdata/coder_scaletest_workspace-traffic_--help.golden
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
Usage: coder scaletest workspace-traffic [flags]

Generate traffic to scaletest workspaces through coderd

Options
--bytes-per-tick int, $CODER_SCALETEST_WORKSPACE_TRAFFIC_BYTES_PER_TICK (default: 1024)
How much traffic to generate per tick.

--cleanup-concurrency int, $CODER_SCALETEST_CLEANUP_CONCURRENCY (default: 1)
Number of concurrent cleanup jobs to run. 0 means unlimited.

--cleanup-job-timeout duration, $CODER_SCALETEST_CLEANUP_JOB_TIMEOUT (default: 5m)
Timeout per job. Jobs may take longer to complete under higher
concurrency limits.

--cleanup-timeout duration, $CODER_SCALETEST_CLEANUP_TIMEOUT (default: 30m)
Timeout for the entire cleanup run. 0 means unlimited.

--concurrency int, $CODER_SCALETEST_CONCURRENCY (default: 1)
Number of concurrent jobs to run. 0 means unlimited.

--duration duration, $CODER_SCALETEST_WORKSPACE_TRAFFIC_DURATION (default: 10s)
How long to generate traffic for.

--job-timeout duration, $CODER_SCALETEST_JOB_TIMEOUT (default: 5m)
Timeout per job. Jobs may take longer to complete under higher
concurrency limits.

--output string-array, $CODER_SCALETEST_OUTPUTS (default: text)
Output format specs in the format "<format>[:<path>]". Not specifying
a path will default to stdout. Available formats: text, json.

--tick-interval duration, $CODER_SCALETEST_WORKSPACE_TRAFFIC_TICK_INTERVAL (default: 100ms)
How often to send traffic.

--timeout duration, $CODER_SCALETEST_TIMEOUT (default: 30m)
Timeout for the entire test run. 0 means unlimited.

--trace bool, $CODER_SCALETEST_TRACE
Whether application tracing data is collected. It exports to a backend
configured by environment variables. See:
https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/protocol/exporter.md.

--trace-coder bool, $CODER_SCALETEST_TRACE_CODER
Whether opentelemetry traces are sent to Coder. We recommend keeping
this disabled unless we advise you to enable it.

--trace-honeycomb-api-key string, $CODER_SCALETEST_TRACE_HONEYCOMB_API_KEY
Enables trace exporting to Honeycomb.io using the provided API key.

--trace-propagate bool, $CODER_SCALETEST_TRACE_PROPAGATE
Enables trace propagation to the Coder backend, which will be used to
correlate server-side spans with client-side spans. Only enable this
if the server is configured with the exact same tracing configuration
as the client.

---
Run `coder --help` for a list of global options.
1 change: 1 addition & 0 deletions docs/cli/scaletest.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,3 +16,4 @@ coder scaletest
| ------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [<code>cleanup</code>](./scaletest_cleanup.md) | Cleanup scaletest workspaces, then cleanup scaletest users. |
| [<code>create-workspaces</code>](./scaletest_create-workspaces.md) | Creates many users, then creates a workspace for each user and waits for them finish building and fully come online. Optionally runs a command inside each workspace, and connects to the workspace over WireGuard. |
| [<code>workspace-traffic</code>](./scaletest_workspace-traffic.md) | Generate traffic to scaletest workspaces through coderd |
149 changes: 149 additions & 0 deletions docs/cli/scaletest_workspace-traffic.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
<!-- DO NOT EDIT | GENERATED CONTENT -->

# scaletest workspace-traffic

Generate traffic to scaletest workspaces through coderd

## Usage

```console
coder scaletest workspace-traffic [flags]
```

## Options

### --bytes-per-tick

| | |
| ----------- | -------------------------------------------------------------- |
| Type | <code>int</code> |
| Environment | <code>$CODER_SCALETEST_WORKSPACE_TRAFFIC_BYTES_PER_TICK</code> |
| Default | <code>1024</code> |

How much traffic to generate per tick.

### --cleanup-concurrency

| | |
| ----------- | ------------------------------------------------- |
| Type | <code>int</code> |
| Environment | <code>$CODER_SCALETEST_CLEANUP_CONCURRENCY</code> |
| Default | <code>1</code> |

Number of concurrent cleanup jobs to run. 0 means unlimited.

### --cleanup-job-timeout

| | |
| ----------- | ------------------------------------------------- |
| Type | <code>duration</code> |
| Environment | <code>$CODER_SCALETEST_CLEANUP_JOB_TIMEOUT</code> |
| Default | <code>5m</code> |

Timeout per job. Jobs may take longer to complete under higher concurrency limits.

### --cleanup-timeout

| | |
| ----------- | --------------------------------------------- |
| Type | <code>duration</code> |
| Environment | <code>$CODER_SCALETEST_CLEANUP_TIMEOUT</code> |
| Default | <code>30m</code> |

Timeout for the entire cleanup run. 0 means unlimited.

### --concurrency

| | |
| ----------- | ----------------------------------------- |
| Type | <code>int</code> |
| Environment | <code>$CODER_SCALETEST_CONCURRENCY</code> |
| Default | <code>1</code> |

Number of concurrent jobs to run. 0 means unlimited.

### --duration

| | |
| ----------- | -------------------------------------------------------- |
| Type | <code>duration</code> |
| Environment | <code>$CODER_SCALETEST_WORKSPACE_TRAFFIC_DURATION</code> |
| Default | <code>10s</code> |

How long to generate traffic for.

### --job-timeout

| | |
| ----------- | ----------------------------------------- |
| Type | <code>duration</code> |
| Environment | <code>$CODER_SCALETEST_JOB_TIMEOUT</code> |
| Default | <code>5m</code> |

Timeout per job. Jobs may take longer to complete under higher concurrency limits.

### --output

| | |
| ----------- | ------------------------------------- |
| Type | <code>string-array</code> |
| Environment | <code>$CODER_SCALETEST_OUTPUTS</code> |
| Default | <code>text</code> |

Output format specs in the format "<format>[:<path>]". Not specifying a path will default to stdout. Available formats: text, json.

### --tick-interval

| | |
| ----------- | ------------------------------------------------------------- |
| Type | <code>duration</code> |
| Environment | <code>$CODER_SCALETEST_WORKSPACE_TRAFFIC_TICK_INTERVAL</code> |
| Default | <code>100ms</code> |

How often to send traffic.

### --timeout

| | |
| ----------- | ------------------------------------- |
| Type | <code>duration</code> |
| Environment | <code>$CODER_SCALETEST_TIMEOUT</code> |
| Default | <code>30m</code> |

Timeout for the entire test run. 0 means unlimited.

### --trace

| | |
| ----------- | ----------------------------------- |
| Type | <code>bool</code> |
| Environment | <code>$CODER_SCALETEST_TRACE</code> |

Whether application tracing data is collected. It exports to a backend configured by environment variables. See: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/protocol/exporter.md.

### --trace-coder

| | |
| ----------- | ----------------------------------------- |
| Type | <code>bool</code> |
| Environment | <code>$CODER_SCALETEST_TRACE_CODER</code> |

Whether opentelemetry traces are sent to Coder. We recommend keeping this disabled unless we advise you to enable it.

### --trace-honeycomb-api-key

| | |
| ----------- | ----------------------------------------------------- |
| Type | <code>string</code> |
| Environment | <code>$CODER_SCALETEST_TRACE_HONEYCOMB_API_KEY</code> |

Enables trace exporting to Honeycomb.io using the provided API key.

### --trace-propagate

| | |
| ----------- | --------------------------------------------- |
| Type | <code>bool</code> |
| Environment | <code>$CODER_SCALETEST_TRACE_PROPAGATE</code> |

Enables trace propagation to the Coder backend, which will be used to correlate server-side spans with client-side spans. Only enable this if the server is configured with the exact same tracing configuration as the client.
5 changes: 5 additions & 0 deletions docs/manifest.json
Original file line number Diff line number Diff line change
Expand Up @@ -630,6 +630,11 @@
"description": "Creates many users, then creates a workspace for each user and waits for them finish building and fully come online. Optionally runs a command inside each workspace, and connects to the workspace over WireGuard.",
"path": "cli/scaletest_create-workspaces.md"
},
{
"title": "scaletest workspace-traffic",
"description": "Generate traffic to scaletest workspaces through coderd",
"path": "cli/scaletest_workspace-traffic.md"
},
{
"title": "schedule",
"description": "Schedule automated start and stop times for workspaces",
Expand Down