• Adds httpapi.Duration which wraps time.Duration with sensible JSON marshaling and unmarshaling
• Adds loadtest/harness package with tests which is the overall runner of a collection of tests
  • Concurrency can be controlled by using a different execution strategy
• Adds loadtest/placebo which has an implementation of harness.Runnable that sleeps for a given duration with jitter and failure chance
• Adds loadtest/workspacebuild which has an implementation of harness.Runnable that builds a workspace, waits for it to become available
  • Also implements harness.Cleanable to delete the workspace on test completion
• Adds hidden coder loadtest subcommand which is only available to the site owner
  • Configured by a JSON file (accepts STDIN as well)
  • Runs any variety of tests with any given concurrency settings
  • Prints error message and logs for all failed test runs
  • Cleans up afterwards

TODO (in future PRs):

• Add a loadtest for workspace connections via STUN or DERP
• Better output formatting, including HTML and JSON output
• Live progress/failure output instead of only outputting information at the end
• Github actions workflows for periodic load testing in different configurations
• Documentation, json schema and unhidden loadtest command

{
  "strategy": {
    "type": "concurrent"
  },
  "tests": [
    {
      "type": "placebo",
      "count": 500,
      "placebo": {
        "sleep": "1s",
        "failure_chance": 0.005
      }
    }
  ],
  "timeout": "1m"
}

$ coder loadtest --config ./loadtest.config.json
Running load test...

== FAIL: placebo-0/322

        Error: test failed due to configured failure chance

        Log:
                sleeping for 1s
                failure chance is 0.005000
                rolling the dice of fate...
                rolled: 0.001123
                :(


Test results:
        Pass:  499
        Fail:  1
        Total: 500

        Total duration: 5.003371453s
        Avg. duration:  1.000593179s

Cleaning up...