feat(scaletest/dashboard): integrate chromedp #9927
Conversation
This commit adds a set of actions to automatically interact with a Coder instance using chromedp.
This commit integrates the chromedp actions in the previous commit into the scaletest dashboard command, and re-enables the previously disabled unit test. Note that this unit test does not actually run headless chrome, nor does it test the actual scaletest actions yet, as coderdtest only exposes an API and does not expose the actual site.
| { | ||
| Label: "template_files", | ||
| ClickOn: `a[href^="/templates/"][href$="/docs"]:not([aria-current])`, | ||
| WaitFor: `.monaco-editor`, |
There was a problem hiding this comment.
nit: should we wrap with an extra coder-editor div or .monaco-editor introduced by us?
There was a problem hiding this comment.
I'm not sure, it seems to work for the moment?
Under the hood, this probably does a very similar thing to what Playwright does. |
cli/exp_scaletest.go
Outdated
| Description: "Minimum wait between fetches.", | ||
| Value: clibase.DurationOf(&minWait), | ||
| }, | ||
| { | ||
| Flag: "max-wait", | ||
| Env: "CODER_SCALETEST_DASHBOARD_MAX_WAIT", | ||
| Default: "1s", | ||
| Default: "10s", |
There was a problem hiding this comment.
Any chance we may be able to eliminate these options? It'd be nice if you didn't need to adjust these parameters to have a working load test, and that it'd continue working even if coderd was under too much load, etc.
We could instead track user experience impact when things slow down to a crawl.
There was a problem hiding this comment.
This more just controls the interval between actions. Between 1 and 10 seconds I think is a reasonable default. I'll rename it to better reflect its semantics.
| @@ -244,6 +244,9 @@ require ( | |||
| github.com/bep/godartsass/v2 v2.0.0 // indirect | |||
| github.com/bep/golibsass v1.1.1 // indirect | |||
| github.com/cespare/xxhash/v2 v2.2.0 // indirect | |||
| github.com/chromedp/cdproto v0.0.0-20230802225258-3cf4e6d46a89 // indirect | |||
| github.com/chromedp/chromedp v0.9.2 // indirect | |||
| github.com/chromedp/sysutil v1.0.0 // indirect | |||
There was a problem hiding this comment.
These new imports add 5MB to the slim binary, perhaps we need to consider breaking off exp scaletest from slim 🤔
There was a problem hiding this comment.
I can do this here 👍
Edit: better done in a follow-up PR
Fixes #9131
This PR replaces the existing implementation of dashboard load test generation with chromedp.
The 10,000 foot view:
Notes/future work:
I initially tried to make a tight "state machine" and model the valid transitions from one page to another. It quickly became evident that this would break quickly, so I refactored into the more "stateless" form that you see here.
There is definitely some room for more "intelligent" choice of elements to click, as opposed to completely random.
Edit: you can specify a seed with
--rand-seedto get a deterministic test.