Suggestion

🙃 I hate this 🙃

Example failure: https://github.com/typescript-eslint/typescript-eslint/actions/runs/3201746626/jobs/5230076005 in #5762:

  1) Index
       has no accessibility issues detected by aXe:
     CypressError: `cy.visit()` failed trying to load:

http://localhost:3000/

We attempted to make an http request to this URL but the request failed without a response.

We received this error at the network level:

  > Error: ESOCKETTIMEDOUT

These failures are happening pretty consistently now. They're making the end-to-end tests a useless signal for PR changes.

My primary theory is that we're not properly waiting for the Docusaurus server (yarn start) before running tests. Maybe the timeout is N seconds and the server often takes N+1 or more seconds to start.

My second theory is that this isn't the sole fault of any of the testing dependencies in our package.jsons, but rather GitHub Actions having a flaky network layer. And maybe one of those dependencies (Cypress?) not handling GitHub Actions well.

At this point, my steps to try would be:

1. Bumping Cypress to the latest version again
2. If we can identify how to, fixing Docusaurus server start-and-wait-ing
3. Switching to Playwright

For each, if test pass on multiple re-runs, hooray! We're fixed!

Historical context: we have a long trail of going back and forth on test flake and Cypress versions. Previously we opted to pin the Cypress version to 8.3.0 because newer versions were flaky. I would hope that we can upgrade Cypress to the latest eventually. If the issue ends up being our waiting for the Docusaurus server, I'd still want to try bumping Cypress.

1. chore: Bump cypress from 8.3.0 to 9.5.2 #4710
2. fix: revert "chore: Bump cypress from 8.3.0 to 9.5.2", and ignore cypress for now #4740
3. Website tests flake: ESOCKETTIMEDOUT #4739
4. chore: downgraded Cypress to 8.3.0 again #5006
5. test: fix Cypress test flakes #5086