Skip to content

Fix flaky system-tests-firefox job #18644

@flotwig

Description

@flotwig

system-tests-firefox seems to fail frequently on snapshot matches when it takes longer than normal to launch Firefox:

https://app.circleci.com/pipelines/github/cypress-io/cypress/25418/workflows/c6f39a26-4ae6-4c34-a562-730b7d5e9001/jobs/953049

Two ways to fix it:

  1. Support some env var like CYPRESS_CDP_CONNECTION_TIMEOUT in protocol.ts and pass it in system-tests.ts
  2. Filter out the noisy stdout in system-tests.ts like we do for video snapshot errors in FF:
    try {
    if (options.originalTitle) {
    systemTests.snapshot(options.originalTitle, str, { allowSharedSnapshot: true })
    } else {
    systemTests.snapshot(str)
    }
    } catch (err) {
    // firefox has issues with recording video. for now, ignore snapshot diffs that only differ in this error.
    // @see https://github.com/cypress-io/cypress/pull/16731
    if (!(options.browser === 'firefox' && isVideoSnapshotError(err))) {
    throw err
    }
    console.log('(system tests warning) Firefox failed to process the video, but this is being ignored due to known issues with video processing in Firefox.')
    }

Metadata

Metadata

Labels

process: flaky testRelated to test(s) that have flake in our internal tests

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions