Symfony version(s) affected

6.0

Description

We run our tests in random order (vendor/bin/phpunit --order-by=random) to detect tests that depend on previous tests.

This leads to a problem when deprecations notices are triggered while autoloading classes.

For example:

  1x: Method "ArrayAccess::offsetUnset()" might add "void" as a native return type declaration in the future. Do the same in implementation "GuzzleHttp\Ring\Future\FutureArray" now to avoid errors or add an explicit @return annotation to suppress this message.
    1x in ElasticsearchAnalyzersTest::it_should_analyze_input from Functional\Core\SearchBundle

This error is only reported once.

If you use this together with a baseline, the error is written like this:

{
    "location": "Functional\\Core\\SearchBundle\\ElasticsearchAnalyzersTest::it_should_analyze_input",
    "message": "Method \"ArrayAccess::offsetUnset()\" might add \"void\" as a native return type declaration in the future. Do the same in implementation \"GuzzleHttp\\Ring\\Future\\CompletedFutureArray\" now to avoid errors or add an explicit @return annotation to suppress this message.",
    "count": 1
}

The next time this test suit runs, it's not the ElasticsearchAnalyzersTest that triggers the first autoload, but another test.

Therefore, the location no longer matches.

How to reproduce

• Run tests with vendor/bin/phpunit --order-by=random
• Make sure your test autoload files that trigger a deprecation
• Use SYMFONY_DEPRECATIONS_HELPER=generateBaseline=true&baselineFile=./tests/baseline.json&max[total]=0&max[direct]=0&max[self]=0 and run the tests
• Check that the location in the baseline is the first that that did the autoload
• Now set generateBaseline=false and re-run the test a few times, it will error as soon as your order changes

Possible Solution

Deprecations that are triggered during autoloading should never specify the location key in the baseline.

Additional Context

• [PHPUnitBridge] fail only for "new" deprecations / "baseline" feature #34496
• [PhpUnitBridge] Add ability to set a baseline for deprecation testing #37733