Can you provide a link to the failing test?

If we need to turn off xdist we can set PYTEST_XDIST_VERSION: none in azure-pipelines.yml.

Can you provide a link to the failing test?

If we need to turn off xdist we can set PYTEST_XDIST_VERSION: none in azure-pipelines.yml.

The link to the failing test is here. Indeed, I was just testing if the problem was related with the parallel testing (and, indeed, it is).

I turn off xdist with PYTEST_XDIST_VERSION: 'none' but still need to investigate the failing test.

Ah its the fetch_20newsgroups_vectorized_fxt one. This fails because the fetch_* functions are not threadsafe, so when multiple processes tries to download the dataset, it could break. I think the "simple fix" would be to fetch all the data before pytest runs when we need the network for testing.

Edit: I had solution somewhere in #17553 which I pull out to resolve the issue directly.

I prefer the solution of #19118 (pre-downloading sequentially the datasets to avoid issues with pytest-xdist).

I prefer the solution of #19118 (pre-downloading sequentially the datasets to avoid issues with pytest-xdist).

+1 about the @thomasjpfan solution. Nevertheless, we still need to solve the remaining test failure.

+1 about the @thomasjpfan solution. Nevertheless, we still need to solve the remaining test failure.

Indeed.

There is still a remaining failure in sklearn.metrics.cluster.tests.test_supervised.test_exactly_zero_info_score which looks like a real regression triggered by adjusted_mutual_info_score that deserves investigation.

Closing this PR now that is #19118 merged in favor of a dedicated issue to track the remaining test failure: #19165.