Let me try to push a follow-up debug commit that I plan to undo so check whether we can reproduce the problem on the CI with a large number of seeds.

2 of the ARM runs failed on travis because the debug test with 1000 random seeds latested more than 10 minutes without output.

The latest failure https://app.travis-ci.com/github/scikit-learn/scikit-learn/jobs/521780414 seems unrelated with many numpy.linalg.LinAlgError: Eigenvalues did not converge on other tests (which is worrying but not in the scope of this PR...).

The macos builds failed with:

wget https://homebrew.bintray.com/bottles/libomp-11.0.0.high_sierra.bottle.tar.gz
--2021-07-05 12:40:00--  https://homebrew.bintray.com/bottles/libomp-11.0.0.high_sierra.bottle.tar.gz
Resolving homebrew.bintray.com (homebrew.bintray.com)... 52.88.131.165, 34.210.204.177
Connecting to homebrew.bintray.com (homebrew.bintray.com)|52.88.131.165|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2021-07-05 12:40:01 ERROR 403: Forbidden.

which is also annoying but also not related to this PR.

The linux 32 bit python builds failed for some seeds such as:

  seed = 95
  
      @pytest.mark.parametrize("seed", range(1000))
      def test_multi_target_regression(seed):
          X, y = datasets.make_regression(n_targets=3, random_state=seed)
          X_train, y_train = X[:50], y[:50]
          X_test, y_test = X[50:], y[50:]
      
          references = np.zeros_like(y_test)
          for n in range(3):
              rgr = GradientBoostingRegressor(random_state=0)
              rgr.fit(X_train, y_train[:, n])
              references[:, n] = rgr.predict(X_test)
      
          rgr = MultiOutputRegressor(GradientBoostingRegressor(random_state=0))
          rgr.fit(X_train, y_train)
          y_pred = rgr.predict(X_test)
      
  >       assert_almost_equal(references, y_pred)
  E       AssertionError: 
  E       Arrays are not almost equal to 7 decimals
  E       
  E       Mismatched elements: 2 / 150 (1.33%)
  E       Max absolute difference: 0.03136504
  E       Max relative difference: 0.00051924
  E        x: array([[ -72.3890004,  157.8077157,   34.6056844],
  E              [  -4.8907783,   40.3488598,    3.7400021],
  E              [  46.3881095, -160.8625623,    3.3398913],...
  E        y: array([[ -72.3890004,  157.8077157,   34.6056844],
  E              [  -4.8907783,   40.3488598,    3.7400021],
  E              [  46.3881095, -160.8625623,    3.3398913],...

but the seeds with failures are not the same for different builds with different versions for the dependencies (Python, numpy...).

So maybe there are datasets that generate splits with near machine precision ties that cause rare platform specific behaviors...

I am not sure why this problem would impact 32 bit Python more than 64 bit Python though...

I reverted the debug commit to make it possible to merge this PR even if we do not understand the precise cause of unstable test.

Maybe related to: #8853

We could use another estimator than the GradientBoostingRegressor.

We could use another estimator than the GradientBoostingRegressor.

Fixing the random seed is enough for now. But it would still be interested to investigate #8853 in more details.

And it could also be linked to #12259 where we pick up a random feature with some tie-breaking even if the random_state is fixed if I recall properly.

Merging. We would need to further investigate the underlying cause and the other failures.