This isn't working quickly. Strangely getting 403 just doing a basic Python run on Travis.

We should probably be using https://github.com/matthew-brett/multibuild.git

We're unable to detect it :) The kmeans mac failure is due to #12648

@jnothman We're able to reproduce the error if we only update scipy from 0.17.0 to 1.1.0, so seems that it's indeed a scipy issue.

@jnothman We're able to reproduce the error if we only update scipy from 0.17.0 to 1.1.0, so seems that it's indeed a scipy issue.

Thanks for checking that while I was out for lunch

Now, do we disable the fit_predict tests and add this to our regular CI?

Now, do we disable the fit_predict tests and add this to our regular CI?

merge #12648 to keep consistent with 0.20.X branch? I think we need this if we want to add mac CI.

kmeans tests pass, seems that we don't need #12648 :)

DictionaryLearning docstring fails due to a different sign in the second row of output now.

So somehow they pass with latest scipy??

scipy 1.2.0 wheels seems to include openblas. scipy 1.1.0 do not (I checked the contents of the .dylibs folder of the wheel archive with unzip). Therefore scipy 1.1.0 would use the macOS accelerate system BLAS which might be the source of this non-deterministic behavior although that seems weird.

@jeremiedbb any opinion?

scipy 1.2.0 wheels seems to include openblas. scipy 1.1.0 do not (I checked the contents of the .dylibs folder of the wheel archive with unzip). Therefore scipy 1.1.0 would use the macOS accelerate system BLAS which might be the source of this non-deterministic behavior although that seems weird.

Actually, travis uses anaconda on the macOS build so this is unrelated. However the version of the travis build with the PCA test that fails runs openblas 0.2.20 while the latest build that does not have this failure uses openblas 0.3.3 so it might indeed be a numerical precision issue in old versions of openblas.

My first though was that the test was unstable due to the way inertia is computed, which can lead to different inertia for only a permutation of the labels. But I tried with many seeds and the test seems stable.

I Also tried with OpenBLAS 0.2.20 and OpenBLAS 0.3.3 and it's stable with both, so I does not seem to be blas instabilities either...

DictionaryLearning docstring fails due to a different sign in the second row of output now.

The DictionaryLearning doctest was reverted in master. I merged master into this branch to check whether CI can pass now.

Is it possible to remove the 3rd Travis job?
Maybe it's better to put the mac job at the end.

@qinhanmin2014 I pushed a new commit with your suggestions. Also I think it's better to test the anaconda mac setup with MKL as most anaconda users on mac will probably use MKL.

I have no idea why the previous mac build is marked failed by travis:

https://travis-ci.org/scikit-learn/scikit-learn/jobs/469979217

Here is the end of the log that shows a success:

[...]
==== 10538 passed, 116 skipped, 3 xfailed, 10004 warnings in 759.59 seconds ====
The command "bash build_tools/travis/test_script.sh" exited with 0.
[...]
============= 36 passed, 4 skipped, 100 warnings in 21.60 seconds ==============
The command "bash build_tools/travis/test_docs.sh" exited with 0.
0.01s$ bash build_tools/travis/test_pytest_soft_dependency.sh
The command "bash build_tools/travis/test_pytest_soft_dependency.sh" exited with 0.
cache.2
store build cache
0.01s3.62schanges detected, packing new archive
uploading PR.12824/cache-osx-1660d3122ceb6c19de16ed2ec2206da034564d90d940bc6b41e622063713f077.tgz
cache uploaded
$ source build_tools/travis/after_success.sh

Thanks @jnothman. The cd command in zsh has an issue on travis under mac (there some kind of rvm-related zsh callback that fails silently). I have pushed a workaround. I will remove the -x now.

Because the mac build tends to be a little bit slower than the linux build, maybe we should put it at the top of the list to maximize effective parallelization and limiting the straggler effect. WDYT?

Alright the macOS build seems stable enough. Let's merge.