Skip to content

[MRG] Add Windows continuous integration with AppVeyor CI #3363

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 14, 2014

Conversation

ogrisel
Copy link
Member

@ogrisel ogrisel commented Jul 11, 2014

AppVeyor is a system similar to Travis CI that provides a Windows based build environment with MSVC++ Express 2008 and 2010 pre-installed along with the matching Windows SDKs for 64 bit builds. This PR configures how to build scikit-learn, generate .whl and .exe packages for 4 supported target platforms: Python 3.4 & 2.7, each with 32 and 64 bit.

This PR also has a workaround for #3255: some common tests that call fit_transform twice (with a fixed random state) do not deterministic output on 32 bit Python (both 2 and 3) for some estimators that use LAPACK routines like SVD or eigen solvers. Note the numpy / scipy libraries tested here come from Christoph Gohlke's distribution and are therefore linked with MKL. However I also had non-deterministic fit_transform for the same bunch of estimators when I manually built numpy and scipy with OpenBLAS on a Rackspace windows VM a while ago so the problem is not tied to the use of MKL. I therefore decided to catch AssertionError and raise SkipTest instead for this specific test_common check under 32 bit platforms.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.01%) when pulling a62d661 on ogrisel:appveyor-ci into 178d0b6 on scikit-learn:master.

@ogrisel
Copy link
Member Author

ogrisel commented Jul 11, 2014

I forgot to link to a working sample output of this build setup. Here is the report I get when I configured AppVeyor to watch my own fork of the sklearn github repo:

https://ci.appveyor.com/project/ogrisel/scikit-learn

Note that you can click on the artifact tab of each sub-entry of the build matrix to download the generated .whl and .exe packages.

@ogrisel
Copy link
Member Author

ogrisel commented Jul 11, 2014

Once this is merge in master, I will reconfigure https://ci.appveyor.com/project/ogrisel/scikit-learn to point to https://github.com/scikit-learn/scikit-learn . Then existing PRs will have to be rebased on top of master to get properly tested under windows.

Ideally I would also like to backport that to 0.15.X to be able to use AppVeyor to build the binary release artifacts for Windows for scikit-learn 0.15.0.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.01%) when pulling 35c3f74 on ogrisel:appveyor-ci into 1fe371e on scikit-learn:master.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.01%) when pulling 98c9f52 on ogrisel:appveyor-ci into e33b3b8 on scikit-learn:master.

@@ -603,3 +604,18 @@ def check_skip_network():


with_network = with_setup(check_skip_network)


def is_32bit():
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would add this to the __ALL__constant and add a small docstring.

@ogrisel
Copy link
Member Author

ogrisel commented Jul 13, 2014

@arjoly I addressed your comment and decided to disable the 32 bit windows skip to see exactly which estimators are failing. I just tried to run the tests on Python 3.4 32-bit on a Rackspace windows box and could not reproduce the failures any more in 10 consecutive runs. Let see if appveyor is failing or not.

@coveralls
Copy link

Coverage Status

Coverage remained the same when pulling 157b997 on ogrisel:appveyor-ci into 851d914 on scikit-learn:master.

@coveralls
Copy link

Coverage Status

Coverage remained the same when pulling 6ed1e38 on ogrisel:appveyor-ci into 851d914 on scikit-learn:master.

@arjoly
Copy link
Member

arjoly commented Jul 13, 2014

Hm, there is one failed build.

@coveralls
Copy link

Coverage Status

Coverage remained the same when pulling ba7cb6a on ogrisel:appveyor-ci into fbfcdc5 on scikit-learn:master.

@ogrisel
Copy link
Member Author

ogrisel commented Jul 13, 2014

Yes I am fixing a bunch of other heisen tests before tackling #3255.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.0%) when pulling 6477ade on ogrisel:appveyor-ci into 514f073 on scikit-learn:master.

@ogrisel
Copy link
Member Author

ogrisel commented Jul 14, 2014

I rebased on top of the current master where I pushed some fixes for other tests and rewrote this PR to only skip the fit_transform check for 3 specific transformers with Python 32 bit under Windows.

# To avoid spurious builds and notification as long as not merged in master
branches:
only:
- appveyor-ci
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about we remove this for the merge!

@GaelVaroquaux
Copy link
Member

👍 to merge with these 2 comments, if the tests pass.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.0%) when pulling 1b296fd on ogrisel:appveyor-ci into d8a8a8d on scikit-learn:master.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.0%) when pulling dbcc3ff on ogrisel:appveyor-ci into d8a8a8d on scikit-learn:master.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.0%) when pulling 16dae39 on ogrisel:appveyor-ci into d8a8a8d on scikit-learn:master.

@@ -0,0 +1,6 @@
--find-links http://28daf2247a33ed269873-7b1aad3fab3cc330e1fd9d109892382a.r6.cf2.rackcdn.com/index.html
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this scary-looking url?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a temporary wheelhouse we host on the rackspace cloud account of the sklearn project to host numpy and scipy wheels built from Christoph Gohlke's installers. I will add an inline comment.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/we ca delete/we can delete
Thanks for the clarification. The note about how they're obtained is definitely useful.

@coveralls
Copy link

Coverage Status

Changes Unknown when pulling 1481e53 on ogrisel:appveyor-ci into * on scikit-learn:master*.

@ogrisel
Copy link
Member Author

ogrisel commented Jul 14, 2014

I rebased on master and it's all green (both appveyor and travis, although travis should not be affected).

@GaelVaroquaux
Copy link
Member

OK, merging this guy.

GaelVaroquaux added a commit that referenced this pull request Jul 14, 2014
[MRG] Add Windows continuous integration with AppVeyor CI
@GaelVaroquaux GaelVaroquaux merged commit a9ad5a2 into scikit-learn:master Jul 14, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants