Skip to content

MNT Add more architectur in CRON job in Travis #15650

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

xnox
Copy link

@xnox xnox commented Nov 18, 2019

Enable arm64, ppc64le (little endian power9), s390x (mainframe)

@xnox
Copy link
Author

xnox commented Nov 18, 2019

Hm, so conda exists on ppc64le but scripts need to be tweaked to use that.

And there is no conda for s390x/arm64, as if we need a new "ports-arch" python tests.

@rth
Copy link
Member

rth commented Nov 19, 2019

Thanks, interesting, I wasn't aware Travis supported those!

Not sure about running them at each build, but we could certainly run them in a cron job (or at least some of these arch) cc @glemaitre

And there is no conda for s390x/arm64, as if we need a new "ports-arch" python tests.

Hmm, if we have to build scipy/numpy from sources that will be slow..

@cmarmo
Copy link
Contributor

cmarmo commented Nov 19, 2019

@rth, @glemaitre, @thomasjpfan sorry for jumping in...
wasn't planned to no longer use Travis?
See #15185 and #15189 .

@rth
Copy link
Member

rth commented Nov 19, 2019

wasn't planned to no longer use Travis?

We were because we no longer needed it (aside for the cron job maybe). But at the same time some time ago, we were considering how to run CI on ARM. #13073 proposed using another CI service (Shippable), but if we can do that in Travis we might as well just keep it since we already have the setup.

@xnox
Copy link
Author

xnox commented Nov 19, 2019

There does not seem to be an arm64 build of conda, as used by travis setup scripts.
I'm not sure if you have some other ways to deploy the required deps on arm64. Maybe just use the available xenial/bionic packages?

@xnox
Copy link
Author

xnox commented Nov 19, 2019

numpy scipy are available pre-built, but not sure about version numbers though.

@glemaitre
Copy link
Member

numpy scipy are available pre-built, but not sure about version numbers though.

We still use Travis as a CRON job to install nightly built. I think it could be worth to add the new platform until we are able to use wheels for these platforms or that we are not having a timeout.
A daily CRON job for these platforms would be enough.

@glemaitre glemaitre changed the title travis: Add more architectures MAINT Add more architectur in CRON job in Travis Nov 19, 2019
@glemaitre glemaitre changed the title MAINT Add more architectur in CRON job in Travis MNT Add more architectur in CRON job in Travis Nov 19, 2019
@glemaitre
Copy link
Member

I think that on these systems we should the packaging system as much as we can and pip otherwise. This would go faster.

@thomasjpfan
Copy link
Member

I experimented with travis ci with arm using the packaging system by the OS and I have been hitting the 50 minute timeout limit. The build takes around 20 minutes, and the tests can take more than 30 minutes long.

Reference: https://travis-ci.org/thomasjpfan/scikit-learn/jobs/614672030

@glemaitre
Copy link
Member

The only way is to have 2 CRON jobs. One to build and cache the install and one to run the tests.

@cmarmo
Copy link
Contributor

cmarmo commented Aug 3, 2020

@rth, @ogrisel, @thomasjpfan, after #17996 is this PR still relevant? Thanks!

@ogrisel
Copy link
Member

ogrisel commented Aug 3, 2020

Let's close this but if someone wants to build on top of #17996 to add build config to test on ppc64le (POWER8/9), feel free to do so. Miniforge should be able to provide the scikit-learn build and test dependencies for that platform:

https://github.com/conda-forge/miniforge/#download

@ogrisel ogrisel closed this Aug 3, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants