I have recently tried doing this. I found that I had to update pyproject.toml because numpy only has arm wheels for 1.19. If we go lower, pip wheel will spend time building numpy. Here is how I updated the pyproject.toml

I also had a travis-ci build that can builld the wheel but stalls during pytest. link here

Is it possible to use arm64-gravtion2 to build and test the wheels? It will go much faster.

Travis-CI Graviton2 support: https://blog.travis-ci.com/2020-09-11-arm-on-aws

Travis-CI config to use Graviton2 instances:

arch: arm64-graviton2
dist: focal
virt: vm
group: edge

Is it possible to use arm64-gravtion2 to build and test the wheels? It will go much faster.

Travis-CI Graviton2 support: https://blog.travis-ci.com/2020-09-11-arm-on-aws

Travis-CI config to use Graviton2 instances:

arch: arm64-graviton2
dist: focal
virt: vm
group: edge

Thank you @janaknat for the suggestion.

I will have a look as soon as #18761 gets merged 😉.

I have recently tried doing this. I found that I had to update pyproject.toml because numpy only has arm wheels for 1.19. If we go lower, pip wheel will spend time building numpy. Here is how I updated the pyproject.toml

I will wait for #18761 to get merged before continuing working on this PR to use the NumPy ARM wheels.

I also had a travis-ci build that can builld the wheel but stalls during pytest. link here

Same for me. We need to extend the waiting time with travis_wait before executing pytest (30 minutes approximately?).

You might want to try to set the environment variables of #18807 and increase the verbosity of cibuildwheel to see how far it could make progress.

I think it's time to re-explore using the graviton2 processors which might be significantly faster as suggested in #18782 (comment) .

I think it's time to re-explore using the graviton2 processors which might be significantly faster as suggested in #18782 (comment) .

Let us try ;)

The suggestion of @janaknat allows the Travis jobs to finish (thank you for the suggestion), but some tests are failing:

• test_explained_variance[randomized-20-dense]
• test_explained_variance_components_10_20[randomized-dense]
• test_lle_simple_grid
• test_fit
• test_partial_fit
• test_small_sparse_partial_fit
• test_sample_hiddens
• test_gibbs_smoke
• test_score_samples
• test_rbm_verbose
• test_convergence_dtype_consistency
• test_randomized_svd_power_iteration_normalizer
• test_svd_flip

This are probably the same failures than those reported by @janaknat in MacPython/scikit-learn-wheels#66. It seems that they are all happening on models that use BLAS / LAPACK operations which makes me suspect a bug on the BLAS implementation for ARM64 on graviton2 processors...

The suggestion of @janaknat allows the Travis jobs to finish

We still get a timeout after 10min without any output on the Python 3.9 job though...

I pushed a change to print the implementation and version of BLAS used by numpy and scipy in those wheels.

I am afraid that we will have to start interactive debugging sessions to craft minimal reproduction cases that only depend on numpy and/or scipy to be able to report the issues upstream for that platform.

Check to see if the bugs disappear with different implementations of BLAS and LAPACK.

Alternatives:
You can use OpenBLAS (it's the default when using pip install numpy scipy.)
You can replace BLIS for BLAS, and you will still need OpenBLAS for LAPACK. Instructions on how to install numpy and scipy with BLIS: https://github.com/aws/aws-graviton-getting-started/blob/master/python.md#22-install-numpy-and-scipy-with-blis-on-ubuntu-and-debian
You also can use ArmPL instead of BLAS and LAPACK.
ArmPL can be freely downloaded from:
https://developer.arm.com/tools-and-software/server-and-hpc/downloads/arm-performance-libraries

• to build NumPy with ArmPL: https://gitlab.com/arm-hpc/packages/-/wikis/packages/numpy
• to build SciPy with ArmPL: https://gitlab.com/arm-hpc/packages/-/wikis/packages/scipy

We have enough to build around ~ 17 times, which means we can not run it every day as a cron job. We can conservatively have it run once or twice a week.

I think we do it only manually for ARM64 wheels.

And we should move the ICC and scipy-dev builds out of travis (e.g. to github actions or Azure). If someone wants to volunteer :)

We have enough to build around ~ 17 times, which means we can not run it every day as a cron job. We can conservatively have it run once or twice a week.

I think we do it only manually for ARM64 wheels.

Indeed, I will try to move the wheel builder for ARM64 to GitHub Actions: #19027.

And we should move the ICC and scipy-dev builds out of travis (e.g. to github actions or Azure). If someone wants to volunteer :)

I think that the scipy-dev job is already in Azure with Linux_Nightly pylatest_pip_scipy_dev.

Am I wrong?

I think that the scipy-dev job is already in Azure with Linux_Nightly pylatest_pip_scipy_dev.

Am I wrong?

It is. I moved it to azure-pipelines a few months ago to get the ball rolling.

I opened #19036 to remove scipy-dev from travis.

FYI, the ARM64 wheels are available online: https://anaconda.org/scikit-learn-wheels-staging/scikit-learn/files.

@alfaro96 The latest version doesn't have the aarch64 wheels.

cibuildwheel 1.8.0 now supports building ARM via emulation on GHA and others. :)

@alfaro96 The latest version doesn't have the aarch64 wheels.

Hey @janaknat,

We are out of travis credits and we were not able to build the wheels for the latest version.

@alfaro96 I highly recommend asking Travis, they may have the ability to increase the free allotment for special cases. They have done that for at least one group here.

@alfaro96 I highly recommend asking Travis, they may have the ability to increase the free allotment for special cases. They have done that for at least one group here.

CC @thomasjpfan and @ogrisel.

I think @adrinjalali has already emailed travis about this.

Yes. I have sent two emails, no luck so far!

@alfaro96 Did Travis get in contact wrt credits?

@alfaro96 Did Travis get in contact wrt credits?

We already have some travis credits and, although the wheels are not available in pypi, you can install from the anaconda repository with:

pip install -i https://pypi.anaconda.org/scikit-learn-wheels-staging/simple scikit-learn

CC @ogrisel.

Any idea on when the wheels will show up in PyPI?

@alfaro96 Will the aarch64 wheels show up in pypi?

Since the aarch64 wheels are already available in the anaconda repository, I think that we can run the Publish to Pypi workflow to upload these wheels to the pypi repository.

WDYT @ogrisel and @thomasjpfan.

I think we are waiting for the 0.24.2 release. I do not think the script would work now because it will try to upload all the wheels and fail because pypi does not allow one to reload a wheel with the same version. (unless we give it a "post1" tag)

Technically, we can run build-wheel on the 0.24.X branch and then upload the few arm wheels ourselves. (I do not have access to pypi)

@thomasjpfan Is your user name on pypi the same as your github's?

@adrinjalali It is the same as my github handle.

Sent you an invite.

Thank you @adrinjalali! I have uploaded the wheels aarch64 files onto pypi.

Thanks @thomasjpfan and sorry @alfaro96 for not replying to your pings, I had too many notifications in my inbox.