It seems something feasible.

You can clear the cache of the build when building and cache it at the end of the job and reuse the cache after.

Kinda but without automation for the checking.

I would set up 2 distinct CRON jobs with a 1-hour difference. The first one to build (erasing the cache first), the second one to test the version which has been stored in the cache.

Then, we still spent 15 minutes to make the scikit-learn wheel and the remaining time for the tests

Maybe using ccache could help some.

and the remaining time for the tests (the tests almost complete ~99%).

Sounds like we are almost there. Would pytest-xdist help enough to get it finished in 1 job (assuming there are multiple cores on the ARM VM)?

There were a few PRs to make tests faster, so this Shippable build might now complete.

Also, those VM have 96 CPU cores, I don't know how many we can access, it would be interesting to run,

python -c "import joblib; print(joblib.effective_n_jobs())"

there. If it's more than 1, can now use pytest-xdist, with pytest -n <n-cpu>, that might also help.

Also pytest-xdist is handy for catching segfaults in tests as well.

Actually conda-forge is doing this on Azure, though I don't see aarch in the public Agent pools. Not sure how they got it, but there is a possibility of asking for it as well.

They are using vmImage: ubuntu-16.04 with a condaforge/linux-anvil-aarch64 as a docker image. We can set up a similar test environment here for ARM.

OK, so it's an Azure VM that runs Docker, that runs QEMU that emulates ARM. No wonder the test suite is taking 2 hours 20min.

If we can get real ARM hardware that would be best, and preliminary results above indicate that it's at least 2x faster.

I was considering using Gitlab CI for Github and configure gitlab runners to be our own raspberry pis. (And take advantage of the new raspberry pi 4)

FYI: Drone.io ( https://drone.io/ ) have an AArch64 and AArch32 build service that might be helpful.

Hi,

Since manylinux2014 has given Linux AArch64 support, I am interested in building/uploading Linux AArch64 wheel to PyPI repository. Please let me know if you are interested.

Thanks

MacPython/scikit-learn-wheels#61 could be one way to do it long term. But we would still need a CI on the main repo. drone.io seems promising (it is in particular use by conda-forge). If we can manage to build scikit-learn and run tests there it would already be a step forward. CI is likely a pre-requisite for building wheels. A PR to invetigate would be very welcome you should be able to activate that CI on your fork.

@rth, I will work on it and update you.

Azure Pipelines now supposedly supports the Linux/ARM64 platform as announced here:

https://azure.microsoft.com/en-in/updates/azure-devops-pipelines-introduces-support-for-linuxarm64/

However I could not find any reference for this platform in the documentation. Maybe it's just a matter of trying to use it.

Looking more into it, I think azure added an arm64 agent so one can run an self-hosted arm agent. I do not think microsoft is hosting any arm agents themselves. Here is a list of microsoft hosted agents.

It's not explicitly mentioned here, but Travis CI also has some ARM CI I think.