Skip to content

Build aarch64 wheels #19402

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Mar 10, 2021
Merged

Build aarch64 wheels #19402

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
e260069
Build aarch64 wheels
janaknat Feb 9, 2021
3748039
Build and cache numpy=1.16 aarch64 wheels
janaknat Feb 19, 2021
47263cc
Use global var for numpy version
janaknat Mar 8, 2021
3907180
ci: Move NumPy version into a variable.
QuLogic Mar 9, 2021
c842a29
ci: Fix cache key for NumPy wheels.
QuLogic Mar 9, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 39 additions & 6 deletions .github/workflows/cibuildwheel.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,24 @@ jobs:
build_wheels:
name: Build wheels on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
env:
min-numpy-version: "1.16.0"
min-numpy-hash: "04/b6/d7faa70a3e3eac39f943cc6a6a64ce378259677de516bd899dd9eb8f9b32"
strategy:
matrix:
os: [ubuntu-18.04, windows-latest, macos-latest]
cibw_archs: ["auto"]
include:
- os: ubuntu-18.04
cibw_archs: "aarch64"

steps:
- name: Set up QEMU
if: matrix.cibw_archs == 'aarch64'
uses: docker/setup-qemu-action@v1
with:
platforms: arm64

- uses: actions/checkout@v2
with:
fetch-depth: 0
Expand All @@ -26,9 +39,26 @@ jobs:
with:
python-version: '3.7'

- uses: actions/cache@v2
id: numpy-cache
with:
path: numpy-aarch64-cache/
key: numpy-${{ matrix.cibw_archs }}-cache-${{ env.min-numpy-version }}

- name: Install cibuildwheel
run: |
python -m pip install cibuildwheel==1.6.3
python -m pip install cibuildwheel==1.9.0

- name: Build minimum NumPy for aarch64
if: matrix.cibw_archs == 'aarch64' && steps.numpy-cache.outputs.cache-hit != 'true'
run: |
wget https://files.pythonhosted.org/packages/${{ env.min-numpy-hash }}/numpy-${{ env.min-numpy-version }}.zip
unzip numpy-${{ env.min-numpy-version }}.zip
cd numpy-${{ env.min-numpy-version }}
python -m cibuildwheel --output-dir ../numpy-aarch64-cache
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we make this a hidden directory? I assume that the folder needs to exist in the repo for the caching to work correctly?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. I prefer to have the wheels stored in a separate directory so as to not confuse pip and cibuildwheel when searching for wheels to install.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That seems reasonable, but I do not want to clutter the top level of the repo with empty folders if we do not have.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Understood. How about under the 'requirements' directory, seeing as numpy is a requirement for matplotlib?. I can make a hidden directory there.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is made in the build only, not checked in? It seems like build/ or dist/ would be a better location.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran a few tests with cibuildwheel. There doesn't need to be any separate directory for the numpy wheels. cibuildwheel create a directory as part of its run. That should be sufficient.

env:
CIBW_BUILD: "cp36-* cp37-* cp38-*"
CIBW_ARCHS: aarch64

- name: Copy setup.cfg to configure wheel
run: |
Expand All @@ -43,17 +73,18 @@ jobs:
CIBW_MANYLINUX_I686_IMAGE: manylinux1
CIBW_BEFORE_BUILD: pip install certifi numpy==1.19.3
MPL_DISABLE_FH4: "yes"
CIBW_ARCHS: ${{ matrix.cibw_archs }}

- name: Build wheels for CPython
run: |
python -m cibuildwheel --output-dir dist
env:
CIBW_BUILD: "cp3?-*"
CIBW_SKIP: "cp35-* cp36-* cp39-*"
CIBW_BUILD: "cp37-* cp38-*"
CIBW_MANYLINUX_X86_64_IMAGE: manylinux1
CIBW_MANYLINUX_I686_IMAGE: manylinux1
CIBW_BEFORE_BUILD: pip install certifi numpy==1.16
CIBW_BEFORE_BUILD: pip install certifi; pip install --find-links=numpy-aarch64-cache/ numpy==${{ env.min-numpy-version }}
MPL_DISABLE_FH4: "yes"
CIBW_ARCHS: ${{ matrix.cibw_archs }}

- name: Build wheels for CPython 3.6
run: |
Expand All @@ -62,8 +93,9 @@ jobs:
CIBW_BUILD: "cp36-*"
CIBW_MANYLINUX_X86_64_IMAGE: manylinux1
CIBW_MANYLINUX_I686_IMAGE: manylinux1
CIBW_BEFORE_BUILD: pip install certifi numpy==1.16
CIBW_BEFORE_BUILD: pip install certifi; pip install --find-links=numpy-aarch64-cache/ numpy==${{ env.min-numpy-version }}
MPL_DISABLE_FH4: "yes"
CIBW_ARCHS: ${{ matrix.cibw_archs }}
if: >
startsWith(github.ref, 'refs/heads/v3.3') ||
startsWith(github.ref, 'refs/tags/v3.3')
Expand All @@ -73,7 +105,8 @@ jobs:
python -m cibuildwheel --output-dir dist
env:
CIBW_BUILD: "pp3?-*"
CIBW_BEFORE_BUILD: pip install certifi numpy==1.16
CIBW_BEFORE_BUILD: pip install certifi numpy==${{ env.min-numpy-version }}
CIBW_ARCHS: ${{ matrix.cibw_archs }}
if: >
runner.os != 'Windows' && (
startsWith(github.ref, 'refs/heads/v3.3') ||
Expand Down