Skip to content

rework github action jobs #19154

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 7 commits into from
Closed

rework github action jobs #19154

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
4dc2f13
rework cibuildwheel job
mattip Dec 19, 2020
f595fe5
add pypy test run
mattip Dec 19, 2020
82d4db5
add libraries needed to build lxml
mattip Dec 20, 2020
b278f67
add libraries needed to build pikepdf
mattip Dec 20, 2020
367a210
fix yml syntax
mattip Dec 20, 2020
aae8741
skip pikepdf on pypy since no wheel is available
mattip Dec 20, 2020
9878f83
numerous failures on PyPy, skip the test
mattip Dec 20, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 28 additions & 8 deletions .github/workflows/cibuildwheel.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@ on:
- v[0-9]+.[0-9]+.x
tags:
- v*
schedule:
# every Sunday at midnight, builds master and uploads to
# https://anaconda.org/scipy-wheels-nightly/
- cron: '0 0 * * 0'

jobs:
build_wheels:
Expand All @@ -15,6 +19,7 @@ jobs:
strategy:
matrix:
os: [ubuntu-18.04, windows-latest, macos-latest]


steps:
- uses: actions/checkout@v2
Expand All @@ -39,8 +44,8 @@ jobs:
python -m cibuildwheel --output-dir dist
env:
CIBW_BUILD: "cp39-*"
CIBW_MANYLINUX_X86_64_IMAGE: manylinux1
CIBW_MANYLINUX_I686_IMAGE: manylinux1
CIBW_MANYLINUX_X86_64_IMAGE: manylinux2010
CIBW_MANYLINUX_I686_IMAGE: manylinux2010
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the minimum version of pip we are likly to find in the wild going to recognize manylinux2010 now? Is there any compelling reason to move away from manylinux1?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wrote a blog post about it :). manylinux1 is based on CentOS5, which went EOL in 2017. manylinux2010 is based on CentOS6 which is now EOL. The problem was that pip did not keep up. Pip began supporting manylinux2010 wheels with version 19.0, released in Jan 2019. Pip 18 is the version shipped with cpython3.6.12 (the final release), which means users of python3.6 may still have a pip that only recognizes manylinux1 wheels. Users of cpython3.7.3 (released March 2019) and up will have at least pip 19.0, so manylinux2010 is definitely safe for cpython3.9

As for why use a later manylinux: the advantages are bugfixes in the glibc used in the base image, and later versions of the gcc compiler which are not available for Centos5. Some of those bugs are in the math routines, which is why NumPy blocklists older glibc versions.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the default actually, so doesn't need to be specified.

CIBW_BEFORE_BUILD: pip install certifi numpy==1.19.3
MPL_DISABLE_FH4: "yes"

Expand Down Expand Up @@ -68,16 +73,14 @@ jobs:
startsWith(github.ref, 'refs/heads/v3.3') ||
startsWith(github.ref, 'refs/tags/v3.3')

- name: Build wheels for PyPy
- name: Build wheels for PyPy3.7
run: |
python -m cibuildwheel --output-dir dist
env:
CIBW_BUILD: "pp3?-*"
CIBW_BUILD: "pp37-*"
CIBW_BEFORE_BUILD: pip install certifi numpy==1.16
if: >
runner.os != 'Windows' && (
startsWith(github.ref, 'refs/heads/v3.3') ||
startsWith(github.ref, 'refs/tags/v3.3') )
CIBW_MANYLINUX_X86_64_IMAGE: manylinux2010
CIBW_MANYLINUX_I686_IMAGE: manylinux2010

- name: Validate that LICENSE files are included in wheels
run: |
Expand All @@ -87,3 +90,20 @@ jobs:
with:
name: wheels
path: ./dist/*.whl

- name: Set token
if: github.event_name == "schedule"
run: |
echo "TOKEN=${{ secrets.SCIPY_WHEELS_NIGHTLY_ACCESS }}" >> $GITHUB_ENV;
echo "ANACONDA_ORG=https://anaconda.org/scipy-wheels-nightly" >> $GITHUB_ENV;

Comment on lines +94 to +99
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you need to set this in the environment if it's passed by argument anyway? These can be used directly in the next step.

- name: Upload weekly wheels
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The regular Upload artifact step should be disabled to not upload to PyPI with the cron job.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, that's not PyPI, as I do that part. I guess they'll be garbage collected from Actions eventually, so that should be fine.

if: env.TOKEN
run: |
set -ex
pip install git+https://github.com/Anaconda-Server/anaconda-client;
# The first -t option refers to the token, the second is the "type"
# option to the "upload" command
anaconda -t ${TOKEN} upload \
--no-progress --force -u $ANACONDA_ORG \
./dist/*.whl;
7 changes: 7 additions & 0 deletions .github/workflows/tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ jobs:
runs-on: ${{ matrix.os }}

strategy:
fail-fast: false
matrix:
include:
- name-suffix: "(Minimum Versions)"
Expand All @@ -30,6 +31,10 @@ jobs:
python-version: 3.7
extra-requirements: '-r requirements/testing/travis_extra.txt'
XVFB_RUN: xvfb-run -a
# - os: ubuntu-18.04
# python-version: pypy-3.7
# extra-requirements: '-r requirements/testing/travis_extra.txt'
# XVFB_RUN: xvfb-run -a
- os: ubuntu-16.04
python-version: 3.8
extra-requirements: '-r requirements/testing/travis_extra.txt'
Expand Down Expand Up @@ -76,6 +81,8 @@ jobs:
libxcb-randr0 \
libxcb-render-util0 \
libxcb-xinerama0 \
libxml2-dev \
libxslt1-dev \
Comment on lines +84 to +85
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't use these?

lmodern \
fonts-freefont-otf \
texlive-pictures \
Expand Down
2 changes: 1 addition & 1 deletion requirements/testing/travis_extra.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,5 @@ ipykernel
nbconvert[execute]!=6.0.0,!=6.0.1
nbformat!=5.0.0,!=5.0.1
pandas!=0.25.0
pikepdf
pikepdf; platform_python_implementation != 'PyPy'
pytz
5 changes: 5 additions & 0 deletions src/_path.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1085,6 +1085,11 @@ void quad2cubic(double x0, double y0,
outy[2] = y2;
}

/* PyPy before v7.3.3 does not have Py_DTSF_ADD_DOT_0 */
#ifndef Py_DTSF_ADD_DOT_0
#define Py_DTSF_ADD_DOT_0 0x02
#endif


void __add_number(double val, char format_code, int precision,
std::string& buffer)
Expand Down