Skip to content

[MRG] MNT Re-enable PyPy CI #12039

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Sep 18, 2018
Merged

[MRG] MNT Re-enable PyPy CI #12039

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
5e5dfe3
Revert "MAINT Temporarly remove PyPy3 build in CircleCI"
rth Sep 8, 2018
442a482
Catch segfaults
rth Sep 8, 2018
9dbdf3c
Ignore deprecation warnings and skip new tests that use hashing
rth Sep 10, 2018
ca3c1be
Use latest numpy and some tests as known failure
rth Sep 16, 2018
c14542c
Skip test_show_versions_with_blas
rth Sep 16, 2018
8e31357
Use LOKY_MAX_CPU_COUNT and python -m pytest
rth Sep 17, 2018
ace507a
Disable pytest-xdist
rth Sep 17, 2018
2c6816d
Skip test_docstring_parameters.py::test_docstring_parameters on PyPy
rth Sep 17, 2018
80cfe61
A few more fixes
rth Sep 17, 2018
b82b416
Address Joel's comments
rth Sep 18, 2018
ba688b7
Setup CircleCI cron job
rth Sep 18, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions .circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,21 @@ jobs:
path: ~/log.txt
destination: log.txt

pypy3:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're going to do it on CRON in master, do that here, and make a separate PR to 0.20.X?

docker:
- image: pypy:3-6.0.0
steps:
- restore_cache:
keys:
- pypy3-ccache-{{ .Branch }}
- pypy3-ccache
- checkout
- run: ./build_tools/circle/build_test_pypy.sh
- save_cache:
key: pypy3-ccache-{{ .Branch }}-{{ .BuildNum }}
paths:
- ~/.ccache
- ~/.cache/pip

deploy:
docker:
Expand All @@ -89,6 +104,21 @@ workflows:
jobs:
- python3
- python2
- pypy3:
filters:
branches:
only:
- 0.20.X
- deploy:
requires:
- python3
pypy:
triggers:
- schedule:
cron: "0 0 * * *"
filters:
branches:
only:
- master
jobs:
- pypy3
9 changes: 6 additions & 3 deletions build_tools/circle/build_test_pypy.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,16 @@ source pypy-env/bin/activate
python --version
which python

pip install --extra-index https://antocuni.github.io/pypy-wheels/ubuntu numpy==1.14.4 Cython pytest
pip install --extra-index https://antocuni.github.io/pypy-wheels/ubuntu numpy Cython pytest
pip install "scipy>=1.1.0" sphinx numpydoc docutils

ccache -M 512M
export CCACHE_COMPRESS=1
export PATH=/usr/lib/ccache:$PATH
export LOKY_MAX_CPU_COUNT="2"

pip install -e .
pip install -vv -e .
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it still useful to keep the verbose mode here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's still useful, because sometimes the install can take more than 10min, in which case CircleCI will kill the job if there is no output to stdout. It happened once in this PR.


make test
python -m pytest sklearn/
python -m pytest doc/sphinxext/
python -m pytest $(find doc -name '*.rst' | sort)
4 changes: 3 additions & 1 deletion conftest.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,9 @@ def pytest_collection_modifyitems(config, items):
skip_marker = pytest.mark.skip(
reason='FeatureHasher is not compatible with PyPy')
for item in items:
if item.name == 'sklearn.feature_extraction.hashing.FeatureHasher':
if item.name in (
'sklearn.feature_extraction.hashing.FeatureHasher',
'sklearn.feature_extraction.text.HashingVectorizer'):
item.add_marker(skip_marker)

# Skip tests which require internet if the flag is provided
Expand Down
3 changes: 3 additions & 0 deletions sklearn/feature_extraction/tests/test_text.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1108,6 +1108,8 @@ def test_vectorizers_invalid_ngram_range(vec):
message = ("Invalid value for ngram_range=%s "
"lower boundary larger than the upper boundary."
% str(invalid_range))
if isinstance(vec, HashingVectorizer):
pytest.xfail(reason='HashingVectorizer not supported on PyPy')

assert_raise_message(
ValueError, message, vec.fit, ["good news everyone"])
Expand All @@ -1119,6 +1121,7 @@ def test_vectorizers_invalid_ngram_range(vec):
ValueError, message, vec.transform, ["good news everyone"])


@fails_if_pypy
def test_vectorizer_stop_words_inconsistent():
if PY2:
lstr = "[u'and', u'll', u've']"
Expand Down
5 changes: 4 additions & 1 deletion sklearn/tests/test_docstring_parameters.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@
from sklearn.utils.testing import ignore_warnings
from sklearn.utils.deprecation import _is_deprecated

import pytest

PUBLIC_MODULES = set([pckg[1] for pckg in walk_packages(prefix='sklearn.',
path=sklearn.__path__)
if not ("._" in pckg[1] or ".tests." in pckg[1])])
Expand All @@ -45,7 +47,8 @@

# numpydoc 0.8.0's docscrape tool raises because of collections.abc under
# Python 3.7
@ignore_warnings(category=DeprecationWarning)
@pytest.mark.filterwarnings('ignore::DeprecationWarning')
@pytest.mark.skipif(IS_PYPY, reason='test segfaults on PyPy')
def test_docstring_parameters():
# Test module docstring formatting

Expand Down