MNT Update dependencies versions in PyPy CI #17994

rth · 2020-07-26T08:34:29Z

Updates dependencies versions for PyPy, numpy, scipy in PyPy CI

Let's see if this fixes the segfaults we are currently seeing there.

rth · 2020-07-26T09:05:08Z

OK, still getting the segfault at import time,

>>> import sklearn.manifold
Fatal error in cpyext, CPython compatibility layer, calling PyMemoryView_FromObject
Either report a bug or consider not using this particular extension
<OperationError object at 0x5360278>
RPython traceback:
  File "pypy_module_cpyext.c", line 7762, in wrapper_second_level__star_1_6
  File "pypy_module_cpyext_1.c", line 31676, in make_ref
  File "pypy_module_cpyext_2.c", line 2886, in create_ref
  File "pypy_module_cpyext_3.c", line 17934, in memory_attach
  File "pypy_module_cpyext_3.c", line 51824, in fill_Py_buffer
  File "pypy_module_cpyext_1.c", line 32374, in unwrapper__StdObjSpaceConst_SomeInstance_star_1
  File "pypy_interpreter.c", line 37344, in BuiltinCodePassThroughArguments0_funcrun
  File "pypy_module__frozen_importlib.c", line 70, in import_with_frames_removed
  File "rpython_rlib_rvmprof.c", line 105, in __vmprof_eval_vmprof
  File "pypy_interpreter_1.c", line 31682, in execute_frame
  File "rpython_jit_metainterp.c", line 8553, in ll_portal_runner__Unsigned_Bool_pypy_interpreter
  File "pypy_module_pypyjit.c", line 181, in portal_7
  File "pypy_interpreter_2.c", line 34130, in handle_bytecode__AccessDirect_None
  File "pypy_interpreter_3.c", line 13476, in dispatch_bytecode__AccessDirect_None
  File "pypy_interpreter_3.c", line 37359, in CALL_FUNCTION__AccessDirect_None
  File "rpython_rlib_rvmprof.c", line 105, in __vmprof_eval_vmprof
  File "pypy_interpreter_1.c", line 31682, in execute_frame
  File "rpython_jit_metainterp.c", line 8553, in ll_portal_runner__Unsigned_Bool_pypy_interpreter
  File "pypy_module_pypyjit.c", line 181, in portal_7
  File "pypy_interpreter_2.c", line 34130, in handle_bytecode__AccessDirect_None
  File "pypy_interpreter_3.c", line 13476, in dispatch_bytecode__AccessDirect_None
  File "pypy_interpreter_3.c", line 37359, in CALL_FUNCTION__AccessDirect_None
  File "rpython_rlib_rvmprof.c", line 105, in __vmprof_eval_vmprof
  File "pypy_interpreter_1.c", line 31682, in execute_frame
  File "rpython_jit_metainterp.c", line 8579, in ll_portal_runner__Unsigned_Bool_pypy_interpreter
  File "rpython_jit_metainterp.c", line 21543, in handle_jitexception_7
Segmentation fault (core dumped)

will to import cython version and if it doesn't help report upstream.

@ogrisel The output of,

python -m threadpoolctl -i sklearn

is

[
  {
    "filepath": "/usr/lib/x86_64-linux-gnu/libgomp.so.1.0.0",
    "prefix": "libgomp",
    "user_api": "openmp",
    "internal_api": "openmp",
    "version": null,
    "num_threads": 36
  },
  {
    "filepath": "/root/project/pypy-env/site-packages/numpy.libs/libopenblasp-r0-ae94cfde.3.9.dev.so",
    "prefix": "libopenblas",
    "user_api": "blas",
    "internal_api": "openblas",
    "version": "0.3.9.dev",
    "num_threads": 36,
    "threading_layer": "pthreads"
  },
  {
    "filepath": "/root/project/pypy-env/site-packages/scipy.libs/libopenblas-r0-da994f36.3.3.so",
    "prefix": "libopenblas",
    "user_api": "blas",
    "internal_api": "openblas",
    "version": null,
    "num_threads": 1,
    "threading_layer": "disabled"
  }
]

The issue was with openblas 3.10 only right?

ogrisel · 2020-07-26T09:23:39Z

The regression I mentioned IRL is impacting OpenBLAS 0.3.10 indeed. It was not present in 0.3.9 but I don't know about 0.3.9*.dev* though.

ogrisel · 2020-07-26T09:25:07Z

See: #17980 and linked issues.

ogrisel · 2020-07-26T09:26:45Z

If the numpy openblas is the culprit then there is the following minimal reproducer:

import numpy as np

np.ones(shape=(300000, 5)) @ np.ones(shape=(5, 5))

mattip · 2020-10-14T08:31:54Z

.circleci/config.yml

@@ -105,16 +105,16 @@ jobs:

  pypy3:
    docker:
-      - image: pypy:3.6-7.2.0
+      - image: pypy:3.6-7.3.1


PyPy released 7.3.2, is there a image for it yet?

I have downloaded 7.3.2 from the download page and the segfaults are still there.

mattip · 2020-10-14T08:33:36Z

build_tools/circle/build_test_pypy.sh

@@ -21,10 +21,8 @@ which python
 pip install -U pip

 # pins versions to install wheel from https://antocuni.github.io/pypy-wheels/manylinux2010
-pip install --extra-index-url https://antocuni.github.io/pypy-wheels/manylinux2010 numpy==1.18.0 scipy==1.3.2
+pip install --extra-index-url https://antocuni.github.io/pypy-wheels/manylinux2010 Cython numpy==1.19.1 scipy==1.5.2


The NumPy wheel is available on PyPI (at the bottom of the page) now

mattip · 2020-10-14T08:34:15Z

build_tools/circle/build_test_pypy.sh


-# Install Cython directly
-pip install https://antocuni.github.io/pypy-wheels/ubuntu/Cython/Cython-0.29.14-py3-none-any.whl


I think this is can also be installed from PyPI

lesteve · 2020-10-27T16:47:36Z

FWIW I can reproduce the problem locally:

install pypy 3.6 from https://github.com/conda-forge/miniforge#miniforge-pypy3
make sure the right conda is used:

# default installation path, correct it if needed
source ~/miniforge-pypy3/etc/profile.d/conda.sh

install dependencies

pip install --extra-index-url https://antocuni.github.io/pypy-wheels/manylinux2010 Cython numpy==1.19.1 scipy==1.5.2 Cython
pip install sphinx numpydoc docutils joblib pillow pytest

build scikit-learn

python setup.py build_ext --inplace -j 3       
pip install --no-build-isolation -e .

Reproduce segmentation fault (it seems to happen at import time with sklearn.tree somewhere looking at python -v output the last succesful import mentions sklearn.tree._utils):

python -v -c 'import sklearn.tree'

Tail of the output:

Fatal error in cpyext, CPython compatibility layer, calling PyMemoryView_FromObject
Either report a bug or consider not using this particular extension
<OperationError object at 0x561564943c90>
RPython traceback:
  File "pypy_module_cpyext.c", line 5650, in wrapper_second_level__star_1
  File "pypy_module_cpyext_1.c", line 35424, in make_ref
  File "pypy_module_cpyext_2.c", line 10168, in create_ref
  File "pypy_module_cpyext_3.c", line 23729, in memory_attach
  File "pypy_module_cpyext_4.c", line 9653, in fill_Py_buffer
  File "pypy_module_cpyext_1.c", line 41590, in unwrapper__StdObjSpaceConst_SomeInstance_star_1_2
  File "pypy_interpreter.c", line 22498, in BuiltinCodePassThroughArguments0_funcrun
  File "pypy_module__frozen_importlib.c", line 70, in import_with_frames_removed
  File "rpython_rlib_rvmprof.c", line 105, in __vmprof_eval_vmprof
  File "pypy_interpreter_1.c", line 29197, in execute_frame
  File "rpython_jit_metainterp.c", line 8574, in ll_portal_runner__Unsigned_Bool_pypy_interpreter
  File "pypy_module_pypyjit.c", line 181, in portal_7
  File "pypy_interpreter_2.c", line 33958, in handle_bytecode__AccessDirect_None
  File "pypy_interpreter_3.c", line 14663, in dispatch_bytecode__AccessDirect_None
  File "pypy_interpreter_3.c", line 38521, in CALL_FUNCTION__AccessDirect_None
  File "rpython_rlib_rvmprof.c", line 105, in __vmprof_eval_vmprof
  File "pypy_interpreter_1.c", line 29197, in execute_frame
  File "rpython_jit_metainterp.c", line 8574, in ll_portal_runner__Unsigned_Bool_pypy_interpreter
  File "pypy_module_pypyjit.c", line 181, in portal_7
  File "pypy_interpreter_2.c", line 33958, in handle_bytecode__AccessDirect_None
  File "pypy_interpreter_3.c", line 14663, in dispatch_bytecode__AccessDirect_None
  File "pypy_interpreter_3.c", line 38521, in CALL_FUNCTION__AccessDirect_None
  File "rpython_rlib_rvmprof.c", line 105, in __vmprof_eval_vmprof
  File "pypy_interpreter_1.c", line 29197, in execute_frame
  File "rpython_jit_metainterp.c", line 8600, in ll_portal_runner__Unsigned_Bool_pypy_interpreter
  File "rpython_jit_metainterp.c", line 21618, in handle_jitexception_7
[1]    13880 segmentation fault  python -v -c 'import sklearn.tree'

so very similar to #17994 (comment).

I can run fine the snippet in:
#17994 (comment) so this problem is probably not related to the OpenBLAS one.

mattip · 2020-10-28T06:42:10Z

Thanks for the detailed report. I can reproduce and am looking at the problem.

mattip · 2020-10-28T08:59:52Z

xref PyPy issue https://foss.heptapod.net/pypy/pypy/-/issues/3336. There is a bug in PyPy which currently only allows the format string of memoryviews to be 128 bytes, the Cell ends up with this "T{q:parent:(8)q:children:q:cell_id:q:point_index:i:is_leaf:f:squared_max_width:q:depth:q:cumulative_size:(3)f:center:(3)f:barycenter:(3)f:min_bounds:(3)f:max_bounds:}" which has 168 chars.

cmarmo · 2020-10-29T11:09:57Z

@mattip thanks for investigating this! I have curated a list of sklearn import throwing segfault with pypy (with the error message reported by @lesteve above).
I'm wondering if they could all be related to the pypy format limitation you have reported.
I'm running pypy 7.3.1 packaged for Fedora.

mattip · 2020-10-29T13:35:52Z

Should be fixed on PyPy HEAD, and there is a temporary fix bubbling through the conda-forge soup, linked above.

rth · 2021-07-29T16:18:44Z

I think this PR is no longer relevant since we switched to installing PyPi via conda-forge.

Update dependencies versions in PyPy CI

957d1ee

rth added the pypy label Jul 26, 2020

Fix synax error

5bc8081

rth marked this pull request as draft July 26, 2020 08:55

Install latest Cython

7b2ed57

rth mentioned this pull request Aug 3, 2020

BUG: segfaults in testsuite with PyPy on ppc64le #16956

Closed

mattip reviewed Oct 14, 2020

View reviewed changes

alfaro96 mentioned this pull request Oct 15, 2020

CircleCI fails on pypy3 build #18624

Closed

mattip mentioned this pull request Oct 29, 2020

temporary fix in buffer format size conda-forge/pypy3.6-feedstock#30

Merged

5 tasks

Base automatically changed from master to main January 22, 2021 10:52

rth closed this Jul 29, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MNT Update dependencies versions in PyPy CI #17994

MNT Update dependencies versions in PyPy CI #17994

rth commented Jul 26, 2020

rth commented Jul 26, 2020

ogrisel commented Jul 26, 2020

ogrisel commented Jul 26, 2020

ogrisel commented Jul 26, 2020

mattip Oct 14, 2020

cmarmo Oct 29, 2020

mattip Oct 14, 2020

mattip Oct 14, 2020

lesteve commented Oct 27, 2020 •

edited

Loading

mattip commented Oct 28, 2020

mattip commented Oct 28, 2020

cmarmo commented Oct 29, 2020

mattip commented Oct 29, 2020

rth commented Jul 29, 2021


		# Install Cython directly
		pip install https://antocuni.github.io/pypy-wheels/ubuntu/Cython/Cython-0.29.14-py3-none-any.whl

MNT Update dependencies versions in PyPy CI #17994

MNT Update dependencies versions in PyPy CI #17994

Conversation

rth commented Jul 26, 2020

rth commented Jul 26, 2020

ogrisel commented Jul 26, 2020

ogrisel commented Jul 26, 2020

ogrisel commented Jul 26, 2020

mattip Oct 14, 2020

Choose a reason for hiding this comment

cmarmo Oct 29, 2020

Choose a reason for hiding this comment

mattip Oct 14, 2020

Choose a reason for hiding this comment

mattip Oct 14, 2020

Choose a reason for hiding this comment

lesteve commented Oct 27, 2020 • edited Loading

mattip commented Oct 28, 2020

mattip commented Oct 28, 2020

cmarmo commented Oct 29, 2020

mattip commented Oct 29, 2020

rth commented Jul 29, 2021

lesteve commented Oct 27, 2020 •

edited

Loading