Skip to content

KNeighborsClassifier test failures with PyPy with loky backend #12554

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
ogrisel opened this issue Nov 9, 2018 · 1 comment
Closed

KNeighborsClassifier test failures with PyPy with loky backend #12554

ogrisel opened this issue Nov 9, 2018 · 1 comment
Labels
Bug Build / CI pypy

Comments

@ogrisel
Copy link
Member

ogrisel commented Nov 9, 2018 •
edited
Loading

Here is a minimal case to reproduce the cause of the failure (to reproduce with pypy3):

import numpy as np
from sklearn.neighbors import KNeighborsClassifier
from sklearn.utils import parallel_backend
import cloudpickle
import pickle
parallel_backend('loky')

X = np.random.randn(3, 1)
y = X[:, 0] > 0
knn = KNeighborsClassifier(algorithm="ball_tree").fit(X, y)

pickle.loads(cloudpickle.dumps(knn._tree.query))

which results in:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-36-9fc0a20fa010> in <module>
----> 1 pickle.loads(cloudpickle.dumps(knn._tree.query))

~/pypy3-v6.0.0-linux64/lib-python/3/pickle.py in _loads(s, fix_imports, encoding, errors)
   1585     file = io.BytesIO(s)
   1586     return _Unpickler(file, fix_imports=fix_imports,
-> 1587                       encoding=encoding, errors=errors).load()
   1588 
   1589 # Use the faster _pickle if possible

~/pypy3-v6.0.0-linux64/lib-python/3/pickle.py in load(self)
   1061                     raise EOFError
   1062                 assert isinstance(key, bytes_types)
-> 1063                 dispatch[key[0]](self)
   1064         except _Stop as stopinst:
   1065             return stopinst.value

~/pypy3-v6.0.0-linux64/lib-python/3/pickle.py in load_newobj(self)
   1345         args = self.stack.pop()
   1346         cls = self.stack.pop()
-> 1347         obj = cls.__new__(cls, *args)
   1348         self.append(obj)
   1349     dispatch[NEWOBJ[0]] = load_newobj

TypeError: object.__new__(getset_descriptor) is not safe, use getset_descriptor.__new__()

This is probably a bug in cloudpickle but I don't have time to investigate further tonight. We could skip the failing tests under PyPy in the mean time if they are to noisy.
@rth rth added the pypy label Nov 9, 2018
@ogrisel
Copy link
Member Author

ogrisel commented Nov 12, 2018

Simpler reproduction case:

import numpy as np
import cloudpickle, pickle
from sklearn.neighbors.ball_tree import BallTree

tree = BallTree(np.zeros(shape=(2, 1)))
pickled_query = cloudpickle.dumps(tree.query)
pickle.loads(pickled_query)

yields:

Traceback (most recent call last):
  File "<ipython-input-9-4a50517f33c7>", line 1, in <module>
    pickle.loads(pickled_query)
  File "/home/ogrisel/pypy3-v6.0.0-linux64/lib-python/3/pickle.py", line 1587, in _loads
    encoding=encoding, errors=errors).load()
  File "/home/ogrisel/pypy3-v6.0.0-linux64/lib-python/3/pickle.py", line 1063, in load
    dispatch[key[0]](self)
  File "/home/ogrisel/pypy3-v6.0.0-linux64/lib-python/3/pickle.py", line 1347, in load_newobj
    obj = cls.__new__(cls, *args)
TypeError: object.__new__(getset_descriptor) is not safe, use getset_descriptor.__new__()

@ogrisel ogrisel mentioned this issue Nov 12, 2018
Merged
@ogrisel ogrisel closed this as completed Nov 16, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug Build / CI pypy
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ogrisel @rth