Skip to content

[MRG] Workaround limitation of cloudpickle under PyPy #12566

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

ogrisel
Copy link
Member

@ogrisel ogrisel commented Nov 12, 2018

This is a fix for #12554.

I have tried to have a look as at the dissembled pickle payload of the example in #12554 (comment) to try to understand what is the cause of the problem and how it differs from the case where we we use pickle with PyPy or cloudpickle with CPython (which both work) but I don't know where to start. But I am lost in PyPy internals

So instead here is a simple workaround.

@ogrisel ogrisel added this to the 0.20.1 milestone Nov 12, 2018
@ogrisel ogrisel changed the title Workaround limitation of cloudpickle under PyPy [MRG] Workaround limitation of cloudpickle under PyPy Nov 12, 2018
@amueller
Copy link
Member

looks ok. Where's the failure in the tests?

"""Helper for the Parallel calls in KNeighborsMixin.kneighbors

The Cython method tree.query is not directly picklable by cloudpickle
under PyPy.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting that this works while pickling directly fails. I don't really have a good understanding of cloudpickle (or pickling in general) to have much to say about this. LGTM since it fixes the original failure.

@jnothman
Copy link
Member

Not sure if this deserves a what's new. I'm going with no.

@jnothman jnothman merged commit 32e5fd4 into scikit-learn:master Nov 13, 2018
thoo added a commit to thoo/scikit-learn that referenced this pull request Nov 13, 2018
…ybutton

* upstream/master:
  Fix max_depth overshoot in BFS expansion of trees (scikit-learn#12344)
  TST don't test utils.fixes docstrings (scikit-learn#12576)
  DOC Fix typo (scikit-learn#12563)
  FIX Workaround limitation of cloudpickle under PyPy (scikit-learn#12566)
  MNT bare asserts (scikit-learn#12571)
  FIX incorrect error when OneHotEncoder.transform called prior to fit (scikit-learn#12443)
thoo pushed a commit to thoo/scikit-learn that referenced this pull request Nov 13, 2018
thoo added a commit to thoo/scikit-learn that referenced this pull request Nov 13, 2018
…ikit-learn into add_codeblock_copybutton

* 'add_codeblock_copybutton' of https://github.com/thoo/scikit-learn:
  Move an extension under sphinx_copybutton/
  Move css/js file under sphinxext/
  Fix max_depth overshoot in BFS expansion of trees (scikit-learn#12344)
  TST don't test utils.fixes docstrings (scikit-learn#12576)
  DOC Fix typo (scikit-learn#12563)
  FIX Workaround limitation of cloudpickle under PyPy (scikit-learn#12566)
  MNT bare asserts (scikit-learn#12571)
  FIX incorrect error when OneHotEncoder.transform called prior to fit (scikit-learn#12443)
  Retrigger travis:max time limit error
  DOC: Clarify `cv` parameter description in `GridSearchCV` (scikit-learn#12495)
  FIX remove FutureWarning in _object_dtype_isnan and add test (scikit-learn#12567)
  DOC Add 's' to "correspond" in docs for Hamming Loss. (scikit-learn#12565)
  EXA Fix comment in plot-iris-logistic example (scikit-learn#12564)
  FIX stop words validation in text vectorizers with custom preprocessors / tokenizers (scikit-learn#12393)
  DOC Add skorch to related projects (scikit-learn#12561)
  MNT Don't change self.n_values in OneHotEncoder.fit (scikit-learn#12286)
  MNT Remove unused assert_true imports (scikit-learn#12560)
  TST autoreplace assert_true(...==...) with plain assert (scikit-learn#12547)
  DOC: add a testimonial from JP Morgan (scikit-learn#12555)
thoo pushed a commit to thoo/scikit-learn that referenced this pull request Nov 14, 2018
jnothman pushed a commit to jnothman/scikit-learn that referenced this pull request Nov 14, 2018
jnothman pushed a commit to jnothman/scikit-learn that referenced this pull request Nov 14, 2018
@ogrisel ogrisel deleted the fix-parallel-tree-query-pypy-cloudpickle branch November 16, 2018 08:38
xhluca pushed a commit to xhluca/scikit-learn that referenced this pull request Apr 28, 2019
xhluca pushed a commit to xhluca/scikit-learn that referenced this pull request Apr 28, 2019
xhluca pushed a commit to xhluca/scikit-learn that referenced this pull request Apr 28, 2019
koenvandevelde pushed a commit to koenvandevelde/scikit-learn that referenced this pull request Jul 12, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants