Skip to content

Remove fixes/compat for Python 2 #11991

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
3 of 5 tasks
jnothman opened this issue Sep 3, 2018 · 16 comments
Closed
3 of 5 tasks

Remove fixes/compat for Python 2 #11991

jnothman opened this issue Sep 3, 2018 · 16 comments
Labels
Easy Well-defined and straightforward way to resolve

Comments

@jnothman
Copy link
Member

jnothman commented Sep 3, 2018

This can be broken up into multiple pull requests.

Remove:

  • from __future__ imports
  • sklearn.externals.six
  • super(cls, self) is now super()
  • any code explicitly commented to be for Python 2 support
  • anything else in sklearn.utils.fixes related to this

Admittedly I'm not sure it is wise to rush this change, given that we might still be backporting fixes to 0.20.X for a little while...

@jnothman jnothman added Easy Well-defined and straightforward way to resolve help wanted labels Sep 3, 2018
@rth
Copy link
Member

rth commented Sep 4, 2018

I also can't wait for this to happen, but I think we might want to wait for the final 0.20.0 release or even 0.20.1 before removing Py2 code to make backports easier.

@Carreau
Copy link
Contributor

Carreau commented Sep 18, 2018

From experience with IPython/Jupyter and backport: do not rush to remove code, it will make backport way harder, and trip git bisect/git blame.

We still have some py2 compat code here and there close to 18 month after dropping 2.7 and only remove it when it bother us.

Also feel free to use our backport bot for trivial backport.

@rth rth removed the help wanted label Sep 18, 2018
@adrinjalali
Copy link
Member

Should we add some tests to make sure at least the new PRs don't include things such as from __future__ ..., etc.?

@Carreau
Copy link
Contributor

Carreau commented Sep 18, 2018

If master is Py3only, the change to get python2-ism is slim.

"I'm going to start implementing this feature by making sure it works with Python 2 even if the whole library is Python3 only" said no-one ever.

There are few construct that will likely survive the transition (not making use of general unpacking... etc), but nothing easy to detect.

One thing you might be able to do is turn more deprecation warnings into fatal errors.

@rth
Copy link
Member

rth commented Sep 18, 2018

Thanks for sharing your experience on this @Carreau !

@jnothman
Copy link
Member Author

jnothman commented May 1, 2019

There are a few further references to Py2:

sklearn/datasets/base.py:    """Ensure different filenames for Python 2 and Python 3 pickles
sklearn/datasets/base.py:    An object pickled under Python 3 cannot be loaded under Python 2. An object
sklearn/datasets/base.py:    pickled under Python 2 can sometimes not be loaded correctly under Python 3
sklearn/datasets/base.py:    because some Python 2 strings are decoded as Python 3 strings which can be
sklearn/datasets/base.py:    problematic for objects that use Python 2 strings as byte buffers for
sklearn/datasets/base.py:    manages by Python 2 and Python 3 in the same SCIKIT_LEARN_DATA folder so as
sklearn/datasets/base.py:      - /path/to/folder/filename.pkl under Python 2
sklearn/datasets/tests/data/svmlight_invalid.txt:python 2:2.5 10:-5.2 15:1.5
sklearn/ensemble/tests/test_forest.py:    # Using a Python 2.x list as the sample_weight parameter used to raise
sklearn/utils/_unittest_backport.py:            # don't switch to '{}' formatting in Python 2.X
sklearn/utils/deprecation.py:        # on function arguments in Python 2 (already works in Python 3)
sklearn/utils/fixes.py:    """Workaround for Python 2 limitations of pickling instance methods
sklearn/utils/testing.py:    Copied from Python 2.7.5 and modified as required.
sklearn/utils/testing.py:    We may not need to do this any more when getting rid of Python 2, not

@sbehren
Copy link
Contributor

sbehren commented Oct 13, 2019

I am working on sklearn/datasets/base.py

sbehren pushed a commit to sbehren/scikit-learn that referenced this issue Oct 13, 2019
The function
sklearn.datasets.base._pkl_filepath
is a compatibility function for Python 2 support.

The function expects path components as *args and constructs and returns the path to a pickel from it.

It modifies the last path component as follows:
- For Python 3, it adds a suffix ('_py3' by default, or the value of the kwarg 'py3_suffix' if specified).
- For Python 2, it does nothing .
It then concatenates its arguments using os.path.join and returns the
result.

If Python 2 support is to be deprecated, we can
- remove the function _pkl_filepath and
- replace all calls to it by direct calls to os.path.join.
@sbehren
Copy link
Contributor

sbehren commented Oct 13, 2019

Working on sklearn/ensemble/tests/test_forest.py

sbehren pushed a commit to sbehren/scikit-learn that referenced this issue Oct 13, 2019
Issue 'Remove fixes/compat for Python 2'

An extra test with a list raised an exception in Python2.
Since Python2 is deprecated we do not need this extra test any more.
sbehren pushed a commit to sbehren/scikit-learn that referenced this issue Oct 13, 2019
The function
sklearn.datasets.base._pkl_filepath
is a compatibility function for Python 2 and Python 3 support.

The function expects path components as *args and constructs and returns the path to a pickel from it.

It modifies the last path component as follows:

For Python 3, it adds a suffix ('_py3' by default, or the value of the kwarg 'py3_suffix' if specified).
For Python 2, it does nothing .
It then concatenates its arguments using os.path.join and returns the
result.

If Python 2 support is to be deprecated, we can simplify the code: Just
always add the suffix. We cannot get rid of the function entirely since
some users still might have Python 2 pickles on their filesystem.

We can also shorten the docstring.
@albertvillanova
Copy link
Contributor

Is there any pending task in this issue?

@adrinjalali
Copy link
Member

@albertvillanova you could check and see if everything in #11991 (comment) is done, or at least a PR is open for them.

@albertvillanova
Copy link
Contributor

@adrinjalali that is the reason why I was asking if there are any additional tasks, because the ones mentioned in this issue seem to have been already addressed.

@adrinjalali
Copy link
Member

Then I think it's safe to close this one. We can always have PRs which do further cleanups, but seems like this is mostly done. Feel free to re-open @jnothman if you see obvious leftovers.

@rth
Copy link
Member

rth commented Aug 31, 2020

I think all of the points were indeed addressed here.

@albertvillanova thanks! If you are looking for for another issue to contribute, you can also have a look at stalled PRs starting with those that are short and not too controversial as they are more likely to get merged faster.

@albertvillanova
Copy link
Contributor

@rth thanks for the advice. However it's easier said than done. It seems that most of the PR are waiting for reviews, aren't they?

@cmarmo
Copy link
Contributor

cmarmo commented Sep 2, 2020

@albertvillanova I've put some effort in checking stalled PR and labelling them as "help wanted": perhaps you will find your way in this pool.

@albertvillanova
Copy link
Contributor

Thank you @cmarmo. That is very useful!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Easy Well-defined and straightforward way to resolve
Projects
None yet
Development

No branches or pull requests

7 participants