MNT Release 0.19.2 with python 3.7 support #11422

jnothman · 2018-07-03T23:38:34Z

This is to check how easy it would be to release a version 0.19.2 with Python 3.7 support. Is it just a matter of regenerating the C code with a more recent Cython?

rth · 2018-07-04T07:52:27Z

Is it just a matter of regenerating the C code with a more recent Cython?

It needs to be cython >=0.27.3,<0.28 I think (#11320 (comment))

There will be several test failures that would need to be solved I think (cf link above). Backporting #11256 might help with a few.

jnothman · 2018-07-04T07:54:31Z

Without yet looking at the logs, I'm going to assume that conda can't find appropriate dependencies and I need to use dev wheels

jnothman · 2018-07-04T07:57:40Z

I'd not seen your comment @rth. My conjectures were wrong? Good. I'll add the baxkport

jnothman · 2018-07-04T08:02:13Z

(it seems our Cron Travis should use python RCs so that we can anticipate this issue better)

)

jnothman · 2018-07-04T08:21:37Z

Circle fails due to lack of py 3.7 matplotlib release

rth · 2018-07-04T08:28:34Z

Circle fails due to lack of py 3.7 matplotlib release

It's being worked on in matplotlib/matplotlib#11533

MacPython/scikit-learn-wheels#5 would also be needed to make a release with Py 3.7..

ogrisel · 2018-07-04T08:47:22Z

I have recently fixed wheelhouse-uploader to work with Python 3.7 on appveyor. So if we have numpy and scipy wheels for Python 3.7 on all platforms we should be able to use MacPython/scikit-learn-wheels to generate the scikit-learn 0.19.2 wheels.

ogrisel · 2018-07-04T08:52:56Z

.travis.yml

@@ -42,9 +42,9 @@ matrix:
           CYTHON_VERSION="0.25.2" COVERAGE=true
    # This environment use pytest to run the tests. It uses the newest
    # supported Anaconda release (4.4.0). It also runs tests requiring Pandas.


We should remove "(4.4.0)" from the comment.

ogrisel · 2018-07-04T08:54:01Z

appveyor.yml

@@ -26,11 +26,11 @@ environment:
      PYTHON_ARCH: "64"

    - PYTHON: "C:\\Python35"


C:\\Python35 needs to be changed to C:\\Python37 otherwise the preinstalled folder is used IIRC.

ogrisel · 2018-07-04T08:54:19Z

appveyor.yml

@@ -26,11 +26,11 @@ environment:
      PYTHON_ARCH: "64"

    - PYTHON: "C:\\Python35"
-      PYTHON_VERSION: "3.5.0"
+      PYTHON_VERSION: "3.7.0"
      PYTHON_ARCH: "32"

    - PYTHON: "C:\\Python35-x64"


C:\\Python37-x64

ogrisel · 2018-07-04T08:56:36Z

BTW, I am +1 to release a minimal 0.19.2 to add support for Python 3.7.

ogrisel · 2018-07-04T09:05:48Z

I do not understand why we get a cython error on travis for the old configurations that have not changed (Python 2.7 and such). We did not have that error at the time of the 0.19.1 release.

ogrisel · 2018-07-04T09:09:51Z

export CYTHON_VERSION="0.23.4" is ignored for the first entry of the build matrix: 0.28 is installed with pip instead.

jnothman · 2018-07-04T11:25:14Z

Appveyor is failing while trying to install numpy with a rather mysterious error:

        libraries)
      File "c:\python37\lib\site-packages\setuptools\msvc.py", line 199, in msvc14_gen_lib_options
        if LegacyVersion(np.__version__) < LegacyVersion('1.11.2'):
    AttributeError: module 'numpy' has no attribute '__version__'

I'll change the numpy dependency to the most recent version...

ogrisel · 2018-07-04T11:28:46Z

Thanks I was about to do it :)

ogrisel · 2018-07-04T12:39:06Z

The failure in test_logistic.test_max_iter on appveyor with Python 2.7 is fishy:

https://ci.appveyor.com/project/sklearn-ci/scikit-learn/build/1.0.23007/job/gexvcabi6n396fdt#L4021

We did not change anything for this specific environment, right?

ogrisel · 2018-07-04T12:41:27Z

Actually this could be caused by the change in the version of scipy on appveyor. Maybe we would reproduce the issue on master if we bumbed up the versions of the dependencies on appveyor. Let's ignore this issue for now.

jnothman · 2018-07-04T12:45:21Z

Yes, there was a change in the number of iterations reported by fmin_l_bfgs_b. We can probably ignore that assertion and leave a note in what's new. Or try find the right commit to cherry-pick.

jnothman · 2018-07-04T12:46:06Z

Here: #10723 / 2aba6e2

ogrisel · 2018-07-04T12:50:09Z

Great, let's cherry-pick that.

ogrisel · 2018-07-04T12:51:51Z

Actually this is not exactly the same problem: it says n_iter_ might exceed max_iter which is not what we observe in the failure message.

jnothman · 2018-07-04T12:56:03Z

The issue is that scipy was reporting the n_iter off by one (in some cases?). I can't remember which direction. But rather than hack consistency in our interface depending on which version of scipy, we went with reporting whatever scipy reports, but making the test weaker.

jnothman · 2018-07-04T13:05:45Z

Okay. That's a pain. Doctests will fail for the version of numpy that supports Python 3.7. Just as we disable doctests for earlier versions of numpy in master, we should disable doctests here for recent versions of numpy...

ogrisel · 2018-07-04T13:07:28Z

FYI I have opened #11425: we will see if the test_logistic.test_max_iter failure is still there on master or not.

…cikit-learn#10723)

jnothman · 2018-07-04T13:15:47Z

I'm leaving this alone for the night. Feel free to hack it if something turns up.

…ing (scikit-learn#10851)

sklearn-lgtm · 2018-07-04T13:41:56Z

This pull request fixes 1 alert when merging 857ccd9 into a24c8b4 - view on LGTM.com

fixed alerts:

1 for Except block handles 'BaseException'

Comment posted by LGTM.com

ogrisel · 2018-07-04T15:57:29Z

Appveyor looks good. Let's merge and I will try to generate the wheels.

jnothman · 2018-07-04T21:35:32Z

We'll need to fix circle if we want the docs to upload themselves...

ogrisel · 2018-07-05T13:54:41Z

I won't have time to fix circle today but I would like to try to get the wheels up and running. I pushed a stopgap to try to get them to build:

1d3bf73

I will be off-line this tonight and this WE and then be traveling to Austin for scipy. Feel free (@jnothman or someone else) to do the 0.19.2 release without me in the meantime.

amueller · 2018-07-12T18:36:00Z

What's the status on this now?

rth · 2018-07-12T18:56:31Z

Can't comment on the overall status, but for my part,

I would like to make a PR backporting [MRG] Fix DeprecationWarning due to collections.abc in Python 3.7 #11431 (comment) to 0.19.2. A second review of that PR would be appreciated.
@ogrisel suggested double-checking that .tar.gz installation from source for 0.19.2 works as expected in a clean env

I haven't got a chance to get to it yet, but will try later today.

rth · 2018-07-14T16:59:34Z

Some updates on this,

tested that we can build the latest commit on 0.19.X in conda-forge without errors ([MRG] Version 0.19.2 conda-forge/scikit-learn-feedstock#67) (this also runs tests)
tested that after the .tar.gz is build with setup.py sdist on Linux (using Cython 0.27.3) it can be installed in a clean virutalenv with just latest numpy and scipy and that tests pass. Tested this with conda for py2.7, 3.6, 3.7.
backport of [MRG] Fix DeprecationWarning due to collections.abc in Python 3.7 #11431 done in [MRG] Fix collections.abc deprecations with Python 3.7 (0.19.X backport) #11509

* tag '0.19.2': Release 0.19.2 [MRG] Fix collections.abc deprecations with Python 3.7 (0.19.X backport) (scikit-learn#11509) FIX need to explicitly raise SkipTest MAINT disabled some tests failing under macOS and 32-bit Python on windows. MNT Release 0.19.2 with python 3.7 support (scikit-learn#11422) DOC previous versions -> all available versions (scikit-learn#10179) MAINT only build versions list on master branch [MRG+1] MAINT Use magic to list documentation versions (scikit-learn#9841) DOC Add Examples heading minor fixes to whatsnew fix release 0.19.1 whatsnew link (scikit-learn#9981)

* releases: Release 0.19.2 [MRG] Fix collections.abc deprecations with Python 3.7 (0.19.X backport) (scikit-learn#11509) FIX need to explicitly raise SkipTest MAINT disabled some tests failing under macOS and 32-bit Python on windows. MNT Release 0.19.2 with python 3.7 support (scikit-learn#11422) DOC previous versions -> all available versions (scikit-learn#10179) MAINT only build versions list on master branch [MRG+1] MAINT Use magic to list documentation versions (scikit-learn#9841) DOC Add Examples heading minor fixes to whatsnew fix release 0.19.1 whatsnew link (scikit-learn#9981)

* dfsg: Release 0.19.2 [MRG] Fix collections.abc deprecations with Python 3.7 (0.19.X backport) (scikit-learn#11509) FIX need to explicitly raise SkipTest MAINT disabled some tests failing under macOS and 32-bit Python on windows. MNT Release 0.19.2 with python 3.7 support (scikit-learn#11422) DOC previous versions -> all available versions (scikit-learn#10179) MAINT only build versions list on master branch [MRG+1] MAINT Use magic to list documentation versions (scikit-learn#9841) DOC Add Examples heading minor fixes to whatsnew fix release 0.19.1 whatsnew link (scikit-learn#9981)

Try upgrading python version requirements in CI

c103d3b

Wenhao Zhang and others added 3 commits July 4, 2018 18:06

Compatibility with python 3.7.0b5 scikit-learn#11224 (scikit-learn#11256

31b975c

)

Pin Cython to 0.27.3

841c414

Update version to 0.19.2

350cce0

jnothman changed the title ~~MNT Try upgrading python version requirements in CI~~ MNT Release 0.19.2 with python 3.7 support Jul 4, 2018

ogrisel reviewed Jul 4, 2018

View reviewed changes

ogrisel and others added 6 commits July 4, 2018 11:11

FIX use C:\Python37 on appveyor

6cd3077

Fix the cython version on travis/ubuntu

f54ef63

DOC better comments

1c07b84

FIX doctest ellipsis pattern

45d980d

Install Cython and Pandas with pip on Travis

19e559f

Let Appveyor use most recent versions

579124c

jnothman and others added 3 commits July 4, 2018 21:54

Travis needs to stall scipy with pip too

ae7c676

MAINT Install nose and pytest with pip

5e946c2

Don't test pandas for now

ff7bcd0

jnothman added 3 commits July 4, 2018 23:09

FIX n_iter_ should be less than max_iter in when using lbfgs solver(s…

76e2491

…cikit-learn#10723)

TST Skip doctests for numpy < 1.14

f55f8e9

Revert circle changes so that docs build on working config

a1a7f4a

MAINT: correct cythonization of sklearn.metrics.cluster to avoid warn…

857ccd9

…ing (scikit-learn#10851)

ogrisel merged commit 7c3944f into scikit-learn:0.19.X Jul 4, 2018

rth mentioned this pull request Jul 13, 2018

[MRG] Fix collections.abc deprecations with Python 3.7 (0.19.X backport) #11509

Merged

jnothman mentioned this pull request Aug 6, 2018

DeprecationWarning in sklearn/feature_extraction/text.py #11757

Closed

		@@ -26,11 +26,11 @@ environment:
		PYTHON_ARCH: "64"

		- PYTHON: "C:\\Python35"

Uh oh!

MNT Release 0.19.2 with python 3.7 support #11422

MNT Release 0.19.2 with python 3.7 support #11422

Uh oh!

Conversation

jnothman commented Jul 3, 2018

Uh oh!

rth commented Jul 4, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jnothman commented Jul 4, 2018

Uh oh!

jnothman commented Jul 4, 2018

Uh oh!

jnothman commented Jul 4, 2018

Uh oh!

jnothman commented Jul 4, 2018

Uh oh!

rth commented Jul 4, 2018

Uh oh!

ogrisel commented Jul 4, 2018

Uh oh!

ogrisel Jul 4, 2018

Choose a reason for hiding this comment

Uh oh!

ogrisel Jul 4, 2018

Choose a reason for hiding this comment

Uh oh!

ogrisel Jul 4, 2018

Choose a reason for hiding this comment

Uh oh!

ogrisel commented Jul 4, 2018

Uh oh!

ogrisel commented Jul 4, 2018

Uh oh!

ogrisel commented Jul 4, 2018

Uh oh!

jnothman commented Jul 4, 2018

Uh oh!

ogrisel commented Jul 4, 2018

Uh oh!

ogrisel commented Jul 4, 2018

Uh oh!

ogrisel commented Jul 4, 2018

Uh oh!

jnothman commented Jul 4, 2018

Uh oh!

jnothman commented Jul 4, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ogrisel commented Jul 4, 2018

Uh oh!

ogrisel commented Jul 4, 2018

Uh oh!

jnothman commented Jul 4, 2018 via email

Uh oh!

jnothman commented Jul 4, 2018

Uh oh!

ogrisel commented Jul 4, 2018

Uh oh!

jnothman commented Jul 4, 2018

Uh oh!

sklearn-lgtm commented Jul 4, 2018

Uh oh!

ogrisel commented Jul 4, 2018

Uh oh!

jnothman commented Jul 4, 2018

Uh oh!

ogrisel commented Jul 5, 2018

Uh oh!

amueller commented Jul 12, 2018

Uh oh!

rth commented Jul 12, 2018

Uh oh!

rth commented Jul 14, 2018

Uh oh!

Uh oh!

rth commented Jul 4, 2018 •

edited

Loading

jnothman commented Jul 4, 2018 •

edited

Loading