Fix testing with tests.py on Py3.6. #8281

anntzer · 2017-03-13T06:54:32Z

Invoking tests.py used to fail on Py3.6 due to the use of deprecated
invalid escapes both in the codebase and in third-party code (dateutil).
Fix the codebase, skip the checks on dateutil.

It is strange (and should be worth investigating...) that the CI did not
fail before, as it also uses tests.py on Py3.6...

xref #8271.

Invoking `tests.py` used to fail on Py3.6 due to the use of deprecated invalid escapes both in the codebase and in third-party code (dateutil). Fix the codebase, skip the checks on dateutil. It is strange (and should be worth investigating...) that the CI did not fail before, as it also uses `tests.py` on Py3.6...

tacaswell · 2017-03-13T14:55:27Z

I have also not been seeing these failures, and have been testing using tests.py + py3.6 for a while now...

tacaswell · 2017-03-13T14:57:23Z

lib/matplotlib/tests/test_dates.py

@@ -94,12 +94,6 @@ def test_too_many_date_ticks():
    # setting equal datetimes triggers and expander call in
    # transforms.nonsingular which results in too many ticks in the
    # DayLocator.  This should trigger a Locator.MAXTICKS RuntimeError
-    warnings.filterwarnings(


Does this warning not come out of singular dates anymore?

Nope (that's why the tests pass :-)).

anntzer · 2017-03-13T15:08:43Z

Do you get a warning if you do $ python -Wd -c 'import dateutil.parser'?

Do you get an error on one of the escape sequences I fixed if you install the filter as in tests.py and import matplotlib in an interactive session?

dstansby · 2017-03-14T13:08:33Z

I can confirm that on a fresh install of python 3.6 there are a lot of escaped characters errors, which are fixed by this PR.

tacaswell · 2017-03-14T19:43:13Z

with arch system python:

15:37 $ python
Python 3.6.0 (default, Jan 16 2017, 12:12:55) 
[GCC 6.3.1 20170109] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import dateutil.parser
>>> import dateutil
>>> dateutil.__version__
'2.6.0'
>>>

and python -Wd -c 'import dateutil.parser' does not raise. I am greatly confused by this whole thing...

anntzer · 2017-03-14T19:55:19Z

woot

$ pacman -Q python{,-dateutil}          
python 3.6.0-2
python-dateutil 2.6.0-2
$ which python && python -minspect -d dateutil.parser
/usr/bin/python
Target: dateutil.parser
Origin: /usr/lib/python3.6/site-packages/dateutil/parser.py
Cached: /usr/lib/python3.6/site-packages/dateutil/__pycache__/parser.cpython-36.pyc
Loader: <_frozen_importlib_external.SourceFileLoader object at 0x7f6c5b71b278>
$ python -Wd -c 'import dateutil.parser'
/usr/lib/python3.6/site-packages/dateutil/parser.py:50: DeprecationWarning: invalid escape sequence \.
  _split_decimal = re.compile("([\.,])")

tacaswell · 2017-03-14T20:05:21Z

16:02 $ pacman -Q python{,-dateutil}
python 3.6.0-2
python-dateutil 2.6.0-2
✔ ~ 
16:02 $ which python && python -minspect -d dateutil.parser
/usr/bin/python
Target: dateutil.parser
Origin: /usr/lib/python3.6/site-packages/dateutil/parser.py
Cached: /usr/lib/python3.6/site-packages/dateutil/__pycache__/parser.cpython-36.pyc
Loader: <_frozen_importlib_external.SourceFileLoader object at 0x7f046b1672e8>


✔ ~ 
16:02 $ python -Wd -c 'import dateutil.parser'
✔ ~ 
16:02 $

I am even more confused now...

anntzer · 2017-03-14T20:08:58Z

$ md5sum /var/cache/pacman/pkg/python-3.6.0-2-x86_64.pkg.tar.xz  
931e87301d4c750a5bd66ff3507da484  /var/cache/pacman/pkg/python-3.6.0-2-x86_64.pkg.tar.xz
$ md5sum /var/cache/pacman/pkg/python-dateutil-2.6.0-2-any.pkg.tar.xz 
d4a81b5b18cf55bf4949a7b7be57f12c  /var/cache/pacman/pkg/python-dateutil-2.6.0-2-any.pkg.tar.xz

tacaswell · 2017-03-14T20:11:39Z

16:04 $ md5sum /var/cache/pacman/pkg/python-3.6.0-2-x86_64.pkg.tar.xz                                                                                                                                                                                                                                                                                                                                                                   
931e87301d4c750a5bd66ff3507da484  /var/cache/pacman/pkg/python-3.6.0-2-x86_64.pkg.tar.xz                                                                                                                                                                                                                                                                                                                                                
✔ ~                                                                                                                                                                                                                                                                                                                                                                                                                                     
16:10 $ md5sum /var/cache/pacman/pkg/python-dateutil-2.6.0-2-any.pkg.tar.xz                                                                                                                                                                                                                                                                                                                                                             
d4a81b5b18cf55bf4949a7b7be57f12c  /var/cache/pacman/pkg/python-dateutil-2.6.0-2-any.pkg.tar.xz                                                                                                                                                                                                                                                                                                                                          
✔ ~                                                                                                                                                                                                                                                                                                                                                                                                                                     
16:10 $

anntzer · 2017-03-14T20:16:12Z

Anything in env | grep '^PYTHON'?

tacaswell · 2017-03-14T20:21:31Z

Nope, I can get the warning via:

16:19 $ python -Wd
Python 3.6.0 (default, Jan 16 2017, 12:12:55) 
[GCC 6.3.1 20170109] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import dateutil.parser
>>> s = '\['
<stdin>:1: DeprecationWarning: invalid escape sequence \[
>>>

anntzer · 2017-03-14T20:24:05Z

Ah, I found it.

If you do e.g. sudo touch /usr/lib/python3.6/site-packages/dateutil/parser.py, you will see the warning. (rather, I probably sudo-edited that file at some point just to quickly check on #8271, which changed the mtime, even though I later reverted the change). On the other hand, after I reinstall dateutil using sudo pacman -S --asdeps python-dateutil, the warning goes away.

I guess this is because the warning is only triggered at byte-compilation time, which only occurs if the mtime of the .py is more recent than the mtime of the .pyc -- which is not the case when you install from the package manager, but is the case after my change (and then it was always going to stay like that, as python did not have the rights to rewrite the pyc, so it had to re-byte-compile every time).

tacaswell · 2017-03-14T20:30:46Z

That does make sense. It is re-assuring that the interpreter is, as usual, correct.

QuLogic · 2017-03-14T20:55:48Z

tests.py

@@ -14,15 +14,16 @@

 if __name__ == '__main__':

+    import dateutil.parser


It's good to get rid of these in Matplotlib itself, but it's hacks like these that make me a bit wary to keep this test setup around. Maybe when we figure out the cron jobs, we can run this on a nightly build or something.

FWIW I think this test really belongs in something like pycodestyle.

anntzer added the topic: testing label Mar 13, 2017

tacaswell reviewed Mar 13, 2017

View reviewed changes

tacaswell added this to the 2.1 (next point release) milestone Mar 13, 2017

dstansby approved these changes Mar 14, 2017

View reviewed changes

QuLogic reviewed Mar 14, 2017

View reviewed changes

anntzer changed the title ~~Fix testing with tests.py on Py3.6.~~ [MRG] Fix testing with tests.py on Py3.6. Mar 15, 2017

NelleV changed the title ~~[MRG] Fix testing with tests.py on Py3.6.~~ [MRG+1] Fix testing with tests.py on Py3.6. Mar 16, 2017

QuLogic changed the title ~~[MRG+1] Fix testing with tests.py on Py3.6.~~ Fix testing with tests.py on Py3.6. Mar 19, 2017

QuLogic merged commit c59edad into matplotlib:master Mar 19, 2017

anntzer deleted the fix-deprecated-syntax-py36 branch March 19, 2017 21:23

anntzer mentioned this pull request May 26, 2017

matplotlib does not build on python 3.6 #8271

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix testing with tests.py on Py3.6. #8281

Fix testing with tests.py on Py3.6. #8281

anntzer commented Mar 13, 2017

tacaswell commented Mar 13, 2017

tacaswell Mar 13, 2017

anntzer Mar 13, 2017

anntzer commented Mar 13, 2017

dstansby commented Mar 14, 2017

tacaswell commented Mar 14, 2017

anntzer commented Mar 14, 2017

tacaswell commented Mar 14, 2017

anntzer commented Mar 14, 2017

tacaswell commented Mar 14, 2017

anntzer commented Mar 14, 2017

tacaswell commented Mar 14, 2017

anntzer commented Mar 14, 2017 •

edited

Loading

tacaswell commented Mar 14, 2017

QuLogic Mar 14, 2017

anntzer Mar 14, 2017

		@@ -14,15 +14,16 @@

		if __name__ == '__main__':

		import dateutil.parser

Fix testing with tests.py on Py3.6. #8281

Fix testing with tests.py on Py3.6. #8281

Conversation

anntzer commented Mar 13, 2017

tacaswell commented Mar 13, 2017

tacaswell Mar 13, 2017

Choose a reason for hiding this comment

anntzer Mar 13, 2017

Choose a reason for hiding this comment

anntzer commented Mar 13, 2017

dstansby commented Mar 14, 2017

tacaswell commented Mar 14, 2017

anntzer commented Mar 14, 2017

tacaswell commented Mar 14, 2017

anntzer commented Mar 14, 2017

tacaswell commented Mar 14, 2017

anntzer commented Mar 14, 2017

tacaswell commented Mar 14, 2017

anntzer commented Mar 14, 2017 • edited Loading

tacaswell commented Mar 14, 2017

QuLogic Mar 14, 2017

Choose a reason for hiding this comment

anntzer Mar 14, 2017

Choose a reason for hiding this comment

anntzer commented Mar 14, 2017 •

edited

Loading