Skip to content

Max recursion errors for mathtext on Python 3.6 / OSX #7799

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
matthew-brett opened this issue Jan 11, 2017 · 13 comments
Closed

Max recursion errors for mathtext on Python 3.6 / OSX #7799

matthew-brett opened this issue Jan 11, 2017 · 13 comments

Comments

@matthew-brett
Copy link
Contributor

See: MacPython/matplotlib-wheels#3

Errors occur testing 1.5.3 and current v2.x branch on Python 3.6, OSX.

======================================================================
ERROR: matplotlib.tests.test_mathtext.test_mathtext_cm_53.test
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/travis/build/MacPython/matplotlib-wheels/venv/lib/python3.6/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/Users/travis/build/MacPython/matplotlib-wheels/venv/lib/python3.6/site-packages/matplotlib/testing/decorators.py", line 53, in failer
    result = f(*args, **kwargs)
  File "/Users/travis/build/MacPython/matplotlib-wheels/venv/lib/python3.6/site-packages/matplotlib/testing/decorators.py", line 207, in do_test
    figure.savefig(actual_fname, **self._savefig_kwarg)
  File "/Users/travis/build/MacPython/matplotlib-wheels/venv/lib/python3.6/site-packages/matplotlib/figure.py", line 1563, in savefig

[snip]

 File "/Users/travis/build/MacPython/matplotlib-wheels/venv/lib/python3.6/site-packages/pyparsing.py", line 1510, in _parseCache
    value = cache.get(lookup)
  File "/Users/travis/build/MacPython/matplotlib-wheels/venv/lib/python3.6/site-packages/pyparsing.py", line 1459, in get
    return cache.get(key, not_in_cache)
RecursionError: maximum recursion depth exceeded

https://s3.amazonaws.com/archive.travis-ci.org/jobs/187592465/log.txt

Further investigation reveals:

  • error does not occur when running the individual mathtext tests, e.g. with python tests.py matplotlib.tests.test_mathtext, but only when running these tests as part of a full test suite python tests.py;
  • when dropping into the debugger on the test failure - python tests.py --pdb - this sequence of commands also cause the recursion error:
from matplotlib.font_manager import FontProperties
from matplotlib import mathtext as mt

my_p = mt.Parser()
backend = mt.MathtextBackendPath()
font_output = mt.BakomaFonts(FontProperties(), backend)
my_str = '$\\sqrt{1+\\sqrt{1+\\sqrt{1+\\sqrt{1+\\sqrt{1+\\sqrt{1+\\sqrt{1+x}}}}}}}$'
my_p._state_stack = [my_p.State(font_output, 'default', 'rm', 100, 72)]
my_p._em_width_cache = {}
result = my_p._expression.parseString(my_str)
print(result)

The same sequence of commands run on its own in Python 3.6, passes.

This string also generates the recursion errors, when dropped into the debugger:

r'$\cos{1 + \sqrt{1 + \sqrt{1 + \sqrt{1 + \sqrt{1 + \sqrt{1+\cos{1+x}}}}}}}$'

but not:

r'$\cos{1 + \sqrt{1 + \sqrt{1 + \sqrt{1 + \sqrt{1 + \cos{1+\cos{1+x}}}}}}}$'
r'$\cos{1 + \cos{1 + \cos{1 + \cos{1 + \cos{1 + \cos{1+\cos{1+x}}}}}}}$'
r'$\sqrt{1 + \sqrt{1 + \sqrt{1 + \sqrt{1 + \sqrt{1+\sqrt{1+x}}}}}}$'

It looks like there is some horrible global state in the MathText module or pyparsing that is being triggered only under certain circumstances.

@mdboom - can you reproduce? Any insights into the problem?
@tomkooij tomkooij mentioned this issue Jan 11, 2017
Merged
@tacaswell tacaswell added this to the 2.0.1 (next bug fix release) milestone Jan 11, 2017
@matthew-brett
Copy link
Contributor Author

@mdboom - do you have any time to look into this one? Or any suggestions as to how to investigate further?

@tacaswell
Copy link
Member

Can the recursion limit on travis be pushed up? The test runner adds a whole bunch of layers, I wonder if we were always sitting on the limit?

@matthew-brett
Copy link
Contributor Author

Had a shot at that - but now tests all failing - https://travis-ci.org/MacPython/matplotlib-wheels/jobs/192055053#L1757 - something to do with the results images change?

@jkseppan
Copy link
Member

That Travis log looks like it's installing matplotlib from a wheel, and as of recently the wheels don't include test images.

matthew-brett added a commit to matthew-brett/matplotlib that referenced this issue Jan 16, 2017
@matthew-brett
MAINT: add ability to specify recursionlimit
c013ed7 
We're getting a test failure on OSX and Python 3.6 where parsing reaches the
recursion limit: matplotlib#7799

Upping the recursionlimit resolves the error.

Add command line argument to tests to up the recursion limit for the
test run.
@matthew-brett matthew-brett mentioned this issue Jan 16, 2017
Merged
matthew-brett added a commit to matthew-brett/matplotlib that referenced this issue Jan 16, 2017
@matthew-brett
MAINT: add ability to specify recursionlimit
3434eb5 
We're getting a test failure on OSX and Python 3.6 where parsing reaches the
recursion limit: matplotlib#7799

Upping the recursionlimit resolves the error.

Add command line argument to tests to up the recursion limit for the
test run.
@matthew-brett matthew-brett mentioned this issue Jan 16, 2017
Merged
@matthew-brett
Copy link
Contributor Author

OK - I have solved the test images problem - see discussion over at #7757 (comment) .

Upping the recursion limit does make the tests pass on Python 3.6 - see https://travis-ci.org/MacPython/matplotlib-wheels/jobs/192072933

I've submitted a couple of PRs to add the ability to specify the recursion limit to the test function, see #7843, #7849.

Once those are merged, I think wheel building will be OK for master and for the 2.x branch.

@tacaswell
Copy link
Member

I can now reproduce this on linux as will, but only if I run the test suite not parallel.

@tacaswell
Copy link
Member

Also, I wonder if that very deep recursion a performance bottle neck that we should be looking into.

@matthew-brett
Copy link
Contributor Author

It's certainly crazy slow to do the parsing - at one point I ran 1000 parses of that multiply nested sqrt string, and it took something like 30 seconds.

@dstansby
Copy link
Member

I've just run a plain pytest on python 3.6, OSX, and I'm not getting the above errors. Perhaps this was fixed by all the recent changes to testing?

@QuLogic
Copy link
Member

QuLogic commented Feb 22, 2017

Plain pytest does not run any of @matthew-brett's changes since those are part of matplotlib.test which isn't called that way. I don't recall much that would have had an effect here, though. And 3.6 on Travis/Linux never seemed to have the problem...

@tacaswell
Copy link
Member

What does running plain pytest miss?

@QuLogic
Copy link
Member

QuLogic commented Feb 22, 2017

Anything custom in tests.py and matplotlib.test(), so the recursion limit increase and the warning filter thing. But the default recursion limit is to not change anything anyway, and I think we've already fixed everything that triggers the warnings.

@QuLogic QuLogic modified the milestones: 2.0.1 (next bug fix release), 2.0.2 (next bug fix release) May 3, 2017
@tacaswell tacaswell modified the milestones: 2.1.1 (next bug fix release), 2.2 (next feature release) Oct 9, 2017
@QuLogic
Copy link
Member

QuLogic commented Nov 23, 2021

We don't support Python 3.6, Matplotlib 2(ish), and the recursion limit stuff was removed, so I think there's nothing to do here.

@QuLogic QuLogic closed this as completed Nov 23, 2021
@story645 story645 removed this from the future releases milestone Oct 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@matthew-brett @jkseppan @tacaswell @QuLogic @story645 @dstansby