Skip to content

check_figures_equal decorator reuses figures if called multiple times inside a single test. #15079

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
andrewzwicky opened this issue Aug 18, 2019 · 4 comments · Fixed by #15589
Closed

check_figures_equal decorator reuses figures if called multiple times inside a single test. #15079

andrewzwicky opened this issue Aug 18, 2019 · 4 comments · Fixed by #15589
Labels
Good first issue Open a pull request against these issues if there are no active ones! topic: testing
Milestone
v3.3.0

Comments

@andrewzwicky
Copy link

Bug report

Bug summary

When running a test suite with multiple tests using check_figures_equal, figures are re-used, causing plot elements to become "stacked", causing tests to unexpectedly fail.

Code for reproduction

from matplotlib.testing.decorators import check_figures_equal
from matplotlib import pyplot as plt

@check_figures_equal(extensions=["png"])
def test_red_line_first(test_figure, reference_figure):
    red_circle = plt.Circle((0, 0), 0.2, color='r', clip_on=False)
    reference_figure.add_artist(red_circle)
    reference_figure.subplots().plot([0,1,2], [3,4,5], color='red')    


@check_figures_equal(extensions=["png"])
def test_blue_line_second(test_figure, reference_figure):
    blue_circle = plt.Circle((1, 1), 0.2, color='b', clip_on=False)
    reference_figure.add_artist(blue_circle)
    reference_figure.subplots().plot([0,1,2], [4,4,4], color='blue')


@check_figures_equal(extensions=["png"])
def test_green_line_third(test_figure, reference_figure):
    green_circle = plt.Circle((1, 0), 0.2, color='g', clip_on=False)
    reference_figure.add_artist(green_circle)
    reference_figure.subplots().plot([0,1,2], [5,4,3], color='green')

running as:
pytest .\mpl_figure_equal_test.py

Actual outcome
Reference Screenshots:
test_red_line_first-expected
test_blue_line_second-expected
test_green_line_third-expected

Expected outcome
Reference Screenshots:
test_red_line_first-expected
test_blue_line_second-expected
test_green_line_third-expected

Matplotlib version

  • Operating system: Windows 10
  • Matplotlib version: matplotlib 3.1.0
  • Matplotlib backend (print(matplotlib.get_backend())): TkAgg
  • Python version: 3.7.3
  • Jupyter version (if applicable):
  • Other libraries: pytest 4.5.0

everything installed via pip

matplotlib/lib/matplotlib/testing/decorators.py

Lines 394 to 395 in e86c9cd

fig_test = plt.figure("test")
fig_ref = plt.figure("reference")

I'm guessing these lines are at fault, and might need some kind of other information to distinguish them?
@anntzer
Copy link
Contributor

anntzer commented Aug 18, 2019 •
edited
Loading

Probably one should just wrap the whole thing in a try... finally: plt.close(fig_test); plt.close(fitg_ref)? In our own test suite this is done by an autouse fixture but I guess adding the close() to check_figures_equal makes it more usable by third-party packages.

@QuLogic
Copy link
Member

QuLogic commented Aug 18, 2019

All our tests autouse a fixture from matplotlib.testing.conftest that closes all figure at the end of a test. If you're not running these as Matplotlib tests, you'll need to replicate that behaviour.

@andrewzwicky
Copy link
Author

andrewzwicky commented Aug 18, 2019 •
edited
Loading

Both of those make sense. Is check_figures_equal intended for use in third-party projects? It seems quite useful, so it might make sense to either update the documentation re: extra testing steps for closing figures or include that functionality to close just those two figures. Do maintainers have a preference? It seems like a relatively small thing I can contribute.

I just tested a small fix with try/finally and it worked as I expected. I think having to add extra figure closing stuff in my own test suite would make me scratch my head a bit, so I'd prefer the plt.close included in the wrapper, but I understand if that's not desirable for some other reason.

@anntzer
Copy link
Contributor

anntzer commented Aug 18, 2019

I'm personally fine with a pr adding the call to close.

@anntzer anntzer added the Good first issue Open a pull request against these issues if there are no active ones! label Aug 26, 2019
@MarcoGorelli MarcoGorelli mentioned this issue Nov 2, 2019
Merged
6 tasks
@tacaswell tacaswell added this to the v3.3.0 milestone Nov 2, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Good first issue Open a pull request against these issues if there are no active ones! topic: testing
Projects
None yet
Milestone
v3.3.0
Development

Successfully merging a pull request may close this issue.

Make sure that figures are closed when check_figures_equal finishes MarcoGorelli/matplotlib
5 participants
@tacaswell @QuLogic @anntzer @andrewzwicky @dstansby