Skip to content

FIX: Decrease figure refcount on close of a macosx figure #23059

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Apr 26, 2023

Conversation

greglucas
Copy link
Contributor

PR Summary

We need to take the refcount of the figuremanager down when we close a window. (apparently dealloc on the Window isn't being called?) We could call this within the FigureManager_dealloc method, or just inline it here because close gets called during the FigureManager_dealloc already, so I decided this simplifies the code.

This was causing leaking figures before, per: #19769
and this PR does fix the issue on v3.5.x, but unfortunately there is a new leak that I introduced in #21790 :( There are no leaks when we plt.show() figures and fire up the GUI mainloop. However, when we are calling savefig() without ever showing we are now creating singleshot Timers in the app, but the callback never gets triggered due to the event loop not being active/listening for these timers. This is then keeping a reference from the Timer to the Figure on every close, causing a leak when calling a bunch of savefig's.

PR Checklist

Tests and Styling

  • [N/A] Has pytest style unit tests (and pytest passes).
  • [N/A] Is Flake 8 compliant (install flake8-docstrings and run flake8 --docstring-convention=all).

Documentation

  • [N/A] New features are documented, with examples if plot related.
  • [N/A] New features have an entry in doc/users/next_whats_new/ (follow instructions in README.rst there).
  • [N/A] API changes documented in doc/api/next_api_changes/ (follow instructions in README.rst there).
  • [N/A] Documentation is sphinx and numpydoc compliant (the docs should build without error).

@jklymak
Copy link
Member

jklymak commented May 18, 2022

Is there a way to test this that you used?

@greglucas
Copy link
Contributor Author

I was modifying: #19769 (comment)
then:
mprof run script.py

Which will send some count information out, which should show FigureManager in it before and fewer of them now. If you cherry-pick this commit over to v3.5.x and test you will see a major difference though and no more memory leak/FigureManager at all, but I still need to figure out how to fix the second Timer memory leak now... So, this fixes one of the leaks, but not both unfortunately. A step in the right direction...

@greglucas
Copy link
Contributor Author

I just pushed a new commit: 0822fd0 that briefly runs the event loop before destroying the figure. This seems to clean up the memory leak. The idea was taken from: #23061 (comment), or at least I think that is what @tacaswell meant! I tried to use "flush_events", which didn't work, so maybe that should be modified (?) as well. However, if I tried to put the start_event_loop call within flush_events in the objective-c layer, even that didn't seem to trigger this, so we may have to call the start_event_loop from Python to get a different event loop going... This is definitely getting out of my area now, so if others have a better idea to try, let me know!

To test this, you should be able to run mprof run script.py followed by mprof plot and get a slowly growing memory count, rather than a rapidly growing one (around 500 MB before this fix).
image

There may be another memory leak in FontProperties, as I'm seeing that hang around in the growing counts here, so I don't get that jagged line like others were upon garbage collection, even with Agg as the backend.
image

@greglucas
Copy link
Contributor Author

Just putting some notes down here about the mac leaks I've found so far:

  • Setting the backend to "agg" on a mac currently has a slow leak when installing from source and using the older pinned Freetype version. I grabbed the newest Freetype (2.12.1) and the leak went away on the Agg backend. There was likely an (apparently mac only) leak that was fixed in Freetype sometime over the past several versions, but I couldn't find anything immediately obvious in their changelog, although they have fixed multiple memory leaks.
  • With the updates here the macosx backend still slowly leaks (even with an updated Freetype) as shown in the Figure above, but it is much lower than before.
  • Tracking down memory leaks between C and Python code is not trivial, especially when multiple external libraries and leaks are involved :)

ysasano added a commit to ysasano/gaii that referenced this pull request Jun 7, 2022
MacOSXでsavefigするとメモリリークする問題(※)への対応
# [Bug]: macosx timers don't fire if plt.show() hasn't been called
#    matplotlib/matplotlib#23061
# FIX: Decrease figure refcount on close of a macosx figure #23059
#    matplotlib/matplotlib#23059
#
# 解決策: 描画処理を別プロセスに隔離するように変更
# 参考にしたページ: https://qiita.com/Masahiro_T/items/037c118c9dd26450f8c8
@timhoffm
Copy link
Member

timhoffm commented Jun 25, 2022

Can you please rebase to confirm that all (likely unrelated) CI problems are gone?

Copy link
Member

@tacaswell tacaswell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we are going to run the event loop on behalf of the user we should probably wrap it in a check that the event loop is not already running like we do with Tk.

In the tests we can just force the event loop to run more aggressively.

@jklymak
Copy link
Member

jklymak commented Sep 8, 2022

Popping to draft pending revision...

@QuLogic QuLogic modified the milestones: v3.6.0, v3.6.1 Sep 14, 2022
@QuLogic QuLogic modified the milestones: v3.6.1, v3.6.2 Oct 6, 2022
@QuLogic QuLogic modified the milestones: v3.6.2, v3.6.3 Oct 27, 2022
@QuLogic QuLogic modified the milestones: v3.6.3, v3.7.0 Jan 11, 2023
@greglucas greglucas modified the milestones: v3.7.0, v3.8.0 Jan 12, 2023
@greglucas greglucas force-pushed the macosx-manager-refcount branch from 0822fd0 to 364194e Compare March 19, 2023 15:12
@greglucas greglucas marked this pull request as ready for review March 19, 2023 15:12
@greglucas
Copy link
Contributor Author

OK, I've reworked this now to keep track of the timers on the canvas and then iterate through them on close to clear out any pending timers, let me know your thoughts on this approach instead of the event loop spinning.

The problem is that we don't ever start the mainloop when just calling savefig() and thus all the draw_idle() calls have built up pending single-shot timers that we don't want to hold any back-references after the figure is destroyed. In the previous attempt, I was running the event loop for the user to trigger all these timers (and thus clear them) which @tacaswell rightfully said we shouldn't do for users who don't want us doing that!

@dstansby pinging you as well to see if this fixes what you were seeing since you were looking into some of this recently: #19769 (comment)

We need to take the refcount of the figuremanager down when we
close a window. This was leaking figures before.
Running the macosx backend without calling show() would cause
Timers to pile up and not fire because the event loop was not running.
This leaked objects when closing/opening multiple figures. To fix this,
we keep track of the timers on the canvas and remove all of the pending
timers when destroying the figure to clear all back-references.
@greglucas
Copy link
Contributor Author

@tacaswell since you're currently blocking, I'll ping for another look at the event loop work now that I've updated it with a set of timers and get your thoughts on that approach.

@tacaswell tacaswell dismissed their stale review April 26, 2023 15:13

Very old.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants