Skip to content

Memory leak with CallbackRegistry #19474

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
vallsv opened this issue Feb 7, 2021 · 6 comments
Closed

Memory leak with CallbackRegistry #19474

vallsv opened this issue Feb 7, 2021 · 6 comments
Milestone

Comments

@vallsv
Copy link
Contributor

vallsv commented Feb 7, 2021

Hi,

Here is a report with a possible patch.

Bug report

We had memory leak issue with our library (see silx-kit/silx#3372).

CallbackRegistry can sometimes not clean up stored weakref from released object.

Bug summary

_func_cid_map is only cleaned up if a callback name if fully empty.

We can think of cases where objects are released in order to never make this callback empty.

As result dead weakrefs are still stored in the structure.

Code for reproduction

from matplotlib.cbook import CallbackRegistry

callbacks = CallbackRegistry()

class Foo:
    def callback(x):
        pass

for i in range(100):
    a = Foo()
    callbacks.connect("units", a.callback)
    b = Foo()
    callbacks.connect("units", b.callback)

    print("Is that grow up?", len(callbacks._func_cid_map['units']))

Expected outcome

We expect _func_cid_map to be not bigger than 2.

Matplotlib version

  • 3.3.4
  • git-blame tells it was there 6 years ago

Fix

Without thinking much, i did this hotfix.

     def _remove_proxy(self, proxy, *, _is_finalizing=sys.is_finalizing):
         if _is_finalizing():
             # Weakrefs can't be properly torn down at that point anymore.
             return
         for signal, proxies in list(self._func_cid_map.items()):
             try:
                 del self.callbacks[signal][proxies[proxy]]
+                del self._func_cid_map[signal][proxy]
             except KeyError:
                 pass
             if len(self.callbacks[signal]) == 0:
                 del self.callbacks[signal]
                 del self._func_cid_map[signal]
@anntzer
Copy link
Contributor

anntzer commented Feb 8, 2021

Looks related to #19345 (comment) as well?

@vallsv
Copy link
Contributor Author

vallsv commented Feb 8, 2021

Sounds like.
I will probably create a PR.

@anntzer
Copy link
Contributor

anntzer commented Feb 8, 2021

Thanks, please have a go at it :)

@tacaswell
Copy link
Member

I think that this bug was introduced in 5257c4f when we switch to using normal dicts from WeakKeyDictionary() for the elements in self._func_cid_map (I missed this on review, 🐑 sorry).

@vallsv
Copy link
Contributor Author

vallsv commented Feb 15, 2021

That's life. My PR provides a bit stronger tests.

@QuLogic
Copy link
Member

QuLogic commented Feb 19, 2021

Fixed by #19480.

@QuLogic QuLogic closed this as completed Feb 19, 2021
@QuLogic QuLogic added this to the v3.4.0 milestone Feb 19, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants