Skip to content

1.4.0rc1: Python-level memory "leak" (internal font cache?) #3264

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
letmaik opened this issue Jul 15, 2014 · 2 comments
Closed

1.4.0rc1: Python-level memory "leak" (internal font cache?) #3264

letmaik opened this issue Jul 15, 2014 · 2 comments
Labels
Release critical For bugs that make the library unusable (segfaults, incorrect plots, etc) and major regressions.
Milestone
v1.4.0

Comments

@letmaik
Copy link
Contributor

letmaik commented Jul 15, 2014

I'm currently testing the 1.4.0rc1 (had to change the min_version of freetype2 to 1.3, see #3262). It seems that the memory leak reported in #3197 is gone, but I still see a very slow but considerable increase in resident memory, and in fact I can observe a constant Python object count growth of the following classes:

__builtin__.list 14645  +114
__builtin__.dict    18076   +114
matplotlib.font_manager.FontProperties  761 +114

(measured using objgraph)

So I think between 1.3.1 and 1.4.0rc1 some kind of new "leak" appeared on Python level. It's not a real leak as the objects are accessible, so I guess it's a kind of internal font (property) cache which doesn't clear up old entries.

Test script:

import numpy as np
import matplotlib as mpl
mpl.use('agg')
import matplotlib.pyplot as plt

for i in range(100000):
    fig, ax = plt.subplots()         

    circle1=plt.Circle((0,0),2,color='r',rasterized=True)
    ax.add_artist(circle1)

    fig.savefig('test.svg')
    plt.close(fig)

EDIT: I had measured object count growth for 1.3.1 also, and there was no growth of any class there.
@letmaik letmaik changed the title 1.4.0rc1: Python-level "leak" (internal font cache?) 1.4.0rc1: Python-level memory "leak" (internal font cache?) Jul 15, 2014
@WeatherGod
Copy link
Member

I think this is a fantastic example of the "many eyes" situation in
software development. We have been so focused on finding actual leaks using
valgrind that we didn't really think to see if there were any other
increases in memory usage. This is certainly a very interesting result.
Thank you (and thanks for mentioning objgraph, I never heard of it and it
looks useful).

On Tue, Jul 15, 2014 at 10:45 AM, Maik Riechert notifications@github.com
wrote:

I'm currently testing the 1.4.0rc1 (had to change the min_version of
freetype2 to 1.3, see #3262
#3262). It seems that
the memory leak reported in #3197
#3197 is gone, but I
still see a very slow but considerable increase in resident memory, and in
fact I can observe a constant object count growth of the following classes:

builtin.list 14645 +114
builtin.dict 18076 +114
matplotlib.font_manager.FontProperties 761 +114

(measured using objgraph)

So I think between 1.3.1 and 1.4.0rc1 some kind of new "leak" appeared on
Python level. It's not a real leak as the objects are accessible, so I
guess it's a kind of internal font (property) cache which doesn't clear up
old entries.

Test script:

import numpy as npimport matplotlib as mplmpl.use('agg')import matplotlib.pyplot as plt
for i in range(100000):
fig, ax = plt.subplots()

circle1=plt.Circle((0,0),2,color='r',rasterized=True)
ax.add_artist(circle1)

fig.savefig('test.svg')
plt.close(fig)

—
Reply to this email directly or view it on GitHub
#3264.

@tacaswell tacaswell added this to the v1.4.0 milestone Jul 15, 2014
@tacaswell
Copy link
Member

This is probably related to the change to re-setting the font-cache (#3077).

tacaswell added a commit to tacaswell/matplotlib that referenced this issue Jul 18, 2014
@tacaswell
BUG : fixes FontProperties memory leak
a3b18a2 
According to the data
model (https://docs.python.org/2/reference/datamodel.html#object.__hash__)
objects that define __hash__ need to define __eq__ or __cmp__.  By
default, all user defined classes have a __cmp__ which evaluates to
False for all other objects.

Previously we do not define either __cmp__ or __eq__ on FontProperties,

This results in never finding the property
cached in the dict, hence the growth in the number of
FontProperties (and I assume the stuff in them).

By adding __eq__ and __ne__ we complete the data model and adding
FontProperties to dictionaries should work as expected.

This was not a problem before, but in matplotlib#3077 the caching key was changed
from hash(prop) -> prop.

Closes matplotlib#3264
@tacaswell tacaswell mentioned this issue Jul 18, 2014
Merged
joferkington pushed a commit to joferkington/matplotlib that referenced this issue Jul 23, 2014
@tacaswell @joferkington
BUG : fixes FontProperties memory leak
4285ede 
According to the data
model (https://docs.python.org/2/reference/datamodel.html#object.__hash__)
objects that define __hash__ need to define __eq__ or __cmp__.  By
default, all user defined classes have a __cmp__ which evaluates to
False for all other objects.

Previously we do not define either __cmp__ or __eq__ on FontProperties,

This results in never finding the property
cached in the dict, hence the growth in the number of
FontProperties (and I assume the stuff in them).

By adding __eq__ and __ne__ we complete the data model and adding
FontProperties to dictionaries should work as expected.

This was not a problem before, but in matplotlib#3077 the caching key was changed
from hash(prop) -> prop.

Closes matplotlib#3264
@letmaik letmaik mentioned this issue Jul 28, 2014
Closed
jbmohler pushed a commit to jbmohler/matplotlib that referenced this issue Aug 14, 2014
@tacaswell
BUG : fixes FontProperties memory leak
4665ed6 
According to the data
model (https://docs.python.org/2/reference/datamodel.html#object.__hash__)
objects that define __hash__ need to define __eq__ or __cmp__.  By
default, all user defined classes have a __cmp__ which evaluates to
False for all other objects.

Previously we do not define either __cmp__ or __eq__ on FontProperties,

This results in never finding the property
cached in the dict, hence the growth in the number of
FontProperties (and I assume the stuff in them).

By adding __eq__ and __ne__ we complete the data model and adding
FontProperties to dictionaries should work as expected.

This was not a problem before, but in matplotlib#3077 the caching key was changed
from hash(prop) -> prop.

Closes matplotlib#3264
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Release critical For bugs that make the library unusable (segfaults, incorrect plots, etc) and major regressions.
Projects
None yet
Milestone
v1.4.0
Development

No branches or pull requests
3 participants
@tacaswell @WeatherGod @letmaik