Reduce number of font file handles opened #5295
Conversation
|
Test failure is due to functools32 not working with Python 2.6. Since this is slated for matplotlib 2.0, which is currently planned to drop support for Python 2.6, I don't think it matters. |
|
👍 I think this needs an entry in api_changes.rst |
|
What's the API change? Everything is backward compatible here -- I've just replaced a bunch of independent caches with a central one (that also works correctly). |
|
I mean that backends should now use |
|
That's not strictly true, though. That's only if they want to have the font file cached in memory for later use. There are places still in the matplotlib code base that instantiate an I guess what's at issue is that I see |
|
Status update: This is ready to go once we turn off Python 2.6 (#5215) |
4952394 to
e05d646
Compare
This should hopefully address the long-reported "Too many open files" error message (Fix matplotlib#3315). To reproduce: On a Mac or Windows box with starvation for file handles (Linux has a much higher file handle limit by default), build the docs, then immediately build again. This will trigger the caching bug. The font cache in the mathtext renderer was broken. It was caching a font file once for every *combination* of font properties, including things like size. Therefore, in a complex math expression containing many different sizes of the same font, the font file was opened once for each of those sizes. Font files are opened and kept open (rather than opened, read, and closed) so that FreeType only needs to load the actual glyphs that are used, rather than the entire font. In an era of cheap memory and fast disk, it probably doesn't matter for our current fonts, but once matplotlib#5214 is merged, we will have larger font files with many more glyphs and this loading time will matter more. The solution here is to do all font file loading in one place and to use `lru_cache` (available since Python 3.2) to do the caching, and to use only the file name and hinting parameters as a cache key. For earlier versions of Python, the functools32 backport package is required. (Or we can discuss whether we want to vendor it).
e05d646 to
5e93dfc
Compare
MNT: Reduce number of font file handles opened
|
Backported to v2.0.x as cdea77c |
MNT: Reduce number of font file handles opened
Reduce number of font files opened
This should hopefully address the long-reported "Too many open files"
error message (Fix #3315).
To reproduce: On a Mac or Windows box with starvation for file
handles (Linux has a much higher file handle limit by default), build
the docs, then immediately build again. This will trigger the caching
bug.
The font cache in the mathtext renderer was broken. It was caching a
font file once for every combination of font properties, including
things like size. Therefore, in a complex math expression containing
many different sizes of the same font, the font file was opened once for
each of those sizes.
Font files are opened and kept open (rather than opened, read,
and closed) so that FreeType only needs to load the actual glyphs that
are used, rather than the entire font. In an era of cheap memory and
fast disk, it probably doesn't matter for our current fonts, but once
#5214 is merged, we will have larger font files with many more glyphs
and this loading time will matter more.
The solution here is to do all font file loading in one place and to use
lru_cache(available since Python 3.2) to do the caching, and to useonly the file name and hinting parameters as a cache key. For earlier
versions of Python, the functools32 backport package is required. (Or
we can discuss whether we want to vendor it).
Milestoning as 2.0, since it should be tied to #5214.