Skip to content

Simplify file handling in ft2font. #15104

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
May 12, 2020
Merged

Conversation

anntzer
Copy link
Contributor

@anntzer anntzer commented Aug 22, 2019

Just call the Python-level seek(), read() and close() instead of trying
to play with C-level FILE*.

Note that unlike the png case (#15095), we can't just pass restrict ourselves to
passing in file-like objects because FT2Font is public API.

PR Summary

PR Checklist

  • Has Pytest style unit tests
  • Code is Flake 8 compliant
  • New features are documented, with examples if plot related
  • Documentation is sphinx and numpydoc compliant
  • Added an entry to doc/users/next_whats_new/ if major new feature (follow instructions in README.rst there)
  • Documented in doc/api/api_changes.rst if API changed in a backward-incompatible way

open_args->flags = FT_OPEN_STREAM;
open_args->stream = &self->stream;
} else {
if (PyObject_HasAttrString(py_file_arg, "read") &&
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

... or alternatively we can just read the whole file in memory, as previously the case in this branch...

@timhoffm
Copy link
Member

Note that unlike the png case (#15095), we can't just pass restrict ourselves to
passing in file-like objects because FT2Font is public API.

Does it have to be? Otherwise it might be reasonable to deprecate its use and clean up later.

@anntzer
Copy link
Contributor Author

anntzer commented Aug 22, 2019

well, that can happen later :)

}

return 0;
memcpy(buffer, tmpbuf, n_read);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have any insights how often libpng calls this callback?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My guess is just once to load the file, but no guarantees here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is FreeType, not libpng. It's called pretty repeatedly, maybe for every glyph? For example, in examples/misc/ftface_props.py, it's called 68 times, while in examples/showcase/anatomy.py, it's called 8064 times. There are not 8000 glyphs in that figure, so I guess it's even more than once per glyph. Maybe it's improved in newer versions, but over half (4574) of those are called with 0 count.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like calls with count=0 are simply used, well, to seek() the file, but given that FT only interacts with the file via this callback, and that read offsets are always given from the file start, we could even just skip pure seek()s (i.e. immediately return if count=0) as later read()s will anyways pass an absolute offset. Adding the early return doesn't seem to break anything, do you want that?

}

return 0;
memcpy(buffer, tmpbuf, n_read);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is FreeType, not libpng. It's called pretty repeatedly, maybe for every glyph? For example, in examples/misc/ftface_props.py, it's called 68 times, while in examples/showcase/anatomy.py, it's called 8064 times. There are not 8000 glyphs in that figure, so I guess it's even more than once per glyph. Maybe it's improved in newer versions, but over half (4574) of those are called with 0 count.

@anntzer
Copy link
Contributor Author

anntzer commented Mar 10, 2020

Even with the latest FT there's a gazillion reads.
So the original version was (essentially) relying on C-level fread()s, whereas this one cycles through Python for each read. (I guess we don't want to read the whole file into memory at once per the discussion at #15410.) So I guess I need to profile to check how much of a performance cost this is? Or alternatively deprecate the FT2Font(<file-like>) API (which we don't actually use) (or just accept that it'll go through Python reads), only support FT2Font(<path-like>), and open the file from C?

@efiring
Copy link
Member

efiring commented May 4, 2020

It's not clear to me that this PR is solving a problem or improving performance. Is there an expected benefit other than reduced lines of code?

@anntzer
Copy link
Contributor Author

anntzer commented May 4, 2020

It's mostly just deleting quite a bit of code (so basically maintenance) and should(?) not have effect on performance (although @QuLogic points out this may actually be a bit slower, but I'm not sure?). And may or may not close #15410, at least it deletes the line that crashes...

@efiring
Copy link
Member

efiring commented May 7, 2020

Closes #15410, based on a test supplied there.

@QuLogic
Copy link
Member

QuLogic commented May 9, 2020

FWIW, running asv with just the basic tests, it does not seem to think there are significant changes here:

· Fetching recent changes.
· Fetching recent changes.
· Creating environments
· Discovering benchmarks
·· Uninstalling from conda-py3.7
·· Installing 36540bf7 <pull/15104/head> into conda-py3.7.
· Running 8 total benchmarks (2 commits * 1 environments * 4 benchmarks)
[  0.00%] · For matplotlib commit 54072564 <pull/16108/head~1> (round 1/2):
[  0.00%] ·· Building for conda-py3.7.
[  0.00%] ·· Benchmarking conda-py3.7
[  6.25%] ··· Running (basic.time_plot--)....
[ 25.00%] · For matplotlib commit 36540bf7 <pull/15104/head> (round 1/2):
[ 25.00%] ·· Building for conda-py3.7..
[ 25.00%] ·· Benchmarking conda-py3.7
[ 31.25%] ··· Running (basic.time_plot--)....
[ 50.00%] · For matplotlib commit 36540bf7 <pull/15104/head> (round 2/2):
[ 50.00%] ·· Benchmarking conda-py3.7
[ 56.25%] ··· basic.time_plot                                       62.0±0.3ms
[ 62.50%] ··· basic.time_projection                                         ok
[ 62.50%] ··· ============= ============
                   proj                 
              ------------- ------------
               rectilinear   44.2±0.2ms 
                  polar      68.4±0.1ms 
                  aitoff     68.8±0.2ms 
                  hammer     69.8±0.6ms 
                mollweide    74.8±0.5ms 
                 lambert     61.2±0.7ms 
              ============= ============

[ 68.75%] ··· basic.time_savefig                                    56.3±0.1ms
[ 75.00%] ··· basic.time_subplots                                           ok
[ 75.00%] ··· ==== ============
               N               
              ---- ------------
               1    44.1±0.1ms 
               2    156±0.8ms  
               10   2.46±0.02s 
              ==== ============

[ 75.00%] · For matplotlib commit 54072564 <pull/16108/head~1> (round 2/2):
[ 75.00%] ·· Building for conda-py3.7.
[ 75.00%] ·· Benchmarking conda-py3.7
[ 81.25%] ··· basic.time_plot                                       61.5±0.4ms
[ 87.50%] ··· basic.time_projection                                         ok
[ 87.50%] ··· ============= ============
                   proj                 
              ------------- ------------
               rectilinear   43.8±0.4ms 
                  polar      68.4±0.2ms 
                  aitoff     68.4±0.7ms 
                  hammer     69.1±0.7ms 
                mollweide    74.0±0.4ms 
                 lambert      62.5±4ms  
              ============= ============

[ 93.75%] ··· basic.time_savefig                                    56.1±0.3ms
[100.00%] ··· basic.time_subplots                                           ok
[100.00%] ··· ==== ============
               N               
              ---- ------------
               1    43.9±0.2ms 
               2    155±0.6ms  
               10   2.50±0.02s 
              ==== ============


BENCHMARKS NOT SIGNIFICANTLY CHANGED.

But note these are just simple plots and don't heavily tax the text rendering routines.

@tacaswell
Copy link
Member

By executive decision, @QuLogic can merge this when happy.

anntzer added 2 commits May 11, 2020 22:05
Just call the Python-level seek(), read() and close() instead of trying
to play with C-level FILE*.

Note that unlike the png case, we can't just pass restrict ourselves to
passing in file-like objects because FT2Font is public API.
@anntzer
Copy link
Contributor Author

anntzer commented May 11, 2020

I handled the non-performance related comments in a separate commit (please squash when merging).

  • close_file was removed from PyFT2Font, we simply set the close callback to NULL when the file shouldn't be closed.
  • Correctly use Py_CLEAR instead of Py_DECREF -- and also do so for init failures, which lets us get rid of PyFT2Font_fail by inlining it. This means that if an error occurrent, we can't pass self->py_file to PyErr_WriteUnraisable anymore as it'll be possibly NULL, so we pass self instead.

@anntzer
Copy link
Contributor Author

anntzer commented May 11, 2020

... and inlined convert_open_args as it's now short enough that I think inlining it is actually clearer.

@QuLogic QuLogic merged commit 965e533 into matplotlib:master May 12, 2020
@anntzer anntzer deleted the ft2fontfile branch May 12, 2020 07:19
@Anthchirp
Copy link

Would this PR (and #17510) be suitable for a backport to the 3.2.x release? I'm asking because we are still running into #15410 in the wild, and the 3.3.0 release is some time away.

@QuLogic
Copy link
Member

QuLogic commented Jul 10, 2020

It's not a bug fix, so no, that's unlikely. And 3.3 is soon, you can already install the release candidate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants