Skip to content

[Feature Request] Better default font family that supports all unicode chars #20251

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
tddschn opened this issue May 18, 2021 · 13 comments
Closed

Comments

@tddschn
Copy link

tddschn commented May 18, 2021

Problem

I'm working with a dataset that contains lots of east Asian characters. When trying to plot the data, I got a flood of warnings like this: RuntimeWarning: Glyph <number> missing from current font. font.set_text(s, 0.0, flags=flags), and the generated plot had no readable texts.

Since I'm new to matplotlib and was calling plot from a pandas DataFrame, I couldn't find a trivial way to get the chars displayed correctly. Some suggest downloading a font file and configuring the font family with the path to that file, all seems quite complicated and I couldn't figure how to do that directly from DataFrame.plot arguments.

A few quick searches lead me to this issue from May 2019: #14269 (comment)

That is because the default font (DejaVu Sans) does not have CJKV

I don't think the current default font is a sensible choice since it only support latin characters.

Proposed Solution

  • Dynamically determine the default font family by choosing from a list of available fonts that supports unicode characters.
  • Only fallback to font families with poor or no unicode support when no such fonts are available on users' system.
  • Allow people to work with matplotlib with minimal friction when it comes to unicode text or data.

Additional context and prior art

Warning screenshot:
image

Plot generated:
image

matplotlib version: 3.4.2 (from conda-forge)
OS: macOS 11.3.1

@jklymak
Copy link
Member

jklymak commented May 18, 2021

Does https://matplotlib.org/stable/gallery/text_labels_and_annotations/font_family_rc_sgskip.html help?

This SO post is also quite good: https://stackoverflow.com/questions/45877746/changing-fonts-in-matplotlib

Google doesn't turn up more complete docs on our side, which seems a shame. Are there more @aitikgupta ?

@tddschn
Copy link
Author

tddschn commented May 18, 2021

@jklymak Thanks for the reply!

Does matplotlib.org/stable/gallery/text_labels_and_annotations/font_family_rc_sgskip.html help?

I set font.family to Hack Nerd Font, and this was the output:

from matplotlib import rcParams
rcParams['font.family'] = 'Hack Nerd Font'
rcParams['font.sans-serif'] = ['Hack Nerd Font']
import matplotlib.pyplot as plt
fig, ax = plt.subplots()
ax.plot([1, 2, 3], label='test')

ax.legend()
ax.set_xlabel('测试')
plt.show()

MPL didn't complain about missing glyphs, but the characters are not rendered correctly.

This SO post is also quite good: stackoverflow.com/questions/45877746/changing-fonts-in-matplotlib

I was able to get a list of fonts files (font_list) installed on my machine, but the last line of the code

font_names = [fm.FontProperties(fname=fname).get_name() for fname in font_list]

RuntimeError: In FT2Font: Could not set the fontsize (error code 0x17)

didn't work.

@aitikgupta
Copy link
Contributor

aitikgupta commented May 18, 2021

MPL didn't complain about missing glyphs, but the characters are not rendered correctly.

If there are missing glyphs (text not rendering correctly) - Matplotlib is supposed to emit out the warnings, you might want to try it again?

Are you certain that the font you're using (Hack Nerd Font) contains these glyphs - 测试? I tried reproducing with https://github.com/ryanoasis/nerd-fonts/tree/master/patched-fonts/Hack, it did warn me about the missing glyphs from this font too.

You can follow this blog to use custom fonts with Pandas, another SO discussion you might want to take a look at.

Basically there's 2 ways:

  1. import matplotlib, set custom font as global font.
  2. import matplotlib, define an ax with custom font, and pass that to pandas.

@aitikgupta
Copy link
Contributor

aitikgupta commented May 18, 2021

Google doesn't turn up more complete docs on our side

Unfortunately, we don't have a solid entry point which talks in-depth about using custom fonts - we might be tracking this here: #5941

I'd vote for closing this issue in favour of other similar issues (eg, #17498, #18883) that basically track the proposed solutions mentioned by @tddschn:

  • Dynamically determine the default font family by choosing from a list of available fonts that supports unicode characters.
  • Only fallback to font families with poor or no unicode support when no such fonts are available on users' system.
  • Allow people to work with matplotlib with minimal friction when it comes to unicode text or data.

@jklymak jklymak closed this as completed May 18, 2021
@jklymak
Copy link
Member

jklymak commented May 18, 2021

Sure let's continue discussion at those issues. Thanks!

@tddschn
Copy link
Author

tddschn commented May 19, 2021

@aitikgupta

Unfortunately, we don't have a solid entry point which talks in-depth about using custom fonts

I wonder if there is a MPL predefined font (i.e. non-custom font) that has good unicode support?

@QuLogic
Copy link
Member

QuLogic commented May 19, 2021

Nerd Fonts simply adds a bunch of fancy symbols, so we can ignore that part of it. Going to the Hack fonts (https://sourcefoundry.org/hack/) details, they state:

Over 1500 glyphs that include lovingly tuned extended Latin, modern Greek, and Cyrillic character sets.

That is nowhere near enough glyphs to cover CJK, and if you check it in e.g., fontforge, you will find no CJK glyphs. Your terminal supports glyph fallback and is picking a different font, as is discussed in the other issues.

@tddschn
Copy link
Author

tddschn commented May 19, 2021

@QuLogic You're right.

I found a working solution but it's ugly (PingFang is a system font that supports CJK chars shipped with macOS):

@tddschn
Copy link
Author

tddschn commented May 19, 2021

@aitikgupta

Basically there's 2 ways:
import matplotlib, set custom font as global font.
import matplotlib, define an ax with custom font, and pass that to pandas.

Those are great ideas!
I tried to set the font family for the ax but with no success.
Can you show me how to do both? Thanks!

@tacaswell
Copy link
Member

@jklymak
Copy link
Member

jklymak commented May 19, 2021

Oh good find. I knew we had something like that, but it's not easy to find.

@thedirk
Copy link

thedirk commented Nov 17, 2022

Any progress here? I'm having the same issue, my fonts display properly in my Pandas DataFrame, but if I set the font family, only my non-standard characters display (and the Latin characters do not!).

image

matplotlib.rcParams['font.family'] = ['Noto Sans Tibetan', 'sans-serif']

image

@tacaswell
Copy link
Member

@thedirk Yes, see https://matplotlib.org/stable/users/prev_whats_new/whats_new_3.6.0.html#font-fallback that lets you set up multiple fonts and we will fallback through them until we find the glyph requested.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants