Use DejaVu fonts as default for text and mathtext #5214

zblz · 2015-10-08T18:10:45Z

In preparation for #4636 and #5128, this PR adds options to use DejaVu fonts as mathtext fonts. In case glyphs are missing, the fallback is STIX, given that they will have a more similar weight than CM.

DejaVu fonts are not included here, they should be added in a PR that sets DejaVu as default replacing Vera (#4636 or replacement). Do not merge before #4636!

edit: this initial message is outdated, the PR does add the DejaVu fonts and mathtext DejaVu options and sets them as default. See below messages for details.

hetland · 2015-10-08T19:56:22Z

@mdboom asked to do a PR to add DejaVu, but did not have a chance to do that yet. I believe this takes care of that, so I won't duplicate. Also, I agree that STIX should be the fallback for missing fonts, when DejaVu does not have them. There is also a LaTeX math font called arev that I use for presentations (using LaTeXit) that looks nice (http://mirrors.rit.edu/CTAN/info/Free_Math_Font_Survey/en/survey.pdf). However, I am not sure how complicated it would be to use in matplotlib. But that may be yet another option that could be used instead of falling back to computer modern.

mdboom · 2015-10-08T20:04:40Z

Looks good, thanks!

zblz · 2015-10-08T20:16:47Z

@hetland: No, this PR does not actually add DejaVu or make it default, it only allows to use mathtext.fontset = dejavusans in the configuration (or do plt.rcParams['mathtext.fontset']='dejavusans') and DejaVu Sans will be used in the math sections if usetex = False. Do you want me to do it or will you modify #4636? Note that with this PR the only changes you would need in the mathtext section of rcsetup.py is to set the default of mathtext.fontset to dejavusans.

AFAIK, the arev latex package is just a packaging of Arev (with other glyphs from mathdesign), so should be equivalent to using DejaVu.

hetland · 2015-10-08T20:23:19Z

I think the idea was to make DejaVu (was Arev) the default font for text
and math. It would be easier if you could modify your pull request to make
this small change, so it is all in one place.

I agree that the area package is identical to DejaVu (actually, DejaVu may
have more by now), but it may be useful to have latex render fonts as pdf
snippets using arev as a fallback. I'm not sure how may people still use
this option -- I think mathtext should definitely remain the preferred
default rendering engine.

On Thu, Oct 8, 2015 at 3:16 PM, Victor Zabalza notifications@github.com
wrote:

@hetland https://github.com/hetland: No, this PR does not actually add
DejaVu or make it default, it only allows to use mathtext.fontset =
dejavusans in the configuration (or do
plt.rcParams['mathtext.fontset']='dejavusans') and DejaVu Sans will be
used in the math sections if usetex = False. Do you want me to do it or
will you modify #4636 #4636?
Note that with this PR the only changes you would need in the mathtext
section of rcsetup.py is to set the default of mathtext.fontset to
dejavusans.

AFAIK, the arev latex package is just a packaging of Arev (with other
glyphs from mathdesign), so should be equivalent to using DejaVu.

—
Reply to this email directly or view it on GitHub
#5214 (comment)
.

Prof. Rob Hetland
Texas A&M Univ. – Dept. of Oceanography
http://pong.tamu.edu/~rob

zblz · 2015-10-08T20:38:35Z

Ok, I'll close your PR and change the title for this one.

I'll take a look at consistency with selecting the font for usetex, but if
arev is not in the core TeX distribution, it will mean trouble when users
find LaTeX errors because they don't have arev installed. Right now
sans-serif TeX output defaults to computer modern sans serif (which is
obviously in all TeX distributions), and the only alternative sans-serif
fonts in core TeX are Helvetica and Avant Garde, and I'm not sure about
their math coverage.

Mathtext is definitely preferred --it's much faster and portable-- but
usetex has a clear use case for complex math and the typography of
mathtext will never be at the level of TeX. I often use it when I want a
final plot to have the exact same fonts as the paper it will be embedded in.

jkseppan · 2015-10-09T05:45:24Z

Just a comment about TeX fonts: I think http://www.tug.dk/FontCatalogue/mathfonts.html is the most complete list of fonts that have TeX math support.

mdboom · 2015-10-12T20:20:08Z

lib/matplotlib/mpl-data/fonts/ttf/local.conf

-                        <family>Bitstream Vera Sans Mono</family>
-                </prefer>
-        </alias>
-</fontconfig>


I actually didn't realize this was here. I think this should be updated to use DejaVu rather than just deleted. It sets the default font types if a user is opting for the font-config mode rather than using matplotlib's own font lookup system.

It's not strictly necessary, but removing this will certainly change behavior for those that are relying on it.

I thought it was part of the Vera release and not being actually used. If it is used I will definitely put it back and modify it.

Is there any particular reason for removing the old fonts is stead of just adding the new ones and changing the defaults. For all other changes we have made them backwards compatible and added the old default to the classic style

DejaVu are a superset of Bitstream Vera, so it would be redundant to ship them both. The classic style can still have mathtext.fontset : cm. However, it is true that it might cause problems with matplotlibrc that have been modified to have a single font.sans-serif : Bitstream Vera Sans rather than a list...

Regarding shipping them both, I just realised that even though the DejaVu fonts are 5.3 MB, the Vera fonts are only 600 KB, so from a distribution size perspective it is not a huge difference shipping them both rather than removing Vera.

@jenshnielsen: As @zblz said, since DejaVu is a superset of Vera, using DejaVu whenever Vera is requested should have no visible impact other than missing characters now being present, which I'd call a bugfix, not a behavior change. I don't think there's any benefit to keeping the old font, particularly with font subsetting working so well in all backends.

That said, it might be worth adding a built-in mapping from Vera to DejaVu, so that if a user was explicitly setting Vera (rather than just leaving the defaults), and they don't have it installed system wide, they will get DejaVu and things continue to work. This is the kind of thing where some extra release candidate testing may be necessary to catch all of the environmental gotchas about this change.

mdboom · 2015-10-13T12:36:20Z

Note to distributors: @sandrotosi, @jspaleta, others: I think Debian and Fedora are removing the fonts vendored with matplotlib and requiring the system packages of those fonts instead. Just a heads up that we are changing our main vendored font from "Bitstream Vera" to "DejaVu" and that package requirement will need to be updated as well. (And this is expected to land in matplotlib 2.0)

@zblz: This should get a "What's New" entry.

toddrjen · 2015-10-13T12:50:45Z

On Oct 13, 2015 2:36 PM, "Michael Droettboom" notifications@github.com
wrote:

Note to distributors: @sandrotosi, @jspaleta, others: I think Debian and
Fedora are removing the fonts vendored with matplotlib and requiring the
system packages of those fonts instead. Just a heads up that we are
changing our main vendored font from "Bitstream Vera" to "DejaVu" and that
package requirement will need to be updated as well. (And this is expected
to land in matplotlib 2.0)

@zblz: This should get a "What's New" entry.

Is this something they are doing by hand? If so, might this be something
setup.py could support?

sandrotosi · 2015-10-13T13:02:10Z

thanks a ton @mdboom for the heads up! we are indeed (trying to) use the debian font files where possible instead of distributed ones, so it's important to know. @toddrjen yes, at least in debian we are doing it by hand, as we inspect the shipped fonts, then we need to match with the currenctly available debian fonts, add the dependencies to the binary package and then create the symlinks in the mpl namespace

zblz · 2015-10-14T11:06:30Z

I have added the mathtext baseline images, squashed a few commits, rebased with master to fix a doc-building warning, and added a what's new file.

I have changed a bit the mathtext fontset classes to reuse code and fix the fallback for alternative sized symbols, which was broken (and one of the main reasons to fallback onto STIX).

zblz · 2015-10-20T10:28:11Z

The advance of the point glyph in mathtext with this PR is slightly too large. It doesn't happen on regular text. I'll look a bit deeper and try to find the reason.

Tillsten · 2015-10-20T10:36:39Z

To support the classic mpl style Vera should not be removed. Btw wonderful work, i really disliked vera.

zblz · 2015-10-20T11:13:30Z

@Tillsten -- DejaVu is a superset of Vera, so that the only difference between the new and old default (for regular text) will be when Vera falls back to CM. That could be argued to be a bug, so I'm not sure we should retain the option of a small character set in the classic mpl style.

mdboom · 2015-10-20T11:39:49Z

@Tillsten: Deja Vu is the same font as Vera under a new name, with additional glyphs added and bugs fixed (I think historically it was a fork). There is no need to retain the old font for the classic style.

This should hopefully address the long-reported "Too many open files" error message (Fix matplotlib#3315). To reproduce: On a Mac or Windows box with starvation for file handles (Linux has a much higher file handle limit by default), build the docs, then immediately build again. This will trigger the caching bug. The font cache in the mathtext renderer was broken. It was caching a font file once for every *combination* of font properties, including things like size. Therefore, in a complex math expression containing many different sizes of the same font, the font file was opened once for each of those sizes. Font files are opened and kept open (rather than opened, read, and closed) so that FreeType only needs to load the actual glyphs that are used, rather than the entire font. In an era of cheap memory and fast disk, it probably doesn't matter for our current fonts, but once matplotlib#5214 is merged, we will have larger font files with many more glyphs and this loading time will matter more. The solution here is to do all font file loading in one place and to use `lru_cache` (available since Python 3.2) to do the caching, and to use only the file name and hinting parameters as a cache key. For earlier versions of Python, the functools32 backport package is required. (Or we can discuss whether we want to vendor it).

zblz · 2015-11-05T16:42:29Z

Yes, I was waiting to rebase on #5306 and regenerate the images.

jenshnielsen · 2015-11-05T17:21:57Z

#5306 is in so this can be rebased now

Add mathtext options to use dejavu use dejavusans as default subsuper metric for custom set dejavusans as default mathtext.fontset fix sized alternatives in dejavu mathtext

Add DejaVu as default font change other mentions to default font to DejaVu modify rcsetup and rc template add DejaVu to seaborn styles and examples

see comments in matplotlib#4873 and matplotlib#4987: Primes in stix and DejaVu are already superscripted, so the TeX way of setting a prime as $f^\prime$ results in doubly-superscripted primes. For consistency with TeX, here we use the CM prime symbol.

zblz · 2015-11-05T17:36:14Z

I rebased, rebuilt with local freetype, and regenerated images, but just realized that there is a problem with some characters in dejavuserif mathtext showing a box rather than falling back to STIX serif. I'll look into it tomorrow.

mdboom · 2015-11-05T17:44:25Z

Thanks, @zblz.

zblz · 2015-11-06T00:31:29Z

I think that the problem was that under FreeType 2.6.1 the TrueType font would return a .notdef symbol name and a square glyph when the glyph was not found, rather than the old behaviour of returning a KeyError. Therefore, it was not being caught as a missing glyph. Commit 0ef2cd4 should also fix this for all truetype fonts that fall back onto CM.

mdboom · 2015-11-06T00:47:21Z

Actually I think the relevant change was ~~#5410~~ #5299, which replaced the cmap dictionary with calls to get_char_index. get_char_index returns 0 when the char doesn't exist -- it doesn't throw a KeyError like a dictionary would. I think you can just check for gind == 0, don't need the additional call to get_glyph_name and don't need the try/except. That should work for all versions of freetype.

zblz · 2015-11-06T11:09:57Z

@mdboom - that makes more sense, I had not checked your latest font PRs. I changed it as you suggested and now everything should be fine.

Use DejaVu fonts as default for text and mathtext

mdboom · 2015-11-06T17:46:25Z

Thanks!

tacaswell · 2015-11-07T04:18:33Z

This needs to be backported to 2.0.x?

mdboom · 2015-11-07T13:35:12Z

Backported to 2.0.x as 4a2a186

tacaswell added the status: needs review label Oct 8, 2015

zblz mentioned this pull request Oct 8, 2015

Add Arev Sans fonts and license #4636

Closed

mdboom added this to the proposed next point release (2.1) milestone Oct 8, 2015

zblz changed the title ~~Add classes to use DejaVu fonts in mathtext~~ Use DejaVu fonts as default for text and mathtext Oct 8, 2015

mdboom modified the milestones: next major release (2.0), proposed next point release (2.1) Oct 8, 2015

jenshnielsen added the API: default changes label Oct 9, 2015

mdboom reviewed Oct 12, 2015
View reviewed changes

zblz force-pushed the dejavu-mathtext branch from b3f8755 to b4f6c1d Compare October 13, 2015 12:54

zblz force-pushed the dejavu-mathtext branch 3 times, most recently from 9282dfc to e1c6003 Compare October 14, 2015 11:02

zblz force-pushed the dejavu-mathtext branch from e1c6003 to 871d1d9 Compare October 14, 2015 12:53

zblz added 6 commits November 5, 2015 17:22

Add DejaVu mathtext fontset, make it default

6a5df0b

Add mathtext options to use dejavu use dejavusans as default subsuper metric for custom set dejavusans as default mathtext.fontset fix sized alternatives in dejavu mathtext

Add DejaVu as default font

d1a2f3b

Add DejaVu as default font change other mentions to default font to DejaVu modify rcsetup and rc template add DejaVu to seaborn styles and examples

remove Vera ttf files

3a75f7a

add whats_new file

2c39c12

change mentions to Vera in docs

9a99bd5

Use prime symbol from bakoma

be43ee5

see comments in matplotlib#4873 and matplotlib#4987: Primes in stix and DejaVu are already superscripted, so the TeX way of setting a prime as $f^\prime$ results in doubly-superscripted primes. For consistency with TeX, here we use the CM prime symbol.

zblz force-pushed the dejavu-mathtext branch from 496ee9c to 5330f39 Compare November 5, 2015 17:34

zblz force-pushed the dejavu-mathtext branch from 5330f39 to 6e871b9 Compare November 6, 2015 00:26

zblz added 2 commits November 6, 2015 11:05

add mathtext dejavu baseline images

3610db1

fix not found chars

23e04d0

zblz force-pushed the dejavu-mathtext branch from 6e871b9 to 23e04d0 Compare November 6, 2015 11:09

mdboom added a commit that referenced this pull request Nov 6, 2015

Merge pull request #5214 from zblz/dejavu-mathtext

e9b03f8

Use DejaVu fonts as default for text and mathtext

mdboom merged commit e9b03f8 into matplotlib:master Nov 6, 2015

mdboom removed the status: needs review label Nov 6, 2015

mdboom added a commit that referenced this pull request Nov 6, 2015

Merge pull request #5214 from zblz/dejavu-mathtext

4a2a186

Use DejaVu fonts as default for text and mathtext

zblz mentioned this pull request Nov 6, 2015

ENH: Better default font #5128

Closed

This was referenced Nov 8, 2015

Use DejaVu Sans as default fallback font #5439

Merged

Make "classic" style have effect #5437

Merged

QuLogic mentioned this pull request Feb 14, 2016

\math??{} font styles are ignored in 2.x #5998

Closed

anntzer mentioned this pull request Oct 18, 2016

MEP 29: Markup text #7291

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use DejaVu fonts as default for text and mathtext #5214

Use DejaVu fonts as default for text and mathtext #5214

zblz commented Oct 8, 2015

hetland commented Oct 8, 2015

mdboom commented Oct 8, 2015

zblz commented Oct 8, 2015

hetland commented Oct 8, 2015

zblz commented Oct 8, 2015

jkseppan commented Oct 9, 2015

mdboom Oct 12, 2015

zblz Oct 13, 2015

jenshnielsen Oct 13, 2015

zblz Oct 13, 2015

zblz Oct 13, 2015

mdboom Oct 13, 2015

mdboom commented Oct 13, 2015

toddrjen commented Oct 13, 2015

sandrotosi commented Oct 13, 2015

zblz commented Oct 14, 2015

zblz commented Oct 20, 2015

Tillsten commented Oct 20, 2015

zblz commented Oct 20, 2015

mdboom commented Oct 20, 2015

zblz commented Nov 5, 2015

jenshnielsen commented Nov 5, 2015

zblz commented Nov 5, 2015

mdboom commented Nov 5, 2015

zblz commented Nov 6, 2015

mdboom commented Nov 6, 2015

zblz commented Nov 6, 2015

mdboom commented Nov 6, 2015

tacaswell commented Nov 7, 2015

mdboom commented Nov 7, 2015

Use DejaVu fonts as default for text and mathtext #5214

Use DejaVu fonts as default for text and mathtext #5214

Conversation

zblz commented Oct 8, 2015

hetland commented Oct 8, 2015

mdboom commented Oct 8, 2015

zblz commented Oct 8, 2015

hetland commented Oct 8, 2015

zblz commented Oct 8, 2015

jkseppan commented Oct 9, 2015

mdboom Oct 12, 2015

Choose a reason for hiding this comment

zblz Oct 13, 2015

Choose a reason for hiding this comment

jenshnielsen Oct 13, 2015

Choose a reason for hiding this comment

zblz Oct 13, 2015

Choose a reason for hiding this comment

zblz Oct 13, 2015

Choose a reason for hiding this comment

mdboom Oct 13, 2015

Choose a reason for hiding this comment

mdboom commented Oct 13, 2015

toddrjen commented Oct 13, 2015

sandrotosi commented Oct 13, 2015

zblz commented Oct 14, 2015

zblz commented Oct 20, 2015

Tillsten commented Oct 20, 2015

zblz commented Oct 20, 2015

mdboom commented Oct 20, 2015

zblz commented Nov 5, 2015

jenshnielsen commented Nov 5, 2015

zblz commented Nov 5, 2015

mdboom commented Nov 5, 2015

zblz commented Nov 6, 2015

mdboom commented Nov 6, 2015

zblz commented Nov 6, 2015

mdboom commented Nov 6, 2015

tacaswell commented Nov 7, 2015

mdboom commented Nov 7, 2015