Skip to content

Support {lua,xe}tex as alternative usetex engine. #29807

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

anntzer
Copy link
Contributor

@anntzer anntzer commented Mar 25, 2025

Currently, this PR is mostly a proof of concept; it only implements the
dvi generation and parsing parts, but does not implement rendering in
any of the builtin backends, except svg (under rcParams["svg.fonttype"]
= "none", the default). However, there is a companion branch on the
mplcairo repository, also named "luadvi", which implements support.

Example (requiring both this PR, and mplcairo installed from its luadvi
branch; or try saving to svg with the builtin svg backend):

import matplotlib as mpl; mpl.use("module://mplcairo.qt")
from matplotlib import pyplot as plt

plt.rcParams["text.latex.engine"] = "lualatex"  # or "xelatex"
plt.rcParams["text.latex.preamble"] = (
    # {lua,xe}tex can use any font installed on the system, spec'd using its
    # "normal" name.  Try e.g. DejaVu Sans instead.
    r"\usepackage{fontspec}\setmainfont{TeX Gyre Pagella}")
plt.figtext(.5, .5, r"\textrm{gff\textwon}", usetex=True)
plt.show()

TODO:

  • Fix many likely remaining bugs.
  • Rework font selection in texmanager, which is currently very ad-hoc
    due to the limited number of fonts supported by latex.
  • Implement rendering support in the (other) builtin backends. In
    particular, the Agg (and, if we care, cairo) backend will require
    significant reworking because dvipng, currently used to rasterize dvi
    to png, doesn't support luatex-generated dvi; instead we will need to
    proceed as with the other backends, reading the glyphs one at a time
    from the dvi file and rasterizing them one at a time to the output
    buffer. Working on the other backends is not very high on my priority
    list (as I already have mplcairo as playground...) so it would be nice
    if others showed some interest for it :-)

See #20262, #22715, #28131.

PR summary

PR checklist

@anntzer anntzer changed the title Support luatex as alternative usetex engine. Support {lua,xe}tex as alternative usetex engine. Mar 26, 2025
@anntzer anntzer force-pushed the luadvi branch 2 times, most recently from 4d488f4 to 697d106 Compare March 26, 2025 23:42
@anntzer anntzer force-pushed the luadvi branch 3 times, most recently from ff9ecdd to 4f93376 Compare March 27, 2025 22:39
@anntzer anntzer force-pushed the luadvi branch 2 times, most recently from 85a73b8 to f1b16f2 Compare March 28, 2025 10:48
@anntzer anntzer force-pushed the luadvi branch 2 times, most recently from c134a34 to ccf927c Compare March 30, 2025 13:49
anntzer added 3 commits April 29, 2025 15:20
In 89a7e19, an API for converting "dvi glyph indices" (as stored
in a dvi file) to FreeType-compatible keys (either "indices into
the native charmap" or "glyph names") was introduced.  It was
intended that end users (i.e., backends) would check the type of
`text.glyph_name_or_index` ((A) int or (B) str) and load the glyph
accordingly ((A) `FT_Set_Charmap(native_cmap); FT_Load_Char(index);` or
(B) `FT_Load_Glyph(FT_Get_Name_Index(name));`); however, with the future
introduction of {xe,lua}tex support, this kind of type checking becomes
inconvenient, because {xe,lua}tex's "dvi glyph indices", which are
directly equal to FreeType glyph indices (i.e. they would be loaded with
`FT_Load_Glyph(index);`), would normally also be converted to ints.

This PR introduces a new API (`Text.index`) that performs this mapping
(via the new `DviFont._index_dvi_to_freetype`), always mapping to
FreeType glyph indices (i.e. one can always just call `FT_Load_Glyph`
on the result).  To do so, in case (A) it loads itself the native
charmap (something the end user needed to do by themselves previously)
and performs the cmap-to-index conversion (`FT_Get_Char_Index`)
previously implicit in `FT_Load_Char`; in case (B) it performs itself
the name-to-index conversion (`FT_Get_Name_Index`).  When {xe,lua}tex
support is introduced in the future, `_index_dvi_to_freetype` will
just return the index as is.

The old APIs are not deprecated yet, as other changes will occur for
{xe,lua}tex support (e.g. font_effects will go away and be replaced by
`.font.effects`, as {xe,lua}tex don't store that info in the pdftexmap
entry), and grouping all API changes together seems nicer (to only
require a single adaptation step by the API consumer).
Currently, this PR is mostly a proof of concept; only the svg backend
is supported (under rcParams["svg.fonttype"] = "none", the default).
However, there is a companion branch on the mplcairo repository, also
named "luadvi", which implements support for all output formats.

Example (requiring both this PR, and mplcairo installed from its luadvi
branch):
```
import matplotlib as mpl; mpl.use("module://mplcairo.qt")
from matplotlib import pyplot as plt

plt.rcParams["text.latex.engine"] = "lualatex"  # or "xelatex"
plt.rcParams["text.latex.preamble"] = (
    # {lua,xe}tex can use any font installed on the system, spec'd using its
    # "normal" name.  Try e.g. DejaVu Sans instead.
    r"\usepackage{fontspec}\setmainfont{TeX Gyre Pagella}")
plt.figtext(.5, .5, r"\textrm{gff\textwon}", usetex=True)
plt.show()
```
Font effects are supported by mplcairo, e.g.
`\fontspec{DejaVu Sans}[FakeSlant=0.2] abc`.

TODO:

- Fix many likely remaining bugs.
- Rework font selection in texmanager, which is currently very ad-hoc
  due to the limited number of fonts supported by latex.
- Implement rendering support in the (other) builtin backends.  In
  particular, the Agg (and, if we care, cairo) backend will require
  significant reworking because dvipng, currently used to rasterize dvi
  to png, doesn't support luatex-generated dvi; instead we will need to
  proceed as with the other backends, reading the glyphs one at a time
  from the dvi file and rasterizing them one at a time to the output
  buffer.  Working on the other backends is not very high on my priority
  list (as I already have mplcairo as playground...) so it would be nice
  if others showed some interest for it :-)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants