Fix baseline alignment when using usetex. #16476

anntzer · 2020-02-11T23:51:48Z

Previously we inferred the baseline of usetex strings with the baseline
of the character with the lowest baseline, which is wrong in the
presence of indices (see usetex_baseline_test.py). There was an option
to use the "preview" latex package to fix that, but that was never made
the default and likely cannot as the package is GPL.

Instead I can infer the baseline position by "reverse-engineering"
the starting instructions of the dvi stream (this was done without
consulting the approach in "preview"). The results can be checked
using usetex_baseline_test.py (preview=False now looks the same as
preview=True; likewise, the output of test_usetex was changed.
(see #16373)

The text.latex.preview rc can now be deprecated (in a future PR).

The test was also updated to include the strings of the
usetex_baseline_test example (which may go away once text.latex.preview
is removed). (The baseline image was already changing, in any case...)

I think the test is "spuriously" failing because my local version of latex tools are not compiled against the same freetype version as on CI :/ Basically the baseline image will need to be regen'd by someone who can otherwise already pass test_usetex.py locally.

Edit: Now also includes #16410 as first commit (that PR has been closed), so closes #16409.

... but still can't repro the failure locally.

PR Summary

PR Checklist

Has Pytest style unit tests
Code is Flake 8 compliant
New features are documented, with examples if plot related
Documentation is sphinx and numpydoc compliant
Added an entry to doc/users/next_whats_new/ if major new feature (follow instructions in README.rst there)
Documented in doc/api/api_changes.rst if API changed in a backward-incompatible way

tacaswell · 2020-03-10T04:56:10Z

lib/matplotlib/tests/test_usetex.py

 def test_usetex():
    mpl.rcParams['text.usetex'] = True
    fig = plt.figure()
    ax = fig.add_subplot(111)
-    ax.text(0.1, 0.2,
+    kwargs = {"verticalalignment": "baseline", "size": 24,


This test is significantly better 👍

anntzer · 2020-03-10T05:02:24Z

(I'll combine this with #16410 as a single PR for the usual baseline image changes reason.)

tacaswell

Feel free to ignore my comments, but this is full of so much state....

tacaswell · 2020-03-10T05:04:17Z

lib/matplotlib/dviread.py

+        #       push  <=  here, v is the baseline position.
+        #         etc.
+        # (dviasm is useful to explore this structure.)
+        self._baseline_v = None


do we want to put this in __init__ as well? Are we sure that _output will never be called without _read being called first?

we'd already have errored out earlier anyways, because self.text wouldn't have been defined either.

lib/matplotlib/dviread.py

anntzer · 2020-03-10T17:45:40Z

@tacaswell can you repro the failure locally?

QuLogic · 2020-03-24T22:38:23Z

I can reproduce locally, though it's a slightly different RMS.

anntzer · 2020-03-24T22:39:47Z

Can you try regen'ing the baseline image locally and force-push? probably something about the version of freetype that dvipng is using...

QuLogic · 2020-04-02T09:36:43Z

Problem is, now it's also doing:

__________________________________________________________________ test_empty[png] ___________________________________________________________________
[gw0] linux -- Python 3.6.8 /var/container/conda/envs/mpl36/bin/python

args = (), kwargs = {}, ext = 'png', request = <FixtureRequest for <Function 'test_empty[png]'>>, file_name = 'test_empty[png]'
fig_test = <Figure size 640x480 with 0 Axes>, fig_ref = <Figure size 640x480 with 0 Axes>
test_image_path = PosixPath('matplotlib/result_images/test_usetex/test_empty[png].png')
ref_image_path = PosixPath('matplotlib/result_images/test_usetex/test_empty[png]-expected.png')

    @pytest.mark.parametrize("ext", extensions)
    def wrapper(*args, **kwargs):
        ext = kwargs['ext']
        if 'ext' not in old_sig.parameters:
            kwargs.pop('ext')
        request = kwargs['request']
        if 'request' not in old_sig.parameters:
            kwargs.pop('request')
    
        file_name = "".join(c for c in request.node.name
                            if c in ALLOWED_CHARS)
        try:
            fig_test = plt.figure("test")
            fig_ref = plt.figure("reference")
            func(*args, fig_test=fig_test, fig_ref=fig_ref, **kwargs)
            test_image_path = result_dir / (file_name + "." + ext)
            ref_image_path = result_dir / (file_name + "-expected." + ext)
>           fig_test.savefig(test_image_path)

lib/matplotlib/testing/decorators.py:417: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
lib/matplotlib/figure.py:2171: in savefig
    self.canvas.print_figure(fname, **kwargs)
lib/matplotlib/backend_bases.py:2105: in print_figure
    **kwargs)
lib/matplotlib/backends/backend_agg.py:493: in print_png
    FigureCanvasAgg.draw(self)
lib/matplotlib/backends/backend_agg.py:382: in draw
    self.figure.draw(self.renderer)
lib/matplotlib/artist.py:41: in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
lib/matplotlib/figure.py:1729: in draw
    renderer, self, artists, self.suppressComposite)
lib/matplotlib/image.py:132: in _draw_list_compositing_images
    a.draw(renderer)
lib/matplotlib/artist.py:41: in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
lib/matplotlib/text.py:688: in draw
    bbox, info, descent = textobj._get_layout(renderer)
lib/matplotlib/text.py:363: in _get_layout
    xmin = corners_rotated[:, 0].min()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

a = array([ nan,   0., -inf,  nan]), axis = None, out = None, keepdims = False, initial = <no value>

    def _amin(a, axis=None, out=None, keepdims=False,
              initial=_NoValue):
>       return umr_minimum(a, axis, None, out, keepdims, initial)
E       RuntimeWarning: invalid value encountered in reduce

.../lib/python3.6/site-packages/numpy/core/_methods.py:32: RuntimeWarning

anntzer · 2020-04-02T09:59:35Z

Strange, I can't repro that failure locally. Tried rebasing on master, and also deleting ~/.cache/mpl/test_cache, still passing...

anntzer · 2020-04-02T10:58:14Z

I guess an additional

diff --git i/lib/matplotlib/dviread.py w/lib/matplotlib/dviread.py
index bb14480e4..dcaba415a 100644
--- i/lib/matplotlib/dviread.py
+++ w/lib/matplotlib/dviread.py
@@ -278,6 +278,9 @@ class Dvi:
             maxy_pure = self._baseline_v  # This should normally be the case.
             self._baseline_v = None
 
+        if not self.text and not self.boxes:  # Avoid infs/nans from inf+/-inf.
+            return Page(text=[], boxes=[], width=0, height=0, descent=0)
+
         if self.dpi is None:
             # special case for ease of debugging: output raw dvi coordinates
             return Page(text=self.text, boxes=self.boxes,

may help, but I still don't know why I don't have an issue locally. numpy version, perhaps?

QuLogic · 2020-04-02T22:13:05Z

My environment has lowest NumPy supported, 1.15.0.

anntzer · 2020-04-02T22:14:49Z

Does the proposed patch help?

QuLogic · 2020-04-02T22:30:52Z

Yes, it works.

anntzer · 2020-04-02T22:55:26Z

Looks like this still failed on the image diff. Do you know if we can download the test artefacts from e.g. azure to check the image and possibly just dump it in the baselines directory?

QuLogic · 2020-04-02T23:57:39Z

But then it would just fail for everyone else off CI?

#17011 for Azure artifacts, if it works.

anntzer · 2020-04-03T00:16:43Z

At least I want to know if it's just a freetype version problem (in which case we can just bump the tolerance) or something more serious.

anntzer · 2020-04-05T23:40:43Z

Looks like freetype-version related, will work on that...

anntzer · 2020-04-06T10:15:41Z

Looks like forcing freetype-less dvipng mode fixes the issue. It's a bit annoying to expose testing bits in texmanager, but I'm not really sure I want to expose that more because I have a further plan to just get rid of the dependency on dvipng altogether (for raster output we can actually just use the parsed dvi (as we already do for vector output) to extract the glyphs and draw them ourselves using our own freetype renderer -- that's what's done in mplcairo for example). This will skip a subprocess call (always good on Windows in particular), and of the FreeType version problem (we don't control the FreeType version used by dvipng, and FreeType-less mode is broken in dvipng 1.16 -- fixed in the most recent 1.17 and was good before 1.16 too, but version bounds on dvipng for tests only may be a bit painful...).

Or we can first wait for that feature to go in...

lib/matplotlib/dviread.py

lib/matplotlib/tests/test_usetex.py

Unfortunately at least some dvipngs appear to be statically linked to FreeType so we can't play tricks with LD_PRELOAD/LD_LIBRARY_PATH.

Currently, `figtext(.5, .5, "%foo", usetex=True)` results in an obscure exception (`FileNotFoundError: No such file or directory: blah.dvi`). Fix that by forcing tex to always generate a page, by inserting at least an empty \hbox{}.

Previously we inferred the baseline of usetex strings with the baseline of the character with the lowest baseline, which is wrong in the presence of indices (see usetex_baseline_test.py). There was an option to use the "preview" latex package to fix that, but that was never made the default and likely cannot as the package is GPL. Instead I can infer the baseline position by "reverse-engineering" the starting instructions of the dvi stream (this was done without consulting the approach in "preview"). The results can be checked using usetex_baseline_test.py (preview=False now looks the same as preview=True; likewise, the output of test_usetex was changed. The text.latex.preview rc can now be deprecated (in a future PR). The test was also updated to include the strings of the usetex_baseline_test example (which may go away once text.latex.preview is removed).

Fix baseline alignment when using usetex. Conflicts: lib/matplotlib/tests/test_usetex.py - import np and formatting issues on decorator

anntzer added the topic: text/usetex label Feb 11, 2020

anntzer added this to the v3.3.0 milestone Feb 11, 2020

anntzer force-pushed the fix-usetex-baseline branch from 698df42 to 497738c Compare February 12, 2020 00:23

anntzer mentioned this pull request Mar 10, 2020

Fix support for empty usetex strings. #16410

Closed

6 tasks

tacaswell reviewed Mar 10, 2020

View reviewed changes

tacaswell approved these changes Mar 10, 2020

View reviewed changes

anntzer force-pushed the fix-usetex-baseline branch 2 times, most recently from b1c5b53 to a494553 Compare March 10, 2020 09:11

QuLogic force-pushed the fix-usetex-baseline branch from a494553 to ce91174 Compare April 2, 2020 22:28

anntzer force-pushed the fix-usetex-baseline branch from ce91174 to 7e09018 Compare April 5, 2020 16:13

anntzer added the status: needs revision label Apr 5, 2020

anntzer force-pushed the fix-usetex-baseline branch from 7e09018 to d4aeacc Compare April 6, 2020 09:19

anntzer removed the status: needs revision label Apr 6, 2020

QuLogic reviewed Apr 14, 2020

View reviewed changes

lib/matplotlib/dviread.py Outdated Show resolved Hide resolved

lib/matplotlib/tests/test_usetex.py Outdated Show resolved Hide resolved

anntzer force-pushed the fix-usetex-baseline branch from d4aeacc to ccabb61 Compare April 14, 2020 08:11

anntzer added 3 commits April 14, 2020 13:08

Disable use of FreeType in dvipng for test reproducibility.

b7c9d85

Unfortunately at least some dvipngs appear to be statically linked to FreeType so we can't play tricks with LD_PRELOAD/LD_LIBRARY_PATH.

Fix support for empty usetex strings.

0651abb

Currently, `figtext(.5, .5, "%foo", usetex=True)` results in an obscure exception (`FileNotFoundError: No such file or directory: blah.dvi`). Fix that by forcing tex to always generate a page, by inserting at least an empty \hbox{}.

anntzer force-pushed the fix-usetex-baseline branch from ccabb61 to 8472fbb Compare April 14, 2020 11:09

QuLogic approved these changes Apr 14, 2020

View reviewed changes

QuLogic merged commit b44ecb6 into matplotlib:master Apr 14, 2020

anntzer deleted the fix-usetex-baseline branch April 14, 2020 22:27

anntzer mentioned this pull request Apr 16, 2020

Deprecate text.latex.preview rcParam. #17156

Merged

6 tasks

tacaswell mentioned this pull request Apr 28, 2020

Fix eps + usetex combo. #17255

Merged

6 tasks

tacaswell modified the milestones: v3.3.0, v3.2.2 Apr 28, 2020

anntzer mentioned this pull request May 3, 2020

Matplotlib git master version fails to pass serveral pytest's tests. #17310

Closed

darthoctopus mentioned this pull request Jan 4, 2021

tick labels displaced vertically with text.usetex and xcolor #19234

Closed

anntzer mentioned this pull request May 23, 2021

a bug that latex expression with subscripts exceeds bbox #14177

Open

anntzer mentioned this pull request Sep 19, 2024

[Bug]: Alignment of minus sign when using LaTeX #28766

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix baseline alignment when using usetex. #16476

Fix baseline alignment when using usetex. #16476

anntzer commented Feb 11, 2020 •

edited

Loading

tacaswell Mar 10, 2020

anntzer commented Mar 10, 2020

tacaswell left a comment

tacaswell Mar 10, 2020

anntzer Mar 10, 2020

anntzer commented Mar 10, 2020

QuLogic commented Mar 24, 2020

anntzer commented Mar 24, 2020

QuLogic commented Apr 2, 2020

anntzer commented Apr 2, 2020

anntzer commented Apr 2, 2020

QuLogic commented Apr 2, 2020

anntzer commented Apr 2, 2020

QuLogic commented Apr 2, 2020

anntzer commented Apr 2, 2020

QuLogic commented Apr 2, 2020

anntzer commented Apr 3, 2020

anntzer commented Apr 5, 2020

anntzer commented Apr 6, 2020

Fix baseline alignment when using usetex. #16476

Fix baseline alignment when using usetex. #16476

Conversation

anntzer commented Feb 11, 2020 • edited Loading

PR Summary

PR Checklist

tacaswell Mar 10, 2020

Choose a reason for hiding this comment

anntzer commented Mar 10, 2020

tacaswell left a comment

Choose a reason for hiding this comment

tacaswell Mar 10, 2020

Choose a reason for hiding this comment

anntzer Mar 10, 2020

Choose a reason for hiding this comment

anntzer commented Mar 10, 2020

QuLogic commented Mar 24, 2020

anntzer commented Mar 24, 2020

QuLogic commented Apr 2, 2020

anntzer commented Apr 2, 2020

anntzer commented Apr 2, 2020

QuLogic commented Apr 2, 2020

anntzer commented Apr 2, 2020

QuLogic commented Apr 2, 2020

anntzer commented Apr 2, 2020

QuLogic commented Apr 2, 2020

anntzer commented Apr 3, 2020

anntzer commented Apr 5, 2020

anntzer commented Apr 6, 2020

anntzer commented Feb 11, 2020 •

edited

Loading