Skip to content

Implement a consistent behavior in TkAgg backend for bad blit bbox #20887

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jan 26, 2022

Conversation

richardsheridan
Copy link
Contributor

PR Summary

This PR seeks to make the failure modes more consistent and manageable when blitting on the TkAgg backend. Draft for now because I don't think it is release critical and some feedback is needed on design choices.

Background

The thread safety changes of #18565 interact a bit awkwardly with the memory safety changes of #14478, namely when _tkagg.blit raises, it pops directly out of the tkinter mainloop call and poisons the current instance of tkapp so that pumping the event loop always raises the ValueError forever. This isn't a big deal for bad values of comp_rule or photoimage since those are matplotlib's responsibilities, however as a user-facing API this prevents a client application from gracefully handling bad bboxes; they are basically fatal.

Why, if this is so bad, did nobody notice? In #14461 the bbox limits are clipped to a mostly-safe range, so only very bad bboxes that fall entirely outside the canvas trigger the issue. (And only my app is crazy enough to generate such bboxes 🤦)

Proposed fix(es)

Extend safe-clipping

The initial commit in this PR makes the most minimal possible change to existing behavior, extending the safe-clipping to boxes outside the canvas; they are simply dropped, and with a fast path optimization to boot. This way if ValueErrors are generated, they are 100% our fault. The if expression comes from the check in _tkagg.cpp but stripping the rules that are redundant to the clipping logic. The disadvantage is that the only way a user knows that their bbox is being clipped is by weird visual artifacts.

Raise aggressively

An alternative fix is to promote the _tkagg.cpp check into the first lines of _backend_tk.blit:

def blit(photoimage, aggimage, offsets, bbox=None):
    ...
    if bbox is not None:
        # Does anyone know why we are calling __array__ directly?
        (x1, y1), (x2, y2) = bbox.__array__().astype(int).tolist()
        if (0 > y1 or y1 > y2 or y2 > height or 0 > x1 or x1 > x2 or x2 > width):
            raise ValueError("Attempting to draw out of bounds")
        bboxptr = (x1, x2, y1, y2)
        comp_rule = TK_PHOTO_COMPOSITE_OVERLAY
    ...

This raise is now catchable and the check inside _tkagg.blit is redundant (though it is super cheap so it should remain in both places). Adding tests for this would finally cover this branch of the if statement. Also ALL bad bbox blits are errors now.

I see two disadvantages to this alternative. The integer conversion has to be different (truncated) so that full-canvas blits are not liable to pick up an extra pixel and error out, however this floor rounding behavior is consistent with other backends. Some user code that worked by silently clipping will now raise.

Open questions

Should we consider this alternative option as exposing latent bugs that should have been caught before, or as making blit "less robust"? Should we reconsider the current rounding/clipping rules? How strict are other backends to weird bboxes and can we make things consistent across all of them?

PR Checklist

  • Has pytest style unit tests (and pytest passes).
  • Is Flake 8 compliant (run flake8 on changed files to check).
  • New features are documented, with examples if plot related.
  • Documentation is sphinx and numpydoc compliant (the docs should build without error).
  • Conforms to Matplotlib style conventions (install flake8-docstrings and run flake8 --docstring-convention=all).
  • New features have an entry in doc/users/next_whats_new/ (follow instructions in README.rst there).
  • API changes documented in doc/api/next_api_changes/ (follow instructions in README.rst there).

@anntzer
Copy link
Contributor

anntzer commented Aug 24, 2021

I think it would be nice to spec exactly what is the expected rounding/clipping behavior expected by blit(); for mplcairo I basically had to reverse engineer what I think is the correct behavior (implemented at https://github.com/matplotlib/mplcairo/blob/9ad105795ef106c74fe37a165b96ca9b02e204ac/src/_mplcairo.cpp#L1531).

@richardsheridan
Copy link
Contributor Author

Perhaps in such a spec, instead of simply clipping the bbox, we could inspect if the rounding was a round-up from e.g. width+eps to width+1 where 0<eps<1. it is not actually a problem on the lower limits, right? That way more bad bboxes could be identified.

@QuLogic QuLogic added the status: needs comment/discussion needs consensus on next step label Oct 21, 2021
@richardsheridan richardsheridan marked this pull request as ready for review December 4, 2021 01:28
@richardsheridan
Copy link
Contributor Author

richardsheridan commented Dec 4, 2021

After stewing on this for a few months, I feel like the best option is to just merge this patch and make a new issue about harmonizing the blit spec across backends.

@anntzer
Copy link
Contributor

anntzer commented Dec 5, 2021

Sorry for letting this wait for so long. I didn't realize that there are actually a few different points here.

  • I think that usually, matplotlib blits back areas after first restoring a background generated via copy_from_bbox. copy_from_bbox itself has to decide how to round the floating-point bbox it gets as input (i.e., what rows and columns does it actually copy); we could be careful to stash that information somewhere (thus including the rounding) and just use that as bbox when blitting back. But perhaps that's too ambitious for the small change here. (However, long-term, it would make sense for blit() to have the same behavior as copy_from_bbox() for rounding, invalid inputs, and all other kinds of edge cases.)
  • It seems like other backends don't handle negative w/h properly anyways: qt ultimately calls https://doc.qt.io/qt-5/qwidget.html#repaint-1 ("If w is negative, it is replaced with width() - x, and if h is negative, it is replaced width height() - y.") and gtk3 calls https://docs.gtk.org/gtk3/method.Widget.queue_draw_area.html ("Negative values for width and height are not allowed."). wx calls https://wxpython.org/Phoenix/docs/html/wx.DC.html#wx.DC.Blit which says nothing about bad inputs.

Intuitively, I think I would rather error out on invalid inputs, but I understand that this erroring out currently occurs at an uncatchable place, and having this error out consistently across backends is a bit of work; so, at least as a stopgap, I agree with silently ignorining such bboxes for now (but perhaps this may get revisited in the future).

@tacaswell tacaswell added this to the v3.6.0 milestone Jan 26, 2022
@tacaswell tacaswell merged commit d615cf1 into matplotlib:main Jan 26, 2022
@richardsheridan richardsheridan deleted the blit-bbox-valueerror branch January 26, 2022 20:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants