Skip to content

ENH/TST: add deadband to image comparison #28923

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

tacaswell
Copy link
Member

This allows us to ignore small images in colors using a more precise threshold than tol provides.

This is motivated due to a new set of semi-random failures on OSX on azure. A simpler fix may be to use tol like we have in many other cases and defer this to 3.11.

PR checklist

@tacaswell tacaswell added this to the v3.11.0 milestone Oct 2, 2024
@tacaswell tacaswell modified the milestones: v3.11.0, v3.10.0 Oct 2, 2024
This allows us to ignore small images in colors using a more precise
threshold than tol provides.
@tacaswell tacaswell force-pushed the tst/add_image_compare_deadband branch from 9aed127 to cc5e65e Compare October 2, 2024 20:36
@QuLogic
Copy link
Member

QuLogic commented Oct 2, 2024

It doesn't look like this fixed the test_axes3d_primary_views failure. Also note that we'll have to fix this on the v3.9.x branch too, so we might just have to bump the tolerance on that one.

@scottshambaugh
Copy link
Contributor

This is a very welcome change. In the 3D plots especially there can be miniscule rendering differences between different platforms which are unrelated to code diffs and are extremely difficult to debug and get tests passing. Only real way I've found is to continuously push up to CI, wait 20 min for tests to run through, then download the image diffs on the failed platforms and enhance contrast in an image editing program (because the diff looks black initially since the diff is so small). Then invariably the fix is to tweak the test slightly and hope the rendering comes out the same. I've gone 10+ cycles of this workflow on some PRs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants