Skip to content

BUG: ensure that errorbar does not error on masked negative errors. #29897

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Apr 16, 2025

Conversation

mhvk
Copy link
Contributor

@mhvk mhvk commented Apr 10, 2025

errorbar checks that errors are not negative, but a bit convolutedly, in order to avoid triggering on nan. Unfortunately, the work-around for nan means that possible masks get discarded, and hence passing in a masked error array that has a negative but masked value leads to an exception. This PR solves that by simply combining the test for
negative values with the indirect isnan test (err == err), so that if a masked array is passed, the test values are masked and ignored in the check.

As a bonus, this also means that astropy's Masked arrays can now be used -- those refuse to write output of tests to unmasked values since that would discard the mask (which is indeed the underlying problem here) -- see
astropy/astropy#17996

p.s. I checked that the test case that I added fails on current main. The test case is based on the one for nan right above - let me know if this is an OK location for it.

EDIT: updated to simplify the check without using any storing of outputs. This means we can also continue to use astropy's Quantity for error bars (and just shows how damn careful one has to be, sigh).

EDIT 2: second commit prevents any comparison of nan < nan, which causes RuntimeError that I did not see locally.

PR checklist

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for opening your first PR into Matplotlib!

If you have not heard from us in a week or so, please leave a new comment below and that should bring it to our attention. Most of our reviewers are volunteers and sometimes things fall through the cracks.

You can also join us on gitter for real-time discussion.

For details on testing, writing docs, and our review process, please see the developer guide

We strive to be a welcoming and open project. Please follow our Code of Conduct.

@mhvk mhvk force-pushed the allow_masked_negative_errors branch from c22cac0 to ae1edf8 Compare April 10, 2025 16:23
errorbar checks that errors are not negative, but a bit convolutedly,
in order to avoid triggering on nan. Unfortunately, the work-around
for nan means that possible masks get discarded, and hence passing in
a masked error array that has a negative but masked value leads to
an exception. This PR solves that by simply combining the test for
negative values with the indirect isnan test (err == err), so that
if a masked array is passed, the test values are masked and ignored
in the check.

As a bonus, this also means that astropy's ``Masked`` arrays can now
be used -- those refuse to write output of tests to unmasked values
since that would discard the mask (which is indeed the underlying
problem here).
@tacaswell
Copy link
Member

The underlying (underlying) problem is that np.where does not respect the mask and __getitem__ does?

@mhvk
Copy link
Contributor Author

mhvk commented Apr 10, 2025

The underlying (underlying) problem is that np.where does not respect the mask and __getitem__ does?

No, the problem is that when writing directly to a regular array, as happened in np.less(err, -err, out=res) (where res was a regular array created with np.zeros), the mask gets dropped (more specifically, np.less(err, -err) would normally return a masked boolean array, with the mask a copy of the mask of err, but writing that to res drops the mask).

p.s. What was a surprise is that np.less raises a RuntimeWarning for np.nan unless that particular element is not included in where -- a surprise mostly because I know from my ufunc work that generally the where argument does not actually prevent the function to be done on the arguments, it just avoids the result being written back. But perhaps it is different for the comparisons - nan are of course notorious and there is a lot of extra code to work around them... Anyway, unrelated to the issue at hand!

@mhvk
Copy link
Contributor Author

mhvk commented Apr 10, 2025

Actually, I guess I misread your question: yes, it is also true that the where argument ignores the mask on the output of err == err.

@mhvk mhvk force-pushed the allow_masked_negative_errors branch from 5c42d71 to ebeb83a Compare April 13, 2025 12:18
@mhvk
Copy link
Contributor Author

mhvk commented Apr 13, 2025

@dstansby - comments addressed (I hope).

Copy link
Member

@dstansby dstansby left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@QuLogic QuLogic merged commit e1887b8 into matplotlib:main Apr 16, 2025
39 of 41 checks passed
@QuLogic
Copy link
Member

QuLogic commented Apr 16, 2025

Thanks @mhvk! Congratulations on your first PR to Matplotlib 🎉 We hope to hear from you again.

meeseeksmachine pushed a commit to meeseeksmachine/matplotlib that referenced this pull request Apr 16, 2025
@mhvk mhvk deleted the allow_masked_negative_errors branch April 16, 2025 11:14
@bsipocz
Copy link
Contributor

bsipocz commented Apr 16, 2025

Thanks @mhvk for fixing this!

QuLogic added a commit that referenced this pull request Apr 17, 2025
…897-on-v3.10.x

Backport PR #29897 on branch v3.10.x (BUG: ensure that errorbar does not error on masked negative errors.)
@ksunden ksunden mentioned this pull request May 9, 2025
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants