ENH: Improve error message in numpy.testing.assert_array_compare #29112
Conversation
…, showing the position of differing elements.
|
Thanks, looks like a nice idea on first thought! Could you show before/after outputs? Also at least the documentation test failures (circleci) are very much real here. |
|
Sure, here is an example with outputs before and after my changes. Example code a = np.random.rand(100, 100) Before: After: |
|
I like where this is going, but I think care needs to go into edge cases so we don't unnecessarily add noise. If all the values are different in two 1000 element arrays, it's not particularly useful to print the first five indices, for example. Maybe there should be a cutoff where the index display starts showing up? Also for this to be mergeable you're going to need to get CI to pass and you're going to need to add a release note describing the change. Try to get the tests, docs build, doctests, and linter to pass locally before pushing here to run CI. |
… improved error message.
|
I hope to have addressed all the edge cases now and have also adapted the tests and docs to the new error message. |
|
Thanks, @polaris-3. |
|
Nice! I have a similar feature in the pipeline for #28827 |
Recently, numpy#29112 added showing first mismatches indices, but assert_array_almost_equal.compare trips it up by returning a shape different from its input, causing an IndexError: ``` <...> if invalids.ndim != 0: if flagged.ndim > 0: > positions = np.argwhere(np.asarray(~flagged))[invalids] E IndexError: boolean index did not match indexed array along axis 0; size of axis is 3 but size of corresponding boolean axis is 2 ``` (traceback shown using pytest --full-trace)
Recently, numpy#29112 added showing first mismatches indices, but assert_array_almost_equal.compare trips it up by returning a shape different from its input, causing an IndexError: ``` <...> if invalids.ndim != 0: if flagged.ndim > 0: > positions = np.argwhere(np.asarray(~flagged))[invalids] E IndexError: boolean index did not match indexed array along axis 0; size of axis is 3 but size of corresponding boolean axis is 2 ``` (traceback shown using pytest --full-trace)
Recently, numpy#29112 added showing first mismatches indices, but assert_array_almost_equal.compare trips it up by returning a shape different from its input, causing an IndexError: ``` <...> if invalids.ndim != 0: if flagged.ndim > 0: > positions = np.argwhere(np.asarray(~flagged))[invalids] E IndexError: boolean index did not match indexed array along axis 0; size of axis is 3 but size of corresponding boolean axis is 2 ``` (traceback shown using pytest --full-trace)
Recently, numpy#29112 added showing first mismatches indices, but assert_array_almost_equal.compare trips it up by returning a shape different from its input, causing an IndexError: ``` <...> if invalids.ndim != 0: if flagged.ndim > 0: > positions = np.argwhere(np.asarray(~flagged))[invalids] E IndexError: boolean index did not match indexed array along axis 0; size of axis is 3 but size of corresponding boolean axis is 2 ``` (traceback shown using pytest --full-trace)
* TST: Add failing test showing shape mismatch issue Recently, #29112 added showing first mismatches indices, but assert_array_almost_equal.compare trips it up by returning a shape different from its input, causing an IndexError: ``` <...> if invalids.ndim != 0: if flagged.ndim > 0: > positions = np.argwhere(np.asarray(~flagged))[invalids] E IndexError: boolean index did not match indexed array along axis 0; size of axis is 3 but size of corresponding boolean axis is 2 ``` (traceback shown using pytest --full-trace) * MNT: Remove assert_array_almost_equal old infs logic, but tests fail A nice cleanup (newer, similar inf handling already exists now in assert_array_compare), and resolves the shape mismatch issue. However, the removed logic was handling complex infs while the one isn't, causing the new test and an existing one (TestInterp::test_complex_interp) to now fail with RuntimeWarnings attempting to subtract the complex infs. * MNT: assert_array_compare handles all inf values assert_array_compare now tests all inf values for matching position and value, including complex infs. Fixes the failing tests. * TST: Test array_allclose behavior for inf values The behavior for real infs is the same is before. For complex infs, demonstrates that the behavior for mismatching values is now cleaner, showing a concise error message vs. previously displaying nan max errors. For complex infs with matching values, the behavior is the same as before, accepting them as equal (although internally they would now be filtered ahead of being passed to isclose, like real infs already had been). * TST: assert_allclose behavior with nans that are also infs * MNT: Extract robust_any_difference() helper to DRY
Improve the error message in numpy.testing.assert_array_compare to show the position of differing elements.
Currently, if there is an assertion error in functions like
numpy.testing.assert_allclose, both arrays are printed. Typically the arrays are so large that they are truncated before being printed, making it hard to spot the differences. I suggest to print the indices of the differing elements as well as the corresponding values. To avoid excessive output, the number of printed out differences is limited to five.