documentation, tests, and bugs in seterr behavior for integer types #3199

ivanov · 2013-04-06T03:19:16Z

this is a work in progress. it started off as a documentation improvement for the fact, currently captured in #593 that np.seterr does not affect error handling behavior for arrays of integers (although it does mostly work for integer scalars).

But in writing a test for this behavior (which I was intending to submit and mark as a KnownFailure), I discovered two other bugs in the scalar handling. These examples assume that np.seterr(all='raise') has already been called.

Integer scalar underflow (for all int types) is reported as overflow. Here's a bit of misinformation - should be UNDER-flow, not overflow

    In [28]: np.int8(-128) - np.int8(1)
    ---------------------------------------------------------------------------
    FloatingPointError                        Traceback (most recent call last)
    <ipython-input-28-2b810c89b4d5> in <module>()
    ----> 1 np.int8(-128) - np.int8(1)

    FloatingPointError: overflow encountered in byte_scalars

int64 overflow is ignored during multiplication

    In [14]: d = np.int8; i = np.iinfo(d); d(i.max) * d(2)
    ---------------------------------------------------------------------------
    FloatingPointError                        Traceback (most recent call last)
    <ipython-input-14-3822c5509024> in <module>()
    ----> 1 d = np.int8; i = np.iinfo(d); d(i.max) * d(2)

    FloatingPointError: overflow encountered in byte_scalars

Good, we also get errors for int16,  int32, but not int64

In [17]: d = np.int64; i = np.iinfo(d); d(i.max) * d(2)
Out[17]: -2

ivanov · 2013-04-06T05:38:40Z

Here's the output of running nosetests test_numeric:TestFloatExceptions -v

Test basic arithmetic function errors ... ERROR
Seterr should apply to arrays of integer types, see gh-593 ... FAIL
Seterr should apply to integer types ... FAIL

======================================================================
ERROR: Test basic arithmetic function errors
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/pi/.local/lib/python2.7/site-packages/numpy/testing/decorators.py", line 216, in knownfailer
    raise KnownFailureTest(msg)
KnownFailureTest: See ticket 1755

======================================================================
FAIL: Seterr should apply to arrays of integer types, see gh-593
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/pi/code/numpy/numpy/core/tests/test_numeric.py", line 382, in test_seterr_for_arrays_of_ints
    assert_(len(raised) == 0, pretty_repr)
  File "/home/pi/.local/lib/python2.7/site-packages/numpy/testing/utils.py", line 37, in assert_
    raise AssertionError(msg)
AssertionError: 36 errors:
  Type <type 'numpy.ndarray'>int8 did not raise fpe error 'underflow'.
  Type <type 'numpy.ndarray'>int8 did not raise fpe error 'overflow'.
  Type <type 'numpy.ndarray'>int8 did not raise fpe error 'overflow'.
  Type <type 'numpy.ndarray'>uint8 did not raise fpe error 'underflow'.
  Type <type 'numpy.ndarray'>uint8 did not raise fpe error 'overflow'.
  Type <type 'numpy.ndarray'>uint8 did not raise fpe error 'overflow'.
  Type <type 'numpy.ndarray'>int16 did not raise fpe error 'underflow'.
  Type <type 'numpy.ndarray'>int16 did not raise fpe error 'overflow'.
  Type <type 'numpy.ndarray'>int16 did not raise fpe error 'overflow'.
  Type <type 'numpy.ndarray'>uint16 did not raise fpe error 'underflow'.
  Type <type 'numpy.ndarray'>uint16 did not raise fpe error 'overflow'.
  Type <type 'numpy.ndarray'>uint16 did not raise fpe error 'overflow'.
  Type <type 'numpy.ndarray'>int32 did not raise fpe error 'underflow'.
  Type <type 'numpy.ndarray'>int32 did not raise fpe error 'overflow'.
  Type <type 'numpy.ndarray'>int32 did not raise fpe error 'overflow'.
  Type <type 'numpy.ndarray'>uint32 did not raise fpe error 'underflow'.
  Type <type 'numpy.ndarray'>uint32 did not raise fpe error 'overflow'.
  Type <type 'numpy.ndarray'>uint32 did not raise fpe error 'overflow'.
  Type <type 'numpy.ndarray'>int64 did not raise fpe error 'underflow'.
  Type <type 'numpy.ndarray'>int64 did not raise fpe error 'overflow'.
  Type <type 'numpy.ndarray'>int64 did not raise fpe error 'overflow'.
  Type <type 'numpy.ndarray'>uint64 did not raise fpe error 'underflow'.
  Type <type 'numpy.ndarray'>uint64 did not raise fpe error 'overflow'.
  Type <type 'numpy.ndarray'>uint64 did not raise fpe error 'overflow'.
  Type <type 'numpy.ndarray'>int64 did not raise fpe error 'underflow'.
  Type <type 'numpy.ndarray'>int64 did not raise fpe error 'overflow'.
  Type <type 'numpy.ndarray'>int64 did not raise fpe error 'overflow'.
  Type <type 'numpy.ndarray'>uint64 did not raise fpe error 'underflow'.
  Type <type 'numpy.ndarray'>uint64 did not raise fpe error 'overflow'.
  Type <type 'numpy.ndarray'>uint64 did not raise fpe error 'overflow'.
  Type <type 'numpy.ndarray'>int64 did not raise fpe error 'underflow'.
  Type <type 'numpy.ndarray'>int64 did not raise fpe error 'overflow'.
  Type <type 'numpy.ndarray'>int64 did not raise fpe error 'overflow'.
  Type <type 'numpy.ndarray'>uint64 did not raise fpe error 'underflow'.
  Type <type 'numpy.ndarray'>uint64 did not raise fpe error 'overflow'.
  Type <type 'numpy.ndarray'>uint64 did not raise fpe error 'overflow'.

======================================================================
FAIL: Seterr should apply to integer types
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/pi/code/numpy/numpy/core/tests/test_numeric.py", line 399, in test_seterr_for_ints
    assert_(len(raised) == 0, pretty_repr)
  File "/home/pi/.local/lib/python2.7/site-packages/numpy/testing/utils.py", line 37, in assert_
    raise AssertionError(msg)
AssertionError: 18 errors:
  Type <type 'numpy.int8'> raised wrong fpe error 'overflow encountered in byte_scalars', not 'underflow'.
  Type <type 'numpy.uint8'> raised wrong fpe error 'overflow encountered in ubyte_scalars', not 'underflow'.
  Type <type 'numpy.int16'> raised wrong fpe error 'overflow encountered in short_scalars', not 'underflow'.
  Type <type 'numpy.uint16'> raised wrong fpe error 'overflow encountered in ushort_scalars', not 'underflow'.
  Type <type 'numpy.int32'> raised wrong fpe error 'overflow encountered in int_scalars', not 'underflow'.
  Type <type 'numpy.uint32'> raised wrong fpe error 'overflow encountered in uint_scalars', not 'underflow'.
  Type <type 'numpy.int64'> raised wrong fpe error 'overflow encountered in long_scalars', not 'underflow'.
  Type <type 'numpy.int64'> did not raise fpe error 'overflow'.
  Type <type 'numpy.uint64'> raised wrong fpe error 'overflow encountered in ulong_scalars', not 'underflow'.
  Type <type 'numpy.uint64'> did not raise fpe error 'overflow'.
  Type <type 'numpy.int64'> raised wrong fpe error 'overflow encountered in longlong_scalars', not 'underflow'.
  Type <type 'numpy.int64'> did not raise fpe error 'overflow'.
  Type <type 'numpy.uint64'> raised wrong fpe error 'overflow encountered in ulonglong_scalars', not 'underflow'.
  Type <type 'numpy.uint64'> did not raise fpe error 'overflow'.
  Type <type 'numpy.int64'> raised wrong fpe error 'overflow encountered in long_scalars', not 'underflow'.
  Type <type 'numpy.int64'> did not raise fpe error 'overflow'.
  Type <type 'numpy.uint64'> raised wrong fpe error 'overflow encountered in ulong_scalars', not 'underflow'.
  Type <type 'numpy.uint64'> did not raise fpe error 'overflow'.

----------------------------------------------------------------------
Ran 3 tests in 0.007s

FAILED (errors=1, failures=2)

njsmith · 2013-04-06T13:27:41Z

My gut feeling is that the overflow versus underflow thing isn't worth fixing; instead we should create a new error type "wraparound" that means, integer overflow/underflow. (This has other advantages too -- integer and floating overflow are very different, and the code for manipulating the floating point error flags is stupidly expensive, so it'd be better if the integer operations didn't do it.)

I'm pretty sure there's code that's supposed to detect int64 multiply wraparound -- guess it's buggy!

charris · 2014-03-28T19:51:17Z

@ivanov Are you going to pursue this? Otherwise I'm going to close it.

ivanov · 2014-03-28T21:27:33Z

Wow, blast from the past - had no idea this was still open. I'll see what I can do

njsmith · 2014-03-28T21:34:02Z

It would really make more sense to add a new error type for integer
wraparound...
On 28 Mar 2014 22:27, "Paul Ivanov" notifications@github.com wrote:

Wow, blast from the past - had no idea this was still open. I'll see what
I can do

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/3199#issuecomment-38969956
.

charris · 2015-01-18T00:32:36Z

@ivanov I'm closing this. Let me know if you still want to work on it.

ivanov added 4 commits April 5, 2013 16:44

DOC: spell out behaviour of seterr for int arrays

009d166

TST: added tests for seterr behavior with integers

74740c7

ENH: more detailed error messages

2b18fc4

FIX: previously referred to incorrect gh issue

47c3de6

ewmoore mentioned this pull request Oct 15, 2014

matrix_power fails/overflows for large exponents. #5166

Closed

charris added the component: documentation label Jan 17, 2015

charris closed this Jan 18, 2015

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

documentation, tests, and bugs in seterr behavior for integer types #3199

documentation, tests, and bugs in seterr behavior for integer types #3199

Uh oh!

ivanov commented Apr 6, 2013

Uh oh!

ivanov commented Apr 6, 2013

Uh oh!

njsmith commented Apr 6, 2013

Uh oh!

charris commented Mar 28, 2014

Uh oh!

ivanov commented Mar 28, 2014

Uh oh!

njsmith commented Mar 28, 2014

Uh oh!

charris commented Jan 18, 2015

Uh oh!

Uh oh!

Uh oh!

documentation, tests, and bugs in seterr behavior for integer types #3199

documentation, tests, and bugs in seterr behavior for integer types #3199

Uh oh!

Conversation

ivanov commented Apr 6, 2013

Uh oh!

ivanov commented Apr 6, 2013

Uh oh!

njsmith commented Apr 6, 2013

Uh oh!

charris commented Mar 28, 2014

Uh oh!

ivanov commented Mar 28, 2014

Uh oh!

njsmith commented Mar 28, 2014

Uh oh!

charris commented Jan 18, 2015

Uh oh!

Uh oh!