Skip to content

BUG: integer promoted to float during calculation #23476

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
burlen opened this issue Mar 28, 2023 · 4 comments
Closed

BUG: integer promoted to float during calculation #23476

burlen opened this issue Mar 28, 2023 · 4 comments
Labels
00 - Bug

Comments

@burlen
Copy link
Contributor

burlen commented Mar 28, 2023 •
edited
Loading

Describe the issue:

I have an array of integers representing the index space of a Cartesian mesh. From this I calculate the dimensions of the mesh. The calculation should be integer operations. Both the operands and result should be integers.

Reproduce the code example:

import numpy as np                                                                                                                                               

ext = np.array([0,3,0,4], dtype=np.uint64) # an array representing the index
                                           # space of a Cartesian mesh indexed by i and j
                                           # in the format [i0, i1, j0, j1].

nx = ext[1] - ext[0] + 1 # the mesh dimension in the i direction. this should
                         # be an integer but numpy unexpectedly promotes to
                         # float64 during the calculation.


print('type of ext[0] = %s'%(str(type(ext[0]))))
print('type of nx = %s'%(str(type(nx))))

Error message:

no imediate crash or errors, however, promoting to float during operations on integers is unexpected behavior and causes issues in later calculations which should be integer operations and relay on the behavior of integer operations for correctness.

Runtime information:

1.24.2
3.11.2 (main, Feb 8 2023, 00:00:00) [GCC 12.2.1 20221121 (Red Hat 12.2.1-4)]
[{'simd_extensions': {'baseline': ['SSE', 'SSE2', 'SSE3'],
'found': ['SSSE3',
'SSE41',
'POPCNT',
'SSE42',
'AVX',
'F16C'],
'not_found': ['FMA3',
'AVX2',
'AVX512F',
'AVX512CD',
'AVX512_KNL',
'AVX512_KNM',
'AVX512_SKX',
'AVX512_CLX',
'AVX512_CNL',
'AVX512_ICL']}},
{'architecture': 'Sandybridge',
'filepath': '/home/bloring/work/teca/py3k-mpich-cuda/lib/python3.11/site-packages/numpy.libs/libopenblas64_p-r0-15028c96.3.21.so',
'internal_api': 'openblas',
'num_threads': 20,
'prefix': 'libopenblas',
'threading_layer': 'pthreads',
'user_api': 'blas',
'version': '0.3.21'},
{'filepath': '/usr/lib64/libgomp.so.1.0.0',
'internal_api': 'openmp',
'num_threads': 20,
'prefix': 'libgomp',
'user_api': 'openmp',
'version': None}]
None

Context for the issue:

The unexpected promotion from int to float leads to incorrect results in later calculations that depend on the standard behavior of integer operations. It also results in issues when interoperate with C++ codes that expect integers.

If I instead use a Python list, rather than a numpy array the behavior is as expected, integer arithmetic operations are used and the result is an integer.

ext = [0,3,0,4]  # and array representing the index
                 # space of a Cartesian mesh indexed by i and j
                 # in the format [i0, i1, j0, j1].

nx = ext[1] - ext[0] + 1 # the mesh dimension in the i direction. 
                         # Python does not promote this to a float.
                         # This is the correct  and expected behavior.

print('type of ext[0] = %s'%(str(type(ext[0]))))
print('type of nx = %s'%(str(type(nx))))
@seberg
Copy link
Member

seberg commented Mar 28, 2023

There are two possible angles to this:

  1. You are using a Python integer, and we hope to change how that would be handled in this context, see BUG: Non-weak promotion of Python integers can be cumbersome for unsigned integers #22624 or maybe the issues linked within (NPY_PROMOTION_STATE=weak would test ride such changes, but there are still some loose ends see NEP 50 and discussion welcome :). E.g. should this apply mainly to operators, how do we define "cast safety" for a python integer?).
  2. int64 and uint64 (a bit more generic) promote to float64, which is pretty weird and BUG: Binary operations between uint64 and intX results in float64  #20905 discusses it for example.

The second one is maybe more to the point (I guess this is a duplicate of it). Unfortunately, it's complicated (mostly w.r.t. to backcompat), but there is some discussion over there.

@burlen
Copy link
Contributor Author

burlen commented Mar 28, 2023

OK, I see a bunch of others have suffered from this.

The crux of this issue is that promotion from integer to floating point type during arithmetic should only occur if one of the operands are floating point. This is essential behavior since the difference between integer and floating point arithmetic is useful and we'd like to exploit that in our codes. Promoting to floating point when all operands are integers prevents the exploitation of this behavior. At the very least it's confusing and unexpected.

Why wouldn't numpy implement the behavior defined by C/C++ relating to these issues? (ref)

@seberg
Copy link
Member

seberg commented Mar 29, 2023

Why wouldn't numpy implement the behavior defined by C/C++ relating to these issues?

I don't think you should care about the why here (yes, Chestertons fence, but overall...). These things presumably pre-date NumPy. There is at least one more reason: The fact that until recently numpy used to clump together the concept of promotion and "safe casting" (which is probably the argument for the original promotion behavior).

The problem is not about convincing anyone that the status-quo is not a great definition. It is about having a plan of how to enact change and to what extend (compared to e.g. C rules which are quite broadly different!).

@burlen burlen mentioned this issue Mar 31, 2023
Open
@seberg
Copy link
Member

seberg commented Feb 3, 2024

Closing, the examples are fixed in main/NumPy 2.0 by NEP 50. For the mixing of uint/int see the issue linkes above (gh-20905).

@seberg seberg closed this as completed Feb 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
00 - Bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@seberg @burlen