Description
Since the numpy 1.19.0 release, calculating ABS on a complex field of a structured dtype gives incorrect results.
The exact output looks as if there is an indexing problem.
I am able to reliably reproduce following bug on Travis-CI (but not other machines, such as my personal laptop and local compute cluster). It is unclear if this is hardware or environment specific in some way. I've created as simple an example
as I could. One possible hint is that it seems that Travis has AVX512 in its environment, whereas my test machines do not.
- https://github.com/ahnitz/test-numpy
- https://travis-ci.org/github/ahnitz/test-numpy/builds/700834047
Reproducing code example:
import numpy
import sys
val = [(0, 0., 0, -31.666483, 200, 0., 0., 1. , -2.8499086-4.852268j , 613090),
(1, 0., 0, 260.91525 , 42, 0., 0., 1. , 11.672331 -0.39883116j, 787315),
(1, 0., 0, 52.15155 , 42, 0., 0., 1. , -3.9071944-3.9876528j , 806641),
(1, 0., 0, 52.430195, 42, 0., 0., 1. , 3.5721765-4.4356694j , 1363540),
(2, 0., 0, 304.43646 , 58, 0., 0., 1. , -0.9019805+6.379318j , 787323),
(3, 0., 0, 299.42108 , 52, 0., 0., 1. , 6.365695 +3.1266918j , 787332),
(4, 0., 0, 39.4836 , 28, 0., 0., 9.182192, -16.804018 +9.028097j , 787304),
(4, 0., 0, 76.83787 , 28, 0., 0., 1. , 3.2854116-5.3171387j , 1321869),
(5, 0., 0, 143.26366 , 24, 0., 0., 10.996129, 14.33073 +1.0027167j , 787299)]
dtype = [('template_id', '<i8'), ('bank_chisq', '<f4'), ('bank_chisq_dof', '<i8'), ('chisq', '<f4'), ('chisq_dof', '<i8'), ('cont_chisq', '<f4'), ('psd_var_val', '<f4'), ('sg_chisq', '<f4'), ('snr', '<c8'), ('time_index', '<i8')]
vec = numpy.array(val, dtype=dtype)
a = vec['snr']
g = abs(a)
b = vec['snr'].copy()
h = abs(b)
print ((h-g).sum(), a, b, g, h)
# Uh, oh, we got a different answer!
if (h-g).sum() != 0:
sys.exit(1)Error message:
Output of the above test script when BROKEN (i.e. numpy 1.19.0)
-52.852776 [ -2.8499086-4.852268j 11.672331 -0.39883116j -3.9071944-3.9876528j
3.5721765-4.4356694j -0.9019805+6.379318j 6.365695 +3.1266918j
-16.804018 +9.028097j 3.2854116-5.3171387j 14.33073 +1.0027167j ] [ -2.8499086-4.852268j 11.672331 -0.39883116j -3.9071944-3.9876528j
3.5721765-4.4356694j -0.9019805+6.379318j 6.365695 +3.1266918j
-16.804018 +9.028097j 3.2854116-5.3171387j 14.33073 +1.0027167j ] [5.6272984e+00 1.1715089e+01 1.0000000e+00 0.0000000e+00 0.0000000e+00
7.2867520e-44 3.9483601e+01 7.6837868e+01 0.0000000e+00] [ 5.6272984 11.679143 5.58279 5.6952267 6.442769 7.092128
19.07568 6.2502713 14.3657675]
Output the above test script when WORKING (i.e. numpy 1.18.5)
0.0 [ -2.8499086-4.852268j 11.672331 -0.39883116j -3.9071944-3.9876528j
3.5721765-4.4356694j -0.9019805+6.379318j 6.365695 +3.1266918j
-16.804018 +9.028097j 3.2854116-5.3171387j 14.33073 +1.0027167j ] [ -2.8499086-4.852268j 11.672331 -0.39883116j -3.9071944-3.9876528j
3.5721765-4.4356694j -0.9019805+6.379318j 6.365695 +3.1266918j
-16.804018 +9.028097j 3.2854116-5.3171387j 14.33073 +1.0027167j ] [ 5.6272984 11.679143 5.58279 5.6952267 6.442769 7.092128
19.07568 6.2502713 14.3657675] [ 5.6272984 11.679143 5.58279 5.6952267 6.442769 7.092128
19.07568 6.2502713 14.3657675]
The 'broken' values looks suspiciously like an indexing error, as the second value output is what abs would be if using the wrong fields, and so on.
Numpy/Python version information:
Broken -> numpy == 1.19.0
Last Working -> numpy == 1.18.5
see https://travis-ci.org/github/ahnitz/test-numpy/builds/700834047