-
-
Notifications
You must be signed in to change notification settings - Fork 10.8k
near-infinite loop on ppc64 in np.random.hypergeometric #14457
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
I'll take a look, but I don't know if I'll be able to reproduce the problem. The call
The platform being |
The new code runs no problem. |
It does seem to be some kind of cython/multiiter endianness issue. Here's some simple cython code that seems to be failing (I don't write much cython so I haven't figured out the exact problem yet). This is debug code I added in it = np.PyArray_MultiIterNew4(randoms, a_arr, b_arr, c_arr)
with lock:
for i in range(n):
a_val = (<int64_t*>np.PyArray_MultiIter_DATA(it, 1))[0]
b_val = (<int64_t*>np.PyArray_MultiIter_DATA(it, 2))[0]
c_val = (<int64_t*>np.PyArray_MultiIter_DATA(it, 3))[0]
print(c_val, c_arr) The last line prints |
All right, I think I see the bug. The hypergeometric code assumes that So the following code in # This cast to long is required to ensure that the values are inbounds
ongood = <np.ndarray>np.PyArray_FROM_OTF(ngood, np.NPY_LONG, np.NPY_ALIGNED)
onbad = <np.ndarray>np.PyArray_FROM_OTF(nbad, np.NPY_LONG, np.NPY_ALIGNED)
onsample = <np.ndarray>np.PyArray_FROM_OTF(nsample, np.NPY_LONG, np.NPY_ALIGNED) The fix might be as simple as replacing |
Hmm, |
I think my description of the size of For reasons I don't really remember, my gcc on my ppc64 system is in 32-bit ELF mode for executables/libraries, which is why long is 4 bytes. I think if it was in 64-bit mode long should be 8 bytes. In any case we should support 32-bit executables on ppc64. |
I don't think it's a cython bug since |
This sounds surprising to me. I was under the impression that python2's |
I think it is also 4 bytes in 32-bit executables on linux 64-bit systems, and is probably controlled by compiler flags too. I think the reason my ppc64(be) system is using 32-bit mode has to do with ubuntu support. I remember spending a while investigating ubuntu versions available on this system (a VM provided by OSUOSL, who did the ubuntu install for us), and ubuntu only provided 32-bit binaries for big-endian ppc when I asked for it in 2016. I think ubuntu used to support "powerpc" (which was 32-bit big-endian) but dropped support 2 years ago. It currently supports ppc64el (little endian, 64-bit) used by new IBM computers. |
Running the following command on ppc64(be) hangs due to a near-infinite loop:
Appears to be related to new code in #13761, perhaps @WarrenWeckesser has an idea. See in particular frame #2 below. It appears a value is negative but the code expects it not to be.
gdb stack trace after breaking during hang:
A little bit of gdb exploration so far:
The failing line 266 in the last bit of code is from #13761, but I haven't read it well enough yet to know what it should do.
The text was updated successfully, but these errors were encountered: