Skip to content

Segfault from using probably invalid dtype #2865

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
charris opened this issue Dec 28, 2012 · 8 comments
Closed

Segfault from using probably invalid dtype #2865

charris opened this issue Dec 28, 2012 · 8 comments
Labels
00 - Bug component: numpy._core

Comments

@charris
Copy link
Member

charris commented Dec 28, 2012

On a 64 bit machine this leads to a segfault.

In [1]: a = np.array([1], dtype='O').astype(('O', [('name', 'Q')]))

In [2]: a['name']

The type on the right should probably be illegal.
@charris
Copy link
Member Author

charris commented Dec 28, 2012

Simpler version of problem:

In [1]: dt = ('O', [('name', 'Q')])

In [2]: a = np.array([1], dtype=dt)

In [3]: a['name']
-- segfault --

@certik
Copy link
Contributor

certik commented Dec 28, 2012

It segfaults for me too:

In [1]: import numpy as np

In [2]: dt = ('O', [('name', 'Q')])

In [3]: a = np.array([1], dtype=dt)

In [4]: a['name']
*** glibc detected *** /home/ondrej/py27/bin/python: free(): invalid pointer: 0x0000000001b8846f ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x7eb96)[0x7fe289b7db96]
/home/ondrej/py27/bin/python[0x415b48]
/home/ondrej/py27/bin/python(PyParser_ASTFromString+0x79)[0x4c0aa9]
/home/ondrej/py27/bin/python(Py_CompileStringFlags+0x52)[0x4c0c32]
/home/ondrej/py27/bin/python[0x494faf]
/home/ondrej/py27/bin/python(PyEval_EvalFrameEx+0x540d)[0x49cacd]
/home/ondrej/py27/bin/python(PyEval_EvalCodeEx+0x826)[0x49e676]
/home/ondrej/py27/bin/python[0x4ffe2c]
/home/ondrej/py27/bin/python(PyObject_Call+0x47)[0x41eda7]
/home/ondrej/py27/bin/python[0x4224cf]
/home/ondrej/py27/bin/python(PyObject_Call+0x47)[0x41eda7]
/home/ondrej/py27/bin/python[0x421512]
/home/ondrej/py27/bin/python(PyObject_Call+0x47)[0x41eda7]
/home/ondrej/py27/bin/python(PyEval_EvalFrameEx+0x3bfc)[0x49b2bc]
/home/ondrej/py27/bin/python(PyEval_EvalFrameEx+0x60a2)[0x49d762]
/home/ondrej/py27/bin/python(PyEval_EvalCodeEx+0x826)[0x49e676]
/home/ondrej/py27/bin/python[0x4fff23]
/home/ondrej/py27/bin/python(PyObject_Call+0x47)[0x41eda7]
/home/ondrej/py27/bin/python[0x4224cf]
/home/ondrej/py27/bin/python(PyObject_Call+0x47)[0x41eda7]
/home/ondrej/py27/bin/python[0x421512]
/home/ondrej/py27/bin/python(PyObject_Call+0x47)[0x41eda7]
/home/ondrej/py27/bin/python(PyEval_EvalFrameEx+0x3bfc)[0x49b2bc]
/home/ondrej/py27/bin/python(PyEval_EvalFrameEx+0x60a2)[0x49d762]
/home/ondrej/py27/bin/python(PyEval_EvalFrameEx+0x60a2)[0x49d762]
/home/ondrej/py27/bin/python(PyEval_EvalCodeEx+0x826)[0x49e676]
/home/ondrej/py27/bin/python(PyEval_EvalFrameEx+0x4e95)[0x49c555]
/home/ondrej/py27/bin/python(PyEval_EvalCodeEx+0x826)[0x49e676]
/home/ondrej/py27/bin/python(PyEval_EvalFrameEx+0x4e95)[0x49c555]
/home/ondrej/py27/bin/python(PyEval_EvalCodeEx+0x826)[0x49e676]
/home/ondrej/py27/bin/python(PyEval_EvalFrameEx+0x4e95)[0x49c555]
/home/ondrej/py27/bin/python(PyEval_EvalCodeEx+0x826)[0x49e676]
/home/ondrej/py27/bin/python(PyEval_EvalFrameEx+0x4e95)[0x49c555]
/home/ondrej/py27/bin/python(PyEval_EvalCodeEx+0x826)[0x49e676]
/home/ondrej/py27/bin/python(PyEval_EvalCode+0x32)[0x49e7a2]
/home/ondrej/py27/bin/python(PyRun_FileExFlags+0xb0)[0x4c0fa0]
/home/ondrej/py27/bin/python(PyRun_SimpleFileExFlags+0xd7)[0x4c1af7]
/home/ondrej/py27/bin/python(Py_Main+0xadd)[0x4156fd]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed)[0x7fe289b2076d]
/home/ondrej/py27/bin/python[0x414811]
======= Memory map: ========
00400000-0056a000 r-xp 00000000 08:01 15475025                           /home/ondrej/py27/bin/python
00769000-0076a000 r--p 00169000 08:01 15475025                           /home/ondrej/py27/bin/python
0076a000-007a6000 rw-p 0016a000 08:01 15475025                           /home/ondrej/py27/bin/python
007a6000-007b5000 rw-p 00000000 00:00 0 
00a81000-01b95000 rw-p 00000000 00:00 0                                  [heap]
7fe27c000000-7fe27c021000 rw-p 00000000 00:00 0 
7fe27c021000-7fe280000000 ---p 00000000 00:00 0 
7fe280fbd000-7fe280ffb000 r-xp 00000000 08:01 15603261                   /home/ondrej/py27/lib/python2.7/site-packages/numpy/random/mtrand.so
7fe280ffb000-7fe2811fb000 ---p 0003e000 08:01 15603261                   /home/ondrej/py27/lib/python2.7/site-packages/numpy/random/mtrand.so
7fe2811fb000-7fe2811fc000 r--p 0003e000 08:01 15603261                   /home/ondrej/py27/lib/python2.7/site-packages/numpy/random/mtrand.so
7fe2811fc000-7fe281230000 rw-p 0003f000 08:01 15603261                   /home/ondrej/py27/lib/python2.7/site-packages/numpy/random/mtrand.so
7fe281230000-7fe281231000 rw-p 00000000 00:00 0 
7fe281231000-7fe28123a000 r-xp 00000000 08:01 15603266                   /home/ondrej/py27/lib/python2.7/site-packages/numpy/fft/fftpack_lite.so
7fe28123a000-7fe281439000 ---p 00009000 08:01 15603266                   /home/ondrej/py27/lib/python2.7/site-packages/numpy/fft/fftpack_lite.so
7fe281439000-7fe28143a000 r--p 00008000 08:01 15603266                   /home/ondrej/py27/lib/python2.7/site-packages/numpy/fft/fftpack_lite.so
7fe28143a000-7fe28143b000 rw-p 00009000 08:01 15603266                   /home/ondrej/py27/lib/python2.7/site-packages/numpy/fft/fftpack_lite.so
7fe28143b000-7fe281470000 r-xp 00000000 08:01 15344829                   /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0
7fe281470000-7fe28166f000 ---p 00035000 08:01 15344829                   /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0
7fe28166f000-7fe281670000 r--p 00034000 08:01 15344829                   /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0
7fe281670000-7fe281671000 rw-p 00035000 08:01 15344829                   /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0
7fe281671000-7fe281686000 r-xp 00000000 08:01 2228268                    /lib/x86_64-linux-gnu/libgcc_s.so.1
7fe281686000-7fe281885000 ---p 00015000 08:01 2228268                    /lib/x86_64-linux-gnu/libgcc_s.so.1
7fe281885000-7fe281886000 r--p 00014000 08:01 2228268                    /lib/x86_64-linux-gnu/libgcc_s.so.1
7fe281886000-7fe281887000 rw-p 00015000 08:01 2228268                    /lib/x86_64-linux-gnu/libgcc_s.so.1
7fe281887000-7fe28199b000 r-xp 00000000 08:01 15350352                   /usr/lib/x86_64-linux-gnu/libgfortran.so.3.0.0
7fe28199b000-7fe281b9b000 ---p 00114000 08:01 15350352                   /usr/lib/x86_64-linux-gnu/libgfortran.so.3.0.0
7fe281b9b000-7fe281b9c000 r--p 00114000 08:01 15350352                   /usr/lib/x86_64-linux-gnu/libgfortran.so.3.0.0
7fe281b9c000-7fe281b9e000 rw-p 00115000 08:01 15350352                   /usr/lib/x86_64-linux-gnu/libgfortran.so.3.0.0
7fe281b9e000-7fe281efe000 r-xp 00000000 08:01 15606652                   /usr/lib/atlas-base/atlas/libblas.so.3gf.0
7fe281efe000-7fe2820fe000 ---p 00360000 08:01 15606652                   /usr/lib/atlas-base/atlas/libblas.so.3gf.0
7fe2820fe000-7fe282107000 rw-p 00360000 08:01 15606652                   /usr/lib/atlas-base/atlas/libblas.so.3gf.0
7fe282107000-7fe282a12000 r-xp 00000000 08:01 15606653                   /usr/lib/atlas-base/atlas/liblapack.so.3gf.0
7fe282a12000-7fe282c12000 ---p 0090b000 08:01 15606653                   /usr/lib/atlas-base/atlas/liblapack.so.3gf.0
7fe282c12000-7fe282c17000 rw-p 0090b000 08:01 15606653                   /usr/lib/atlas-base/atlas/liblapack.so.3gf.0
7fe282c17000-7fe282d24000 rw-p 00000000 00:00 0 
7fe282d24000-7fe282d29000 r-xp 00000000 08:01 15732509                   /home/ondrej/py27/lib/python2.7/site-packages/numpy/linalg/lapack_lite.so
7fe282d29000-7fe282f29000 ---p 00005000 08:01 15732509                   /home/oAborted (core dumped)

Oops. We should fix this segfault before the final release.

@charris
Copy link
Member Author

charris commented Dec 28, 2012

I'm not convinced it needs to be fixed immediately. There are always going to be bugs and if you wait to fix all of them you will never release. I think it would be better to wait 1.7.1 or 1.8.. This segfault is also occurrs in numpy 1.5.

It makes me thing that there are several spots in numpy where using a standard parser generator and a formal language desciption might be the way to go. It might clarify some of these things and make maintenance easier. I'm thinking dtypes and maybe casting rules.

@certik
Copy link
Contributor

certik commented Dec 28, 2012

Ok, that sounds good to me. In any case, I am now releasing rc1.

@njsmith
Copy link
Member

njsmith commented Jan 2, 2013

Removing the 1.7 milestone because I agree w/ @charris. We've already shipped this bug, so releasing 1.7 with this bug won't make anything worse for anyone.

@charris
Copy link
Member Author

charris commented Feb 15, 2014

Still present in 1.9-devel.

@juliantaylor
Copy link
Contributor

I don't really understand the dtype syntax but what is happening is that somewhere during its parsing it loses the NPY_NEEDS_INIT flag of the object dtype so it works on uninitialized memory later.

ahaldane added a commit that referenced this issue Nov 25, 2016
@ahaldane
Merge pull request #8235 from J-Sand/invalid-structured-dtypes-fix
e80b948 
BUG: add checks for some invalid structured dtypes. Fixes #2865.
@ahaldane
Copy link
Member

Fixed by #8235

@eric-wieser eric-wieser mentioned this issue Apr 13, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
00 - Bug component: numpy._core
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@certik @charris @juliantaylor @njsmith @ahaldane