Skip to content

Various operations fail on recursive object arrays #8306

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
J-Sand opened this issue Nov 24, 2016 · 5 comments
Open

Various operations fail on recursive object arrays #8306

J-Sand opened this issue Nov 24, 2016 · 5 comments
Labels
00 - Bug Priority: low Valid, but not for immediate attention

Comments

@J-Sand
Copy link
Contributor

J-Sand commented Nov 24, 2016

It's possible to create a recursive object array:

>>> a = np.empty(1, dtype=object)
>>> a[0] = a

Many functions that support object arrays fail to check for such situations, resulting in a RecursionError or in some cases a stack overflow:

>>> print(a)
<truncated output>
RecursionError: maximum recursion depth exceeded while calling a Python object
>>> a in a
Fatal Python error: Cannot recover from stack overflow.

Current thread 0x00007fa98491c700 (most recent call first):
  File "<stdin>", line 1 in <module>
Aborted (core dumped)

Python's built-in containers handle recursion gracefully:

>>> a = []
>>> a.append(a)
>>> print(a)
[[...]]
>>> a in a
True

My guess is that this isn't really worth doing for ndarray, but it would be nice to at least prevent stack overflows?
@eric-wieser
Copy link
Member

Printing was fixed in #8963

@eric-wieser
Copy link
Member

I think we need to use Py_EnterRecursiveCall to avoid the stack overflow

@eric-wieser eric-wieser mentioned this issue May 9, 2017
Merged
eric-wieser added a commit to eric-wieser/numpy that referenced this issue May 9, 2017
@eric-wieser
BUG: Prevent stackoverflow on self-containing arrays
00f7710 
Approaches numpy#8306. This still errors, but it does so at a python level, without
a segfault.
@eric-wieser
Copy link
Member

Well, bool(a) now works, but a == a still fails

mherkazandjian pushed a commit to mherkazandjian/numpy that referenced this issue May 30, 2017
@eric-wieser @mherkazandjian
BUG: Prevent stackoverflow on self-containing arrays
c9f5a17 
Approaches numpy#8306. This still errors, but it does so at a python level, without
a segfault.
@gaslitbytech
Copy link

gaslitbytech commented Oct 10, 2017 •
edited
Loading

Just want to verify before creating a new issue as this seems similar.

Using pandas

    dataframe = dataframe.replace('', np.nan)

I'm getting the following error

    File "pandas/lib.pyx", line 1811, in pandas.lib.BlockPlacement.__init__ (pandas/lib.c:30223)
    File "/venv/lib/python3.6/site-packages/numpy/core/numeric.py", line 729, in require requirements = set(possible_flags[x.upper()] for x in requirements)
    RecursionError: maximum recursion depth exceeded

Is it related or should I create a new issue?

Edit. My issue is solved by

dataframe = dataframe.replace('', np.nan, inplace=True)

@eric-wieser
Copy link
Member

Sounds like a pandas bug. I'd create an issue there, with a minimal self-contained snippet that reproduces the problem

@mattip mattip added the Priority: low Valid, but not for immediate attention label May 2, 2019
@eric-wieser eric-wieser mentioned this issue Nov 19, 2019
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
00 - Bug Priority: low Valid, but not for immediate attention
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@eric-wieser @mattip @gaslitbytech @J-Sand