Skip to content

MagicMock a pandas DataFrame causes a segfault #25332

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
roachlord opened this issue Feb 15, 2019 · 5 comments
Closed

MagicMock a pandas DataFrame causes a segfault #25332

roachlord opened this issue Feb 15, 2019 · 5 comments
Labels
Regression Functionality that used to work in a prior pandas version Segfault Non-Recoverable Error

Comments

@roachlord
Copy link

roachlord commented Feb 15, 2019 •
edited
Loading

Code Sample

import pandas as pd
from mock import MagicMock

df = pd.DataFrame(0, [[1, 2], [3, 4]], ['col'])
df['col'] += MagicMock()

Problem description

I have a function that returns a column of dataframe. I want to mock out this function and test it is called with the correct args. The above code segs outs in 0.24.1 but not in 0.23.4.

Expected Output

Not to seg fault

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 2.7.12.final.0
python-bits: 64
OS: Linux
OS-release: 4.4.0-142-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US
LOCALE: None.None

pandas: 0.24.1
pytest: None
pip: 19.0.2
setuptools: 40.8.0
Cython: None
numpy: 1.16.1
scipy: 1.2.1
pyarrow: None
xarray: None
IPython: None
sphinx: None
patsy: None
dateutil: 2.8.0
pytz: 2018.9
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: None
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml.etree: None
bs4: None
html5lib: None
sqlalchemy: 1.2.17
pymysql: None
psycopg2: 2.7.7 (dt dec pq3 ext lo64)
jinja2: None
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None
gcsfs: None
@gfyoung gfyoung added the Regression Functionality that used to work in a prior pandas version label Feb 18, 2019
@gfyoung gfyoung added this to the 0.24.2 milestone Feb 18, 2019
@gfyoung
Copy link
Member

gfyoung commented Feb 18, 2019

A very odd regression indeed, but a regression nonetheless and for a reasonable intention IMO.

@gfyoung gfyoung added the Segfault Non-Recoverable Error label Feb 18, 2019
@gfyoung
Copy link
Member

gfyoung commented Feb 18, 2019 •
edited
Loading

cc @jreback

This stems from #24541 in which MagicMock now gets passed into pandas_dtype from when is_timedelta64_dtype is called (this function is called as well prior to this commit, but the object would not get passed to the central pandas_dtype function).

The breaking logic is here:

https://github.com/pandas-dev/pandas/blob/659e0ca/pandas/core/dtypes/common.py#L2036-L2043

Unfortunately, numpy dtypes do not play nice with MagicMock. In fact, you can replicate the segfault yourself by just doing this:

from unittest.mock import MagicMock

import numpy as np

np.dtype(MagicMock())

@roachlord : Perhaps this might be worthwhile to bring up as an issue on the numpy side as well.

@roachlord roachlord mentioned this issue Feb 18, 2019
Closed
@jreback jreback modified the milestones: 0.24.2, Contributions Welcome Feb 18, 2019
@jreback
Copy link
Contributor

jreback commented Feb 18, 2019

this is not supported in any way; it may have happened to work before

@gfyoung
Copy link
Member

gfyoung commented Feb 18, 2019

@jreback : Fair enough. 👍

@roachlord : I see you are making some headway with your issue to numpy. If that gets resolved, that would also resolve the issue on our end (I think?), so perhaps that would be your best route for now.

@WillAyd
Copy link
Member

WillAyd commented Dec 17, 2019

Doesn't appear to be an issue on master so I think OK to close this one

@WillAyd WillAyd closed this as completed Dec 17, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Regression Functionality that used to work in a prior pandas version Segfault Non-Recoverable Error
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@WillAyd @jreback @gfyoung @roachlord