Skip to content

DEBUG try to reproduce Cython segfault on non-aligned data #21677

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 17 commits into from
Closed

DEBUG try to reproduce Cython segfault on non-aligned data #21677

wants to merge 17 commits into from

Conversation

ogrisel
Copy link
Member

@ogrisel ogrisel commented Nov 15, 2021 •
edited
Loading

Let's see if _test_sum actually crashes with when mmap array is created with aligned=False.

Based on #21654.

@ogrisel ogrisel mentioned this pull request Nov 15, 2021
Merged
@ogrisel
Copy link
Member Author

ogrisel commented Nov 15, 2021

Some interesting observations from the last run:

  • gcc version (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0 is used on the failing build
  • the segfault does not happen with float values, only integer values...
::test_memmap_on_contiguous_data[float32] PASSED                         [ 25%]
::test_memmap_on_contiguous_data[float64] PASSED                         [ 50%]
::test_memmap_on_contiguous_data[int32] Fatal Python error: Segmentation fault

I will re-run without capturing the output of print(x_mmap.flags) to check.

@ogrisel
Copy link
Member Author

ogrisel commented Nov 15, 2021

Confirmed, _test_sum does not crash on float data, even when the data is not aligned:

::test_memmap_on_contiguous_data[float32]   C_CONTIGUOUS : True
  F_CONTIGUOUS : True
  OWNDATA : False
  WRITEABLE : True
  ALIGNED : False
  WRITEBACKIFCOPY : False
  UPDATEIFCOPY : False

PASSED
::test_memmap_on_contiguous_data[float64]   C_CONTIGUOUS : True
  F_CONTIGUOUS : True
  OWNDATA : False
  WRITEABLE : True
  ALIGNED : False
  WRITEBACKIFCOPY : False
  UPDATEIFCOPY : False

PASSED
::test_memmap_on_contiguous_data[int32] Fatal Python error: Segmentation fault

Current thread 0x00007fa05a8cf740 (most recent call first):
  File "/home/vsts/work/1/s/sklearn/utils/tests/test_testing.py", line 736 in test_memmap_on_contiguous_data
[...]

@ogrisel
Copy link
Member Author

ogrisel commented Nov 15, 2021

I have tried to reproduce on ubuntu 18.04 with the same version of gcc (7.5.0) but on a arm64 CPU and I cannot reproduce. So it might be related to SIMD vectorization which is platform specific.

@ogrisel ogrisel mentioned this pull request Nov 15, 2021
Merged
@ogrisel
Copy link
Member Author

ogrisel commented Nov 16, 2021 •
edited
Loading

So the segfault happens on linux wheels built with the manylinux1 compiler but the good news it that it does not happen with the manylinux2010 compiler.

$ docker run --rm quay.io/pypa/manylinux2010_x86_64 gcc --version
gcc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3)

$ docker run --rm quay.io/pypa/manylinux1_x86_64 gcc --version
gcc (GCC) 4.8.2 20140120 (Red Hat 4.8.2-15)

So maybe it's a bug with older versions of GCC when building with SSE / SSE2 / SSE3 vectorization.

@ogrisel ogrisel closed this Nov 19, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
cython module:utils No Changelog Needed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@ogrisel @lorentzenchr