Skip to content

Intermittent badness on Python 3.5, 32-bit #6620

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
matthew-brett opened this issue Jun 21, 2016 · 9 comments
Closed

Intermittent badness on Python 3.5, 32-bit #6620

matthew-brett opened this issue Jun 21, 2016 · 9 comments

Comments

@matthew-brett
Copy link
Contributor

I'm testing the build of matplotlib 2.0.0b1 by building 64 and 32-bit manylinux wheels and then testing on a matching (64 / 32 bit) docker container.

The test runs have picked up a couple of not-replicable errors for Python 3.5.0 and 32-bits. I wanted to flag these here in case y'all have an idea what might be going on, and to record in case anyone else hits the same problems.

One run stalled:

matplotlib.tests.test_axes.test_stackplot.test ... 
No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself.

https://travis-ci.org/matthew-brett/matplotlib-wheels/jobs/138518918#L1069

Another generated a segfault:

matplotlib.tests.test_figure.test_figure.test ... Fatal Python error: Segmentation fault

Current thread 0xf751f700 (most recent call first):
  File "/venv/lib/python3.5/site-packages/numpy/linalg/linalg.py", line 526 in inv
  File "/venv/lib/python3.5/site-packages/matplotlib/transforms.py", line 1796 in inverted
  File "/venv/lib/python3.5/site-packages/matplotlib/transforms.py", line 2400 in inverted
  File "/venv/lib/python3.5/site-packages/matplotlib/transforms.py", line 2400 in inverted
  File "/venv/lib/python3.5/site-packages/matplotlib/axis.py", line 1768 in _get_pixel_distance_along_axis
  File "/venv/lib/python3.5/site-packages/matplotlib/axis.py", line 1027 in _update_ticks

[snipped trace]

Segmentation fault      (core dumped) python $MPL_SRC_DIR/tests.py -sv

https://travis-ci.org/MacPython/matplotlib-wheels/jobs/138682507#L8175

I guess this may be an error in the blas library of numpy (OpenBLAS 0.2.18, 32-bit), but I haven't seen it since:

https://travis-ci.org/matthew-brett/matplotlib-wheels/builds

I guess it would be worth trying to track that one down (sigh).

@matthew-brett matthew-brett changed the title Intermittent badness on Python 3, 32-bit Intermittent badness on Python 3.5, 32-bit Jun 21, 2016
@matthew-brett
Copy link
Contributor Author

Oh dear - that was pretty easy to replicate:

docker run --rm -ti -e PYTHON_VERSION=3.5 -v $PWD:/io matthewbrett/trusty:32 /bin/bash

Then, at that bash prompt:

source /io/test_mpl_py35.sh

Where test_mpl_py35.sh is:

apt-get update
apt-get install -y pkg-config git libpng-dev libz-dev
# Use nipy server for 32-bit wheel
pip install -f https://nipy.bic.berkeley.edu/manylinux numpy
git clone https://github.com/matplotlib/matplotlib.git
export MPLLOCALFREETYPE=1
cd matplotlib
pip install -e .
pip install nose mock sphinx
python tests.py -sv >& test.log

This gives a similar segfault to that above, on the first test run, but on a different test:

matplotlib.tests.test_cycles.test_colorcycle_basic.test ... Fatal Python error: Bus error

Current thread 0xf752c700 (most recent call first):
  File "/venv/lib/python3.5/site-packages/numpy/linalg/linalg.py", line 526 in inv

A second run of the tests gives a similar segfault, but at a yet another test:

matplotlib.tests.test_simplification.test_clipper.test ... Fatal Python error: Bus error

Current thread 0xf74ca700 (most recent call first):
  File "/venv/lib/python3.5/site-packages/numpy/linalg/linalg.py", line 526 in inv

A few tries of nosetests lib/matplotlib/tests/test_cycles.py doesn't generate the segfault, maybe meaning that the process of running the previous tests has made the system prone to crash.

I then tried restoring to a completely vanilla numpy, with system blas and lapack, to see whether it was the OpenBLAS blas / lapack that was causing the segfault:

pip uninstall -y numpy
apt-get install -y libblas-dev liblapack-dev
pip install --no-binary :all: numpy

Rerunning the tests still gives the segfault, but not mentioning linalg this time:

matplotlib.tests.test_mathtext.test_mathtext_cm_53.test ... Fatal Python error: Bus error

Current thread 0xf74a1700 (most recent call first):
  File "/venv/lib/python3.5/site-packages/pyparsing.py", line 170 in __init__

Again, running nosetests lib/matplotlib/tests/test_mathtext.py three times didn't cause the segfault.

The fact that the segfault happened with numpy linked to the system blas / lapack makes me wonder whether this is in fact a matplotlib bug.

@tacaswell
Copy link
Member

does your docker image have enough memory? In the other logs there were a couple of 'out of memory' exceptions, does docker do funny things at the edges?

Is the docker image posted someplace public?

@tacaswell tacaswell added this to the 2.0 (style change major release) milestone Jun 21, 2016
@matthew-brett
Copy link
Contributor Author

I don't think the image would run out of memory - it's on a beast of a machine in a Berkeley server room somewhere. cat /proc/meminfo gives:

MemTotal:       263851876 kB
MemFree:        14519900 kB

Yes the docker image is public on docker hub - if you run the docker run command above it should pull the docker image down for you - or pull it down beforehand with docker pull matthewbrett/trusty:32.

@tacaswell
Copy link
Member

🐑 Sorry, glossed over the docker details

@matthew-brett
Copy link
Contributor Author

Tried on a real 32-bit machine, Ubuntu 12.04, with vanilla blas / lapack numpy, got this:

matplotlib.tests.test_axes.test_stackplot.test ... Fatal Python error: Segmentation fault

Current thread 0xb770db00 (most recent call first):
  File "/home/mb312/tmp/mpl-test/matplotlib/lib/matplotlib/backends/backend_agg.py", line 123 in draw_markers
  File "/home/mb312/tmp/mpl-test/matplotlib/lib/matplotlib/collections.py", line 320 in draw

You're welcome to a login to that machine - send me your public ssh key somehow?

@tacaswell
Copy link
Member

coupled to #6604

I think the way we should deal with this is to make the default threshold value configurable. This might also help with getting the tests to pass for the down-stream packagers (as they can turn the knob up for different freetype versions).

The other option is to have pluggable sets of test images for different architectures and freetype versions.

@efiring
Copy link
Member

efiring commented Nov 11, 2016

Sets of test images? Please, no!

@QuLogic QuLogic modified the milestones: 2.0.1 (next bug fix release), 2.0 (style change major release) Dec 7, 2016
@stefanv
Copy link
Contributor

stefanv commented Feb 15, 2017

@tacaswell @efiring Matplotlib wheels being missing on 32-bit architectures is currently blocking scikit-image from releasing. Is any work planned on this issue?

@QuLogic QuLogic modified the milestones: 2.0.1 (next bug fix release), 2.0.2 (next bug fix release) May 3, 2017
@tacaswell tacaswell modified the milestones: 2.1.1 (next bug fix release), 2.2 (next feature release) Oct 9, 2017
@QuLogic
Copy link
Member

QuLogic commented Jun 3, 2021

We no longer support Python 3.5.

@QuLogic QuLogic closed this as completed Jun 3, 2021
@story645 story645 removed this from the future releases milestone Oct 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants