Skip to content

matplotlib installation from source and numpy incompatibility #10135

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
grzanka opened this issue Dec 30, 2017 · 9 comments
Closed

matplotlib installation from source and numpy incompatibility #10135

grzanka opened this issue Dec 30, 2017 · 9 comments

Comments

@grzanka
Copy link

grzanka commented Dec 30, 2017

Bug report

Bug summary

After installing matplotlib 2.1.1 via pip (using source package , .tar.gz) it fails to run due to incompatibility with numpy API.

Code for reproduction

I've used docker on debian 8 to test it with default python (2.7) and default pip (1.5):

→ docker run --rm -it debian:8 bash
root@49c12ed51a55:/# apt-get update -qq
root@49c12ed51a55:/# apt-get install python-pip libpng12-dev libfreetype6-dev pkg-config 
(...)
root@49c12ed51a55:/# pip install matplotlib 1>stdout.log 2>stderr.log
root@49c12ed51a55:/# python -c "import matplotlib.artist as martist"
RuntimeError: module compiled against API version 0xc but this version of numpy is 0xb
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/matplotlib/artist.py", line 16, in <module>
    from .path import Path
  File "/usr/local/lib/python2.7/dist-packages/matplotlib/path.py", line 25, in <module>
    from . import _path, rcParams
ImportError: numpy.core.multiarray failed to import
root@49c12ed51a55:/# python -V
Python 2.7.9
root@49c12ed51a55:/# python -c "import matplotlib; print(matplotlib.get_backend())"
agg
root@49c12ed51a55:/# cat stdout.log | grep NPY_API_VERSION
    #define NPY_API_VERSION 0x0000000C
    #define NPY_API_VERSION 0x0000000B

Matplotlib version

  • Operating system: Linux, debian 8 (in docker)
  • Matplotlib version: 2.1.1
  • Matplotlib backend (print(matplotlib.get_backend())): agg
  • Python version: 2.7.9
  • Other libraries: docker, debian 8

The problem was triggered by some other package with which I had trouble to produce wheels. See discussion here: pytrip/pytrip#419 (comment)
@QuLogic
Copy link
Member

QuLogic commented Dec 31, 2017

Seems more like a pip problem to me. It downloads NumPy 1.14.0rc1 and installs it temporarily as a build requirement, builds Matplotlib against that, then actually installs NumPy 1.13.3 for normal use.

@QuLogic
Copy link
Member

QuLogic commented Dec 31, 2017

This works fine if you pip install --upgrade pip and then use /usr/local/bin/pip (because that's not on the path for some reason). Of course, the new version defaults to wheels, but if you also pass --no-binary :all: it still works when built from source.

@QuLogic QuLogic closed this as completed Dec 31, 2017
@grzanka
Copy link
Author

grzanka commented Dec 31, 2017

Thank you for the advice @QuLogic , indeed it solved the problem on debian.

The problem still persist on the docker image which is used to build binary manylinux1 wheels (https://github.com/pypa/manylinux).

Here I've used 32bit docker image with Python 3.4 and pip 9.0. As the docker image runs some ancient OS (CentOS 5), with freetype 2.2, I had to manually compile newer freetype library, required by matplotlib. See below:

→ docker run --rm -it quay.io/pypa/manylinux1_i686 linux32 bash
bash-3.2# wget --no-check-certificate http://downloads.sourceforge.net/freetype/freetype-2.3.0.tar.gz -q
bash-3.2# tar -zxf freetype-2.3.0.tar.gz
bash-3.2# cd freetype-2.3.0
bash-3.2# ./configure --prefix=/usr  1>/dev/null 2>/dev/null
bash-3.2# make -j4 -s 1>/dev/null 2>/dev/null
bash-3.2# make install -s 1>/dev/null 2>/dev/null
bash-3.2# cd ..
bash-3.2# yum install libpng-devel -yq >/dev/null
bash-3.2# /opt/python/cp34-cp34m/bin/pip -V
pip 9.0.1 from /opt/_internal/cpython-3.4.6/lib/python3.4/site-packages (python 3.4)
bash-3.2# /opt/python/cp34-cp34m/bin/pip install matplotlib
Collecting matplotlib
  Downloading matplotlib-2.1.1.tar.gz (36.1MB)
    100% |████████████████████████████████| 36.1MB 29kB/s 
Collecting numpy>=1.7.1 (from matplotlib)
  Downloading numpy-1.13.3-cp34-cp34m-manylinux1_i686.whl (12.9MB)
    100% |████████████████████████████████| 12.9MB 78kB/s 
Collecting six>=1.10 (from matplotlib)
  Downloading six-1.11.0-py2.py3-none-any.whl
Collecting python-dateutil>=2.0 (from matplotlib)
  Downloading python_dateutil-2.6.1-py2.py3-none-any.whl (194kB)
    100% |████████████████████████████████| 194kB 4.4MB/s 
Collecting pytz (from matplotlib)
  Downloading pytz-2017.3-py2.py3-none-any.whl (511kB)
    100% |████████████████████████████████| 512kB 1.9MB/s 
Collecting cycler>=0.10 (from matplotlib)
  Downloading cycler-0.10.0-py2.py3-none-any.whl
Collecting pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 (from matplotlib)
  Downloading pyparsing-2.2.0-py2.py3-none-any.whl (56kB)
    100% |████████████████████████████████| 61kB 9.0MB/s 
Building wheels for collected packages: matplotlib
  Running setup.py bdist_wheel for matplotlib ... done
  Stored in directory: /root/.cache/pip/wheels/46/46/09/c96e256b5672080df66ed320966f76c040fcc1d335a47a0e5e
Successfully built matplotlib
Installing collected packages: numpy, six, python-dateutil, pytz, cycler, pyparsing, matplotlib
Successfully installed cycler-0.10.0 matplotlib-2.1.1 numpy-1.13.3 pyparsing-2.2.0 python-dateutil-2.6.1 pytz-2017.3 six-1.11.0
bash-3.2# /opt/python/cp34-cp34m/bin/python -c "import matplotlib.artist as martist"
RuntimeError: module compiled against API version 0xc but this version of numpy is 0xb
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/opt/python/cp34-cp34m/lib/python3.4/site-packages/matplotlib/artist.py", line 16, in <module>
    from .path import Path
  File "/opt/python/cp34-cp34m/lib/python3.4/site-packages/matplotlib/path.py", line 25, in <module>
    from . import _path, rcParams
ImportError: numpy.core.multiarray failed to import

@QuLogic
Copy link
Member

QuLogic commented Jan 1, 2018

Pass -v to pip install and it shows it's doing the same thing, building against the release candidate but installing the released version. The annoying thing is that even in verbose mode, I can't get pip to show that it's downloading anything to be built against. I think this is a bug in pip or the PyPI metadata for NumPy, or maybe in the manylinux1 container.

@grzanka
Copy link
Author

grzanka commented Jan 1, 2018

@QuLogic thanks again for suggestion where to look for source of this problem.

First of all I've realized that recent stable version of matplotlib (2.1.1) doesn't have manylinux1 wheels for 32bit architecture, looking at: https://pypi.org/project/matplotlib/2.1.1/#files . They are however present for Windows. Is there any reason behind it ?

This is forcing the compilation of matplotlib package from sources for 32bit, even when using modern pip (i.e. 9.X) and modern linux distro (i.e. debian 9), see below - it gives same problem as before.

→ docker run --rm -it i386/debian:stable linux32 bash
root@b2cf53793176:/# apt-get update -qq
root@b2cf53793176:/# apt-get install python3-pip libfreetype6-dev pkg-config python3-dev gfortran libatlas-dev libatlas-base-dev subversion -y -qq 1>1.log 2>2.log 
root@b2cf53793176:/# pip3 install matplotlib -vvv 1>stdout.log 2>stderr.log
root@b2cf53793176:/# python3 -c "import matplotlib.artist as martist"
RuntimeError: module compiled against API version 0xc but this version of numpy is 0xb
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.5/dist-packages/matplotlib/artist.py", line 16, in <module>
    from .path import Path
  File "/usr/local/lib/python3.5/dist-packages/matplotlib/path.py", line 25, in <module>
    from . import _path, rcParams
ImportError: numpy.core.multiarray failed to import
root@b2cf53793176:/# cat stdout.log | grep NPY_API_VERSION
    #define NPY_API_VERSION 0x0000000C

Contents of the log file with standard output can be found here:
https://gist.github.com/grzanka/2ed91a170ded7bf0b525fab6064e352b#file-stdout-log

As you mentioned below matplotlib during compilation gets numpy 1.14rc and compiles itself against it, but later installs as dependency numpy 1.13.

@grzanka grzanka mentioned this issue Jan 1, 2018
Closed
@grzanka
Copy link
Author

grzanka commented Jan 1, 2018

@QuLogic could you please take a look at the comment here:
pypa/setuptools#1243 (comment)

when fetching requirements, easy_install allow pre-releases per default, while pip disallow them unless --pre is used.

Do you think that there is some hope that on matplotlib side this problem can be handled somehow ?

@tacaswell
Copy link
Member

First of all I've realized that recent stable version of matplotlib (2.1.1) doesn't have manylinux1 wheels for 32bit architecture, looking at: https://pypi.org/project/matplotlib/2.1.1/#files . They are however present for Windows. Is there any reason behind it ?

There is something wrong with the wheel-builder on 32bit manylinux (it may just be that the tests do not pass due to bit-issues) and no one has asked about it yet.

Do you think that there is some hope that on matplotlib side this problem can be handled somehow ?

No, I think this should go to pypa.

The simplest work around is to just install numpy in a separate step and then install Matplotlib.

@grzanka
Copy link
Author

grzanka commented Jan 2, 2018

@tacaswell thank you for advice - indeed prior installation of numpy fixes the problem:

→ docker run --rm -it i386/debian:stable linux32 bash
root@3bdfb21fe54f:/# apt-get update -qq
root@3bdfb21fe54f:/# apt-get install python3-pip libfreetype6-dev pkg-config python3-dev gfortran libatlas-dev libatlas-base-dev subversion -y -qq 1>1.log 2>2.log 
root@3bdfb21fe54f:/# pip3 install numpy
Collecting numpy
  Downloading numpy-1.13.3-cp35-cp35m-manylinux1_i686.whl (12.8MB)
    100% |################################| 12.8MB 115kB/s 
Installing collected packages: numpy
Successfully installed numpy-1.13.3
root@3bdfb21fe54f:/# pip3 install matplotlib
Collecting matplotlib
  Downloading matplotlib-2.1.1.tar.gz (36.1MB)
    100% |################################| 36.1MB 41kB/s 
Collecting cycler>=0.10 (from matplotlib)
  Downloading cycler-0.10.0-py2.py3-none-any.whl
Requirement already satisfied: numpy>=1.7.1 in /usr/local/lib/python3.5/dist-packages (from matplotlib)
Collecting pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 (from matplotlib)
  Downloading pyparsing-2.2.0-py2.py3-none-any.whl (56kB)
    100% |################################| 61kB 11.0MB/s 
Collecting python-dateutil>=2.0 (from matplotlib)
  Downloading python_dateutil-2.6.1-py2.py3-none-any.whl (194kB)
    100% |################################| 194kB 5.9MB/s 
Collecting pytz (from matplotlib)
  Downloading pytz-2017.3-py2.py3-none-any.whl (511kB)
    100% |################################| 512kB 2.5MB/s 
Requirement already satisfied: six>=1.10 in /usr/lib/python3/dist-packages (from matplotlib)
Building wheels for collected packages: matplotlib
  Running setup.py bdist_wheel for matplotlib ... done
  Stored in directory: /root/.cache/pip/wheels/46/46/09/c96e256b5672080df66ed320966f76c040fcc1d335a47a0e5e
Successfully built matplotlib
Installing collected packages: cycler, pyparsing, python-dateutil, pytz, matplotlib
Successfully installed cycler-0.10.0 matplotlib-2.1.1 pyparsing-2.2.0 python-dateutil-2.6.1 pytz-2017.3
root@3bdfb21fe54f:/# python3 -c "import matplotlib.artist as martist"

@matthew-brett
Copy link
Contributor

matthew-brett commented Jan 2, 2018 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@matthew-brett @tacaswell @QuLogic @grzanka