Skip to content

Port build system to Meson #26621

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Oct 4, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .appveyor.yml
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ install:
test_script:
# Now build the thing..
- set LINK=/LIBPATH:%cd%\lib
- pip install -ve .
- pip install -v --no-build-isolation --config-settings=setup-args="--vsenv" --editable .[dev]
# this should show no freetype dll...
- set "DUMPBIN=%VS140COMNTOOLS%\..\..\VC\bin\dumpbin.exe"
- '"%DUMPBIN%" /DEPENDENTS lib\matplotlib\ft2font*.pyd | findstr freetype.*.dll && exit /b 1 || exit /b 0'
Expand Down
14 changes: 7 additions & 7 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ commands:
graphviz \
inkscape \
lmodern \
ninja-build \
optipng \
texlive-fonts-recommended \
texlive-latex-base \
Expand Down Expand Up @@ -97,6 +98,7 @@ commands:
- run:
name: Install Python dependencies
command: |
python -m pip install --user meson-python pybind11
python -m pip install --user \
numpy<< parameters.numpy_version >> \
-r requirements/doc/doc-requirements.txt
Expand All @@ -114,15 +116,13 @@ commands:
version=${version#v}
python -m pip install matplotlib==${version}
else
python -m pip install --user -ve .
python -m pip install --user --verbose \
--no-build-isolation --editable .[dev]
fi
- save_cache:
key: build-deps-1
key: build-deps-2
paths:
# FreeType 2.6.1 tarball.
- ~/.cache/matplotlib/0a3c7dfbda6da1e8fce29232e8e96d987ababbbf71ebc8c75659e4132c367014
# Qhull 2020.2 tarball.
- ~/.cache/matplotlib/b5c2d7eb833278881b952c8a52d20179eab87766b00b865000469a45c1838b7e
- subprojects/packagecache

doc-build:
steps:
Expand Down Expand Up @@ -218,8 +218,8 @@ jobs:
- fonts-install
- pip-install

- mpl-install
- doc-deps-install
- mpl-install

- doc-build
- doc-show-errors-warnings
Expand Down
2 changes: 0 additions & 2 deletions .flake8
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,6 @@ exclude =
.eggs

per-file-ignores =
setup.py: E402

lib/matplotlib/__init__.py: E402, F401
lib/matplotlib/_animation_data.py: E501
lib/matplotlib/_api/__init__.py: F401
Expand Down
9 changes: 5 additions & 4 deletions .github/workflows/cibuildwheel.yml
Original file line number Diff line number Diff line change
Expand Up @@ -91,16 +91,17 @@ jobs:
runs-on: ${{ matrix.os }}
env:
CIBW_BEFORE_BUILD: >-
pip install certifi numpy>=1.25 &&
pip install numpy>=1.25 &&
rm -rf {package}/build
CIBW_BEFORE_BUILD_WINDOWS: >-
pip install certifi delvewheel numpy>=1.25 &&
pip install delvewheel numpy>=1.25 &&
rm -rf {package}/build
CIBW_REPAIR_WHEEL_COMMAND_WINDOWS: >-
delvewheel repair -w {dest_dir} {wheel}
CIBW_AFTER_BUILD: >-
twine check {wheel} &&
python {package}/ci/check_wheel_licenses.py {wheel}
CIBW_CONFIG_SETTINGS: setup-args="--vsenv"
CIBW_MANYLINUX_X86_64_IMAGE: manylinux2014
CIBW_SKIP: "*-musllinux_aarch64"
CIBW_TEST_COMMAND: >-
Expand Down Expand Up @@ -144,11 +145,11 @@ jobs:
CIBW_ARCHS: ${{ matrix.cibw_archs }}
# Remove this once NumPy with Python 3.12 wheels is not pre-release.
CIBW_BEFORE_BUILD: >-
pip install certifi "pybind11>=2.6" "setuptools>=42" "setuptools_scm>=7" &&
pip install "meson-python>=0.13.1" ninja "pybind11>=2.6" "setuptools>=42" "setuptools_scm>=7" &&
pip install --pre "numpy>=1.25" &&
rm -rf {package}/build
CIBW_BEFORE_BUILD_WINDOWS: >-
pip install certifi delvewheel "pybind11>=2.6" "setuptools>=42" "setuptools_scm>=7" &&
pip install delvewheel "meson-python>=0.13.1" ninja "pybind11>=2.6" "setuptools>=42" "setuptools_scm>=7" &&
pip install --pre "numpy>=1.25" &&
rm -rf {package}/build
CIBW_ENVIRONMENT: PIP_NO_BUILD_ISOLATION=0
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/codeql-analysis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,9 +41,9 @@ jobs:
# dependencies so we don't need another copy here.
# https://github.com/jazzband/pip-tools/pull/1681
python -m pip install --upgrade \
certifi contourpy cycler fonttools kiwisolver importlib_resources \
numpy packaging pillow pyparsing python-dateutil setuptools-scm \
pybind11
build contourpy cycler fonttools kiwisolver \
importlib_resources meson-python numpy packaging pillow pybind11 \
pyparsing python-dateutil setuptools-scm
echo "CODEQL_PYTHON=$(which python)" >> $GITHUB_ENV

- name: Initialize CodeQL
Expand All @@ -56,7 +56,7 @@ jobs:
if: matrix.language == 'cpp'
run: |
mkdir ~/.cache/matplotlib
$CODEQL_PYTHON setup.py build
$CODEQL_PYTHON -m build

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v2
18 changes: 6 additions & 12 deletions .github/workflows/cygwin.yml
Original file line number Diff line number Diff line change
Expand Up @@ -174,8 +174,8 @@ jobs:
- name: Install Python dependencies
shell: bash.exe -eo pipefail -o igncr "{0}"
run: |
python -m pip install --upgrade pip 'setuptools<60' wheel
python -m pip install kiwisolver 'numpy!=1.21.*' pillow importlib_resources
python -m pip install --upgrade pip setuptools wheel
python -m pip install kiwisolver 'numpy>=1.22,<1.26' pillow importlib_resources
grep -v -F -e psutil requirements/testing/all.txt >requirements_test.txt
python -m pip install meson-python pybind11
export PATH="/usr/local/bin:$PATH"
Expand Down Expand Up @@ -203,20 +203,14 @@ jobs:
AUTOCONF: /usr/bin/autoconf-2.69
MAKEFLAGS: dw
run: |
export PATH="/usr/local/bin:$PATH"
ccache -s
git describe
cat <<EOT >> mplsetup.cfg
[rc_options]
backend=Agg

[libs]
system_freetype = False
system_qhull = True
EOT
cat mplsetup.cfg
# All dependencies must have been pre-installed, so that the minver
# constraints are held.
python -m pip install --no-deps -ve .
python -m pip install --no-deps --no-build-isolation --verbose \
--config-settings=setup-args="-DrcParams-backend=Agg" \
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that in the long commit message of e5e0e2f, it was noted that system freetype/qhull were just for quicker testing and should probably have been reverted, so I have not carried that over to here.

--editable .[dev]

- name: Find DLLs to rebase
shell: bash.exe -eo pipefail -o igncr "{0}"
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/mypy-stubtest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ jobs:
run: |
pip3 install -r requirements/testing/mypy.txt \
-r requirements/testing/all.txt
pip3 install -e .
pip3 install .

- name: Set up reviewdog
run: |
Expand All @@ -38,6 +38,7 @@ jobs:
run: |
set -o pipefail
MPLBACKEND=agg python tools/stubtest.py | \
sed -e "s!$pythonLocation/lib/python3.9/site-packages!lib!g" | \
reviewdog \
-efm '%Eerror: %m' \
-efm '%CStub: in file %f:%l' \
Expand Down
29 changes: 7 additions & 22 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,6 @@ jobs:
pyside2-ver: '==5.15.1' # oldest version with working Py3.9 wheel.
pyside6-ver: '==6.0.0'
delete-font-cache: true
no-build-isolation: true
- os: ubuntu-20.04
python-version: 3.9
extra-requirements: '-r requirements/testing/extra.txt'
Expand All @@ -81,7 +80,6 @@ jobs:
python-version: '3.12-dev'
pyside6-ver: '!=6.5.1'
pre: true
no-build-isolation: true
- os: macos-latest
python-version: 3.9
# https://bugreports.qt.io/projects/PYSIDE/issues/PYSIDE-2346
Expand Down Expand Up @@ -131,6 +129,7 @@ jobs:
libxcb-render-util0 \
libxcb-xinerama0 \
lmodern \
ninja-build \
pkg-config \
qtbase5-dev \
texlive-fonts-recommended \
Expand All @@ -150,7 +149,7 @@ jobs:
macOS)
brew install ccache
brew tap homebrew/cask-fonts
brew install font-noto-sans-cjk gobject-introspection gtk4
brew install font-noto-sans-cjk gobject-introspection gtk4 ninja
;;
esac

Expand Down Expand Up @@ -205,17 +204,14 @@ jobs:
fi

# Install dependencies from PyPI.
# Preinstall build requirements to enable no-build-isolation builds.
python -m pip install --upgrade $PRE \
'contourpy>=1.0.1' cycler fonttools kiwisolver importlib_resources \
numpy packaging pillow 'pyparsing!=3.1.0' python-dateutil setuptools-scm \
'meson-python>=0.13.1' 'pybind11>=2.6' \
-r requirements/testing/all.txt \
${{ matrix.extra-requirements }}

# Preinstall pybind11 on no-build-isolation builds.
if [[ "${{ matrix.no-build-isolation }}" == 'true' ]]; then
python -m pip install 'pybind11>=2.6'
fi

# Install optional dependencies from PyPI.
# Sphinx is needed to run sphinxext tests
python -m pip install --upgrade sphinx!=6.1.2
Expand Down Expand Up @@ -300,20 +296,9 @@ jobs:
fi
fi

cat <<EOT >> mplsetup.cfg
[rc_options]
backend=Agg
EOT

cat mplsetup.cfg

if [[ "${{ matrix.no-build-isolation }}" == 'true' ]]; then
# Minimum versions run does not use build isolation so that it
# builds against the pre-installed minver dependencies.
python -m pip install --no-deps --no-build-isolation -ve .
else
python -m pip install --no-deps -ve .
fi
python -m pip install --no-deps --no-build-isolation --verbose \
--config-settings=setup-args="-DrcParams-backend=Agg" \
--editable .[dev]

if [[ "${{ runner.os }}" != 'macOS' ]]; then
unset CPPFLAGS
Expand Down
10 changes: 5 additions & 5 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,10 @@

# Python files #
################
# setup.py working directory
# meson-python working directory
build

# setup.py dist directory
# meson-python/build frontend dist directory
dist
# Egg metadata
*.egg-info
Expand All @@ -41,9 +41,9 @@ dist
pip-wheel-metadata/*
# tox testing tool
.tox
mplsetup.cfg
# generated by setuptools_scm
lib/matplotlib/_version.py
# build subproject files
subprojects/*/
!subprojects/packagefiles/

# OS generated files #
######################
Expand Down
13 changes: 9 additions & 4 deletions azure-pipelines.yml
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,7 @@ stages:
cm-super \
dvipng \
ffmpeg \
fonts-freefont-otf \
fonts-noto-cjk \
fonts-wqy-zenhei \
gdb \
Expand All @@ -105,23 +106,24 @@ stages:
libcairo2 \
libgirepository-1.0-1 \
lmodern \
fonts-freefont-otf \
ninja-build \
poppler-utils \
texlive-pictures \
texlive-fonts-recommended \
texlive-latex-base \
texlive-latex-extra \
texlive-latex-recommended \
texlive-luatex \
texlive-pictures \
texlive-xetex
;;
darwin)
brew install --cask xquartz
brew install pkg-config ffmpeg imagemagick mplayer ccache
brew install ccache ffmpeg imagemagick mplayer ninja pkg-config
brew tap homebrew/cask-fonts
brew install font-noto-sans-cjk-sc
;;
win32)
choco install ninja
;;
*)
exit 1
Expand All @@ -131,12 +133,15 @@ stages:

- bash: |
python -m pip install --upgrade pip
python -m pip install --upgrade meson-python pybind11
python -m pip install -r requirements/testing/all.txt -r requirements/testing/extra.txt ||
[[ "$PYTHON_VERSION" = 'Pre' ]]
displayName: 'Install dependencies with pip'

- bash: |
python -m pip install -ve . ||
python -m pip install \
--no-build-isolation --config-settings=setup-args="--vsenv" \
--verbose --editable .[dev] ||
[[ "$PYTHON_VERSION" = 'Pre' ]]
displayName: "Install self"

Expand Down
46 changes: 46 additions & 0 deletions doc/api/next_api_changes/development/26621-ES.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
Build system ported to Meson
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The build system of Matplotlib has been ported from setuptools to `meson-python
<https://meson-python.readthedocs.io>`_ and `Meson <https://mesonbuild.com>`_.
Consequently, there have been a few changes for development and packaging purposes.

1. Installation by ``pip`` of packages with ``pyproject.toml`` use `build isolation
<https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/#build-isolation>`_
by default, which interferes with editable installation. Thus for developers using
editable installs, it is now necessary to pass the ``--no-build-isolation`` flag to
``pip install``. This means that all build-time requirements must be available in the
environment for an editable install.
2. Build configuration has moved from a custom :file:`mplsetup.cfg` (also configurable
via ``MPLSETUP`` environment variable) to Meson options. These may be specified using
`meson-python's build config settings
<https://meson-python.readthedocs.io/en/stable/how-to-guides/config-settings.html>`_
for ``setup-args``. See :file:`meson_options.txt` for all options. For example, a
:file:`mplsetup.cfg` containing the following::

[rc_options]
backend=Agg

[libs]
system_qhull = True

may be replaced by passing the following arguments to ``pip``::

--config-settings=setup-args="-DrcParams-backend=Agg" \
--config-settings=setup-args="-Dsystem-qhull=true"

Note that you must use ``pip`` >= 23.1 in order to pass more than one setting.
3. Relatedly, Meson's `builtin options <https://mesonbuild.com/Builtin-options.html>`_
are now used instead of custom options, e.g., the LTO option is now ``b_lto``.
4. On Windows, Meson activates a Visual Studio environment automatically. However, it
will not do so if another compiler is available. See `Meson's documentation
<https://mesonbuild.com/Builtin-options.html#details-for-vsenv>`_ if you wish to
change the priority of chosen compilers.
5. Installation of test data was previously controlled by :file:`mplsetup.cfg`, but has
now been moved to Meson's install tags. To install test data, add the ``tests``
tag to the requested install (be sure to include the existing tags as below)::

--config-settings=install-args="--tags=data,python-runtime,runtime,tests"
6. Checking typing stubs with ``stubtest`` does not work easily with editable install.
For the time being, we suggest using a normal (non-editable) install if you wish to
run ``stubtest``.
7 changes: 3 additions & 4 deletions doc/devel/contribute.rst
Original file line number Diff line number Diff line change
Expand Up @@ -266,7 +266,7 @@ A brief overview of the workflow is as follows.

#. Install the local version of Matplotlib with::

python -m pip install -e .
python -m pip install --no-build-isolation --editable .[dev]

See :ref:`installing_for_devs` for detailed instructions.

Expand Down Expand Up @@ -476,9 +476,8 @@ take particular care when adding new API:
New modules and files: installation
-----------------------------------

* If you have added new files or directories, or reorganized existing
ones, make sure the new files are included in the match patterns in
in *package_data* in :file:`setupext.py`.
* If you have added new files or directories, or reorganized existing ones, make sure the
new files are included in the :file:`meson.build` in the corresponding directories.
* New modules *may* be typed inline or using parallel stub file like existing modules.

C/C++ extensions
Expand Down
Loading