Skip to content

[MRG] MAINT moving to modern way of building docs. #9402

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 11 commits into from
75 changes: 63 additions & 12 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,35 @@ apt-run: &apt-install
libgeos-dev \
otf-freefont

conda36-run: &conda36-install
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why switch to conda? It looks like all we are using is python and then pip installing everything else.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I switch to conda only for the environment: we now need to access the scripts "sphinx-build", and not only the library as before. When installing with --user, we either need to change the PYTHONPATH manually, or use conda.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just use python -msphinx instead? AFAIK sphinx-build is a strict synonym to it, except that python -msphinx looks up PYTHONPATH (which is the same as what we did before in make.py) whereas sphinx-build uses PATH (which may be set differently).

This relates to my earlier point about using the shorter/newer makefile generated by recent versions of sphinx.

name: Install conda
command: |
wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh \
-O miniconda.sh
chmod +x miniconda.sh && ./miniconda.sh -b -p $HOME/miniconda
export PATH="$HOME/miniconda/bin:$PATH"
echo $PATH
conda update --yes --quiet conda

# Configure the conda environment and put it in the path using the
# provided versions
conda create -n testenv --yes --quiet python
source activate testenv

conda27-run: &conda27-install
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All of these are templated so you don't need to write out two copies for Python 3 and 2. There should not be a separate 36 and 27 template. You can use an environment variable to differentiate if necessary (see the NUMPY variable for example).

name: Install conda
command: |
wget https://repo.continuum.io/miniconda/Miniconda-latest-Linux-x86_64.sh \
-O miniconda.sh
chmod +x miniconda.sh && ./miniconda.sh -b -p $HOME/miniconda
export PATH="$HOME/miniconda/bin:$PATH"
conda update --yes --quiet conda

# Configure the conda environment and put it in the path using the
# provided versions
conda create -n testenv --yes --quiet python
source activate testenv

fonts-run: &fonts-install
name: Install custom fonts
# We manually install Humor-Sans using the package from Ubuntu 14.10.
Expand All @@ -52,23 +81,40 @@ pip-run: &pip-install
# Upgrade pip and setuptools and wheel to get as clean an install as possible
name: Upgrade pip, setuptools, wheel
command: |
pip install --upgrade --user pip
pip install --upgrade --user wheel
pip install --upgrade --user setuptools
export PATH="$HOME/miniconda/bin:$PATH"
source activate testenv
pip install --upgrade pip
pip install --upgrade wheel
pip install --upgrade setuptools

deps-run: &deps-install
name: Install Python dependencies
command: |
pip install --user python-dateutil numpy${NUMPY_VERSION} pyparsing!=2.1.6 cycler codecov coverage sphinx pillow
pip install --user -r doc-requirements.txt
export PATH="$HOME/miniconda/bin:$PATH"
source activate testenv
pip install python-dateutil numpy${NUMPY_VERSION} pyparsing!=2.1.6 cycler codecov coverage
pip install -r doc-requirements.txt

mpl-run: &mpl-install
name: Install Matplotlib
command: pip install --user -ve .
command: |
export PATH="$HOME/miniconda/bin:$PATH"
source activate testenv
pip install -ve .
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do the docs build with a non-editable installation?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They should. I can switch to pip install -v . if you prefer.


linkchecker-run: &linkchecker-install
name: Install linkchecker
command: |
export PATH="$HOME/miniconda/bin:$PATH"
source activate testenv
pip install $PRE requests==2.9.2 linkchecker

doc-run: &doc-build
name: Build documentation
command: python make.py html
command: |
export PATH="$HOME/miniconda/bin:$PATH"
source activate testenv
make
working_directory: doc

doc-bundle-run: &doc-bundle
Expand All @@ -82,13 +128,14 @@ doc-bundle-run: &doc-bundle
#

jobs:
docs-python35:
docs-python36:
docker:
- image: circleci/python:3.5
- image: circleci/python:3.6
steps:
- checkout

- run: *apt-install
- run: *conda36-install
- run: *fonts-install
- run: *pip-install

Expand Down Expand Up @@ -122,6 +169,7 @@ jobs:
- checkout

- run: *apt-install
- run: *conda27-install
- run: *fonts-install
- run: *pip-install

Expand All @@ -131,7 +179,7 @@ jobs:
NUMPY_VERSION: "==1.7.1"
# Linkchecker only works with python 2.7 for the time being.
# Linkchecker is currently broken with requests 2.10.0 so force an earlier version.
- run: pip install --user $PRE requests==2.9.2 linkchecker
- run: *linkchecker-install
- run: *mpl-install

- run: *doc-build
Expand All @@ -142,7 +190,10 @@ jobs:
# Linkchecker only works with python 2.7 for the time being
- run:
name: linkchecker
command: ~/.local/bin/linkchecker build/html/index.html
command: |
export PATH="$HOME/miniconda/bin:$PATH"
source activate testenv
linkchecker build/html/index.html
working_directory: doc

- run: *doc-bundle
Expand All @@ -165,5 +216,5 @@ workflows:
version: 2
build:
jobs:
- docs-python35
- docs-python36
- docs-python27
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,6 @@ doc/tutorials
doc/modules
doc/pyplots/tex_demo.png
doc/users/installing.rst
doc/_static/depsy_badge.svg
doc/_static/matplotlibrc
lib/dateutil
examples/*/*.pdf
Expand Down
4 changes: 2 additions & 2 deletions doc-requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,10 @@
# Install the documentation requirements with:
# pip install -r doc-requirements.txt
#
sphinx>=1.3,!=1.5.0
sphinx>=1.3,!=1.5.0,!=1.6.4
colorspacious
ipython
mock
numpydoc
pillow
sphinx-gallery>=0.1.12
sphinx-gallery
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why drop the pinning on this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was a mistake… putting it back.

121 changes: 121 additions & 0 deletions doc/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
# Makefile for Sphinx documentation
#

# You can set these variables from the command line.
SPHINXOPTS =
SPHINXBUILD ?= sphinx-build
PAPER =
BUILDDIR = build

# Internal variables.
PAPEROPT_a4 = -D latex_paper_size=a4
PAPEROPT_letter = -D latex_paper_size=letter
ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .

.PHONY: help clean html dirhtml pickle json latex latexpdf changes linkcheck doctest optipng

all: html

help:
@echo "Please use \`make <target>' where <target> is one of"
@echo " html to make standalone HTML files"
@echo " dirhtml to make HTML files named index.html in directories"
@echo " pickle to make pickle files"
@echo " json to make JSON files"
@echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
@echo " latexpdf to make LaTeX files and run them through pdflatex"
@echo " changes to make an overview of all changed/added/deprecated items"
@echo " linkcheck to check all external links for integrity"
@echo " doctest to run all doctests embedded in the documentation (if enabled)"

clean:
-rm -rf $(BUILDDIR)/*
-rm -rf gallery/
-rm -rf tutorials/
-rm -rf api/_as_gen/
-rm -rf generated/*
-rm -rf modules/generated/*
-rm -rf _static/matplotlibrc
-rm -rf _templates/gallery.html
-rm -rf users/installing.rst

html: prepare
# These two lines make the build a bit more lengthy, and the
# the embedding of images more robust
rm -rf $(BUILDDIR)/html/_images
#rm -rf _build/doctrees/
$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) -W $(BUILDDIR)/html
@echo
@echo "Build finished. The HTML pages are in $(BUILDDIR)/html"


html-allow-warnings: prepare
# These two lines make the build a bit more lengthy, and the
# the embedding of images more robust
rm -rf $(BUILDDIR)/html/_images
#rm -rf _build/doctrees/
$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
@echo
@echo "Build finished. The HTML pages are in $(BUILDDIR)/html"


html-noplot: prepare
$(SPHINXBUILD) -D plot_gallery=0 -b html -W $(ALLSPHINXOPTS) $(BUILDDIR)/html
@echo
@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."

dirhtml: prepare
$(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml
@echo
@echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml."

pickle: prepare
$(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle
@echo
@echo "Build finished; now you can process the pickle files."

json: prepare
$(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json
@echo
@echo "Build finished; now you can process the JSON files."

latex: prepare
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo
@echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex."
@echo "Run \`make' in that directory to run these through (pdf)latex" \
"(use \`make latexpdf' here to do that automatically)."

latexpdf: prepare
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo "Running LaTeX files through pdflatex..."
make -C $(BUILDDIR)/latex all-pdf
@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."

changes: prepare
$(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes
@echo
@echo "The overview file is in $(BUILDDIR)/changes."

linkcheck: prepare
$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck
@echo
@echo "Link check complete; look for any errors in the above output " \
"or in $(BUILDDIR)/linkcheck/output.txt."

doctest: prepare
$(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest
@echo "Testing of doctests in the sources finished, look at the " \
"results in $(BUILDDIR)/doctest/output.txt."

prepare: _static/matplotlibrc users/installing.rst

.SECONDARY:

#############################################
# Matplotlib needs to copy paste stuff around
_static/matplotlibrc: ../lib/matplotlib/mpl-data/matplotlibrc
cp $< $@

users/installing.rst: ../INSTALL.rst
cp $< $@
10 changes: 5 additions & 5 deletions doc/README.txt
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ or all of them via conda and pip::
sphinx-gallery
pip install colorspacious

To build the HTML documentation, type ``python make.py html`` in this
To build the HTML documentation, type ``make html`` in this
directory. The top file of the results will be ./build/html/index.html

**Note that Sphinx uses the installed version of the package to build the
Expand All @@ -34,9 +34,9 @@ generated.

You can build the documentation with several options:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

options -> targets


* `--small` saves figures in low resolution.
* `--allowsphinxwarnings`: Don't turn Sphinx warnings into errors.
* `-n N` enables parallel build of the documentation using N process.
* `make html-noplot` doesn't save the gallery's images. Allows for fast build.
* `make html-allow-warnings`: Don't turn Sphinx warnings into errors.
* `make -j N` enables parallel build of the documentation using N process.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry if this a silly question, but what in the makefile deals with this? It still seems to only use one processor when running locally....

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd guess the correct invocation is make SPHINXOPTS=-j$N ...


Organization
-------------
Expand All @@ -57,7 +57,7 @@ python documentation system built on top of ReST. This directory contains
* mpl_toolkits - documentation of individual toolkits that ship with
Matplotlib

* make.py - the build script to build the html or PDF docs
* Makefile - the build script to build the html or PDF docs

* index.rst - the top level include document for Matplotlib docs

Expand Down
6 changes: 0 additions & 6 deletions doc/_templates/badgesidebar.html
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,6 @@

<br/>

<a href="http://depsy.org/package/python/matplotlib">
<img src="{{ pathto('_static/depsy_badge.svg', 1) }}">
</a>

<br/>

Travis-CI: <a href="https://travis-ci.org/matplotlib/matplotlib">
<img src="https://travis-ci.org/matplotlib/matplotlib.svg?branch=master"/>
</a>
Expand Down
24 changes: 10 additions & 14 deletions doc/devel/documenting_mpl.rst
Original file line number Diff line number Diff line change
Expand Up @@ -54,23 +54,19 @@ The documentation sources are found in the :file:`doc/` directory in
the trunk. To build the users guide in html format, cd into
:file:`doc/` and do::

python make.py html
make html

or::
There are many other targets you can pass to ``make``, and you can see the
full list inside that file. Here are some useful ones:

./make.py html

There are many other flags you can pass to ``make.py``, and you can see the
full list inside that file. Here are two useful ones:

* ``clean`` will delete the built Sphinx files. Use this command if you're getting strange
* ``make html-noplots`` will build the documentation without writing the gallery
images. This is much faster than ``make html``.
* ``make clean`` will delete the built Sphinx files. Use this command if you're getting strange
errors about missing paths or broken links, particularly if you move files around.
* ``latex`` builds a PDF of the documentation.

In addition, these are useful flags:

* ``--help`` will (among other things) display the allowed commands for ``make.py``.
* ``--allowsphinxwarnings`` will allow the docs to continue building even if Sphinx
* ``make latex`` builds a PDF of the documentation.
* ``make help`` will (among other things) display the allowed commands for
``make``.
* ``make-allow-warnings`` will allow the docs to continue building even if Sphinx
throws a warning. This is useful for debugging and spot-checking many warnings
at once.

Expand Down
4 changes: 2 additions & 2 deletions doc/devel/release_guide.rst
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ temporarily comment out the include and toctree glob; re-instate these after a
release. Finally, make sure that the docs build cleanly ::

pushd doc
python make.py html latex -n 16
make html latex
popd

After the docs are built, check that all of the links, internal and external, are still
Expand Down Expand Up @@ -214,7 +214,7 @@ build the docs from the ``ver-doc`` branch. An easy way to arrange this is::
git checkout v2.0.0-doc
git clean -xfd
cd doc
python make.py html latex -n 16
make html latex

which will build both the html and pdf version of the documentation.

Expand Down
Loading