Skip to content

DOC: Spin docs gives errors (distutils) on new codespaces (because python 3.12 in environment.yml) #29131

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
bmwoodruff opened this issue Jun 5, 2025 · 9 comments · May be fixed by #29160

Comments

@bmwoodruff
Copy link
Member

Issue with current documentation:

With SciPy 2025 coming up, and the possibility of having new people join for sprints, i figured it would be worth pointing out a possible hiccup.

  • When building the docs from a fresh codespaces environment (or new conda environment using environment.yml), the command spin docs finishes with error messages. This are the same numpy.distutils error message discussed in DOC: docs build warnings #28694.

The issue occurs because of the pinned 3.12 python version.

- python=3.12 # need to pin to avoid issues with builds

Idea or request for content:

The issue disappears by manually changing the python version back to 3.11. These steps should error free build the docs on codespaces.

  1. Make a branch
  2. Change 3.12 to 3.11 in environment.yml and push the changes to your branch.
  3. Create a codespaces on GitHub for the new branch.
  4. conda activate numpy-dev
  5. spin test && pip install -r requirements/doc_requirements.txt && spin docs

My hope is this could help anyone new who wants to help but can't figure out why they keep getting errors. Maybe there is a simpler way to help someone new that doesn't involve changing environment.yml.

@bmwoodruff bmwoodruff changed the title DOC: Spin docs gives errors (distutils) on new codespaces (because python 3.12 in environment.yaml) DOC: Spin docs gives errors (distutils) on new codespaces (because python 3.12 in environment.yml) Jun 5, 2025
@charris
Copy link
Member

charris commented Jun 5, 2025

It seems reasonable to pin to the lowest supported numpy version, want to make a PR?

I wonder why it was pinned to 3.12?

@bmwoodruff
Copy link
Member Author

The change was made in 8a890f1, but not sure what issues where happening with builds.

Any thoughts @rgommers on moving it back to 3.11 for the time being at least till after SciPy? I haven't experienced any issues on codespaces, ubuntu 24, nor a virtual machinie of ubuntu on windows after changing it to 3.11, but I haven't tested macos (which seems connected to the commit above).

I can submit a PR to revert the change in environment.yml, but I'm not sure the macos.yml needs reverting, nor am I sure if the two differences will cause issues. I'll wait for more feedback.

@rgommers
Copy link
Member

rgommers commented Jun 6, 2025

but I haven't tested macos (which seems connected to the commit above).

Indeed. Following back the trail from that commit, it leads to these comments about a flaky test and having to move to 3.12 to resolve them: #27550 (comment)

This problem isn't specific to environment.yml but will show up for pretty much any way new contributors will set up their environment, since it's unlikely they'll get Python 3.11 in that environment. Hence, if we want to fix the warnings rather than live with them for another 6 months, it'd be better to do it by starting to remove some references (that has to be done anyway) and gating the rest behind an .. only:: py311 directive (see https://sublime-and-sphinx-guide.readthedocs.io/en/latest/conditions.html#use-conditional-text), with the py311 tag set in doc/Makefile.

WDYT?

@bmwoodruff
Copy link
Member Author

it'd be better to do it by starting to remove some references (that has to be done anyway) and gating the rest behind an .. only:: py311 directive

Sounds like a fun project to work on. Do you have an example of a reference that needs to be removed and one that needs to be gated behind a directive? I have time to go through and update all of them.

@ngoldbaum
Copy link
Member

Sounds like a fun project to work on. Do you have an example of a reference that needs to be removed and one that needs to be gated behind a directive? I have time to go through and update all of them.

I think you'd have to figure that out by repeatedly building the docs in a Python 3.12 or 3.13 environment and adding the only directives until the docs build is clean.

@bmwoodruff
Copy link
Member Author

Here is a progress update.

The main problem is the following:

.. automodule:: numpy.distutils.misc_util
:members:
:undoc-members:
:exclude-members: Configuration

Once this automodule section is removed, then WARNING: Failed to import numpy.distutils.misc_util disappears entirely when building the docs with python 3.12. However, then 4 new warnings appear (labels added to reference later).

reading sources... [100%] user/whatisnumpy
(1) /../numpy/doc/source/reference/index.rst:39: WARNING: toctree contains reference to excluded document 'reference/distutils'
(2) /../numpy/doc/source/reference/module_structure.rst:55: WARNING: toctree contains reference to excluded document 'reference/distutils'
looking for now-outdated files... none found
pickling environment... done
(3) checking consistency... /../numpy/doc/source/reference/distutils/misc_util.rst: WARNING: document isn't included in any toctree
done
preparing documents... done
copying assets... copying static files... done
copying extra files... done
done
writing output... [100%] user/whatisnumpy
(4) /../numpy/doc/source/reference/module_structure.rst:47: WARNING: undefined label: 'numpy-distutils-refguide'
generating indices... genindex done

Here are the corresponding lines of code that generate these warnings.


- :ref:`numpy.distutils <numpy-distutils-refguide>` (deprecated) - build system support

numpy.distutils <distutils>

Warnings 1,2, and 4 above can all be dealt with by removing the corresponding reference/label. I'm still working on how to remove warning 3 above in a way that allows the build to work with both python 3.11 and >=3.12.

@bmwoodruff
Copy link
Member Author

Looks like adding :orphan: to the top of the page may be all that's needed. Hopefully I'll a PR ready soon. Thanks for the suggestions on how to fix the docs build issue.

@bmwoodruff
Copy link
Member Author

A cleaner way of excluding this orphaned file is to update conf.py from

numpy/doc/source/conf.py

Lines 153 to 154 in 75fd1ff

if sys.version_info[:2] >= (3, 12):
exclude_patterns += ["reference/distutils.rst"]

to

if sys.version_info[:2] >= (3, 12):
    exclude_patterns += [
        "reference/distutils.rst",
        "reference/distutils/misc_util.rst",
    ]

This solves the import warning errors and keeps the exclusions in the same place for simple removal later.

Updating the toctree is more complicated. The only directive (and ifconfig) both apply AFTER a toctree is built. I couldn't use the .. only:: py311 directive to stop automodule for the same reason. Content will be excluded using only or ifconfig, but the toctree still looks for all related content before conditionals are applied. As such, using .. only:: py311 doesn't prevent WARNING: toctree contains reference to excluded document 'reference/distutils' from appearing.

After searching for solutions, this is apparently a known issue that some of you have encountered before (see #16372 (comment)). Anyone have any experience with using only to deal with toctrees and references?

Right now I'm playing around with nitpick_ignore. I have an option that builds the docs without warnings, but requires suppress_warnings += ['toc.excluded'] as you cannot use nitpick to supress toctree warnings (or I just don't know how). I'm not fully happy with this solution, and would love to discuss it more.

I'm going to submit a draft PR and we can move the conversation over there.

bmwoodruff added a commit to bmwoodruff/numpy that referenced this issue Jun 9, 2025
Enables an error free build of the docs for python 3.12+.
Excludes related files, supresses all warnings for excluded files,
and ignores case-by-case individual references. Closes numpy#29131.

[skip azp][skip cirrus][skip actions]
@rgommers
Copy link
Member

Thanks for working through the options for avoiding the warnings @bmwoodruff!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants