Skip to content

DOC/BUILD add ability for conf to skip whole sections #24907

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jan 14, 2023

Conversation

jklymak
Copy link
Member

@jklymak jklymak commented Jan 8, 2023

PR Summary

UPDATE: 11 Jan:

The doc build is painfully slow. This gives devs an easy way to populate exclude_patterns in conf.py with subdirectories they do not want while they are doing development, and to not do the gallery builds if those subdirectories are excluded. Of course the resulting build is broken with dropped references etc, so the developer should put conf.py back to its pristine state, but this should be a handy tool.

Note to build just the user subdirectory using this takes 5s on my machine (if I exclude the prev_whats_new).

Closes #24906

Behaviour:

The interface is make html-skip-subdirs which calls $(SPHINXBUILD) -D skip_sub_dirs=1 -b html ....

It tries to read doc/.mpl_skip_subdirs.yaml. If that does not exist, that file is created and looks like:

comment: For use with make html-skip-subdirs
skip_subdirs:
- users/prev_whats_new/*
- api/*
- gallery/*
- tutorials/*
- plot_types/*
- devel/*

If all of gallery/* (etc) is skipped, then sphinx-gallery does not get run on that subdirectory. OTOH if only subdirectories of it are skipped, then the whole gallery is run (only once though, if gallery not previously generated).

With the above default the users/* directory is built in 5 seconds on my machine, in contrast to the full doc build that takes upwards of 10 minutes.

PR Checklist

Documentation and Tests

  • Has pytest style unit tests (and pytest passes)
  • Documentation is sphinx and numpydoc compliant (the docs should build without error).
  • New plotting related features are documented with examples.

Release Notes

  • New features are marked with a .. versionadded:: directive in the docstring and documented in doc/users/next_whats_new/
  • API changes are marked with a .. versionchanged:: directive in the docstring and documented in doc/api/next_api_changes/
  • Release notes conform with instructions in next_whats_new/README.rst or next_api_changes/README.rst

@jklymak jklymak added the Documentation: build building the docs label Jan 8, 2023
@story645
Copy link
Member

story645 commented Jan 8, 2023

if this was gonna go in, any way to pass in the exclusions on the command line?

@jklymak
Copy link
Member Author

jklymak commented Jan 8, 2023

I think via environment variables is the only way and that is pretty ugly. Even command line is pretty ugly with a bunch of excludes.

@rcomer
Copy link
Member

rcomer commented Jan 8, 2023

I think via environment variables is the only way and that is pretty ugly. Even command line is pretty ugly with a bunch of excludes.

I’m wondering whether it would be easy or desirable to do it with some sort of config file that is git-ignored. Then you could set it to whatever you like for local building and not need to revert each time you push.

Downside would be that I guess it would persist when you check out some other branch. I would likely forget about it and get confused about what’s going on with my doc build…

@story645
Copy link
Member

story645 commented Jan 8, 2023

I would likely forget about it and get confused about what’s going on with my doc build…

could add a line to the conf printing excluded folders/files

@rcomer
Copy link
Member

rcomer commented Jan 8, 2023

I would likely forget about it and get confused about what’s going on with my doc build…

could add a line to the conf printing excluded folders/files

Possibly hard to spot amongst all the other output from the doc build?

@story645
Copy link
Member

story645 commented Jan 8, 2023

Possibly hard to spot amongst all the other output from the doc build?

I'd put it at the very top b/f the sphinx generated output, where there really isn't much:

image

also possibly add a space/ "SPHINX OUTPUT BEGINS NOW" header between these two sections.

@jklymak
Copy link
Member Author

jklymak commented Jan 8, 2023

This commit adds a make html-skip-dirs option, and make html goes back to a full build. Still have to edit the conf.py.

It would be pretty easy to add a config file for this, but I hesitate just because I don't quite understand how that actually is git ignored. Either we distribute an example config file, or we don't. If we don't, then adding it to the gitignore is fine, but then users need to make it themselves by hand. Is that what we want?

@rcomer
Copy link
Member

rcomer commented Jan 8, 2023

@jklymak fair point.

It occurs to me that the contributor could also just not stage their conf.py changes. Though that for me would mean having to break the git commit -a habit!

@jklymak
Copy link
Member Author

jklymak commented Jan 8, 2023

I do commit -a all the time as well. But I'd do git restore conf.py to get it back to its original state.

But I guess having the user make a skip-dir.yml or something isn't very hard and could be in the comments of Makefile

eg.

skip_dirs: users/prev_whats_new/*
   - gallery/*
   - api/*
   - tutorials/*
   - plot_types/*
   - devel/*

should work.

@jklymak jklymak force-pushed the doc-conf-with-shortcuts branch 3 times, most recently from 00b53a6 to 47ad4a2 Compare January 9, 2023 02:32
@jklymak
Copy link
Member Author

jklymak commented Jan 9, 2023

OK, latest commit does this via a yaml. make html-skip-subdirs will make doc/.mpl_skip_subdirs.yaml with a default minimal build. The user can add or subtract entries to this (including commenting things out). Or if they remove it, it will get generated again when make html-skip-subdirs is called. This is .gitignore-d

@story645
Copy link
Member

story645 commented Jan 9, 2023

Any chance you have a windows machine to update the make.bat? Otherwise thats a good first issue follow up. I think here it's gonna be basically the same command? )

@jklymak
Copy link
Member Author

jklymak commented Jan 9, 2023

I just pushed a change to make.bat. No idea if it works ;-)

@jklymak jklymak force-pushed the doc-conf-with-shortcuts branch 3 times, most recently from 8b612e5 to bcb08ea Compare January 9, 2023 04:24
@tacaswell tacaswell added this to the v3.8.0 milestone Jan 9, 2023
@jklymak jklymak force-pushed the doc-conf-with-shortcuts branch from 1290285 to de27da7 Compare January 11, 2023 21:18
@jklymak
Copy link
Member Author

jklymak commented Jan 11, 2023

Rebased so all tests should pass.

I know folks are busy with the release, but I'll agitate for this one a bit. a) it's a huge convenience for folks who are just changing a small part of the docs and b) it's also not user-facing, so the interface need not be set in stone. I've updated the proposal in the description above.

@QuLogic
Copy link
Member

QuLogic commented Jan 12, 2023

As a note, one can set something in conf.py from the command line via -D setting=value. However, I believe that doing so would mean that you could not check that value within the file itself, and you would need to move the processing of the value to another extension.

@jklymak jklymak force-pushed the doc-conf-with-shortcuts branch 2 times, most recently from e2b90ac to 7047f32 Compare January 12, 2023 21:45
@story645 story645 force-pushed the doc-conf-with-shortcuts branch 2 times, most recently from 814f9cd to 0896b1f Compare January 13, 2023 00:53
@jklymak jklymak force-pushed the doc-conf-with-shortcuts branch from 0896b1f to 2541f23 Compare January 13, 2023 02:50
@jklymak
Copy link
Member Author

jklymak commented Jan 13, 2023

squashed...

@jklymak
Copy link
Member Author

jklymak commented Jan 14, 2023

I'll self merge this, because I want to use it ;-) and it's not at all user-facing.

@jklymak jklymak merged commit 651a874 into matplotlib:main Jan 14, 2023
@jklymak jklymak deleted the doc-conf-with-shortcuts branch January 14, 2023 04:25
@rcomer
Copy link
Member

rcomer commented Jan 14, 2023

It did have two approvals. Thanks @jklymak I am using this already 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Documentation: build building the docs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[DOC/BUILD] add ability to selectively build docs
6 participants