Skip to content

docs do not build on py3.7 due to small change in python handling of -m #11838

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
tacaswell opened this issue Aug 10, 2018 · 4 comments · Fixed by #12148
Closed

docs do not build on py3.7 due to small change in python handling of -m #11838

tacaswell opened this issue Aug 10, 2018 · 4 comments · Fixed by #12148
Milestone

Comments

@tacaswell
Copy link
Member

The issue is that the units examples (https://github.com/matplotlib/matplotlib/tree/master/examples/units) import from a local module that provides the unit classes and handlers. However, it turns out that this was working coincidentally due to a sorta-nasty bug where if you start python via python -m some_module (which is how sphinx gets launched internally) '' is put in the path, which ends up not getting expanded so the cwd is always in the path (see https://bugs.python.org/issue33053 and https://docs.python.org/3/whatsnew/3.7.html#other-language-changes).

This means the docs do not build cleanly any more 😞 .

If you run the examples from the cli they work correctly. If you explicitly add the cwd back into the path they also build fine.

Not sure what the best solution here is.

@tacaswell tacaswell added this to the v2.2-doc milestone Aug 10, 2018
@ImportanceOfBeingErnest
Copy link
Member

I do not quite understand how that -m issue affects this. I always thought examples are run through sphinx gallery, which would os.chdir to the directory of the example file. In that case importing from the cwd should work fine, no?

@tacaswell
Copy link
Member Author

Sorry my notes from last night were not super clear.

  • The MakeFile we use invokes the build via
    SPHINXBUILD = python -msphinx
  • previously when using -m the string '' was first in the path which means the cwd of the process is always in the path (even if you change directories)
  • we, via sphinx-gallery, rely on this when we exec the example code to have the local imports work
  • the fix in cpython to the security concern is that the 'cwd' is resolved as the directory you are launching from and that absolute path is put into sys.path. This in turn breaks our imports

ex

(dd36) ✔ /tmp 
13:05 $ python -m test_mod
3.6.6 | packaged by conda-forge | (default, Jul 26 2018, 09:53:17) 
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)]
['', '/home/tcaswell/mc3/envs/dd36/lib/python36.zip', '/home/tcaswell/mc3/envs/dd36/lib/python3.6']
(dd36) ✔ /tmp 
13:09 $ conda activate dd37
(dd37) ✔ /tmp 
13:09 $ python -m test_mod
3.7.0 | packaged by conda-forge | (default, Jul 23 2018, 21:39:36) 
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)]
['/tmp', '/home/tcaswell/mc3/envs/dd37/lib/python37.zip', '/home/tcaswell/mc3/envs/dd37/lib/python3.7']
(dd37) ✔ /tmp 
13:09 $ cat test_mod.py 
import sys
print(sys.version)
print(sys.path[:3])
(dd37) ✔ /tmp 

I think the fix is to in our sphinx conf add if '' not in sys.path: sys.path.append('') (or something to that effect) or to move this logic to sphinx gallery (and have it explicitly add / remove the cwd directory as it moves around the examples).

@ImportanceOfBeingErnest
Copy link
Member

There is already a similar hack present in conf.py

sys.path.append(os.path.abspath('.'))

so that might be the best option for now.

@anntzer
Copy link
Contributor

anntzer commented Aug 10, 2018

From a quick look at it, I'd suggest consolidating all unit examples that import basic_units into a single one together with basic_units itself; having each "this specific plotting method supports units" example in a separate file doesn't seem to be that helpful(?)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants