import pyplot on MacOS without font cache will search entire subtree of current dir #12176

dsentinel · 2018-09-19T23:03:43Z

Bug report

Bug summary

On import if /Users/lloyd/.matplotlib/fontlist-v300.json does not exist, will scan entire tree of current dir.
Looks like this was a typo, and then a flake commit included "" which pathlib interperetes as .
https://github.com/matplotlib/matplotlib/pull/11963/files#r218979454

Code for reproduction

On a mac installed with conda if ~/.matplotlib/fontlist-v300.json doesn't exist

(mat_bug_py37) moonbase:~ lloyd$ ls -al /Users/lloyd/.matplotlib/fontlist-v300.json
-rw-r--r--  1 lloyd  staff  130027 Sep 19 10:45 /Users/lloyd/.matplotlib/fontlist-v300.json
(mat_bug_py37) moonbase:~ lloyd$ python -c 'from matplotlib import pyplot'
(mat_bug_py37) moonbase:~ lloyd$ rm  /Users/lloyd/.matplotlib/fontlist-v300.json
(mat_bug_py37) moonbase:~ lloyd$ python -c 'from matplotlib import pyplot'

Takes minutes.... ctl-c

^CTraceback (most recent call last):
  File "/Users/lloyd/anaconda2/envs/mat_bug_py37/lib/python3.7/site-packages/matplotlib/font_manager.py", line 1353, in <module>
    fontManager = json_load(_fmcache)
  File "/Users/lloyd/anaconda2/envs/mat_bug_py37/lib/python3.7/site-packages/matplotlib/font_manager.py", line 888, in json_load
    with open(filename, 'r') as fh:
FileNotFoundError: [Errno 2] No such file or directory: '/Users/lloyd/.matplotlib/fontlist-v300.json'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/lloyd/anaconda2/envs/mat_bug_py37/lib/python3.7/site-packages/matplotlib/pyplot.py", line 32, in <module>
    import matplotlib.colorbar
  File "/Users/lloyd/anaconda2/envs/mat_bug_py37/lib/python3.7/site-packages/matplotlib/colorbar.py", line 32, in <module>
    import matplotlib.contour as contour
  File "/Users/lloyd/anaconda2/envs/mat_bug_py37/lib/python3.7/site-packages/matplotlib/contour.py", line 18, in <module>
    import matplotlib.font_manager as font_manager
  File "/Users/lloyd/anaconda2/envs/mat_bug_py37/lib/python3.7/site-packages/matplotlib/font_manager.py", line 1363, in <module>
    _rebuild()
  File "/Users/lloyd/anaconda2/envs/mat_bug_py37/lib/python3.7/site-packages/matplotlib/font_manager.py", line 1344, in _rebuild
    fontManager = FontManager()
  File "/Users/lloyd/anaconda2/envs/mat_bug_py37/lib/python3.7/site-packages/matplotlib/font_manager.py", line 978, in __init__
    ttffiles = findSystemFonts(paths) + findSystemFonts()
  File "/Users/lloyd/anaconda2/envs/mat_bug_py37/lib/python3.7/site-packages/matplotlib/font_manager.py", line 270, in findSystemFonts
    fontfiles.update(OSXInstalledFonts(fontext=fontext))
  File "/Users/lloyd/anaconda2/envs/mat_bug_py37/lib/python3.7/site-packages/matplotlib/font_manager.py", line 218, in OSXInstalledFonts
    for directory in directories
  File "/Users/lloyd/anaconda2/envs/mat_bug_py37/lib/python3.7/site-packages/matplotlib/font_manager.py", line 220, in <listcomp>
    for path in list_fonts(directory, ext)]
  File "/Users/lloyd/anaconda2/envs/mat_bug_py37/lib/python3.7/site-packages/matplotlib/font_manager.py", line 157, in list_fonts
    for path in filter(Path.is_file, Path(directory).glob("**/*.*"))
  File "/Users/lloyd/anaconda2/envs/mat_bug_py37/lib/python3.7/site-packages/matplotlib/font_manager.py", line 156, in <listcomp>
    return [str(path)
  File "/Users/lloyd/anaconda2/envs/mat_bug_py37/lib/python3.7/pathlib.py", line 1344, in is_file
    return S_ISREG(self.stat().st_mode)
  File "/Users/lloyd/anaconda2/envs/mat_bug_py37/lib/python3.7/pathlib.py", line 1140, in stat
    return self._accessor.stat(self)
KeyboardInterrupt
^C

Matplotlib version

Operating system: OSX 10.10.5
Matplotlib version: 3.0.0
Python version: 3.6, and 3.7

The text was updated successfully, but these errors were encountered:

tacaswell · 2018-09-20T02:31:08Z

As I said elsewhere, I think my initial reaction to this was wrong and that rather than fixing a bug, adding the , in was a big change in behavior and the correct fix is to remove the empty string.

While including the cwd makes sense at first glance, it does not because a) the result is cached so the next time your cwd will be different but we will not find those files b) the time it takes to search all the files as reported here c) the other 2 platforms do not do this.

Given that the initial change change came in over 7 years ago I am willing to accept that the original reasons are lost to time and restore the previous behavior.

For now for people on macs the best work around is to make an empty directory, cd there and import matplotlib (to get the cache to build side stepping reading your whole file system). The next time you import matplotlib it should find the cache file and not try to re-build it.

While including the cwd makes sense at first glance, it does not because a) the result is cached so the next time your cwd will be different but we will not find those files b) the time it takes to search all the files is causing prolems c) the other 2 platforms do not do this The comma was introduced to fix what looked like a bug (implicit string concatenation instead of adding the empty string to the list) in matplotlib#11963. Original code come in via 4799341 in 2011. closes matplotlib#12176

Sparen · 2018-09-26T02:06:27Z

For now for people on macs the best work around is to make an empty directory, cd there and import matplotlib (to get the cache to build side stepping reading your whole file system). The next time you import matplotlib it should find the cache file and not try to re-build it.

I am experiencing this bug in a Jupyter notebook I'm using for a university course and doing the exact steps you mentioned (cd to new directory, open Python 3, import matplotlib.pyplot as plt to provide a verbose crash log) did not generate a fontlist-v300.json file. This crash has the unfortunate side effect of preventing large portions of the notebook from functioning.

Can you provide a detailed way to generate the missing file so that importing matplotlib.pyplot no longer crashes? Thank you.

jklymak · 2018-09-26T02:15:12Z

@Sparen, This should not cause a crash; the issue above was a long hang.... If it does cause a crash, please open a new issue with full details. Please try and take jupyter out of the equation if possible (try with a pure python shell) Thanks

Sparen · 2018-09-26T02:18:20Z

@Sparen, This should not cause a crash; the issue above was a long hang.... If it does cause a crash, please open a new issue with full details. Please try and take jupyter out of the equation if possible (try with a pure python shell) Thanks

Running the import directly in the Python 3 shell with the commands I ran above causes a crash. I will open a new issue.

dsentinel mentioned this issue Sep 19, 2018

FIX: maintain return type stability in win32InstalledFonts #12174

Closed

6 tasks

dsentinel referenced this issue Sep 19, 2018

add common font paths to font_manager

4799341

tacaswell mentioned this issue Sep 20, 2018

FIX: remove cwd from mac font path search #12177

Merged

tacaswell added this to the v3.0.x milestone Sep 20, 2018

QuLogic closed this as completed in #12177 Sep 20, 2018

tacaswell mentioned this issue Sep 20, 2018

matplotlib 3 pyplot on MacOS bounces rocket icon in dock #12188

Closed

Sparen mentioned this issue Sep 26, 2018

Importing pyplot crashes on macOS due to missing fontlist-v300.json and then Permission denied: '/opt/local/share/fonts' #12291

Closed

QuLogic mentioned this issue Sep 30, 2018

Too many levels of symbolic links #12354

Closed

jklymak mentioned this issue Oct 18, 2018

Matplotlib 3.0.0 import hangs in clean environment #12556

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

import pyplot on MacOS without font cache will search entire subtree of current dir #12176

import pyplot on MacOS without font cache will search entire subtree of current dir #12176

dsentinel commented Sep 19, 2018 •

edited

Loading

tacaswell commented Sep 20, 2018

Sparen commented Sep 26, 2018

jklymak commented Sep 26, 2018

Sparen commented Sep 26, 2018

import pyplot on MacOS without font cache will search entire subtree of current dir #12176

import pyplot on MacOS without font cache will search entire subtree of current dir #12176

Comments

dsentinel commented Sep 19, 2018 • edited Loading

Bug report

tacaswell commented Sep 20, 2018

Sparen commented Sep 26, 2018

jklymak commented Sep 26, 2018

Sparen commented Sep 26, 2018

dsentinel commented Sep 19, 2018 •

edited

Loading