Windows usetex=True error in long usernames #5923

Grillard · 2016-01-27T12:57:11Z

Hello,

If I turn usetex=True, in rcParams and then try to save a figure using plt.savefig("xxx.eps") I get the error message at the end of the post.

I tracked this down, and the issue is with the temporary file created by tempfile.mkstemp() in the function _print_figure_tex on backend_ps.py. This file, when the username in windows is long, it is shortened to 6 characters + "~1", then when running latex, the ~ is interpreted as a line break, therefore giving the error.

I perform a simple fix on backend_ps.py, which is simply replace the short version of the username for the full name. This completely solves the problem for me:

fd, tmpfile = mkstemp()
#Fix# 
tmplist=tmpfile.split("\\")
tmplist[2]=os.getenv('USERNAME').lower()
tmpfile="\\".join(tmplist)
#####

Do not know how to proceed though in order for this "fix" to be checked/included/discarded...

Thanks!

Now the complete error message ( I changed the visible part of my username to xxxxxx):

import matplotlib.pyplot as plt
plt.rc("text", usetex=True)
plt.savefig("D:/test.eps")
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-3-10b5dc142fe5> in <module>()
----> 1 plt.savefig("D:/test.eps")

c:\python27\lib\site-packages\matplotlib\pyplot.pyc in savefig(*args, **kwargs)
    686 def savefig(*args, **kwargs):
    687     fig = gcf()
--> 688     res = fig.savefig(*args, **kwargs)
    689     fig.canvas.draw_idle()   # need this if 'transparent=True' to reset colors
    690     return res

c:\python27\lib\site-packages\matplotlib\figure.pyc in savefig(self, *args, **kwargs)
   1537             self.set_frameon(frameon)
   1538
-> 1539         self.canvas.print_figure(*args, **kwargs)
   1540
   1541         if frameon:

c:\python27\lib\site-packages\matplotlib\backend_bases.pyc in print_figure(self, filename, dpi, facecolor, edgecolor, orientation, format, **kwargs)
   2228                 orientation=orientation,
   2229                 bbox_inches_restore=_bbox_inches_restore,
-> 2230                 **kwargs)
   2231         finally:
   2232             if bbox_inches and restore_bbox:

c:\python27\lib\site-packages\matplotlib\backends\backend_ps.pyc in print_eps(self, outfile, *args, **kwargs)
    993
    994     def print_eps(self, outfile, *args, **kwargs):
--> 995         return self._print_ps(outfile, 'eps', *args, **kwargs)
    996
    997     def _print_ps(self, outfile, format, *args, **kwargs):

c:\python27\lib\site-packages\matplotlib\backends\backend_ps.pyc in _print_ps(self, outfile, format, *args, **kwargs)
   1017             self._print_figure_tex(outfile, format, imagedpi, facecolor, edgecolor,
   1018                                    orientation, isLandscape, papertype,
-> 1019                                    **kwargs)
   1020         else:
   1021             self._print_figure(outfile, format, imagedpi, facecolor, edgecolor,

c:\python27\lib\site-packages\matplotlib\backends\backend_ps.pyc in _print_figure_tex(self, outfile, format, dpi, facecolor, edgecolor, orientation, isLandscape, papertype, **kwargs)
   1378                                          font_preamble,
   1379                                          custom_preamble, paperWidth, paperHeight,
-> 1380                                          orientation)
   1381
   1382         if rcParams['ps.usedistiller'] == 'ghostscript':

c:\python27\lib\site-packages\matplotlib\backends\backend_ps.pyc in convert_psfrags(tmpfile, psfrags, font_preamble, custom_preamble, paperWidth, paperHeight, orientation)
   1476         if exit_status:
   1477             raise RuntimeError('LaTeX was not able to process your file:\
-> 1478     \nHere is the full report generated by LaTeX: \n\n%s'% fh.read())
   1479         else:
   1480             verbose.report(fh.read(), 'debug')

RuntimeError: LaTeX was not able to process your file:
Here is the full report generated by LaTeX:

This is pdfTeX, Version 3.14159265-2.6-1.40.16 (MiKTeX 2.9)
entering extended mode
! I can't find file `c:/users/xxxxxx'.
<to be read again>
                   \protect
<*> c:/users/xxxxxx~
                    1/appdata/local/temp/tmpykvykd.tex
Please type another input file name
! Emergency stop.
<to be read again>
                   \protect
<*> c:/users/xxxxxx~
                    1/appdata/local/temp/tmpykvykd.tex
No pages of output.
Transcript written on texput.log.

The text was updated successfully, but these errors were encountered:

mdboom · 2016-01-27T14:27:09Z

I suspect we just need to do some sort of filename escaping here -- but not having a Windows box I'm not exactly sure what...

Thoughts, @JanSchulz ?

jankatins · 2016-01-27T15:38:08Z

You mean here:

matplotlib/lib/matplotlib/backends/backend_ps.py

Line 1297 in c32bb0d

fd, tmpfile = mkstemp()

Grillard · 2016-01-27T15:40:52Z

Yep!,
there, sorry did not know how to link a line on a file, now I know! thanks

jankatins · 2016-01-27T15:45:03Z

[lines: click on the line number and the URL will be set to that line :-) ]

I can't reproduce that here:

c:\temp
λ mkdir really_long_dirname_which_should_be_long

c:\temp
λ cd really_long_dirname_which_should_be_long\

c:\temp\really_long_dirname_which_should_be_long
λ set TEMP=%CD%

c:\temp\really_long_dirname_which_should_be_long
λ set TMP=%CD%

c:\temp\really_long_dirname_which_should_be_long
λ set TMPDIR=%CD%

c:\temp\really_long_dirname_which_should_be_long
λ python
Python 2.7.11 |Continuum Analytics, Inc.| (default, Jan 19 2016, 12:08:31) [MSC v.1500 64 bit (AMD64)] on win32

Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import tempfile
>>> tempfile.mkstemp()
(3, 'c:\\temp\\really_long_dirname_which_should_be_long\\tmpflstuc')
>>>

Can you sent me the output you get from the last two lines and also the output of echo %TEMP% x %TMP% x %TMPDIR% in a cmd window:

λ echo %TEMP% x %TMP% x %TMPDIR%
c:\temp\really_long_dirname_which_should_be_long x c:\temp\really_long_dirname_which_should_be_long x c:\temp\really_long_dirname_which_should_be_long

Is supect that one of the env vars is set to the short name:

c:\temp\really_long_dirname_which_should_be_long
λ set TMPDIR=c:\temp\REALLY~1

c:\temp\really_long_dirname_which_should_be_long
λ set TMP=c:\temp\REALLY~1

c:\temp\really_long_dirname_which_should_be_long
λ set TEMP=c:\temp\REALLY~1

c:\temp\really_long_dirname_which_should_be_long
λ python
Python 2.7.11 |Continuum Analytics, Inc.| (default, Jan 19 2016, 12:08:31) [MSC v.1500 64 bit (AMD64)] on win32

Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import tempfile
>>> tempfile.mkstemp()
(3, 'c:\\temp\\really~1\\tmpc0uvcd')
>>>

c:\temp\really_long_dirname_which_should_be_long
λ echo %TEMP% x %TMP% x %TMPDIR%
c:\temp\REALLY~1 x c:\temp\REALLY~1 x c:\temp\REALLY~1

Grillard · 2016-01-27T15:55:27Z

I see,
the output of the system variables contain my username trimmed at 6 characteres plus a "~1"... So the problem is in the system variable rather than mkstemp(). It seems also that I do not have the TMPDIR variable defined...

C:\Users\XXXXX~1\AppData\Local\Temp x C:\Users\XXXXXX~1\AppData\Local\Temp x %TMPDIR%

jankatins · 2016-01-27T16:05:00Z

ok...

Can you tests this (it worked in my plain python only conda 2.7 env):

>>> import win32file, tempfile
>>> win32file.GetLongPathName(tempfile.mkstemp()[1])

https://stackoverflow.com/questions/11420689/how-to-get-long-file-system-path-from-python-on-windows

I've actually no idea which package this file is from.

jankatins · 2016-01-27T16:10:08Z

The other idea would be to escape the ~ in the latex call:

https://tex.stackexchange.com/questions/183568/using-8-3-path-names-with-pdflatex

Grillard · 2016-01-27T16:20:02Z

I did not have win32file library installed, and could not install trivially with pip, I guess because I run 64 bit python...

And about escaping the ~ on the latex, not really sure how to do that, is it a setting for latex call, or a system setting?

jankatins · 2016-01-27T16:32:43Z

I did not have win32file library installed

I also didn't install anything, it just came with condas python. Not sure if it is part of python itself?

[Update: a bit of googling tells me that it is part of pywin32 :-( ]

re the latex call:

pdflatex \input{c:/users/xxxxxx~1/appdata/local/temp/tmpykvykd.tex}

[if the tex file is still there, which I hope it is]

jankatins · 2016-01-27T16:52:37Z

Ok, can you check if this helps:

Open c:\python27\lib\site-packages\matplotlib\backends\backend_ps.py
in line ~1470 replace "%s" > "%s" by \input{%s} > "%s"

    if sys.platform == 'win32': precmd = '%s &&'% os.path.splitdrive(tmpdir)[0]
    else: precmd = ''
-    command = '%s cd "%s" && latex -interaction=nonstopmode "%s" > "%s"'\
+    command = '%s cd "%s" && latex -interaction=nonstopmode \\input{%s} > "%s"'\
                %(precmd, tmpdir, latexfile, outfile)
    verbose.report(command, 'debug')

Grillard · 2016-01-27T17:32:45Z

Hello, I see, I tried the changes, but did not work, I got "undefined control sequence" from latex. not really sure how it works, but apparently it think that \ character is now calling a function. On the same page, I tried this:

command=command.replace("~","\\string~")

which I based on https://tex.stackexchange.com/questions/183568/using-8-3-path-names-with-pdflatex

and works for me!.

jankatins · 2016-01-27T18:07:04Z

Wanna do a PR based on command=command.replace("~","\\string~")? :-)

Grillard · 2016-01-27T18:11:41Z

Ok!,

Let me figure out how that works haha

fix for latex call on PS backend closes #5923

fix for latex call on PS backend closes matplotlib#5923

tacaswell mentioned this issue Jan 30, 2016

fix for latex call on PS backend (Issue #5895) #5928

Merged

tacaswell closed this as completed in 190c923 Feb 1, 2016

tacaswell added a commit that referenced this issue Feb 1, 2016

Merge pull request #5928 from Grillard/master

ab7c982

fix for latex call on PS backend closes #5923

QuLogic added this to the 1.5.2 (Critical bug fix release) milestone Feb 1, 2016

tacaswell added a commit to tacaswell/matplotlib that referenced this issue May 22, 2016

Merge pull request matplotlib#5928 from Grillard/master

92f463c

fix for latex call on PS backend closes matplotlib#5923

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Windows usetex=True error in long usernames #5923

Windows usetex=True error in long usernames #5923

Grillard commented Jan 27, 2016

mdboom commented Jan 27, 2016

jankatins commented Jan 27, 2016

Grillard commented Jan 27, 2016

jankatins commented Jan 27, 2016

Grillard commented Jan 27, 2016

jankatins commented Jan 27, 2016

jankatins commented Jan 27, 2016

Grillard commented Jan 27, 2016

jankatins commented Jan 27, 2016

jankatins commented Jan 27, 2016

Grillard commented Jan 27, 2016

jankatins commented Jan 27, 2016

Grillard commented Jan 27, 2016

Windows usetex=True error in long usernames #5923

Windows usetex=True error in long usernames #5923

Comments

Grillard commented Jan 27, 2016

mdboom commented Jan 27, 2016

jankatins commented Jan 27, 2016

Grillard commented Jan 27, 2016

jankatins commented Jan 27, 2016

Grillard commented Jan 27, 2016

jankatins commented Jan 27, 2016

jankatins commented Jan 27, 2016

Grillard commented Jan 27, 2016

jankatins commented Jan 27, 2016

jankatins commented Jan 27, 2016

Grillard commented Jan 27, 2016

jankatins commented Jan 27, 2016

Grillard commented Jan 27, 2016