PyWin32 Post install Error: Missing pywin32_system32 folder in Lib/site-packages #58

mbiskach · 2015-03-06T17:10:36Z

Using the latest commit (5b70c41), I am trying to build a custom winpython installation using Python 3.4.2 32-bit (Windows 7). I received an error that the pywin32_system32 folder does not exist in Lib/site-packages.

At what point should the pywin32_system32 folder be created in Lib/site-packages? I feel like simply checking if the folder exists and creating it if missing is not the correct solution...
The pywin32 wheel was downloaded from Christoph Gohlke's site.

Full output:

Ignoring indexes: https://pypi.python.org/simple
Collecting setuptools
Collecting pip
Installing collected packages: pip, setuptools


Successfully installed pip-6.0.8 setuptools-12.0.5


+-------------------------+
| Making WinPython 32bits |
+-------------------------+

Creating WinPython 3.4 base directory... OK
Extracting Python installer... fname=C:\Users\mbiskach\Documents\code\winpython_new\basedir34\build\wheels_tmp.win32\python-3.4.3.msi
TARGETDIR=C:\Users\mbiskach\Documents\code\winpython_new\basedir34\build\winpython-3.4.3\python-3.4.3
patched C:\Users\mbiskach\Documents\code\winpython_new\basedir34\build\winpython-3.4.3\python-3.4.3\Scripts\easy_install-3.4.exe
patched C:\Users\mbiskach\Documents\code\winpython_new\basedir34\build\winpython-3.4.3\python-3.4.3\Scripts\pip3.4.exe
patched C:\Users\mbiskach\Documents\code\winpython_new\basedir34\build\winpython-3.4.3\python-3.4.3\Scripts\pip3.exe
OK
Checking packages
Adding Microsoft Visual C++ DLLs
Installing required packages
    Installing Wheel setuptools 12.4... wheel install_options ['--upgrade', '--no-deps']
OK
    Installing Wheel pip 6.0.8... wheel install_options ['--upgrade', '--no-deps']
OK
patched C:\Users\mbiskach\Documents\code\winpython_new\basedir34\build\winpython-3.4.3\python-3.4.3\Scripts\easy_install-3.4.exe
patched C:\Users\mbiskach\Documents\code\winpython_new\basedir34\build\winpython-3.4.3\python-3.4.3\Scripts\easy_install.exe
    Installing Wheel pywin32 219... wheel install_options ['--no-index', '--find-links=C:\\Users\\mbiskach\\Documents\\code\\winpython_new\\basedir34\\build\\wheels_tmp.win32']
OK
Traceback (most recent call last):
  File "make.py", line 1157, in <module>
    verbose=False, archis=(32, ))
  File "make.py", line 1149, in make_all
    flavor=flavor)
  File "make.py", line 1122, in make_winpython
    dist.make(remove_existing=remove_existing)
  File "make.py", line 995, in make
    self._install_required_packages()
  File "make.py", line 475, in _install_required_packages
    '%s-([0-9\.]*[a-z\+]*[0-9]?)(.*)(\.exe|\.whl)' % happy_few)
  File "make.py", line 306, in install_package
    install_options=self.install_options)
  File "C:\Users\mbiskach\Documents\code\winpython_new\winpython_git\winpython\wppm.py", line 379, in install
    for name in os.listdir(origin):
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'C:\\Users\\mbiskach\\Documents\\code\\winpython_new\\basedir34\\build\\winpython-3.4.3\\python-3.4.3\\Lib\\site-packages\\pywin32_system32'

The text was updated successfully, but these errors were encountered:

stonebig · 2015-03-06T17:22:08Z

hum,

You may check that you see in your directory basedir34\build\wheels_tmp.win32 a wheel pywin32-219-cp34-none-win32.whl

You can compare what you have with the unzipped directories that you have there (in packages.win32.zip )

https://sourceforge.net/projects/winpython/files/WinPython_Source/Do_It_Yourself/WinPython_2015-02/basedir34/

(most common mistake, in my case, is to download the wrong version of wheel, a python3.3 or a 64bit one, or to download two different versions, and the wrong one is picked)

mbiskach · 2015-03-06T17:40:12Z

I did have pywin32-219-cp-none-win32.whl in basedir32\build\wheels_tmp.win32.

I downloaded packages.src.zip and packages.win32.zip from:
http://sourceforge.net/projects/winpython/files/WinPython_Source/Do_It_Yourself/WinPython_2015-02/basedir34/

I replaced pip, setuptools, and pywin32 wheels with yours just to be safe, but either way I received an error when pip was installed (so it seems to be a pip (un)install issue, not pywin32). Note: I am building winpython for Python 3.4.3 (32-bit).

Installing collected packages: pip
  Found existing installation: pip 6.0.8
    Uninstalling pip-6.0.8:
      Exception:
      Traceback (most recent call last):
        File "C:\Users\mbiskach\Documents\code\winpython_new\basedir34\build\winpython-3.4.3\python-3.4.3\lib\shutil.py", line 523, in move
          os.rename(src, real_dst)
      FileNotFoundError: [WinError 3] The system cannot find the path specified: 'c:\\users\\mbiskach\\documents\\code\\winpython_new\\basedir34\\build\\winpython-3.4.3\\python-3.4.3\\lib\\site-packages\\pip\\_vendor\\requests\\packages\\urllib3\\packages\\ssl_match_hostname\\__pycache__\\__init__.cpython-34.pyc' -> 'C:\\Users\\mbiskach\\AppData\\Local\\Temp\\1\\pip-icnv0unu-uninstall\\users\\mbiskach\\documents\\code\\winpython_new\\basedir34\\build\\winpython-3.4.3\\python-3.4.3\\lib\\site-packages\\pip\\_vendor\\requests\\packages\\urllib3\\packages\\ssl_match_hostname\\__pycache__\\__init__.cpython-34.pyc'

Full verbose output:

Ignoring indexes: https://pypi.python.org/simple
Collecting setuptools
Collecting pip
Installing collected packages: pip, setuptools


Successfully installed pip-6.0.8 setuptools-12.0.5
Processing c:\users\mbiskach\documents\code\winpython_new\basedir34\build\wheels_tmp.win32\setuptools-12.3-py2.py3-none-any.whl
Installing collected packages: setuptools
  Found existing installation: setuptools 12.0.5
    Uninstalling setuptools-12.0.5:
      Successfully uninstalled setuptools-12.0.5

Successfully installed setuptools-12.3
Processing c:\users\mbiskach\documents\code\winpython_new\basedir34\build\wheels_tmp.win32\pip-6.0.8-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 6.0.8
    Uninstalling pip-6.0.8:
      Exception:
      Traceback (most recent call last):
        File "C:\Users\mbiskach\Documents\code\winpython_new\basedir34\build\winpython-3.4.3\python-3.4.3\lib\shutil.py", line 523, in move
          os.rename(src, real_dst)
      FileNotFoundError: [WinError 3] The system cannot find the path specified: 'c:\\users\\mbiskach\\documents\\code\\winpython_new\\basedir34\\build\\winpython-3.4.3\\python-3.4.3\\lib\\site-packages\\pip\\_vendor\\requests\\packages\\urllib3\\packages\\ssl_match_hostname\\__pycache__\\__init__.cpython-34.pyc' -> 'C:\\Users\\mbiskach\\AppData\\Local\\Temp\\1\\pip-icnv0unu-uninstall\\users\\mbiskach\\documents\\code\\winpython_new\\basedir34\\build\\winpython-3.4.3\\python-3.4.3\\lib\\site-packages\\pip\\_vendor\\requests\\packages\\urllib3\\packages\\ssl_match_hostname\\__pycache__\\__init__.cpython-34.pyc'

      During handling of the above exception, another exception occurred:

      Traceback (most recent call last):
        File "C:\Users\mbiskach\Documents\code\winpython_new\basedir34\build\winpython-3.4.3\python-3.4.3\lib\site-packages\pip\basecommand.py", line 232, in main
          status = self.run(options, args)
        File "C:\Users\mbiskach\Documents\code\winpython_new\basedir34\build\winpython-3.4.3\python-3.4.3\lib\site-packages\pip\commands\install.py", line 347, in run
          root=options.root_path,
        File "C:\Users\mbiskach\Documents\code\winpython_new\basedir34\build\winpython-3.4.3\python-3.4.3\lib\site-packages\pip\req\req_set.py", line 543, in install
          requirement.uninstall(auto_confirm=True)
        File "C:\Users\mbiskach\Documents\code\winpython_new\basedir34\build\winpython-3.4.3\python-3.4.3\lib\site-packages\pip\req\req_install.py", line 667, in uninstall
          paths_to_remove.remove(auto_confirm)
        File "C:\Users\mbiskach\Documents\code\winpython_new\basedir34\build\winpython-3.4.3\python-3.4.3\lib\site-packages\pip\req\req_uninstall.py", line 126, in remove
          renames(path, new_path)
        File "C:\Users\mbiskach\Documents\code\winpython_new\basedir34\build\winpython-3.4.3\python-3.4.3\lib\site-packages\pip\utils\__init__.py", line 316, in renames
          shutil.move(old, new)
        File "C:\Users\mbiskach\Documents\code\winpython_new\basedir34\build\winpython-3.4.3\python-3.4.3\lib\shutil.py", line 535, in move
          copy2(src, real_dst)
        File "C:\Users\mbiskach\Documents\code\winpython_new\basedir34\build\winpython-3.4.3\python-3.4.3\lib\shutil.py", line 245, in copy2
          copyfile(src, dst, follow_symlinks=follow_symlinks)
        File "C:\Users\mbiskach\Documents\code\winpython_new\basedir34\build\winpython-3.4.3\python-3.4.3\lib\shutil.py", line 109, in copyfile
          with open(dst, 'wb') as fdst:
      FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\mbiskach\\AppData\\Local\\Temp\\1\\pip-icnv0unu-uninstall\\users\\mbiskach\\documents\\code\\winpython_new\\basedir34\\build\\winpython-3.4.3\\python-3.4.3\\lib\\site-packages\\pip\\_vendor\\requests\\packages\\urllib3\\packages\\ssl_match_hostname\\__pycache__\\__init__.cpython-34.pyc'

C:\Users\mbiskach\Documents\code\winpython_new\basedir34\build\winpython-3.4.3\python-3.4.3\python.exe: No module named pip.__main__; 'pip' is a package and cannot be directly executed


+-------------------------+
| Making WinPython 32bits |
+-------------------------+



+---------------------------------------+
| Creating WinPython 3.4 base directory |
+---------------------------------------+



+-----------------------------+
| Extracting Python installer |
+-----------------------------+

fname=C:\Users\mbiskach\Documents\code\winpython_new\basedir34\build\wheels_tmp.win32\python-3.4.3.msi
TARGETDIR=C:\Users\mbiskach\Documents\code\winpython_new\basedir34\build\winpython-3.4.3\python-3.4.3
patched C:\Users\mbiskach\Documents\code\winpython_new\basedir34\build\winpython-3.4.3\python-3.4.3\Scripts\easy_install-3.4.exe
patched C:\Users\mbiskach\Documents\code\winpython_new\basedir34\build\winpython-3.4.3\python-3.4.3\Scripts\pip3.4.exe
patched C:\Users\mbiskach\Documents\code\winpython_new\basedir34\build\winpython-3.4.3\python-3.4.3\Scripts\pip3.exe
Checking packages
Adding Microsoft Visual C++ DLLs
Installing required packages


+----------------------------------+
| Installing Wheel setuptools 12.3 |
+----------------------------------+

wheel install_options ['--upgrade', '--no-deps']
Installed C:\Users\mbiskach\Documents\code\winpython_new\basedir34\build\wheels_tmp.win32\setuptools-12.3-py2.py3-none-any.whl


+----------------------------+
| Installing Wheel pip 6.0.8 |
+----------------------------+

wheel install_options ['--upgrade', '--no-deps']
Installed C:\Users\mbiskach\Documents\code\winpython_new\basedir34\build\wheels_tmp.win32\pip-6.0.8-py2.py3-none-any.whl
patched C:\Users\mbiskach\Documents\code\winpython_new\basedir34\build\winpython-3.4.3\python-3.4.3\Scripts\easy_install-3.4.exe
patched C:\Users\mbiskach\Documents\code\winpython_new\basedir34\build\winpython-3.4.3\python-3.4.3\Scripts\easy_install.exe


+------------------------------+
| Installing Wheel pywin32 219 |
+------------------------------+

wheel install_options ['--no-index', '--find-links=C:\\Users\\mbiskach\\Documents\\code\\winpython_new\\basedir34\\build\\wheels_tmp.win32']
Installed C:\Users\mbiskach\Documents\code\winpython_new\basedir34\build\wheels_tmp.win32\pywin32-219-cp34-none-win32.whl
Traceback (most recent call last):
  File "make.py", line 1157, in <module>
    verbose=True, archis=(32, ))
  File "make.py", line 1149, in make_all
    flavor=flavor)
  File "make.py", line 1122, in make_winpython
    dist.make(remove_existing=remove_existing)
  File "make.py", line 995, in make
    self._install_required_packages()
  File "make.py", line 475, in _install_required_packages
    '%s-([0-9\.]*[a-z\+]*[0-9]?)(.*)(\.exe|\.whl)' % happy_few)
  File "make.py", line 306, in install_package
    install_options=self.install_options)
  File "C:\Users\mbiskach\Documents\code\winpython_new\winpython_git\winpython\wppm.py", line 379, in install
    for name in os.listdir(origin):
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'C:\\Users\\mbiskach\\Documents\\code\\winpython_new\\basedir34\\build\\winpython-3.4.3\\python-3.4.3\\Lib\\site-packages\\pywin32_system32'

mbiskach · 2015-03-06T17:48:37Z

I can confirm the file exists and the destination folder exists. My guess is perhaps some sort of string length limit for paths somewhere?

I will try to move everything to a root somewhere (like the D:\winpython in the example) which should result in shorter path lengths.

stonebig · 2015-03-06T17:49:23Z

In the past, the \U combination like in '\Users' had been source of issues.
I would suggest you try to build with no \U_whatever directory in your building path

stonebig · 2015-03-06T17:50:15Z

As I try to avoid this historic bug to hit me from the start, by building in 'D:\winpython' it may have surfaced back undetected by me.

Also, to Build myself, I open under a winpython3.4 64bit spyder environnement the make.py file, modify the end of the make.py and click run (pre-removing previous build directory by hand)

Also, in that case, I parametrize Spyder to launch the execution in a dedicated python console

stonebig · 2015-03-06T18:01:12Z

If all the previous indication fails to make you succeed, I start to be out of ideas and may propose broader problems like:

your disk is full,
your antivirus is slow,
... or the procedure forgot an important detail.

stonebig · 2015-03-06T18:12:11Z

Of course, I hope you are not on Windows XP

mbiskach · 2015-03-06T19:24:04Z

XP, psshhh!

Moved base winpython directory to flash drive which addressed three possibilities:

Shorter path lengths
No \u in path
Much slower than my SSD (perhaps addressing the antivirus possibility)

Successfully installed pywin32, build process is ongoing now, yeah!

After I verify completely successful build, I will investigate which of the above 3 above possibilities caused the issue in the first place and let you know. Thanks for the help!

stonebig · 2015-03-06T19:37:20Z

You're welcome.
Let me know the result of your investigations and the work hours it took you to get to the first sucess build.
I you have problem at the nullsoftinstallation binary creation step, you will anyway be able to test your build in basedir34\build\winpython-3.4.3 (and so able to ship it as a '.zip')

stonebig · 2015-03-08T08:18:24Z

Now I fell over the same issue as you... scratching my head
Inquiring bacwkard:

it doesn't install anything in Lib\site-packages
the reason seems that 'pip.exe' isn't created anymore (only 'pip3.exe' and 'pip34.exe')
so package is in fact installing in that situation, so the 'side-effect' message on pywin32
If I remove all pip action on 3.4 (not even a pip package), it goes better again.

Now Inquiring forward, looking why 'pip.exe' is not created

stonebig · 2015-03-08T11:14:57Z

when Python3.4 installer runs ensurepip, it does pre-install the wheels that are in its ensurepip directory:

pip6.0.8
setuptools12.0.5

In the case of pip, it won't install pip.exe shortcut by default, only pip3.exe and pip3.4.exe

As ensurepip was apparently not working in previous python3.4 version, I was re-adding latest pip and setuptools wheels in the process.

Today, apparently, trying to do the same thing creates issues.

==> I don't understand clearly why, why not feb 28th on my PC , and why it would work for you now.

Solution I found at the moment :

remove pip and setuptools wheels wheels from my basedir34\packages.src,
don't include them in make.py for 3.4
patch .exepython launchers after each package installation

make.py patching (around line 480)

    def _install_required_packages(self):
        """Installing required packages"""
        print("Installing required packages")

        # Install First these two packages to support wheel format
        if self.python_version == '3.3':
            self.install_package('get-pip-([0-9\.]*[a-z]*[0-9]?).%s(-py%s)?.exe'
                                 % (self.py_arch, self.python_version))
        if self.python_version == '2.7':
            self.install_package('%s-([0-9\.]*[a-z]*[0-9]?)(.*)(\.exe|\.whl)' %
                             'setuptools', install_options=['--upgrade', '--no-deps'])
            self.install_package('%s-([0-9\.]*[a-z]*[0-9]?)(.*)(\.exe|\.whl)' %
                             'pip', install_options=['--upgrade', '--no-deps'])

wppm.py patching (around line 385):

        if package.name == "get-pip":
            # self.exec_script
            my_script_is = osp.join(self.target, 'Scripts', 'get-pip.py')
            self.install_script(my_script_is, install_options=None)
        # change of method 2014-05-08: 
        # touching pip at installation seems not working anymore
        # so brute force method is applied
        if package.name == "pip" or package.name == "get-pip" or 1==1:
            import glob
            for ffname in glob.glob(r'%s\Scripts\*.exe' % self.target):
                utils.patch_shebang_line(ffname)

        if package.name == "pip" or package.name == "get-pip":
            utils.patch_sourcefile(
              self.target + (
              r"\Lib\site-packages\pip\_vendor\distlib\scripts.py"),
              " executable = get_executable()",
              " executable = os.path.join(os.path.basename(get_executable()))")

stonebig · 2015-03-08T15:37:05Z

pushed to https://github.com/stonebig/winpython

stonebig · 2015-03-15T16:40:16Z

I think i found the root issue: a problem in WPPM + wheel.
WPPM should be reworked entirely, painfull work

stonebig · 2015-03-15T22:02:58Z

Solved on my pc, WPPM was relying on pip to install wheels, but not for their removal.

stonebig · 2015-03-16T21:37:40Z

last commit of #65 seems to solve this random issue (one day there, one day not).

stonebig mentioned this issue Mar 8, 2015

release 2015-03 follow-up #56

Closed

24 tasks

mbiskach closed this as completed Mar 13, 2015

stonebig mentioned this issue Mar 28, 2015

release 2015-04 follow-up #73

Closed

32 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

PyWin32 Post install Error: Missing pywin32_system32 folder in Lib/site-packages #58

PyWin32 Post install Error: Missing pywin32_system32 folder in Lib/site-packages #58

mbiskach commented Mar 6, 2015

stonebig commented Mar 6, 2015

Uh oh!

mbiskach commented Mar 6, 2015

Uh oh!

mbiskach commented Mar 6, 2015

Uh oh!

stonebig commented Mar 6, 2015

Uh oh!

stonebig commented Mar 6, 2015

Uh oh!

stonebig commented Mar 6, 2015

Uh oh!

stonebig commented Mar 6, 2015

Uh oh!

mbiskach commented Mar 6, 2015

Uh oh!

stonebig commented Mar 6, 2015

Uh oh!

stonebig commented Mar 8, 2015

Uh oh!

stonebig commented Mar 8, 2015

Uh oh!

stonebig commented Mar 8, 2015

Uh oh!

stonebig commented Mar 15, 2015

Uh oh!

stonebig commented Mar 15, 2015

Uh oh!

stonebig commented Mar 16, 2015

Uh oh!

PyWin32 Post install Error: Missing pywin32_system32 folder in Lib/site-packages #58

PyWin32 Post install Error: Missing pywin32_system32 folder in Lib/site-packages #58

Comments

mbiskach commented Mar 6, 2015

stonebig commented Mar 6, 2015

Uh oh!

mbiskach commented Mar 6, 2015

Uh oh!

mbiskach commented Mar 6, 2015

Uh oh!

stonebig commented Mar 6, 2015

Uh oh!

stonebig commented Mar 6, 2015

Uh oh!

stonebig commented Mar 6, 2015

Uh oh!

stonebig commented Mar 6, 2015

Uh oh!

mbiskach commented Mar 6, 2015

Uh oh!

stonebig commented Mar 6, 2015

Uh oh!

stonebig commented Mar 8, 2015

Uh oh!

stonebig commented Mar 8, 2015

Uh oh!

stonebig commented Mar 8, 2015

Uh oh!

stonebig commented Mar 15, 2015

Uh oh!

stonebig commented Mar 15, 2015

Uh oh!

stonebig commented Mar 16, 2015

Uh oh!