utf8 mode seems not working while loading site module with Python 3.11/3.12 in Windows #117793
Description
Bug report
Bug description:
I installed python 3.12.2 with rye 0.32.0, and it created a pth file that contains non-ascii file path.
$ cat .venv/Lib/site-packages/_tesuto.pth
C:\Users\XXXXXXXXX\Documents\tmp\テスト\src
Processing this file causes a UnicodeDecodeError and python prompt fails to open. (#87380)
I tried running python in utf-8 mode via $ python -X utf8 and PYTHONUTF8=1 environment variable, but not worked.
(not also with 3.11.8)
Error message is like following:
$ python -X utf8
Fatal Python error: init_import_site: Failed to import the site module
Python runtime state: initialized
Traceback (most recent call last):
File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
File "<frozen importlib._bootstrap>", line 1176, in exec_module
File "<frozen site>", line 624, in <module>
File "<frozen site>", line 607, in main
File "<frozen site>", line 539, in venv
File "<frozen site>", line 394, in addsitepackages
File "<frozen site>", line 236, in addsitedir
File "<frozen site>", line 188, in addpackage
UnicodeDecodeError: 'cp932' codec can't decode byte 0x86 in position 35: illegal multibyte sequence
related part in messages from -v option:
Processing global site-packages
Adding directory: 'C:\\Users\\XXXXXXXX\\Documents\\tmp\\テスト\\.venv'
Adding directory: 'C:\\Users\\XXXXXXXX\\Documents\\tmp\\テスト\\.venv\\Lib\\site-packages'
Processing .pth file: 'C:\\Users\\XXXXXXXX\\Documents\\tmp\\テスト\\.venv\\Lib\\site-packages\\_tesuto.pth'
# C:\Users\XXXXXXXX\.rye\py\cpython@3.12.2\Lib\encodings\__pycache__\cp932.cpython-312.pyc matches C:\Users\XXXXXXXX\.rye\py\cpython@3.12.2\Lib\encodings\cp932.py
tried but not worked
- installing python via pyenv
worked
- using python 3.10 or earlier
- changing the encoding of the pth file to cp932
CPython versions tested on:
3.12
Operating systems tested on:
Windows