Closed
Description
Environment
- Pythonnet version: 3.0.0-dev (latest master on 18/2/2021)
- Python version: 3.8.5
- Operating System: Ubuntu 20.04
- .NET Runtime: .NET 5.0.2
Details
- Describe what you were trying to get done.
I am trying to use the latestpythonnetin a virtual environment. If I call my simple demo script usingpythoncommand in PATH, pythonnet.load() crashes. It is the same with any other PATH-basedpythonXcommand. Using a full path works fine. See the details and the analysis below.
Preparation
# Create virtual env.
python3 -m virtualenv venv
. venv/bin/activate
# Install the latest pythonnet
git clone https://github.com/pythonnet/pythonnet.git
pip install ./pythonnet/
# Get .NET 5 runtime
wget https://download.visualstudio.microsoft.com/download/pr/824175f2-5577-46ec-a675-95d2f652575f/07dafc2398e669afa7d25f2d47398c36/dotnet-runtime-5.0.2-linux-x64.tar.gz
mkdir dotnet_5.0.2
tar xfzv dotnet-runtime-5.0.2-linux-x64.tar.gz -C dotnet_5.0.2/
Runtime config (cat runtimeconfig.json)
{
"runtimeOptions": {
"tfm": "net5.0",
"framework": {
"name": "Microsoft.NETCore.App",
"version": "5.0.2"
}
}
}
Python script (cat bug.py):
import os
import clr_loader
import pythonnet
runtime_config = os.path.join(os.path.dirname(__file__), "runtimeconfig.json")
dotnet_root = os.path.join(os.path.dirname(__file__), "dotnet_5.0.2")
runtime = clr_loader.get_coreclr(runtime_config=runtime_config, dotnet_root=dotnet_root)
pythonnet.set_runtime(runtime)
pythonnet.load()
import System
from System import String
s = String("Hello World from Python.NET for .NET 5!")
System.Console.WriteLine(s)- What commands did you run to trigger this issue? If you can provide a
Minimal, Complete, and Verifiable example
this will help us understand the issue.
> which python
/home/takacs/src/pythonnet/venv/bin/python
>`which python` bug.py
Hello World from Python.NET for .NET 5!
> python bug.py
Traceback (most recent call last):
File "bug.py", line 12, in <module>
pythonnet.load()
File "/home/takacs/src/pythonnet/venv/lib/python3.8/site-packages/pythonnet/__init__.py", line 50, in load
_FFI.dlopen(libpython, posix.RTLD_NODELETE | posix.RTLD_LOCAL)
File "/home/takacs/src/pythonnet/venv/lib/python3.8/site-packages/cffi/api.py", line 150, in dlopen
lib, function_cache = _make_ffi_library(self, name, flags)
File "/home/takacs/src/pythonnet/venv/lib/python3.8/site-packages/cffi/api.py", line 832, in _make_ffi_library
backendlib = _load_backend_lib(backend, libname, flags)
File "/home/takacs/src/pythonnet/venv/lib/python3.8/site-packages/cffi/api.py", line 827, in _load_backend_lib
raise OSError(msg)
OSError: cannot load library '/home/takacs/src/pythonnet/python': /home/takacs/src/pythonnet/python: cannot open shared object file: No such file or directory. Additionally, ctypes.util.find_library() did not manage to locate a library called '/home/takacs/src/pythonnet/python'
- If there was a crash, please include the traceback here.
Traceback (most recent call last):
File "bug.py", line 12, in <module>
pythonnet.load()
File "/home/takacs/src/pythonnet/venv/lib/python3.8/site-packages/pythonnet/__init__.py", line 50, in load
_FFI.dlopen(libpython, posix.RTLD_NODELETE | posix.RTLD_LOCAL)
File "/home/takacs/src/pythonnet/venv/lib/python3.8/site-packages/cffi/api.py", line 150, in dlopen
lib, function_cache = _make_ffi_library(self, name, flags)
File "/home/takacs/src/pythonnet/venv/lib/python3.8/site-packages/cffi/api.py", line 832, in _make_ffi_library
backendlib = _load_backend_lib(backend, libname, flags)
File "/home/takacs/src/pythonnet/venv/lib/python3.8/site-packages/cffi/api.py", line 827, in _load_backend_lib
raise OSError(msg)
OSError: cannot load library '/home/takacs/src/pythonnet/python': /home/takacs/src/pythonnet/python: cannot open shared object file: No such file or directory. Additionally, ctypes.util.find_library() did not manage to locate a library called
Analysis
This issue is in the pythonnet.find_libpython module. The dlinfo.dli_fname string contains the actual command that was used to start the Python interpreter. In the case of pure python command, os.path.realpath fails the properly translate this back to the real executable.
Adding a few debug lines properly demonstrate this:
def _linked_libpython_unix():
libdl = ctypes.CDLL(ctypes.util.find_library("dl"))
libdl.dladdr.argtypes = [ctypes.c_void_p, ctypes.POINTER(Dl_info)]
libdl.dladdr.restype = ctypes.c_int
dlinfo = Dl_info()
retcode = libdl.dladdr(
ctypes.cast(ctypes.pythonapi.Py_GetVersion, ctypes.c_void_p),
ctypes.pointer(dlinfo))
if retcode == 0: # means error
return None
print(f"DEBUG dlinfo.dli_fname.decode(): {dlinfo.dli_fname.decode()}")
path = os.path.realpath(dlinfo.dli_fname.decode())
print(f"DEBUG path: {path}")
if path == os.path.realpath(sys.executable):
return None
return path> ./venv/bin/python bug.py
DEBUG dlinfo.dli_fname.decode(): ./venv/bin/python
DEBUG path: /usr/bin/python3.8
Hello World from Python.NET for .NET 5!
> ../pythonnet/venv/bin/python bug.py
DEBUG dlinfo.dli_fname.decode(): ../pythonnet/venv/bin/python
DEBUG path: /usr/bin/python3.8
Hello World from Python.NET for .NET 5!
>python bug.py
DEBUG dlinfo.dli_fname.decode(): python
DEBUG path: /home/takacs/src/pythonnet/python
Traceback (most recent call last):
...
The last case demonstrates that os.path.realpath just extends python with the current path which is obviously incorrect.
As a result, pythonnet.load() method's libpython variable will point to a library which cannot be loaded (
pythonnet/pythonnet/__init__.py
Line 50 in 0f5e781
However, if this part is disabled, the application will crash later in the .NET
Python.Runtime for the same reason.Metadata
Metadata
Assignees
Labels
No labels