Skip to content

Incorrect installation of lib-dynload for custom builds on openSUSE #131425

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
intel-hm opened this issue Mar 18, 2025 · 16 comments
Open

Incorrect installation of lib-dynload for custom builds on openSUSE #131425

intel-hm opened this issue Mar 18, 2025 · 16 comments
Labels
OS-linux topic-installation type-bug An unexpected behavior, bug, or error

Comments

@intel-hm
Copy link

intel-hm commented Mar 18, 2025

Bug report

Bug description:

from Python-3.13.2.tar.xz

./configure --prefix=$RUN/python/3.13.2
make -j 16
make -j 4 install

dynamic libraries are installed to $RUN/3.13.2/lib64/python3.13/lib-dynload
but internally sys.path uses $RUN/3.13.2/lib/python3.13/lib-dynload

setting a link resolves the

errors that show up:

> python3
**Could not find platform dependent libraries <exec_prefix>**
Python 3.13.2 (main, Mar 18 2025, 10:25:05) [GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
**warning: can't use pyrepl: No module named '_opcode'**

pip3 download pip wheel setuptools 
...
    from _posixsubprocess import fork_exec as _fork_exec
**ModuleNotFoundError: No module named '_posixsubprocess'**

CPython versions tested on:

3.13

Operating systems tested on:

Linux

@intel-hm intel-hm added the type-bug An unexpected behavior, bug, or error label Mar 18, 2025
@picnixz
Copy link
Member

picnixz commented Mar 18, 2025

As an openSUSE user, I had this problem as well. I think I did a combination of --exec-prefix=... and altinstall. I can't remember exactly the command and I won't be back until Sunday. I can check at that time.

@picnixz picnixz changed the title installation form source on "openSUSE Leap 15.5" and "SUSE Linux Enterprise Server 15 SP6" - incorrect location of python3.13/lib-dynload Incorrect installation of lib-dynload for custom builds on openSUSE Mar 18, 2025
@tfpf
Copy link

tfpf commented Mar 22, 2025

This was previously reported and discussed in #108819. There are workarounds suggested in the comments on that issue, but no comments from any maintainers.

@picnixz
Copy link
Member

picnixz commented Mar 22, 2025

I think I actually also specified the libdir explicitly so that it doesn't use lib64 but lib instead. I'll see if I can write better installation instructions for openSUSE

@intel-hm
Copy link
Author

I think that's an error in the configure script. And it's not only OPENSUSE - it's also SLES

@picnixz
Copy link
Member

picnixz commented Mar 22, 2025

Yes I think so as well but I'll need to confirm it properly. I'll also tag those responsible for openSUSE contact

@picnixz
Copy link
Member

picnixz commented Apr 28, 2025

@mcepl From a configure PoV, what's the best? detect that we're on openSUSE or SLES or do something else?

@intel-hm
Copy link
Author

intel-hm commented Apr 28, 2025

could someone figure out what causes this difference?
dynamic libraries are installed to $RUN/3.13.2/lib64/python3.13/lib-dynload
but internally sys.path uses $RUN/3.13.2/lib/python3.13/lib-dynload

how are those 2 paths defined in configure? what is different on SUSE that causes the difference?

@tfpf
Copy link

tfpf commented Apr 28, 2025

could someone figure out what causes this difference? dynamic libraries are installed to $RUN/3.13.2/lib64/python3.13/lib-dynload but internally sys.path uses $RUN/3.13.2/lib/python3.13/lib-dynload

how are those 2 paths defined in configure? what is different on SUSE that causes the difference?

I believe it’s because the configure script doesn’t handle the possibility that CONFIG_SITE may be set. I’m pasting a part of my analysis from #108819.

Since CONFIG_SITE is set to /usr/share/site/x86_64-pc-linux-gnu, said file is sourced (this has the effect of setting libdir to '${exec_prefix}/lib64'), and the makefile generated contains, among other things:

LIBDIR=		${exec_prefix}/lib64
PLATLIBDIR=	lib

which I think is the reason lib-dynload went in the wrong location:

BINLIBDEST=	$(LIBDIR)/python$(VERSION)
DESTSHARED=	$(BINLIBDEST)/lib-dynload

while the remaining files, which depend on PLATLIBDIR (which is hard-coded in configure.ac) went in the expected location.

@FFY00
Copy link
Member

FFY00 commented Apr 29, 2025

I opened GH-133163, which might fix this issue. @intel-hm, if you could test it, it'd be appreciated.

@intel-hm
Copy link
Author

if you give me the right git commands to download the correct files (my git knowledge is barely existant....)

@FFY00
Copy link
Member

FFY00 commented Apr 29, 2025

if you give me the right git commands to download the correct files (my git knowledge is barely existant....)

In that case, I'd recommend you use the Github CLI, which simplifies this quite significantly 😊

Using it, it should be as straightforward as:

$ gh pr checkout 133163

Otherwise, you'll have to manually add a remote for my fork, and then checkout the correct branch.

$ git remote add ffy00 https://github.com/FFY00/cpython.git
$ git fetch ffy00
$ git checkout gh-108819

@intel-hm
Copy link
Author

wouldn't I also need a git clone? Github CLI is not really an option

$ git clone https://github.com/FFY00/cpython.git
$ git remote add ffy00 https://github.com/FFY00/cpython.git
$ git fetch ffy00
$ git checkout gh-108819

@FFY00
Copy link
Member

FFY00 commented Apr 29, 2025

You can either clone the main repo and add my fork as a remote, or clone my fork directly, then you can checkout my gh-108819 branch.

@mcepl
Copy link
Contributor

mcepl commented Apr 29, 2025

I obviously don’t see anything like this with our system packages:

Python 3.13.3 (main, Apr 11 2025, 19:56:42) [GCC]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.31.0 -- An enhanced Interactive Python. Type '?' for help.
In [1]: import sys

In [2]: sys.path
Out[2]: 
['/usr/bin',
 '/usr/lib64/python313.zip',
 '/usr/lib64/python3.13',
 '/usr/lib64/python3.13/lib-dynload',
 '',
 '/home/matej/.local/lib/python3.13/site-packages',
 '/usr/lib/python3.13/site-packages/setuptools/_vendor',
 '/home/matej/archiv/knihovna/repos/osc',
 '/home/matej/archiv/2024/SUSE/projekty/maintenance-toolkit',
 '/home/matej/archiv/knihovna/repos/epy/src',
 '/usr/lib64/python3.13/site-packages',
 '/usr/lib64/python3.13/site-packages/PIL',
 '/usr/lib64/python3.13/_import_failed',
 '/usr/lib/python3.13/site-packages']

In [3]: 

However, we have plenty of patches and configuration. Perhaps looking at https://build.opensuse.org/package/show/devel:languages:python:Factory/python313 would show something interesting. However, to be honest, I don’t care that much about user-build builds; I think we are doing pretty good job in packaging Python ourselves and we support those.

@FFY00
Copy link
Member

FFY00 commented Apr 29, 2025

That'd be because you set --with-platlibdir to match the system libdir, so no mismatch would happen there.

@intel-hm
Copy link
Author

intel-hm commented Apr 29, 2025

looks good:

git clone https://github.com/python/cpython.git
cd cpython/
git remote add ffy00 https://github.com/FFY00/cpython.git
git fetch ffy00
git checkout gh-108819

./configure --prefix=/lfs/lfs19/python/3.12.7
make
make install

~/cpython> find /lfs/lfs19/python/3.12.7/ -name lib-dynload
/lfs/lfs19/python/3.12.7/lib/python3.14/lib-dynload

~/cpython> /lfs/lfs19/python/3.12.7/bin/python3
Python 3.14.0a7+ (heads/gh-108819:a2b43c9d137, Apr 29 2025, 12:15:56) [GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from _posixsubprocess import fork_exec as _fork_exec
>>>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
OS-linux topic-installation type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

5 participants