Skip to content

Install fails building wheel on alpine 3.15 #445

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
cessor opened this issue Dec 1, 2021 · 14 comments
Closed

Install fails building wheel on alpine 3.15 #445

cessor opened this issue Dec 1, 2021 · 14 comments

Comments

@cessor
Copy link

cessor commented Dec 1, 2021

Issue description:

python-ldap does not build on Alpine 3.15.

Wheel Build crashes with error (build log attached)

/usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find -lldap_r

Steps to reproduce:

FROM python:alpine3.15
RUN apk add --update build-base openldap-dev python2-dev python3-dev && rm -rf /var/cache/apk/*
RUN pip install --no-cache-dir python-ldap

Operating system:

  • Alpine 3.15

Python version:

  • 3.10.0

python-ldap version:

  • 3.4.0

Notes:

  • This issue does not exist with Alpine 3.14 (to verify, please change Dockerfile to FROM python:alpine3.14)

broken-on-python-alpine3.15.txt

@tiran
Copy link
Member

tiran commented Dec 1, 2021

This looks like a duplicate of #432. python-ldap does not support OpenLDAP 2.5+ yet.

@cessor
Copy link
Author

cessor commented Dec 1, 2021

Thank you for your quick response @tiran. You're right, Alpine 3.15 picks openldap-dev 2.6.0, so I guess this is a duplicate of #432.

@bytemarcz
Copy link

Yes! So happy to see someone else is having the same relevant issue. I don't think this had ever happened to me before.

I'm pretty much having the same issue

-3.8/Modules/berval.o -L/usr/local/lib -lldap_r -llber -o build/lib.linux-x86_64-3.8/_ldap.cpython-38-x86_64-linux-gnu.so #14 337.2 /usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find -lldap_r #14 337.2 collect2: error: ld returned 1 exit status #14 337.2 error: command 'gcc' failed with exit status 1

I only captured the last part of the output since it seems pretty similar at this point. @cessor correct me if I'm wrong here but I thought the FROM in the Dockerfile was supposed to be something like this: FROM python:3.8-alpine where I thought the syntax was something like [image][version-for-python]-[alpine] or is the version part supposed to be for Alpine and it pulls the latest python by default?

I'm going to try what you do according the the alpine3.14 syntax.

@tiran
Copy link
Member

tiran commented Dec 1, 2021

@cessor is using the Docker's Python container https://hub.docker.com/_/python

Ticket #432 contains a simple workaround for the problem. Create a file /usr/lib/libldap_r.so with content INPUT ( libldap.so ). This linker script will tell gcc to link with libldap.so.

@bytemarcz
Copy link

@tiran thank you! I skipped that and did what @cessor did with the alpine switch. I have a bunch more packages involved so by doing that broke another when creating wheels. None the less I had to manipulate my Dockerfile as such:

FROM python:3.8-alpine3.14

I'll make a note of your suggestion just in case anything else brakes.

@cessor
Copy link
Author

cessor commented Dec 2, 2021

@maczamora Thank you for your response; I was hesitant to post this issue at first because I thought that the case might be too specific. I am glad if I could be of help.

As for your question:

I thought the FROM in the Dockerfile was supposed to be something like this: FROM python:3.8-alpine where I thought the syntax was something like [image][version-for-python]-[alpine] or is the version part supposed to be for Alpine and it pulls the latest python by default?

Yes, the version part is for alpine; the structure is python:<version>-alpine<alpine-version>. The version numbers can be omitted which then gives you coarse default tags. You can find an overview over the current tag structure here: https://hub.docker.com/_/python?tab=description

As of 2. Dec 2021, this tag is an alias for python:3.10.0-alpine3.15; The tag python:alpine was changed a couple of days ago; back then it pointed to alpine3.14. I only encountered this error when my CI environment tried to automatically build my custom docker image, which starts with:

FROM python:alpine

I tend to use these very broad image tags because I want my images to keep up to date. I have my Gitlab set to build this as my base image every night so that I can detect version issues like this early on.

To fix my problem, I changed my Dockerfile to:

FROM python:alpine3.14

That said: I managed to get @tiran 's suggestion to run by putting the alias for the linker into my Dockerfile:

FROM python:alpine

RUN apk add --update --no-cache\
    build-base jpeg-dev zlib-dev libjpeg\
    gettext\
    py3-lxml\
    py3-pillow\
    openldap-dev\
    python3-dev\
    && rm -rf /var/cache/apk/*

RUN echo 'INPUT ( libldap.so )' > /usr/lib/libldap_r.so

RUN pip install python-ldap

This works as of 2. Dec 2021 when :alpine was the same as alpine3.15, so it's the newest version.

I will revisit this issue in some time. For now I don't fully understand the implications of @tiran 's solution. Ideally, I would want to do FROM python:alpine again and have python-ldap work against OpenLDAP Dev 2.6 out of the box, but as @tiran mentioned, this isn't a trivial change.

@mistotebe
Copy link
Contributor

Should be resolved with 3.4.2

@danielbrunt57
Copy link

Should be resolved with 3.4.2

I apologize in advance for my ignorance but v3.4.2 of what?

@tiran
Copy link
Member

tiran commented Jul 20, 2022

Should be resolved with 3.4.2

I apologize in advance for my ignorance but v3.4.2 of what?

python-ldap, https://pypi.org/project/python-ldap/3.4.2/

@danielbrunt57
Copy link

danielbrunt57 commented Jul 20, 2022

Thanks for clarifying that but it did not help resolve the issue...

pip install python-ldap==3.4.2
ERROR: Could not build wheels for python-ldap, which is required to install pyproject.toml-based projects

PRETTY_NAME="Alpine Linux v3.16"

@tiran
Copy link
Member

tiran commented Jul 20, 2022

python-ldap 3.4.2 builds fine on Alpine 3.16 for me

/ # apk add gcc musl-dev python3-dev openldap-dev py3-pip
...
/ # pip install python-ldap
Collecting python-ldap
  Downloading python-ldap-3.4.2.tar.gz (378 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 378.1/378.1 kB 14.2 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting pyasn1-modules>=0.1.5
  Downloading pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 155.3/155.3 kB 31.4 MB/s eta 0:00:00
Collecting pyasn1>=0.3.7
  Downloading pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 77.1/77.1 kB 21.0 MB/s eta 0:00:00
Building wheels for collected packages: python-ldap
  Building wheel for python-ldap (pyproject.toml) ... done
  Created wheel for python-ldap: filename=python_ldap-3.4.2-cp310-cp310-linux_x86_64.whl size=315529 sha256=196408f00bb8e94b125a19dd3fa42203764f673019e3f9afe4adaf8393f8c778
  Stored in directory: /root/.cache/pip/wheels/4f/16/6b/4f59ee64a958a5ad74932f09d32e524c5d0a3b1e8f9b3c8c84
Successfully built python-ldap
Installing collected packages: pyasn1, pyasn1-modules, python-ldap
Successfully installed pyasn1-0.4.8 pyasn1-modules-0.2.8 python-ldap-3.4.2

@danielbrunt57
Copy link

Thanks but this is Home Assistant provided Alpine OS so not sure what they've removed/modified. The issue is with AstroWeather integration which just recently developed install issues. It uses wheel deep down. Author has JUST posted an update claiming problem solved. I will try it tomorrow...

@tiran
Copy link
Member

tiran commented Jul 20, 2022

Could you switch to another Linux distro?
Python upstream does not support Alpine and has known issues. Most Python packages with C extensions only provide binary wheels for glibc-based Linux systems, not for musl-based systems like Alpine, too.

@danielbrunt57
Copy link

danielbrunt57 commented Jul 20, 2022

No, that is not possible. The issue seems to have somehow been resolved though... mawinkler/astroweather#16
Issue #16 and Issue #17: The introduction of the module timezonefinder with it's nested dependency to py-h3 failed while compiling the c-module h3 on some home assistant deployment variants (e.g. Home Assistant Operating System on RPi). The config flow now allows selecting the timezone for the specific AstroWeather instance for which reason the dependency to timezonefinder is not required anymore.
Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants