Skip to content

Add missing contrib modules / pl extensions to alpine image #906

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Nov 15, 2021
Merged

Add missing contrib modules / pl extensions to alpine image #906

merged 2 commits into from
Nov 15, 2021

Conversation

wolfgangwalther
Copy link
Contributor

@wolfgangwalther wolfgangwalther commented Nov 12, 2021

The readme on dockerhub says:

When using the Alpine variants, any postgres extension not listed in postgres-contrib will need to be compiled in your own image (again, see github.com/postgis/docker-postgis for a concrete example).

However, not all extensions in contrib are available. The language extension plperl, plpython and pltcl, as well as related contrib modules (e.g. hstore_*, jsonb_*, ...) are missing.

Only in very rare cases, when the postgres version matches the one distributed in the current alpine distribution, one can install e.g. plpython via apk add postgresql-plpython3 (with a bit of moving files around to their proper location). However, this is not sustainable: It currently works for pg 13.x on alpine 3.14, but not for pg 14.x anymore.

Compiling those extensions from source kind of defeats the purpose of deriving from this image in the first place.

Currently, those extensions seem to be left out on purpose:

# these make our image abnormally large (at least 100MB larger), which seems uncouth for an "Alpine" (ie, "small") variant :)
# --with-krb5 \
# --with-gssapi \
# --with-ldap \
# --with-tcl \
# --with-perl \
# --with-python \

I investigated the potential increase in image size a little bit. It seems like the biggest chunk results from additional dependencies in perl, python and tcl. However, those can be avoided, if we build the extension, but then remove the dependencies for them. This means the above mentioned extensions will only be usable after installing the dependencies in a derived image again - but that's already a huge step forward.

As an example, using plpython would be as simple as:

FROM postgres:14-alpine
RUN apk add --no-cache python3

I looked at image size in 4 cases:

  • master: current master branch
  • +PLfull: compiled language extension, but kept the dependencies
  • +PL: compiled language extensions, but removed dependencies
  • +PL+auth: same as before, but also compiled --with-krb5, --with-gssapi and --with-ldap

The last one was just because those were commented out, too, and I wondered what it would cost to enable them now.

postgres master +PLfull +PL +PL+auth
14.1-alpine 195 280 196 199
13.5-alpine 192 276 192 195
12.9-alpine 190 275 191 194
11.14-alpine 188 273 189 192
10.19-alpine 73 157 73 76
9.6.24-alpine 37 92 51 54

The data shows that indeed the biggest chunk comes from perl, python and tcl. The increase in image size for the other two cases however is really small.

I suggest to enable both the language extensions as well as --with-krb5, --with-gssapi and --with-ldap, but leave out the 3 major dependencies to keep image size small. This will make the alpine image much more usable.

@tianon
Copy link
Member

tianon commented Nov 13, 2021

I think this is going to lead to a non-trivial image size increase. See also #384 for a useful previous discussion on this topic.

@tianon
Copy link
Member

tianon commented Nov 13, 2021

... @yosifkit kindly pointed out that the table in your message covers this 🤦

Looking at the builds themselves, 14-alpine goes from ~195MB up to ~199MB. 😅

@yosifkit yosifkit merged commit bafaf65 into docker-library:master Nov 15, 2021
@wolfgangwalther wolfgangwalther deleted the alpine/add-missing-contrib branch November 15, 2021 17:36
@wolfgangwalther
Copy link
Contributor Author

@yosifkit yosifkit merged commit bafaf65 into docker-library:master 3 minutes ago

🎉

Just a question regarding releases: Am I right to assume that this change will only be released on docker hub once new postgres minor versions are released?

@yosifkit
Copy link
Member

@wolfgangwalther
Copy link
Contributor Author

Nope, it'll go up as soon as we decide to make (and merge) a PR over to official-images.

Ok, cool. I'll watch that closely.

See also https://github.com/docker-library/postgres/tree/bafaf65a2356a05810c12a03d4412e1003cd1bb8#see-a-change-merged-here-that-doesnt-show-up-on-docker-hub-yet.

Yeah, I went through that (and the FAQ the link points to) already, but the change in that example is about a version update. That creates new tags, so it all kinds of makes sense. I couldn't tell from the FAQ what that means for "non-version-update" changes, though. Would those recreate the latest version tag? I always assumed that, once I reference a "full-version" image (e.g. postgres:14.1-alpine, right now), I would always get the same image. But apparently those are replaced, too?

I probably just missed that part in the FAQ.

docker-library-bot added a commit to docker-library-bot/official-images that referenced this pull request Nov 15, 2021
Changes:

- docker-library/postgres@bafaf65: Merge pull request docker-library/postgres#906 from wolfgangwalther/alpine/add-missing-contrib
- docker-library/postgres@e331a5b: Build alpine images --with-krb5, --with-gssapi and --with-ldap
- docker-library/postgres@5d9e5a4: Build plperl, plpython and pltcl in alpine images
@yosifkit
Copy link
Member

Yeah, any tag is mutable. We rebuild images when either a PR is merged to official-images that changes its build context (Dockerfile, shell scripts, etc) or any of its parent images can cause a rebuild (e.g. our rebuilding of Debian every ~30 days causes all dependent images to rebuild). Even when rebuilding for just a Dockerfile change, docker build cache is used as much as possible to minimize change between images where possible.

If anyone needs immutable references, then they should use the content addressable digest (postgres@sha256.....). This is a reference to that exact image build and will always be the same image.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants