Skip to content

Reduce the size of the image #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

Conversation

mwcampbell
Copy link

By combining all build steps, plus some new cleanup steps, into a new RUN command, I reduced the size of the image by about 200 MB.

&& cd .. \
&& rm -rf postgres-build \
&& find /usr/local/pgsql -type f -name "*.a" -delete \
&& ((find /usr/local/pgsql -type f -print | xargs strip --strip-all) || true) \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar comment as on PR#1 for docker-library/docker-mysql.

Thanks for your help.

@tianon
Copy link
Member

tianon commented Jun 19, 2014

As mentioned in the comments on the repo (https://registry.hub.docker.com/_/postgres/), having the compiled source is useful since things like RUN cd contrib/hstore && make && make install work, and this makes that not work anymore (even if you adjust the path to the checked out but not compiled source). Any thoughts on how we can get some of the best of both?

@yosifkit
Copy link
Member

I have been trying to get the 'best of both' by saving space and still being able to RUN cd contrib/hstore && make && make install but can only save ~8MB.

If you have any ideas, here is the relevant part of the Dockerfile that I have changed with a little context:

ADD . /usr/src/postgres
WORKDIR /usr/src/postgres
RUN ./configure \
    && make -j"$(nproc)" \
    && chown -R postgres src/test/regress \
    && { gosu postgres make check \
        || { cat >&2 /usr/src/postgres/src/test/regress/log/initdb.log; false; }; } \
    && make install-strip \
    && find . -type f -name '*.a' -delete \
    && { find . -type f -executable -exec strip --strip-all '{}' + || true; } \
    && find /usr/local/pgsql -type f -name '*.a' -delete \
    && { find /usr/local/pgsql -type f -executable -exec strip --strip-all '{}' + || true; } \
    && rm -rf /usr/local/pgsql/include \
    && rm -rf /usr/local/pgsql/lib/pgxs

WORKDIR /usr/local/pgsql

@yosifkit
Copy link
Member

@jberkus had mentioned that the PostgreSQL community would like to take over the Official image. Perhaps someone familiar with postgres and the extensions can provide better direction for the Dockerfile. We are interested in moving this directly to upstream Postgres so the the official image builds from there.

@jberkus
Copy link

jberkus commented Jun 20, 2014

You shouldn't need the source to build extensions, just the headers and pgxs.

@mwcampbell
Copy link
Author

Then I just went a little too far in stripping things down; just take out my commands for removing the headers and pgxs

@yosifkit
Copy link
Member

This is what we get when we try to build (for example) hstore from a make cleaned source repo (and this is after we do the full install with --prefix=/usr/local for good measure, but the default prefix gives the same error and also without the two lines that remove the headers and pgxs):

root@1a0750f32dcb:/usr/src/postgres# (cd contrib/hstore && make && make install)
gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -fpic -I. -I. -I../../src/include -D_GNU_SOURCE   -c -o hstore_io.o hstore_io.c
In file included from ../../src/include/postgres.h:48:0,
                 from hstore_io.c:4:
../../src/include/utils/elog.h:69:28: fatal error: utils/errcodes.h: No such file or directory
 #include "utils/errcodes.h"
                            ^
compilation terminated.
<builtin>: recipe for target 'hstore_io.o' failed
make: *** [hstore_io.o] Error 1

Here is the relevant Dockerfile lines:

RUN ./configure --prefix=/usr/local \
    && make -j"$(nproc)" \
    && chown -R postgres src/test/regress \
    && { gosu postgres make check \
        || { cat >&2 /usr/src/postgres/src/test/regress/log/initdb.log; false; }; } \
    && make install-strip \
    && make clean \
    && find /usr/local -type f -name '*.a' -delete

@jberkus
Copy link

jberkus commented Jun 20, 2014

Is there a particularly good reason to compile PostgreSQL instead of using the readily-available Debian packages?

@tianon
Copy link
Member

tianon commented Jun 20, 2014

Well, the justification was that if we compile from source, it'll be more attractive to the upstream developers since it'd be useful in their own development processes. Since you're upstream, and you're interested, that might be a moot-point now for this particular image. :)

@jberkus
Copy link

jberkus commented Jun 20, 2014

I'll take a look at the build-from-source setup.

I guess it's a question of why we're building the image. I certainly will build a source image for 9.4 as a testing tool. But if we're maintaining the official image for production use by downstream users, it would be much easier to keep up with security updates from various upstream projects by using the apt.postgresql.org packages. Unless they're too large?

@yosifkit
Copy link
Member

As far as Docker is showing, the official images are for production use. If the apt packages are the best way to ensure security updates, I suggest that is where we move the "official build" toward. There could be a special tagged build like postgres:source or even a separate image postgres-source for those that want the directly from source version.

I am not sure on the installed size of the apt.postgresql.org packages. I'll am looking into it (using the quickstart instructions). Will this still give users an easy way to build and add extensions?

@jberkus
Copy link

jberkus commented Jun 20, 2014

Yes, it will. Let me get you the portion of the Dockerfile I'm using which does exactly that.

@yosifkit
Copy link
Member

The install from the apt packages is much smaller, ~277MB vs the current official ~803MB.

The first problem so far is the tight coupling with init and automatic cluster setup. Docker is made to follow a single foreground command in the container, but the default wants to run 6 background postgres processes.

It would also be beneficial to have the config in the same directory as the data to keep the container ephemeral, making use of volumes.

Any suggestions?

@yosifkit
Copy link
Member

Ok, I have found how to run this the "docker way" with a useful volume. Here is the Dockerfile and entrypoint: https://gist.github.com/yosifkit/92948d82feb4660699de. The entrypoint is unchanged from the one in this repo.

@jberkus
Copy link

jberkus commented Jun 20, 2014

So here's the basics I'm talking about. The apt.postgresql.org.sh script comes from http://apt.postgresql.org; it adds the apt-get keys to sources.list.d and does an apt-get update.

# add in build scripts
ADD test/docker/build /build/

# add apt sources for PostgreSQL
# also does apt-get update
RUN /bin/bash /build/apt.postgresql.org.sh

# install make for source-only extension
RUN apt-get install -y make gcc

# install common Postgresql 9.3 packages
# done as a separate layer in order to have cache layers regardless
# of the version of PostgreSQL
RUN apt-get install -y postgresql-9.3 postgresql-server-dev-9.3 postgresql-contrib-9.3 postgresql-plpython-9.3

# copy replacement config files into place since the defaults are not optimal
RUN cp /build/postgresql.conf /etc/postgresql/9.3/postgresql.conf && cp /build/pg_hba.conf /etc/postgresql/9.3/pg_hba.conf 

@jberkus
Copy link

jberkus commented Jun 21, 2014

Comments added to scripts.

@jberkus
Copy link

jberkus commented Jun 21, 2014

So, just because you've installed the Debian packages doesn't mean you can't go around them. Executing "postgres" to start the server is probably the better way to go.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants