Skip to content

Commit 7cf085f

Browse files
robertmhaasrobertmhaas
committed
Add support for zstd base backup compression.
Both client-side compression and server-side compression are now supported for zstd. In addition, a backup compressed by the server using zstd can now be decompressed by the client in order to accommodate the use of -Fp. Jeevan Ladhe, with some edits by me. Discussion: http://postgr.es/m/CA+Tgmobyzfbz=gyze2_LL1ZumZunmaEKbHQxjrFkOR7APZGu-g@mail.gmail.com
1 parent c28839c commit 7cf085f

File tree

17 files changed

+750
-26
lines changed

17 files changed

+750
-26
lines changed

‎doc/src/sgml/protocol.sgml

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2724,8 +2724,8 @@ The commands accepted in replication mode are:
27242724
<listitem>
27252725
<para>
27262726
Instructs the server to compress the backup using the specified
2727-
method. Currently, the supported methods are <literal>gzip</literal>
2728-
and <literal>lz4</literal>.
2727+
method. Currently, the supported methods are <literal>gzip</literal>,
2728+
<literal>lz4</literal>, and <literal>zstd</literal>.
27292729
</para>
27302730
</listitem>
27312731
</varlistentry>
@@ -2737,7 +2737,8 @@ The commands accepted in replication mode are:
27372737
Specifies the compression level to be used. This should only be
27382738
used in conjunction with the <literal>COMPRESSION</literal> option.
27392739
For <literal>gzip</literal> the value should be an integer between 1
2740-
and 9, and for <literal>lz4</literal> it should be between 1 and 12.
2740+
and 9, for <literal>lz4</literal> between 1 and 12, and for
2741+
<literal>zstd</literal> it should be between 1 and 22.
27412742
</para>
27422743
</listitem>
27432744
</varlistentry>

‎doc/src/sgml/ref/pg_basebackup.sgml

Lines changed: 22 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -417,30 +417,33 @@ PostgreSQL documentation
417417
specify <literal>-Xfetch</literal>.
418418
</para>
419419
<para>
420-
The compression method can be set to <literal>gzip</literal> or
421-
<literal>lz4</literal>, or <literal>none</literal> for no
422-
compression. A compression level can be optionally specified, by
423-
appending the level number after a colon (<literal>:</literal>). If no
424-
level is specified, the default compression level will be used. If
425-
only a level is specified without mentioning an algorithm,
426-
<literal>gzip</literal> compression will be used if the level is
427-
greater than 0, and no compression will be used if the level is 0.
428-
</para>
429-
<para>
430-
When the tar format is used with <literal>gzip</literal> or
431-
<literal>lz4</literal>, the suffix <filename>.gz</filename> or
432-
<filename>.lz4</filename> will automatically be added to all tar
433-
filenames. When the plain format is used, client-side compression may
434-
not be specified, but it is still possible to request server-side
435-
compression. If this is done, the server will compress the backup for
436-
transmission, and the client will decompress and extract it.
420+
The compression method can be set to <literal>gzip</literal>,
421+
<literal>lz4</literal>, <literal>zstd</literal>, or
422+
<literal>none</literal> for no compression. A compression level can
423+
optionally be specified, by appending the level number after a colon
424+
(<literal>:</literal>). If no level is specified, the default
425+
compression level will be used. If only a level is specified without
426+
mentioning an algorithm, <literal>gzip</literal> compression will be
427+
used if the level is greater than 0, and no compression will be used if
428+
the level is 0.
429+
</para>
430+
<para>
431+
When the tar format is used with <literal>gzip</literal>,
432+
<literal>lz4</literal>, or <literal>zstd</literal>, the suffix
433+
<filename>.gz</filename>, <filename>.lz4</filename>, or
434+
<filename>.zst</filename>, respectively, will be automatically added to
435+
all tar filenames. When the plain format is used, client-side
436+
compression may not be specified, but it is still possible to request
437+
server-side compression. If this is done, the server will compress the
438+
backup for transmission, and the client will decompress and extract it.
437439
</para>
438440
<para>
439441
When this option is used in combination with
440442
<literal>-Xstream</literal>, <literal>pg_wal.tar</literal> will
441443
be compressed using <literal>gzip</literal> if client-side gzip
442-
compression is selected, but will not be compressed if server-side
443-
compresion or LZ4 compresion is selected.
444+
compression is selected, but will not be compressed if any other
445+
compression algorithm is selected, or if server-side compression
446+
is selected.
444447
</para>
445448
</listitem>
446449
</varlistentry>

‎src/backend/replication/Makefile

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ OBJS = \
2020
basebackup_copy.o \
2121
basebackup_gzip.o \
2222
basebackup_lz4.o \
23+
basebackup_zstd.o \
2324
basebackup_progress.o \
2425
basebackup_server.o \
2526
basebackup_sink.o \

‎src/backend/replication/basebackup.c

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -64,7 +64,8 @@ typedef enum
6464
{
6565
BACKUP_COMPRESSION_NONE,
6666
BACKUP_COMPRESSION_GZIP,
67-
BACKUP_COMPRESSION_LZ4
67+
BACKUP_COMPRESSION_LZ4,
68+
BACKUP_COMPRESSION_ZSTD
6869
} basebackup_compression_type;
6970

7071
typedef struct
@@ -906,6 +907,8 @@ parse_basebackup_options(List *options, basebackup_options *opt)
906907
opt->compression = BACKUP_COMPRESSION_GZIP;
907908
else if (strcmp(optval, "lz4") == 0)
908909
opt->compression = BACKUP_COMPRESSION_LZ4;
910+
else if (strcmp(optval, "zstd") == 0)
911+
opt->compression = BACKUP_COMPRESSION_ZSTD;
909912
else
910913
ereport(ERROR,
911914
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -1026,6 +1029,8 @@ SendBaseBackup(BaseBackupCmd *cmd)
10261029
sink = bbsink_gzip_new(sink, opt.compression_level);
10271030
else if (opt.compression == BACKUP_COMPRESSION_LZ4)
10281031
sink = bbsink_lz4_new(sink, opt.compression_level);
1032+
else if (opt.compression == BACKUP_COMPRESSION_ZSTD)
1033+
sink = bbsink_zstd_new(sink, opt.compression_level);
10291034

10301035
/* Set up progress reporting. */
10311036
sink = bbsink_progress_new(sink, opt.progress);

0 commit comments

Comments
 (0)