Skip to content

Commit 3500ccc

Browse files
robertmhaasrobertmhaas
committed
Support base backup targets.
pg_basebackup now has a --target=TARGET[:DETAIL] option. If specfied, it is sent to the server as the value of the TARGET option to the BASE_BACKUP command. If DETAIL is included, it is sent as the value of the new TARGET_DETAIL option to the BASE_BACKUP command. If the target is anything other than 'client', pg_basebackup assumes that it will now be the server's job to write the backup in a location somehow defined by the target, and that it therefore needs to write nothing locally. However, the server will still send messages to the client for progress reporting purposes. On the server side, we now support two additional types of backup targets. There is a 'blackhole' target, which just throws away the backup data without doing anything at all with it. Naturally, this should only be used for testing and debugging purposes, since you will not actually have a backup when it finishes running. More usefully, there is also a 'server' target, so you can now use something like 'pg_basebackup -Xnone -t server:/SOME/PATH' to write a backup to some location on the server. We can extend this to more types of targets in the future, and might even want to create an extensibility mechanism for adding new target types. Since WAL fetching is handled with separate client-side logic, it's not part of this mechanism; thus, backups with non-default targets must use -Xnone or -Xfetch. Patch by me, with a bug fix by Jeevan Ladhe. The patch set of which this is a part has also had review and/or testing from Tushar Ahuja, Suraj Kharage, Dipesh Pandit, and Mark Dilger. Discussion: http://postgr.es/m/CA+TgmoaYZbz0=Yk797aOJwkGJC-LK3iXn+wzzMx7KdwNpZhS5g@mail.gmail.com
1 parent f80900b commit 3500ccc

File tree

11 files changed

+677
-64
lines changed

11 files changed

+677
-64
lines changed

‎doc/src/sgml/protocol.sgml

+21-2
Original file line numberDiff line numberDiff line change
@@ -2640,8 +2640,27 @@ The commands accepted in replication mode are:
26402640
</para>
26412641

26422642
<para>
2643-
At present, the only supported value for this parameter is
2644-
<literal>client</literal>.
2643+
If the target is <literal>client</literal>, the backup data is
2644+
sent to the client. If it is <literal>server</literal>, the backup
2645+
data is written to the server at the pathname specified by the
2646+
<literal>TARGET_DETAIL</literal> option. If it is
2647+
<literal>blackhole</literal>, the backup data is not sent
2648+
anywhere; it is simply discarded.
2649+
</para>
2650+
</listitem>
2651+
</varlistentry>
2652+
2653+
<varlistentry>
2654+
<term><literal>TARGET_DETAIL</literal> <replaceable>'detail'</replaceable></term>
2655+
<listitem>
2656+
<para>
2657+
Provides additional information about the backup target.
2658+
</para>
2659+
2660+
<para>
2661+
Currently, this option can only be used when the backup target is
2662+
<literal>server</literal>. It specifies the server directory
2663+
to which the backup should be written.
26452664
</para>
26462665
</listitem>
26472666
</varlistentry>

‎doc/src/sgml/ref/pg_basebackup.sgml

+30
Original file line numberDiff line numberDiff line change
@@ -224,6 +224,36 @@ PostgreSQL documentation
224224
</listitem>
225225
</varlistentry>
226226

227+
<varlistentry>
228+
<term><option>-t <replaceable class="parameter">target</replaceable></option></term>
229+
<term><option>--target=<replaceable class="parameter">target</replaceable></option></term>
230+
<listitem>
231+
232+
<para>
233+
Instructs the server where to place the base backup. The default target
234+
is <literal>client</literal>, which specifies that the backup should
235+
be sent to the machine where <application>pg_basebackup</application>
236+
is running. If the target is instead set to
237+
<literal>server:/some/path</literal>, the backup will be stored on
238+
the machine where the server is running in the
239+
<literal>/some/path</literal> directory. Storing a backup on the
240+
server requires superuser privileges. If the target is set to
241+
<literal>blackhole</literal>, the contents are discarded and not
242+
stored anywhere. This should only be used for testing purposes, as you
243+
will not end up with an actual backup.
244+
</para>
245+
246+
<para>
247+
Since WAL streaming is implemented by
248+
<application>pg_basebackup</application> rather than by the server,
249+
this option cannot be used together with <literal>-Xstream</literal>.
250+
Since that is the default, when this option is specified, you must also
251+
specify either <literal>-Xfetch</literal> or <literal>-Xnone</literal>.
252+
</para>
253+
254+
</listitem>
255+
</varlistentry>
256+
227257
<varlistentry>
228258
<term><option>-T <replaceable class="parameter">olddir</replaceable>=<replaceable class="parameter">newdir</replaceable></option></term>
229259
<term><option>--tablespace-mapping=<replaceable class="parameter">olddir</replaceable>=<replaceable class="parameter">newdir</replaceable></option></term>

‎src/backend/replication/Makefile

+1
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ OBJS = \
1919
basebackup.o \
2020
basebackup_copy.o \
2121
basebackup_progress.o \
22+
basebackup_server.o \
2223
basebackup_sink.o \
2324
basebackup_throttle.o \
2425
repl_gram.o \

‎src/backend/replication/basebackup.c

+68-13
Original file line numberDiff line numberDiff line change
@@ -55,8 +55,10 @@
5555

5656
typedef enum
5757
{
58+
BACKUP_TARGET_BLACKHOLE,
5859
BACKUP_TARGET_COMPAT,
59-
BACKUP_TARGET_CLIENT
60+
BACKUP_TARGET_CLIENT,
61+
BACKUP_TARGET_SERVER
6062
} backup_target_type;
6163

6264
typedef struct
@@ -69,6 +71,7 @@ typedef struct
6971
uint32 maxrate;
7072
bool sendtblspcmapfile;
7173
backup_target_type target;
74+
char *target_detail;
7275
backup_manifest_option manifest;
7376
pg_checksum_type manifest_checksum_type;
7477
} basebackup_options;
@@ -702,6 +705,8 @@ parse_basebackup_options(List *options, basebackup_options *opt)
702705
bool o_manifest = false;
703706
bool o_manifest_checksums = false;
704707
bool o_target = false;
708+
bool o_target_detail = false;
709+
char *target_str = "compat"; /* placate compiler */
705710

706711
MemSet(opt, 0, sizeof(*opt));
707712
opt->target = BACKUP_TARGET_COMPAT;
@@ -847,25 +852,35 @@ parse_basebackup_options(List *options, basebackup_options *opt)
847852
}
848853
else if (strcmp(defel->defname, "target") == 0)
849854
{
850-
char *optval = defGetString(defel);
855+
target_str = defGetString(defel);
851856

852857
if (o_target)
853858
ereport(ERROR,
854859
(errcode(ERRCODE_SYNTAX_ERROR),
855860
errmsg("duplicate option \"%s\"", defel->defname)));
856-
if (strcmp(optval, "client") == 0)
861+
if (strcmp(target_str, "blackhole") == 0)
862+
opt->target = BACKUP_TARGET_BLACKHOLE;
863+
else if (strcmp(target_str, "client") == 0)
857864
opt->target = BACKUP_TARGET_CLIENT;
865+
else if (strcmp(target_str, "server") == 0)
866+
opt->target = BACKUP_TARGET_SERVER;
858867
else
859868
ereport(ERROR,
860869
(errcode(ERRCODE_SYNTAX_ERROR),
861-
errmsg("unrecognized target: \"%s\"", optval)));
870+
errmsg("unrecognized target: \"%s\"", target_str)));
862871
o_target = true;
863872
}
864-
else
865-
ereport(ERROR,
866-
errcode(ERRCODE_SYNTAX_ERROR),
867-
errmsg("option \"%s\" not recognized",
868-
defel->defname));
873+
else if (strcmp(defel->defname, "target_detail") == 0)
874+
{
875+
char *optval = defGetString(defel);
876+
877+
if (o_target_detail)
878+
ereport(ERROR,
879+
(errcode(ERRCODE_SYNTAX_ERROR),
880+
errmsg("duplicate option \"%s\"", defel->defname)));
881+
opt->target_detail = optval;
882+
o_target_detail = true;
883+
}
869884
}
870885
if (opt->label == NULL)
871886
opt->label = "base backup";
@@ -877,6 +892,22 @@ parse_basebackup_options(List *options, basebackup_options *opt)
877892
errmsg("manifest checksums require a backup manifest")));
878893
opt->manifest_checksum_type = CHECKSUM_TYPE_NONE;
879894
}
895+
if (opt->target == BACKUP_TARGET_SERVER)
896+
{
897+
if (opt->target_detail == NULL)
898+
ereport(ERROR,
899+
(errcode(ERRCODE_SYNTAX_ERROR),
900+
errmsg("target '%s' requires a target detail",
901+
target_str)));
902+
}
903+
else
904+
{
905+
if (opt->target_detail != NULL)
906+
ereport(ERROR,
907+
(errcode(ERRCODE_SYNTAX_ERROR),
908+
errmsg("target '%s' does not accept a target detail",
909+
target_str)));
910+
}
880911
}
881912

882913

@@ -908,14 +939,38 @@ SendBaseBackup(BaseBackupCmd *cmd)
908939

909940
/*
910941
* If the TARGET option was specified, then we can use the new copy-stream
911-
* protocol. If not, we must fall back to the old and less capable
912-
* copy-tablespace protocol.
942+
* protocol. If the target is specifically 'client' then set up to stream
943+
* the backup to the client; otherwise, it's being sent someplace else and
944+
* should not be sent to the client.
945+
*
946+
* If the TARGET option was not specified, we must fall back to the older
947+
* and less capable copy-tablespace protocol.
913948
*/
914-
if (opt.target != BACKUP_TARGET_COMPAT)
915-
sink = bbsink_copystream_new();
949+
if (opt.target == BACKUP_TARGET_CLIENT)
950+
sink = bbsink_copystream_new(true);
951+
else if (opt.target != BACKUP_TARGET_COMPAT)
952+
sink = bbsink_copystream_new(false);
916953
else
917954
sink = bbsink_copytblspc_new();
918955

956+
/*
957+
* If a non-default backup target is in use, arrange to send the data
958+
* wherever it needs to go.
959+
*/
960+
switch (opt.target)
961+
{
962+
case BACKUP_TARGET_BLACKHOLE:
963+
/* Nothing to do, just discard data. */
964+
break;
965+
case BACKUP_TARGET_COMPAT:
966+
case BACKUP_TARGET_CLIENT:
967+
/* Nothing to do, handling above is sufficient. */
968+
break;
969+
case BACKUP_TARGET_SERVER:
970+
sink = bbsink_server_new(sink, opt.target_detail);
971+
break;
972+
}
973+
919974
/* Set up network throttling, if client requested it */
920975
if (opt.maxrate > 0)
921976
sink = bbsink_throttle_new(sink, opt.maxrate);

‎src/backend/replication/basebackup_copy.c

+16-5
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,9 @@ typedef struct bbsink_copystream
4444
/* Common information for all types of sink. */
4545
bbsink base;
4646

47+
/* Are we sending the archives to the client, or somewhere else? */
48+
bool send_to_client;
49+
4750
/*
4851
* Protocol message buffer. We assemble CopyData protocol messages by
4952
* setting the first character of this buffer to 'd' (archive or manifest
@@ -131,11 +134,12 @@ const bbsink_ops bbsink_copytblspc_ops = {
131134
* Create a new 'copystream' bbsink.
132135
*/
133136
bbsink *
134-
bbsink_copystream_new(void)
137+
bbsink_copystream_new(bool send_to_client)
135138
{
136139
bbsink_copystream *sink = palloc0(sizeof(bbsink_copystream));
137140

138141
*((const bbsink_ops **) &sink->base.bbs_ops) = &bbsink_copystream_ops;
142+
sink->send_to_client = send_to_client;
139143

140144
/* Set up for periodic progress reporting. */
141145
sink->last_progress_report_time = GetCurrentTimestamp();
@@ -212,8 +216,12 @@ bbsink_copystream_archive_contents(bbsink *sink, size_t len)
212216
StringInfoData buf;
213217
uint64 targetbytes;
214218

215-
/* Send the archive content to the client (with leading type byte). */
216-
pq_putmessage('d', mysink->msgbuffer, len + 1);
219+
/* Send the archive content to the client, if appropriate. */
220+
if (mysink->send_to_client)
221+
{
222+
/* Add one because we're also sending a leading type byte. */
223+
pq_putmessage('d', mysink->msgbuffer, len + 1);
224+
}
217225

218226
/* Consider whether to send a progress report to the client. */
219227
targetbytes = mysink->bytes_done_at_last_time_check
@@ -294,8 +302,11 @@ bbsink_copystream_manifest_contents(bbsink *sink, size_t len)
294302
{
295303
bbsink_copystream *mysink = (bbsink_copystream *) sink;
296304

297-
/* Send the manifest content to the client (with leading type byte). */
298-
pq_putmessage('d', mysink->msgbuffer, len + 1);
305+
if (mysink->send_to_client)
306+
{
307+
/* Add one because we're also sending a leading type byte. */
308+
pq_putmessage('d', mysink->msgbuffer, len + 1);
309+
}
299310
}
300311

301312
/*

0 commit comments

Comments
 (0)