Skip to content

Commit 2bf1d15

Browse files
dmpgprodmpgpro
committed
Merge commit '133573e4da1ebd0e6adf4550204c924792e4e9b3' into PGPRO9_6
2 parents d0ee558 + 133573e commit 2bf1d15

37 files changed

+1918
-1543
lines changed

‎contrib/pg_probackup/.travis.yml

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
sudo: required
2+
3+
services:
4+
- docker
5+
6+
script:
7+
- docker run -v $(pwd):/tests --rm centos:7 /tests/travis/backup_restore.sh

‎contrib/pg_probackup/Makefile

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
PROGRAM = pg_arman
1+
PROGRAM = pg_probackup
22
OBJS = backup.o \
33
catalog.o \
44
data.o \
@@ -7,7 +7,7 @@ OBJS = backup.o \
77
fetch.o \
88
init.o \
99
parray.o \
10-
pg_arman.o \
10+
pg_probackup.o \
1111
restore.o \
1212
show.o \
1313
status.o \
@@ -25,7 +25,7 @@ EXTRA_CLEAN = datapagemap.c datapagemap.h xlogreader.c receivelog.c receivelog.h
2525

2626
REGRESS = init option show delete backup restore
2727

28-
all: checksrcdir datapagemap.h logging.h receivelog.h streamutil.h pg_arman
28+
all: checksrcdir datapagemap.h logging.h receivelog.h streamutil.h pg_probackup
2929

3030
MAKE_GLOBAL="../../src/Makefile.global"
3131
TEST_GLOBAL:=$(shell test -e ../../src/Makefile.global)
@@ -45,7 +45,7 @@ endif
4545
else
4646
#TODO: fix me
4747
REGRESS =
48-
subdir=contrib/pg_arman
48+
subdir=contrib/pg_probackup
4949
top_builddir=../..
5050
include $(top_builddir)/src/Makefile.global
5151
include $(top_srcdir)/contrib/contrib-global.mk

‎contrib/pg_probackup/README.md

Lines changed: 39 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,7 @@
1-
pg_arman fork from Postgres Professional
1+
pg_probackup fork of pg_arman by Postgres Professional
22
========================================
33

4-
This repository contains fork of pg_arman by Postgres Professional with
5-
block level incremental backup support.
6-
7-
pg_arman is a backup and recovery manager for PostgreSQL servers able to do
4+
pg_probackup is a backup and recovery manager for PostgreSQL servers able to do
85
differential and full backup as well as restore a cluster to a
96
state defined by a given recovery target. It is designed to perform
107
periodic backups of an existing PostgreSQL server, combined with WAL
@@ -13,41 +10,52 @@ server because of a reason or another. Its differential backup
1310
facility reduces the amount of data necessary to be taken between
1411
two consecutive backups.
1512

13+
Main features:
14+
* incremental backup from WAL and PTRACK
15+
* backup from replica
16+
* multithreaded backup and restore
17+
* autonomous backup without archive command (will need slot replication)
18+
19+
Requirements:
20+
* >=PostgreSQL 9.5
21+
* >=gcc 4.4 or >=clang 3.6 or >= XLC 12.1
22+
* pthread
23+
1624
Download
1725
--------
1826

1927
The latest version of this software can be found on the project website at
20-
https://github.com/postgrespro/pg_arman. Original fork of pg_arman can be
28+
https://github.com/postgrespro/pg_probackup. Original fork of pg_probackup can be
2129
found at https://github.com/michaelpq/pg_arman.
2230

2331
Installation
2432
------------
2533

26-
Compiling pg_arman requires a PostgreSQL installation to be in place
34+
Compiling pg_probackup requires a PostgreSQL installation to be in place
2735
as well as a raw source tree. Pass the path to the PostgreSQL source tree
2836
to make, in the top_srcdir variable:
2937

3038
make USE_PGXS=1 top_srcdir=<path to PostgreSQL source tree>
3139

3240
In addition, you must have pg_config in $PATH.
3341

34-
The current version of pg_arman is compatible with PostgreSQL 9.5 and
42+
The current version of pg_probackup is compatible with PostgreSQL 9.5 and
3543
upper versions.
3644

3745
Platforms
3846
---------
3947

40-
pg_arman has been tested on Linux and Unix-based platforms.
48+
pg_probackup has been tested on Linux and Unix-based platforms.
4149

4250
Documentation
4351
-------------
4452

45-
All the documentation you can find [here](doc/pg_arman.md).
53+
All the documentation you can find [here](doc/pg_probackup.md).
4654

4755
Regression tests
4856
----------------
4957

50-
The test suite of pg_arman is available in the code tree and can be
58+
The test suite of pg_probackup is available in the code tree and can be
5159
launched in a way similar to common PostgreSQL extensions and modules:
5260

5361
make installcheck
@@ -74,50 +82,45 @@ of these approach is requirement to have WAL archive.
7482
some overhead to PostgreSQL performance. On our experiments it appears to be
7583
less than 3%.
7684

77-
These two approaches were implemented in this fork of pg_arman. The second
78-
approach requires [patch for PostgreSQL 9.5](https://gist.github.com/stalkerg/44703dbcbac1da08f448b7e6966646c0).
85+
These two approaches were implemented in this fork of pg_probackup. The second
86+
approach requires [patch for PostgreSQL 9.5](https://gist.github.com/stalkerg/44703dbcbac1da08f448b7e6966646c0) or
87+
[patch for PostgreSQL 10](https://gist.github.com/stalkerg/ab833d94e2f64df241f1835651e06e4b).
7988

8089
Testing block level incremental backup
8190
--------------------------------------
8291

83-
You need build and install [PGPRO9_5 branch of PostgreSQL](https://github.com/postgrespro/postgrespro) or [apply this patch to PostgreSQL 9.5](https://gist.github.com/stalkerg/44703dbcbac1da08f448b7e6966646c0).
92+
You need build and install [PGPRO9_5 or PGPRO9_6 branch of PostgreSQL](https://github.com/postgrespro/postgrespro) or apply this patch to
93+
[PostgreSQL 9.5](https://gist.github.com/stalkerg/44703dbcbac1da08f448b7e6966646c0) or [PostgreSQL 10](https://gist.github.com/stalkerg/ab833d94e2f64df241f1835651e06e4b).
8494

8595
### Retrieving changed blocks from WAL archive
8696

8797
You need to enable WAL archive by adding following lines to postgresql.conf:
8898

8999
```
90100
wal_level = archive
91-
archive_command = 'test ! -f /home/postgres/backup/arman/wal/%f && cp %p /home/postgres/backup/arman/wal/%f'
92-
wal_log_hints = on
101+
archive_command = 'test ! -f /home/postgres/backup/wal/%f && cp %p /home/postgres/backup/wal/%f'
93102
```
94103

95104
Example backup (assuming PostgreSQL is running):
96105
```bash
97106
# Init pg_aramn backup folder
98-
pg_arman init -B /home/postgres/backup/pgarman
99-
cat << __EOF__ >> /home/postgres/backup/pgarman/pg_arman.ini
100-
ARCLOG_PATH = '/home/postgres/backup/arman/wal'
101-
__EOF__
107+
pg_probackup init -B /home/postgres/backup
102108
# Make full backup with 2 thread and verbose mode.
103-
pg_arman backup -B /home/postgres/backup/pgarman -D /home/postgres/pgdata/arman -b full -v -j 2
104-
# Validate backup
105-
pg_arman validate -B /home/postgres/backup/pgarman -D /home/postgres/pgdata/arman
109+
pg_probackup backup -B /home/postgres/backup -D /home/postgres/pgdata -b full -v -j 2
106110
# Show backups information
107-
pg_arman show -B /home/postgres/backup/pgarman
111+
pg_probackup show -B /home/postgres/backup
108112

109113
# Now you can insert or update some data in your database
110114

111115
# Then start the incremental backup.
112-
pg_arman backup -B /home/postgres/backup/pgarman -D /home/postgres/pgdata/arman -b page -v -j 2
113-
pg_arman validate -B /home/postgres/backup/pgarman -D /home/postgres/pgdata/arman
116+
pg_probackup backup -B /home/postgres/backup -D /home/postgres/pgdata -b page -v -j 2
114117
# You should see that increment is really small
115-
pg_arman show -B /home/postgres/backup/pgarman
118+
pg_probackup show -B /home/postgres/backup
116119
```
117120

118121
For restore after remove your pgdata you can use:
119122
```
120-
pg_arman restore -B /home/postgres/backup/pgarman -D /home/postgres/pgdata/arman -j 4 --verbose
123+
pg_probackup restore -B /home/postgres/backup -D /home/postgres/pgdata -j 4 --verbose
121124
```
122125

123126
### Retrieving changed blocks from ptrack
@@ -128,39 +131,33 @@ The advantage of this approach is that you don't have to save WAL archive. You
128131
ptrack_enable = on
129132
```
130133

131-
Also, some WALs still need to be fetched in order to get consistent backup. pg_arman can fetch them trough the streaming replication protocol. Thus, you also need to [enable streaming replication connection](https://wiki.postgresql.org/wiki/Streaming_Replication).
134+
Also, some WALs still need to be fetched in order to get consistent backup. pg_probackup can fetch them trough the streaming replication protocol. Thus, you also need to [enable streaming replication connection](https://wiki.postgresql.org/wiki/Streaming_Replication).
132135

133136
Example backup (assuming PostgreSQL is running):
134137
```bash
135138
# Init pg_aramn backup folder
136-
pg_arman init -B /home/postgres/backup/pgarman
137-
cat << __EOF__ >> /home/postgres/backup/pgarman/pg_arman.ini
138-
ARCLOG_PATH = '/home/postgres/backup/arman/wal'
139-
__EOF__
139+
pg_probackup init -B /home/postgres/backup
140140
# Make full backup with 2 thread and verbose mode.
141-
pg_arman backup -B /home/postgres/backup/pgarman -D /home/postgres/pgdata/arman -b full -v -j 2 --stream
142-
# Validate backup
143-
pg_arman validate -B /home/postgres/backup/pgarman -D /home/postgres/pgdata/arman
141+
pg_probackup backup -B /home/postgres/backup -D /home/postgres/pgdata -b full -v -j 2 --stream
144142
# Show backups information
145-
pg_arman show -B /home/postgres/backup/pgarman
143+
pg_probackup show -B /home/postgres/backup
146144

147145
# Now you can insert or update some data in your database
148146

149147
# Then start the incremental backup.
150-
pg_arman backup -B /home/postgres/backup/pgarman -D /home/postgres/pgdata/arman -b ptrack -v -j 2 --stream
151-
pg_arman validate -B /home/postgres/backup/pgarman -D /home/postgres/pgdata/arman
148+
pg_probackup backup -B /home/postgres/backup -D /home/postgres/pgdata -b ptrack -v -j 2 --stream
152149
# You should see that increment is really small
153-
pg_arman show -B /home/postgres/backup/pgarman
150+
pg_probackup show -B /home/postgres/backup
154151
```
155152

156153
For restore after remove your pgdata you can use:
157154
```
158-
pg_arman restore -B /home/postgres/backup/pgarman -D /home/postgres/pgdata/arman -j 4 --verbose --stream
155+
pg_probackup restore -B /home/postgres/backup -D /home/postgres/pgdata -j 4 --verbose --stream
159156
```
160157

161158
License
162159
-------
163160

164-
pg_arman can be distributed under the PostgreSQL license. See COPYRIGHT
161+
pg_probackup can be distributed under the PostgreSQL license. See COPYRIGHT
165162
file for more information. pg_arman is a fork of the existing project
166163
pg_rman, initially created and maintained by NTT and Itagaki Takahiro.

0 commit comments

Comments
 (0)