Skip to content

Commit 322becb

Browse files
michaelpqmichaelpq
committed
Switch the regression tests of pg_upgrade to use TAP tests
This simplifies a lot of code in the tests of pg_upgrade without sacrificing its coverage: - Removal of test.sh used for builds with make, that has accumulated over the years tweaks for problems that are solved in a duplicated way by the centralized TAP framework (initialization of the various environment variables PG*, port selection). - Removal of the code in MSVC to test pg_upgrade. This was roughly a duplicate of test.sh adapted for Windows, with an extra footprint of a pg_regress command and all the assumptions behind it. Support for upgrades with older versions is changed, not removed. test.sh was able to set up the regression database on the old instance by launching itself the pg_regress command and a dependency to the source tree of thd old cluster, with tweaks on the command arguments to adapt across the versions used. This created a backward-compatibility dependency with older pg_regress commands, and recent changes like d1029bb have made that much more complicated. Instead, this commit allows tests with older major versions by specifying a path to a SQL dump (taken with pg_dumpall from the old cluster's installation) that will be loaded into the old instance to upgrade instead of running pg_regress, through an optional environment variable called $olddump. This requires a second variable called $oldinstall to point to the base path of the installation of the old cluster. This method is more in line with the buildfarm client that uses a set of static dumps to set up an old instance, so hopefully we will be able to reuse what is introduced in this commit there. The last step of the tests that checks for differences between the two dumps taken still needs to be improved as it can fail, requiring a manual lookup at the dumps. This is not different from the old way of testing where things could fail at the last step. Support for EXTRA_REGRESS_OPTS is kept. vcregress.pl in the MSVC scripts still handles the test of pg_upgrade with its upgradecheck, and bincheck is changed to skip pg_upgrade. Author: Michael Paquier Reviewed-by: Andrew Dunstan, Andres Freund, Rachel Heaton, Tom Lane, Discussion: https://postgr.es/m/YJ8xTmLQkotVLpN5@paquier.xyz
1 parent fb691bb commit 322becb

File tree

6 files changed

+290
-436
lines changed

6 files changed

+290
-436
lines changed

‎src/bin/pg_upgrade/Makefile

+8-13
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,10 @@ OBJS = \
2828
override CPPFLAGS := -I$(srcdir) -I$(libpq_srcdir) $(CPPFLAGS)
2929
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
3030

31+
# required for 002_pg_upgrade.pl
32+
REGRESS_SHLIB=$(abs_top_builddir)/src/test/regress/regress$(DLSUFFIX)
33+
export REGRESS_SHLIB
34+
3135
all: pg_upgrade
3236

3337
pg_upgrade: $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
@@ -47,17 +51,8 @@ clean distclean maintainer-clean:
4751
rm -rf delete_old_cluster.sh log/ tmp_check/ \
4852
reindex_hash.sql
4953

50-
# When $(MAKE) is present, make automatically infers that this is a
51-
# recursive make. which is not actually what we want here, as that
52-
# e.g. prevents output synchronization from working (as make thinks
53-
# that the subsidiary make knows how to deal with that itself, but
54-
# we're invoking a shell script that doesn't know). Referencing
55-
# $(MAKE) indirectly avoids that behaviour.
56-
# See https://www.gnu.org/software/make/manual/html_node/MAKE-Variable.html#MAKE-Variable
57-
NOTSUBMAKEMAKE=$(MAKE)
58-
59-
check: test.sh all temp-install
60-
MAKE=$(NOTSUBMAKEMAKE) $(with_temp_install) bindir=$(abs_top_builddir)/tmp_install/$(bindir) EXTRA_REGRESS_OPTS="$(EXTRA_REGRESS_OPTS)" $(SHELL) $<
54+
check:
55+
$(prove_check)
6156

62-
# installcheck is not supported because there's no meaningful way to test
63-
# pg_upgrade against a single already-running server
57+
installcheck:
58+
$(prove_installcheck)

‎src/bin/pg_upgrade/TESTING

+27-59
Original file line numberDiff line numberDiff line change
@@ -2,25 +2,22 @@ THE SHORT VERSION
22
-----------------
33

44
On non-Windows machines, you can execute the testing process
5-
described below by running
5+
described below by running the following command in this directory:
66
make check
7-
in this directory. This will run the shell script test.sh, performing
8-
an upgrade from the version in this source tree to a new instance of
9-
the same version.
107

11-
To test an upgrade from a different version, you must have a built
12-
source tree for the old version as well as this version, and you
13-
must have done "make install" for both versions. Then do:
8+
This will run the TAP tests to run pg_upgrade, performing an upgrade
9+
from the version in this source tree to a new instance of the same
10+
version.
1411

15-
export oldsrc=...somewhere/postgresql (old version's source tree)
16-
export oldbindir=...otherversion/bin (old version's installed bin dir)
17-
export bindir=...thisversion/bin (this version's installed bin dir)
18-
export libdir=...thisversion/lib (this version's installed lib dir)
19-
sh test.sh
20-
21-
In this case, you will have to manually eyeball the resulting dump
22-
diff for version-specific differences, as explained below.
12+
Testing an upgrade from a different version requires a dump to set up
13+
the contents of this instance, with its set of binaries. The following
14+
variables are available to control the test (see DETAILS below about
15+
the creation of the dump):
16+
export olddump=...somewhere/dump.sql (old version's dump)
17+
export oldinstall=...otherversion/ (old version's install base path)
2318

19+
Finally, the tests can be done by running
20+
make check
2421

2522
DETAILS
2623
-------
@@ -29,51 +26,22 @@ The most effective way to test pg_upgrade, aside from testing on user
2926
data, is by upgrading the PostgreSQL regression database.
3027

3128
This testing process first requires the creation of a valid regression
32-
database dump. Such files contain most database features and are
33-
specific to each major version of Postgres.
29+
database dump that can be then used for $olddump. Such files contain
30+
most database features and are specific to each major version of Postgres.
3431

35-
Here are the steps needed to create a regression database dump file:
32+
Here are the steps needed to create a dump file:
3633

3734
1) Create and populate the regression database in the old cluster.
3835
This database can be created by running 'make installcheck' from
39-
src/test/regress.
40-
41-
2) Use pg_dump to dump out the regression database. Use the new
42-
cluster's pg_dump on the old database to minimize whitespace
43-
differences in the diff.
44-
45-
3) Adjust the regression database dump file
46-
47-
a) Perform the load/dump twice
48-
This fixes problems with the ordering of COPY columns for
49-
inherited tables.
50-
51-
b) Change CREATE FUNCTION shared object paths to use '$libdir'
52-
The old and new cluster will have different shared object paths.
53-
54-
c) Fix any wrapping format differences
55-
Commands like CREATE TRIGGER and ALTER TABLE sometimes have
56-
differences.
57-
58-
Once the dump is created, it can be repeatedly loaded into the old
59-
database, upgraded, and dumped out of the new database, and then
60-
compared to the original version. To test the dump file, perform these
61-
steps:
62-
63-
1) Create the old and new clusters in different directories.
64-
65-
2) Copy the regression shared object files into the appropriate /lib
66-
directory for old and new clusters.
67-
68-
3) Create the regression database in the old server.
69-
70-
4) Load the dump file created above into the regression database;
71-
check for errors while loading.
72-
73-
5) Upgrade the old database to the new major version, as outlined in
74-
the pg_upgrade manual section.
75-
76-
6) Use pg_dump to dump out the regression database in the new cluster.
77-
78-
7) Diff the regression database dump file with the regression dump
79-
file loaded into the old server.
36+
src/test/regress using its source code tree.
37+
38+
2) Use pg_dumpall to dump out the contents of the instance, including the
39+
regression database, in the shape of a SQL file. This requires the *old*
40+
cluster's pg_dumpall so as the dump created is compatible with the
41+
version of the cluster it is dumped into.
42+
43+
Once the dump is created, it can be repeatedly used with $olddump and
44+
`make check`, that automates the dump of the old database, its upgrade,
45+
the dump out of the new database and the comparison of the dumps between
46+
the old and new databases. The contents of the dumps can also be manually
47+
compared.

‎src/bin/pg_upgrade/t/001_basic.pl

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
use strict;
2+
use warnings;
3+
4+
use PostgreSQL::Test::Utils;
5+
use Test::More;
6+
7+
program_help_ok('pg_upgrade');
8+
program_version_ok('pg_upgrade');
9+
program_options_handling_ok('pg_upgrade');
10+
11+
done_testing();

‎src/bin/pg_upgrade/t/002_pg_upgrade.pl

+237
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,237 @@
1+
# Set of tests for pg_upgrade, including cross-version checks.
2+
use strict;
3+
use warnings;
4+
5+
use Cwd qw(abs_path getcwd);
6+
use File::Basename qw(dirname);
7+
8+
use PostgreSQL::Test::Cluster;
9+
use PostgreSQL::Test::Utils;
10+
use Test::More;
11+
12+
# Generate a database with a name made of a range of ASCII characters.
13+
sub generate_db
14+
{
15+
my ($node, $from_char, $to_char) = @_;
16+
17+
my $dbname = '';
18+
for my $i ($from_char .. $to_char)
19+
{
20+
next if $i == 7 || $i == 10 || $i == 13; # skip BEL, LF, and CR
21+
$dbname = $dbname . sprintf('%c', $i);
22+
}
23+
$node->run_log(
24+
[ 'createdb', '--host', $node->host, '--port', $node->port, $dbname ]
25+
);
26+
}
27+
28+
# The test of pg_upgrade requires two clusters, an old one and a new one
29+
# that gets upgraded. Before running the upgrade, a logical dump of the
30+
# old cluster is taken, and a second logical dump of the new one is taken
31+
# after the upgrade. The upgrade test passes if there are no differences
32+
# in these two dumps.
33+
34+
# Testing upgrades with an older version of PostgreSQL requires setting up
35+
# two environment variables, as of:
36+
# - "olddump", to point to a dump file that will be used to set up the old
37+
# instance to upgrade from.
38+
# - "oldinstall", to point to the installation path of the old cluster.
39+
if ( (defined($ENV{olddump}) && !defined($ENV{oldinstall}))
40+
|| (!defined($ENV{olddump}) && defined($ENV{oldinstall})))
41+
{
42+
# Not all variables are defined, so leave and die if test is
43+
# done with an older installation.
44+
die "olddump or oldinstall is undefined";
45+
}
46+
47+
# Temporary location for the dumps taken
48+
my $tempdir = PostgreSQL::Test::Utils::tempdir;
49+
50+
# Initialize node to upgrade
51+
my $oldnode = PostgreSQL::Test::Cluster->new('old_node',
52+
install_path => $ENV{oldinstall});
53+
54+
# To increase coverage of non-standard segment size and group access without
55+
# increasing test runtime, run these tests with a custom setting.
56+
# --allow-group-access and --wal-segsize have been added in v11.
57+
$oldnode->init(extra => [ '--wal-segsize', '1', '--allow-group-access' ]);
58+
$oldnode->start;
59+
60+
# The default location of the source code is the root of this directory.
61+
my $srcdir = abs_path("../../..");
62+
63+
# Set up the data of the old instance with a dump or pg_regress.
64+
if (defined($ENV{olddump}))
65+
{
66+
# Use the dump specified.
67+
my $olddumpfile = $ENV{olddump};
68+
die "no dump file found!" unless -e $olddumpfile;
69+
70+
# Load the dump using the "postgres" database as "regression" does
71+
# not exist yet, and we are done here.
72+
$oldnode->command_ok(
73+
[
74+
'psql', '-X', '-f', $olddumpfile,
75+
'--port', $oldnode->port, '--host', $oldnode->host,
76+
'postgres'
77+
]);
78+
}
79+
else
80+
{
81+
# Default is to use pg_regress to set up the old instance.
82+
83+
# Create databases with names covering most ASCII bytes
84+
generate_db($oldnode, 1, 45);
85+
generate_db($oldnode, 46, 90);
86+
generate_db($oldnode, 91, 127);
87+
88+
# Grab any regression options that may be passed down by caller.
89+
my $extra_opts_val = $ENV{EXTRA_REGRESS_OPT} || "";
90+
my @extra_opts = split(/\s+/, $extra_opts_val);
91+
92+
# --dlpath is needed to be able to find the location of regress.so
93+
# and any libraries the regression tests require.
94+
my $dlpath = dirname($ENV{REGRESS_SHLIB});
95+
96+
# --outputdir points to the path where to place the output files.
97+
my $outputdir = $PostgreSQL::Test::Utils::tmp_check;
98+
99+
# --inputdir points to the path of the input files.
100+
my $inputdir = "$srcdir/src/test/regress";
101+
102+
my @regress_command = [
103+
$ENV{PG_REGRESS}, @extra_opts,
104+
'--dlpath', $dlpath,
105+
'--max-concurrent-tests', '20',
106+
'--bindir=', '--host',
107+
$oldnode->host, '--port',
108+
$oldnode->port, '--schedule',
109+
"$srcdir/src/test/regress/parallel_schedule", '--outputdir',
110+
$outputdir, '--inputdir',
111+
$inputdir
112+
];
113+
114+
$oldnode->command_ok(@regress_command,
115+
'regression test run on old instance');
116+
}
117+
118+
# Before dumping, get rid of objects not existing or not supported in later
119+
# versions. This depends on the version of the old server used, and matters
120+
# only if different major versions are used for the dump.
121+
if (defined($ENV{oldinstall}))
122+
{
123+
# Note that upgrade_adapt.sql from the new version is used, to
124+
# cope with an upgrade to this version.
125+
$oldnode->run_log(
126+
[
127+
'psql', '-X',
128+
'-f', "$srcdir/src/bin/pg_upgrade/upgrade_adapt.sql",
129+
'--port', $oldnode->port,
130+
'--host', $oldnode->host,
131+
'regression'
132+
]);
133+
}
134+
135+
# Initialize a new node for the upgrade.
136+
my $newnode = PostgreSQL::Test::Cluster->new('new_node');
137+
$newnode->init(extra => [ '--wal-segsize', '1', '--allow-group-access' ]);
138+
my $newbindir = $newnode->config_data('--bindir');
139+
my $oldbindir = $oldnode->config_data('--bindir');
140+
141+
# Take a dump before performing the upgrade as a base comparison. Note
142+
# that we need to use pg_dumpall from the new node here.
143+
$newnode->command_ok(
144+
[
145+
'pg_dumpall', '--no-sync',
146+
'-d', $oldnode->connstr('postgres'),
147+
'-f', "$tempdir/dump1.sql"
148+
],
149+
'dump before running pg_upgrade');
150+
151+
# After dumping, update references to the old source tree's regress.so
152+
# to point to the new tree.
153+
if (defined($ENV{oldinstall}))
154+
{
155+
# First, fetch all the references to libraries that are not part
156+
# of the default path $libdir.
157+
my $output = $oldnode->safe_psql('regression',
158+
"SELECT DISTINCT probin::text FROM pg_proc WHERE probin NOT LIKE '\$libdir%';"
159+
);
160+
chomp($output);
161+
my @libpaths = split("\n", $output);
162+
163+
my $dump_data = slurp_file("$tempdir/dump1.sql");
164+
165+
my $newregresssrc = "$srcdir/src/test/regress";
166+
foreach (@libpaths)
167+
{
168+
my $libpath = $_;
169+
$libpath = dirname($libpath);
170+
$dump_data =~ s/$libpath/$newregresssrc/g;
171+
}
172+
173+
open my $fh, ">", "$tempdir/dump1.sql" or die "could not open dump file";
174+
print $fh $dump_data;
175+
close $fh;
176+
177+
# This replaces any references to the old tree's regress.so
178+
# the new tree's regress.so. Any references that do *not*
179+
# match $libdir are switched so as this request does not
180+
# depend on the path of the old source tree. This is useful
181+
# when using an old dump. Do the operation on all the databases
182+
# that allow connections so as this includes the regression
183+
# database and anything the user has set up.
184+
$output = $oldnode->safe_psql('postgres',
185+
"SELECT datname FROM pg_database WHERE datallowconn;");
186+
chomp($output);
187+
my @datnames = split("\n", $output);
188+
foreach (@datnames)
189+
{
190+
my $datname = $_;
191+
$oldnode->safe_psql(
192+
$datname, "UPDATE pg_proc SET probin =
193+
regexp_replace(probin, '.*/', '$newregresssrc/')
194+
WHERE probin NOT LIKE '\$libdir/%'");
195+
}
196+
}
197+
198+
# Upgrade the instance.
199+
$oldnode->stop;
200+
command_ok(
201+
[
202+
'pg_upgrade', '--no-sync', '-d', $oldnode->data_dir,
203+
'-D', $newnode->data_dir, '-b', $oldbindir,
204+
'-B', $newbindir, '-p', $oldnode->port,
205+
'-P', $newnode->port
206+
],
207+
'run of pg_upgrade for new instance');
208+
$newnode->start;
209+
210+
# Check if there are any logs coming from pg_upgrade, that would only be
211+
# retained on failure.
212+
my $log_path = $newnode->data_dir . "/pg_upgrade_output.d/log";
213+
if (-d $log_path)
214+
{
215+
foreach my $log (glob("$log_path/*"))
216+
{
217+
note "###########################";
218+
note "Contents of log file $log";
219+
note "###########################";
220+
my $log_contents = slurp_file($log);
221+
print "$log_contents\n";
222+
}
223+
}
224+
225+
# Second dump from the upgraded instance.
226+
$newnode->run_log(
227+
[
228+
'pg_dumpall', '--no-sync',
229+
'-d', $newnode->connstr('postgres'),
230+
'-f', "$tempdir/dump2.sql"
231+
]);
232+
233+
# Compare the two dumps, there should be no differences.
234+
command_ok([ 'diff', '-q', "$tempdir/dump1.sql", "$tempdir/dump2.sql" ],
235+
'old and new dump match after pg_upgrade');
236+
237+
done_testing();

0 commit comments

Comments
 (0)