Skip to content

Commit ad79180

Browse files
anarazelanarazel
committed
Run catalog reindexing test from 3dbb317 serially, to avoid deadlocks.
The tests turn out to cause deadlocks in some circumstances. Fairly reproducibly so with -DRELCACHE_FORCE_RELEASE -DCATCACHE_FORCE_RELEASE. Some of the deadlocks may be hard to fix without disproportionate measures, but others probably should be fixed - but not in 12. We discussed removing the new tests until we can fix the issues underlying the deadlocks, but results from buildfarm animal markhor (which runs with CLOBBER_CACHE_ALWAYS) indicates that there might be a more severe, as of yet undiagnosed, issue (including on stable branches) with reindexing catalogs. The failure is: ERROR: could not read block 0 in file "base/16384/28025": read only 0 of 8192 bytes Therefore it seems advisable to keep the tests. It's not certain that running the tests in isolation removes the risk of deadlocks. It's possible that additional locks are needed to protect against a concurrent auto-analyze or such. Per discussion with Tom Lane. Discussion: https://postgr.es/m/28926.1556664156@sss.pgh.pa.us Backpatch: 9.4-, like 3dbb317
1 parent d8d5e1a commit ad79180

File tree

6 files changed

+75
-39
lines changed

6 files changed

+75
-39
lines changed

‎src/test/regress/expected/create_index.out

Lines changed: 0 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -2998,24 +2998,6 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
29982998
INFO: index "reindex_verbose_pkey" was reindexed
29992999
DROP TABLE reindex_verbose;
30003000
--
3001-
-- check that system tables can be reindexed
3002-
--
3003-
-- whole tables
3004-
REINDEX TABLE pg_class; -- mapped, non-shared, critical
3005-
REINDEX TABLE pg_index; -- non-mapped, non-shared, critical
3006-
REINDEX TABLE pg_operator; -- non-mapped, non-shared, critical
3007-
REINDEX TABLE pg_database; -- mapped, shared, critical
3008-
REINDEX TABLE pg_shdescription; -- mapped, shared non-critical
3009-
-- Check that individual system indexes can be reindexed. That's a bit
3010-
-- different from the entire-table case because reindex_relation
3011-
-- treats e.g. pg_class special.
3012-
REINDEX INDEX pg_class_oid_index; -- mapped, non-shared, critical
3013-
REINDEX INDEX pg_class_relname_nsp_index; -- mapped, non-shared, non-critical
3014-
REINDEX INDEX pg_index_indexrelid_index; -- non-mapped, non-shared, critical
3015-
REINDEX INDEX pg_index_indrelid_index; -- non-mapped, non-shared, non-critical
3016-
REINDEX INDEX pg_database_oid_index; -- mapped, shared, critical
3017-
REINDEX INDEX pg_shdescription_o_c_index; -- mapped, shared, non-critical
3018-
--
30193001
-- REINDEX SCHEMA
30203002
--
30213003
REINDEX SCHEMA schema_to_reindex; -- failure, schema does not exist

‎src/test/regress/expected/reindex_catalog.out

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
--
2+
-- Check that system tables can be reindexed.
3+
--
4+
-- Note that this test currently has to run without parallel tests
5+
-- being scheduled, as currently reindex catalog tables can cause
6+
-- deadlocks:
7+
--
8+
-- * The lock upgrade between the ShareLock acquired for the reindex
9+
-- and RowExclusiveLock needed for pg_class/pg_index locks can
10+
-- trigger deadlocks.
11+
--
12+
-- * The uniqueness checks performed when reindexing a unique/primary
13+
-- key index possibly need to wait for the transaction of a
14+
-- about-to-deleted row in pg_class to commit. That can cause
15+
-- deadlocks because, in contrast to user tables, locks on catalog
16+
-- tables are routinely released before commit - therefore the lock
17+
-- held for reindexing doesn't guarantee that no running transaction
18+
-- performed modifications in the table underlying the index.
19+
-- Check reindexing of whole tables
20+
REINDEX TABLE pg_class; -- mapped, non-shared, critical
21+
REINDEX TABLE pg_index; -- non-mapped, non-shared, critical
22+
REINDEX TABLE pg_operator; -- non-mapped, non-shared, critical
23+
REINDEX TABLE pg_database; -- mapped, shared, critical
24+
REINDEX TABLE pg_shdescription; -- mapped, shared non-critical
25+
-- Check that individual system indexes can be reindexed. That's a bit
26+
-- different from the entire-table case because reindex_relation
27+
-- treats e.g. pg_class special.
28+
REINDEX INDEX pg_class_oid_index; -- mapped, non-shared, critical
29+
REINDEX INDEX pg_class_relname_nsp_index; -- mapped, non-shared, non-critical
30+
REINDEX INDEX pg_index_indexrelid_index; -- non-mapped, non-shared, critical
31+
REINDEX INDEX pg_index_indrelid_index; -- non-mapped, non-shared, non-critical
32+
REINDEX INDEX pg_database_oid_index; -- mapped, shared, critical
33+
REINDEX INDEX pg_shdescription_o_c_index; -- mapped, shared, non-critical

‎src/test/regress/parallel_schedule

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,11 @@ test: create_misc create_operator
5757
# These depend on the above two
5858
test: create_index create_view
5959

60+
# ----------
61+
# Has to run in isolation, due to deadlock risk
62+
# ----------
63+
test: reindex_catalog
64+
6065
# ----------
6166
# Another group of parallel tests
6267
# ----------

‎src/test/regress/serial_schedule

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,7 @@ test: create_misc
6565
test: create_operator
6666
test: create_index
6767
test: create_view
68+
test: reindex_catalog
6869
test: create_aggregate
6970
test: create_function_3
7071
test: create_cast

‎src/test/regress/sql/create_index.sql

Lines changed: 0 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1033,27 +1033,6 @@ CREATE TABLE reindex_verbose(id integer primary key);
10331033
REINDEX (VERBOSE) TABLE reindex_verbose;
10341034
DROP TABLE reindex_verbose;
10351035

1036-
--
1037-
-- check that system tables can be reindexed
1038-
--
1039-
1040-
-- whole tables
1041-
REINDEX TABLE pg_class; -- mapped, non-shared, critical
1042-
REINDEX TABLE pg_index; -- non-mapped, non-shared, critical
1043-
REINDEX TABLE pg_operator; -- non-mapped, non-shared, critical
1044-
REINDEX TABLE pg_database; -- mapped, shared, critical
1045-
REINDEX TABLE pg_shdescription; -- mapped, shared non-critical
1046-
1047-
-- Check that individual system indexes can be reindexed. That's a bit
1048-
-- different from the entire-table case because reindex_relation
1049-
-- treats e.g. pg_class special.
1050-
REINDEX INDEX pg_class_oid_index; -- mapped, non-shared, critical
1051-
REINDEX INDEX pg_class_relname_nsp_index; -- mapped, non-shared, non-critical
1052-
REINDEX INDEX pg_index_indexrelid_index; -- non-mapped, non-shared, critical
1053-
REINDEX INDEX pg_index_indrelid_index; -- non-mapped, non-shared, non-critical
1054-
REINDEX INDEX pg_database_oid_index; -- mapped, shared, critical
1055-
REINDEX INDEX pg_shdescription_o_c_index; -- mapped, shared, non-critical
1056-
10571036
--
10581037
-- REINDEX SCHEMA
10591038
--

‎src/test/regress/sql/reindex_catalog.sql

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
--
2+
-- Check that system tables can be reindexed.
3+
--
4+
-- Note that this test currently has to run without parallel tests
5+
-- being scheduled, as currently reindex catalog tables can cause
6+
-- deadlocks:
7+
--
8+
-- * The lock upgrade between the ShareLock acquired for the reindex
9+
-- and RowExclusiveLock needed for pg_class/pg_index locks can
10+
-- trigger deadlocks.
11+
--
12+
-- * The uniqueness checks performed when reindexing a unique/primary
13+
-- key index possibly need to wait for the transaction of a
14+
-- about-to-deleted row in pg_class to commit. That can cause
15+
-- deadlocks because, in contrast to user tables, locks on catalog
16+
-- tables are routinely released before commit - therefore the lock
17+
-- held for reindexing doesn't guarantee that no running transaction
18+
-- performed modifications in the table underlying the index.
19+
20+
21+
-- Check reindexing of whole tables
22+
REINDEX TABLE pg_class; -- mapped, non-shared, critical
23+
REINDEX TABLE pg_index; -- non-mapped, non-shared, critical
24+
REINDEX TABLE pg_operator; -- non-mapped, non-shared, critical
25+
REINDEX TABLE pg_database; -- mapped, shared, critical
26+
REINDEX TABLE pg_shdescription; -- mapped, shared non-critical
27+
28+
-- Check that individual system indexes can be reindexed. That's a bit
29+
-- different from the entire-table case because reindex_relation
30+
-- treats e.g. pg_class special.
31+
REINDEX INDEX pg_class_oid_index; -- mapped, non-shared, critical
32+
REINDEX INDEX pg_class_relname_nsp_index; -- mapped, non-shared, non-critical
33+
REINDEX INDEX pg_index_indexrelid_index; -- non-mapped, non-shared, critical
34+
REINDEX INDEX pg_index_indrelid_index; -- non-mapped, non-shared, non-critical
35+
REINDEX INDEX pg_database_oid_index; -- mapped, shared, critical
36+
REINDEX INDEX pg_shdescription_o_c_index; -- mapped, shared, non-critical

0 commit comments

Comments
 (0)