Skip to content

Commit 856bc0c

Browse files
anarazelanarazel
committed
Run catalog reindexing test from 3dbb317 serially, to avoid deadlocks.
The tests turn out to cause deadlocks in some circumstances. Fairly reproducibly so with -DRELCACHE_FORCE_RELEASE -DCATCACHE_FORCE_RELEASE. Some of the deadlocks may be hard to fix without disproportionate measures, but others probably should be fixed - but not in 12. We discussed removing the new tests until we can fix the issues underlying the deadlocks, but results from buildfarm animal markhor (which runs with CLOBBER_CACHE_ALWAYS) indicates that there might be a more severe, as of yet undiagnosed, issue (including on stable branches) with reindexing catalogs. The failure is: ERROR: could not read block 0 in file "base/16384/28025": read only 0 of 8192 bytes Therefore it seems advisable to keep the tests. It's not certain that running the tests in isolation removes the risk of deadlocks. It's possible that additional locks are needed to protect against a concurrent auto-analyze or such. Per discussion with Tom Lane. Discussion: https://postgr.es/m/28926.1556664156@sss.pgh.pa.us Backpatch: 9.4-, like 3dbb317
1 parent 40230f0 commit 856bc0c

File tree

6 files changed

+75
-39
lines changed

6 files changed

+75
-39
lines changed

‎src/test/regress/expected/create_index.out

Lines changed: 0 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -2804,21 +2804,3 @@ explain (costs off)
28042804
Index Cond: ((thousand = 1) AND (tenthous = 1001))
28052805
(2 rows)
28062806

2807-
--
2808-
-- check that system tables can be reindexed
2809-
--
2810-
-- whole tables
2811-
REINDEX TABLE pg_class; -- mapped, non-shared, critical
2812-
REINDEX TABLE pg_index; -- non-mapped, non-shared, critical
2813-
REINDEX TABLE pg_operator; -- non-mapped, non-shared, critical
2814-
REINDEX TABLE pg_database; -- mapped, shared, critical
2815-
REINDEX TABLE pg_shdescription; -- mapped, shared non-critical
2816-
-- Check that individual system indexes can be reindexed. That's a bit
2817-
-- different from the entire-table case because reindex_relation
2818-
-- treats e.g. pg_class special.
2819-
REINDEX INDEX pg_class_oid_index; -- mapped, non-shared, critical
2820-
REINDEX INDEX pg_class_relname_nsp_index; -- mapped, non-shared, non-critical
2821-
REINDEX INDEX pg_index_indexrelid_index; -- non-mapped, non-shared, critical
2822-
REINDEX INDEX pg_index_indrelid_index; -- non-mapped, non-shared, non-critical
2823-
REINDEX INDEX pg_database_oid_index; -- mapped, shared, critical
2824-
REINDEX INDEX pg_shdescription_o_c_index; -- mapped, shared, non-critical

‎src/test/regress/expected/reindex_catalog.out

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
--
2+
-- Check that system tables can be reindexed.
3+
--
4+
-- Note that this test currently has to run without parallel tests
5+
-- being scheduled, as currently reindex catalog tables can cause
6+
-- deadlocks:
7+
--
8+
-- * The lock upgrade between the ShareLock acquired for the reindex
9+
-- and RowExclusiveLock needed for pg_class/pg_index locks can
10+
-- trigger deadlocks.
11+
--
12+
-- * The uniqueness checks performed when reindexing a unique/primary
13+
-- key index possibly need to wait for the transaction of a
14+
-- about-to-deleted row in pg_class to commit. That can cause
15+
-- deadlocks because, in contrast to user tables, locks on catalog
16+
-- tables are routinely released before commit - therefore the lock
17+
-- held for reindexing doesn't guarantee that no running transaction
18+
-- performed modifications in the table underlying the index.
19+
-- Check reindexing of whole tables
20+
REINDEX TABLE pg_class; -- mapped, non-shared, critical
21+
REINDEX TABLE pg_index; -- non-mapped, non-shared, critical
22+
REINDEX TABLE pg_operator; -- non-mapped, non-shared, critical
23+
REINDEX TABLE pg_database; -- mapped, shared, critical
24+
REINDEX TABLE pg_shdescription; -- mapped, shared non-critical
25+
-- Check that individual system indexes can be reindexed. That's a bit
26+
-- different from the entire-table case because reindex_relation
27+
-- treats e.g. pg_class special.
28+
REINDEX INDEX pg_class_oid_index; -- mapped, non-shared, critical
29+
REINDEX INDEX pg_class_relname_nsp_index; -- mapped, non-shared, non-critical
30+
REINDEX INDEX pg_index_indexrelid_index; -- non-mapped, non-shared, critical
31+
REINDEX INDEX pg_index_indrelid_index; -- non-mapped, non-shared, non-critical
32+
REINDEX INDEX pg_database_oid_index; -- mapped, shared, critical
33+
REINDEX INDEX pg_shdescription_o_c_index; -- mapped, shared, non-critical

‎src/test/regress/parallel_schedule

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,11 @@ test: create_misc create_operator
5656
# These depend on the above two
5757
test: create_index create_view
5858

59+
# ----------
60+
# Has to run in isolation, due to deadlock risk
61+
# ----------
62+
test: reindex_catalog
63+
5964
# ----------
6065
# Another group of parallel tests
6166
# ----------

‎src/test/regress/serial_schedule

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,7 @@ test: create_misc
6060
test: create_operator
6161
test: create_index
6262
test: create_view
63+
test: reindex_catalog
6364
test: create_aggregate
6465
test: create_function_3
6566
test: create_cast

‎src/test/regress/sql/create_index.sql

Lines changed: 0 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -951,24 +951,3 @@ RESET enable_indexonlyscan;
951951

952952
explain (costs off)
953953
select * from tenk1 where (thousand, tenthous) in ((1,1001), (null,null));
954-
955-
--
956-
-- check that system tables can be reindexed
957-
--
958-
959-
-- whole tables
960-
REINDEX TABLE pg_class; -- mapped, non-shared, critical
961-
REINDEX TABLE pg_index; -- non-mapped, non-shared, critical
962-
REINDEX TABLE pg_operator; -- non-mapped, non-shared, critical
963-
REINDEX TABLE pg_database; -- mapped, shared, critical
964-
REINDEX TABLE pg_shdescription; -- mapped, shared non-critical
965-
966-
-- Check that individual system indexes can be reindexed. That's a bit
967-
-- different from the entire-table case because reindex_relation
968-
-- treats e.g. pg_class special.
969-
REINDEX INDEX pg_class_oid_index; -- mapped, non-shared, critical
970-
REINDEX INDEX pg_class_relname_nsp_index; -- mapped, non-shared, non-critical
971-
REINDEX INDEX pg_index_indexrelid_index; -- non-mapped, non-shared, critical
972-
REINDEX INDEX pg_index_indrelid_index; -- non-mapped, non-shared, non-critical
973-
REINDEX INDEX pg_database_oid_index; -- mapped, shared, critical
974-
REINDEX INDEX pg_shdescription_o_c_index; -- mapped, shared, non-critical

‎src/test/regress/sql/reindex_catalog.sql

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
--
2+
-- Check that system tables can be reindexed.
3+
--
4+
-- Note that this test currently has to run without parallel tests
5+
-- being scheduled, as currently reindex catalog tables can cause
6+
-- deadlocks:
7+
--
8+
-- * The lock upgrade between the ShareLock acquired for the reindex
9+
-- and RowExclusiveLock needed for pg_class/pg_index locks can
10+
-- trigger deadlocks.
11+
--
12+
-- * The uniqueness checks performed when reindexing a unique/primary
13+
-- key index possibly need to wait for the transaction of a
14+
-- about-to-deleted row in pg_class to commit. That can cause
15+
-- deadlocks because, in contrast to user tables, locks on catalog
16+
-- tables are routinely released before commit - therefore the lock
17+
-- held for reindexing doesn't guarantee that no running transaction
18+
-- performed modifications in the table underlying the index.
19+
20+
21+
-- Check reindexing of whole tables
22+
REINDEX TABLE pg_class; -- mapped, non-shared, critical
23+
REINDEX TABLE pg_index; -- non-mapped, non-shared, critical
24+
REINDEX TABLE pg_operator; -- non-mapped, non-shared, critical
25+
REINDEX TABLE pg_database; -- mapped, shared, critical
26+
REINDEX TABLE pg_shdescription; -- mapped, shared non-critical
27+
28+
-- Check that individual system indexes can be reindexed. That's a bit
29+
-- different from the entire-table case because reindex_relation
30+
-- treats e.g. pg_class special.
31+
REINDEX INDEX pg_class_oid_index; -- mapped, non-shared, critical
32+
REINDEX INDEX pg_class_relname_nsp_index; -- mapped, non-shared, non-critical
33+
REINDEX INDEX pg_index_indexrelid_index; -- non-mapped, non-shared, critical
34+
REINDEX INDEX pg_index_indrelid_index; -- non-mapped, non-shared, non-critical
35+
REINDEX INDEX pg_database_oid_index; -- mapped, shared, critical
36+
REINDEX INDEX pg_shdescription_o_c_index; -- mapped, shared, non-critical

0 commit comments

Comments
 (0)