Skip to content

Commit 809c9b4

Browse files
anarazelanarazel
committed
Run catalog reindexing test from 3dbb317 serially, to avoid deadlocks.
The tests turn out to cause deadlocks in some circumstances. Fairly reproducibly so with -DRELCACHE_FORCE_RELEASE -DCATCACHE_FORCE_RELEASE. Some of the deadlocks may be hard to fix without disproportionate measures, but others probably should be fixed - but not in 12. We discussed removing the new tests until we can fix the issues underlying the deadlocks, but results from buildfarm animal markhor (which runs with CLOBBER_CACHE_ALWAYS) indicates that there might be a more severe, as of yet undiagnosed, issue (including on stable branches) with reindexing catalogs. The failure is: ERROR: could not read block 0 in file "base/16384/28025": read only 0 of 8192 bytes Therefore it seems advisable to keep the tests. It's not certain that running the tests in isolation removes the risk of deadlocks. It's possible that additional locks are needed to protect against a concurrent auto-analyze or such. Per discussion with Tom Lane. Discussion: https://postgr.es/m/28926.1556664156@sss.pgh.pa.us Backpatch: 9.4-, like 3dbb317
1 parent 4b40d40 commit 809c9b4

File tree

6 files changed

+75
-39
lines changed

6 files changed

+75
-39
lines changed

‎src/test/regress/expected/create_index.out

Lines changed: 0 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1939,24 +1939,6 @@ INFO: index "reindex_verbose_pkey" was reindexed
19391939
\set VERBOSITY default
19401940
DROP TABLE reindex_verbose;
19411941
--
1942-
-- check that system tables can be reindexed
1943-
--
1944-
-- whole tables
1945-
REINDEX TABLE pg_class; -- mapped, non-shared, critical
1946-
REINDEX TABLE pg_index; -- non-mapped, non-shared, critical
1947-
REINDEX TABLE pg_operator; -- non-mapped, non-shared, critical
1948-
REINDEX TABLE pg_database; -- mapped, shared, critical
1949-
REINDEX TABLE pg_shdescription; -- mapped, shared non-critical
1950-
-- Check that individual system indexes can be reindexed. That's a bit
1951-
-- different from the entire-table case because reindex_relation
1952-
-- treats e.g. pg_class special.
1953-
REINDEX INDEX pg_class_oid_index; -- mapped, non-shared, critical
1954-
REINDEX INDEX pg_class_relname_nsp_index; -- mapped, non-shared, non-critical
1955-
REINDEX INDEX pg_index_indexrelid_index; -- non-mapped, non-shared, critical
1956-
REINDEX INDEX pg_index_indrelid_index; -- non-mapped, non-shared, non-critical
1957-
REINDEX INDEX pg_database_oid_index; -- mapped, shared, critical
1958-
REINDEX INDEX pg_shdescription_o_c_index; -- mapped, shared, non-critical
1959-
--
19601942
-- REINDEX CONCURRENTLY
19611943
--
19621944
CREATE TABLE concur_reindex_tab (c1 int);

‎src/test/regress/expected/reindex_catalog.out

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
--
2+
-- Check that system tables can be reindexed.
3+
--
4+
-- Note that this test currently has to run without parallel tests
5+
-- being scheduled, as currently reindex catalog tables can cause
6+
-- deadlocks:
7+
--
8+
-- * The lock upgrade between the ShareLock acquired for the reindex
9+
-- and RowExclusiveLock needed for pg_class/pg_index locks can
10+
-- trigger deadlocks.
11+
--
12+
-- * The uniqueness checks performed when reindexing a unique/primary
13+
-- key index possibly need to wait for the transaction of a
14+
-- about-to-deleted row in pg_class to commit. That can cause
15+
-- deadlocks because, in contrast to user tables, locks on catalog
16+
-- tables are routinely released before commit - therefore the lock
17+
-- held for reindexing doesn't guarantee that no running transaction
18+
-- performed modifications in the table underlying the index.
19+
-- Check reindexing of whole tables
20+
REINDEX TABLE pg_class; -- mapped, non-shared, critical
21+
REINDEX TABLE pg_index; -- non-mapped, non-shared, critical
22+
REINDEX TABLE pg_operator; -- non-mapped, non-shared, critical
23+
REINDEX TABLE pg_database; -- mapped, shared, critical
24+
REINDEX TABLE pg_shdescription; -- mapped, shared non-critical
25+
-- Check that individual system indexes can be reindexed. That's a bit
26+
-- different from the entire-table case because reindex_relation
27+
-- treats e.g. pg_class special.
28+
REINDEX INDEX pg_class_oid_index; -- mapped, non-shared, critical
29+
REINDEX INDEX pg_class_relname_nsp_index; -- mapped, non-shared, non-critical
30+
REINDEX INDEX pg_index_indexrelid_index; -- non-mapped, non-shared, critical
31+
REINDEX INDEX pg_index_indrelid_index; -- non-mapped, non-shared, non-critical
32+
REINDEX INDEX pg_database_oid_index; -- mapped, shared, critical
33+
REINDEX INDEX pg_shdescription_o_c_index; -- mapped, shared, non-critical

‎src/test/regress/parallel_schedule

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,11 @@ test: create_misc create_operator create_procedure
5252
# These depend on create_misc and create_operator
5353
test: create_index create_index_spgist create_view index_including index_including_gist
5454

55+
# ----------
56+
# Has to run in isolation, due to deadlock risk
57+
# ----------
58+
test: reindex_catalog
59+
5560
# ----------
5661
# Another group of parallel tests
5762
# ----------

‎src/test/regress/serial_schedule

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,7 @@ test: create_index_spgist
6767
test: create_view
6868
test: index_including
6969
test: index_including_gist
70+
test: reindex_catalog
7071
test: create_aggregate
7172
test: create_function_3
7273
test: create_cast

‎src/test/regress/sql/create_index.sql

Lines changed: 0 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -747,27 +747,6 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
747747
\set VERBOSITY default
748748
DROP TABLE reindex_verbose;
749749

750-
--
751-
-- check that system tables can be reindexed
752-
--
753-
754-
-- whole tables
755-
REINDEX TABLE pg_class; -- mapped, non-shared, critical
756-
REINDEX TABLE pg_index; -- non-mapped, non-shared, critical
757-
REINDEX TABLE pg_operator; -- non-mapped, non-shared, critical
758-
REINDEX TABLE pg_database; -- mapped, shared, critical
759-
REINDEX TABLE pg_shdescription; -- mapped, shared non-critical
760-
761-
-- Check that individual system indexes can be reindexed. That's a bit
762-
-- different from the entire-table case because reindex_relation
763-
-- treats e.g. pg_class special.
764-
REINDEX INDEX pg_class_oid_index; -- mapped, non-shared, critical
765-
REINDEX INDEX pg_class_relname_nsp_index; -- mapped, non-shared, non-critical
766-
REINDEX INDEX pg_index_indexrelid_index; -- non-mapped, non-shared, critical
767-
REINDEX INDEX pg_index_indrelid_index; -- non-mapped, non-shared, non-critical
768-
REINDEX INDEX pg_database_oid_index; -- mapped, shared, critical
769-
REINDEX INDEX pg_shdescription_o_c_index; -- mapped, shared, non-critical
770-
771750
--
772751
-- REINDEX CONCURRENTLY
773752
--

‎src/test/regress/sql/reindex_catalog.sql

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
--
2+
-- Check that system tables can be reindexed.
3+
--
4+
-- Note that this test currently has to run without parallel tests
5+
-- being scheduled, as currently reindex catalog tables can cause
6+
-- deadlocks:
7+
--
8+
-- * The lock upgrade between the ShareLock acquired for the reindex
9+
-- and RowExclusiveLock needed for pg_class/pg_index locks can
10+
-- trigger deadlocks.
11+
--
12+
-- * The uniqueness checks performed when reindexing a unique/primary
13+
-- key index possibly need to wait for the transaction of a
14+
-- about-to-deleted row in pg_class to commit. That can cause
15+
-- deadlocks because, in contrast to user tables, locks on catalog
16+
-- tables are routinely released before commit - therefore the lock
17+
-- held for reindexing doesn't guarantee that no running transaction
18+
-- performed modifications in the table underlying the index.
19+
20+
21+
-- Check reindexing of whole tables
22+
REINDEX TABLE pg_class; -- mapped, non-shared, critical
23+
REINDEX TABLE pg_index; -- non-mapped, non-shared, critical
24+
REINDEX TABLE pg_operator; -- non-mapped, non-shared, critical
25+
REINDEX TABLE pg_database; -- mapped, shared, critical
26+
REINDEX TABLE pg_shdescription; -- mapped, shared non-critical
27+
28+
-- Check that individual system indexes can be reindexed. That's a bit
29+
-- different from the entire-table case because reindex_relation
30+
-- treats e.g. pg_class special.
31+
REINDEX INDEX pg_class_oid_index; -- mapped, non-shared, critical
32+
REINDEX INDEX pg_class_relname_nsp_index; -- mapped, non-shared, non-critical
33+
REINDEX INDEX pg_index_indexrelid_index; -- non-mapped, non-shared, critical
34+
REINDEX INDEX pg_index_indrelid_index; -- non-mapped, non-shared, non-critical
35+
REINDEX INDEX pg_database_oid_index; -- mapped, shared, critical
36+
REINDEX INDEX pg_shdescription_o_c_index; -- mapped, shared, non-critical

0 commit comments

Comments
 (0)