Skip to content

Commit bb16aba

Browse files
macdicemacdice
committed
Enable parallel query with SERIALIZABLE isolation.
Previously, the SERIALIZABLE isolation level prevented parallel query from being used. Allow the two features to be used together by sharing the leader's SERIALIZABLEXACT with parallel workers. An extra per-SERIALIZABLEXACT LWLock is introduced to make it safe to share, and new logic is introduced to coordinate the early release of the SERIALIZABLEXACT required for the SXACT_FLAG_RO_SAFE optimization, as follows: The first backend to observe the SXACT_FLAG_RO_SAFE flag (set by some other transaction) will 'partially release' the SERIALIZABLEXACT, meaning that the conflicts and locks it holds are released, but the SERIALIZABLEXACT itself will remain active because other backends might still have a pointer to it. Whenever any backend notices the SXACT_FLAG_RO_SAFE flag, it clears its own MySerializableXact variable and frees local resources so that it can skip SSI checks for the rest of the transaction. In the special case of the leader process, it transfers the SERIALIZABLEXACT to a new variable SavedSerializableXact, so that it can be completely released at the end of the transaction after all workers have exited. Remove the serializable_okay flag added to CreateParallelContext() by commit 9da0cc3, because it's now redundant. Author: Thomas Munro Reviewed-by: Haribabu Kommi, Robert Haas, Masahiko Sawada, Kevin Grittner Discussion: https://postgr.es/m/CAEepm=0gXGYhtrVDWOTHS8SQQy_=S9xo+8oCxGLWZAOoeJ=yzQ@mail.gmail.com
1 parent 13e8643 commit bb16aba

File tree

19 files changed

+429
-67
lines changed

19 files changed

+429
-67
lines changed

‎doc/src/sgml/monitoring.sgml

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -861,7 +861,7 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
861861

862862
<tbody>
863863
<row>
864-
<entry morerows="63"><literal>LWLock</literal></entry>
864+
<entry morerows="64"><literal>LWLock</literal></entry>
865865
<entry><literal>ShmemIndexLock</literal></entry>
866866
<entry>Waiting to find or allocate space in shared memory.</entry>
867867
</row>
@@ -1121,6 +1121,11 @@ postgres 27093 0.0 0.0 30096 2752 ? Ss 11:34 0:00 postgres: ser
11211121
<entry><literal>predicate_lock_manager</literal></entry>
11221122
<entry>Waiting to add or examine predicate lock information.</entry>
11231123
</row>
1124+
<row>
1125+
<entry><literal>serializable_xact</literal></entry>
1126+
<entry>Waiting to perform an operation on a serializable transaction
1127+
in a parallel query.</entry>
1128+
</row>
11241129
<row>
11251130
<entry><literal>parallel_query_dsa</literal></entry>
11261131
<entry>Waiting for parallel query dynamic shared memory allocation lock.</entry>

‎doc/src/sgml/parallel.sgml

Lines changed: 0 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -184,13 +184,6 @@ EXPLAIN SELECT * FROM pgbench_accounts WHERE filler LIKE '%x%';
184184
using a very large number of processes.
185185
</para>
186186
</listitem>
187-
188-
<listitem>
189-
<para>
190-
The transaction isolation level is serializable. This is
191-
a limitation of the current implementation.
192-
</para>
193-
</listitem>
194187
</itemizedlist>
195188

196189
<para>
@@ -233,16 +226,6 @@ EXPLAIN SELECT * FROM pgbench_accounts WHERE filler LIKE '%x%';
233226
that may be suboptimal when run serially.
234227
</para>
235228
</listitem>
236-
237-
<listitem>
238-
<para>
239-
The transaction isolation level is serializable. This situation
240-
does not normally arise, because parallel query plans are not
241-
generated when the transaction isolation level is serializable.
242-
However, it can happen if the transaction isolation level is changed to
243-
serializable after the plan is generated and before it is executed.
244-
</para>
245-
</listitem>
246229
</itemizedlist>
247230
</sect1>
248231

‎src/backend/access/nbtree/nbtsort.c

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1265,7 +1265,7 @@ _bt_begin_parallel(BTBuildState *buildstate, bool isconcurrent, int request)
12651265
EnterParallelMode();
12661266
Assert(request > 0);
12671267
pcxt = CreateParallelContext("postgres", "_bt_parallel_build_main",
1268-
request, true);
1268+
request);
12691269
scantuplesortstates = leaderparticipates ? request + 1 : request;
12701270

12711271
/*

‎src/backend/access/transam/parallel.c

Lines changed: 7 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,7 @@
3131
#include "optimizer/optimizer.h"
3232
#include "pgstat.h"
3333
#include "storage/ipc.h"
34+
#include "storage/predicate.h"
3435
#include "storage/sinval.h"
3536
#include "storage/spin.h"
3637
#include "tcop/tcopprot.h"
@@ -91,6 +92,7 @@ typedef struct FixedParallelState
9192
BackendId parallel_master_backend_id;
9293
TimestampTz xact_ts;
9394
TimestampTz stmt_ts;
95+
SerializableXactHandle serializable_xact_handle;
9496

9597
/* Mutex protects remaining fields. */
9698
slock_t mutex;
@@ -155,7 +157,7 @@ static void ParallelWorkerShutdown(int code, Datum arg);
155157
*/
156158
ParallelContext *
157159
CreateParallelContext(const char *library_name, const char *function_name,
158-
int nworkers, bool serializable_okay)
160+
int nworkers)
159161
{
160162
MemoryContext oldcontext;
161163
ParallelContext *pcxt;
@@ -166,16 +168,6 @@ CreateParallelContext(const char *library_name, const char *function_name,
166168
/* Number of workers should be non-negative. */
167169
Assert(nworkers >= 0);
168170

169-
/*
170-
* If we are running under serializable isolation, we can't use parallel
171-
* workers, at least not until somebody enhances that mechanism to be
172-
* parallel-aware. Utility statement callers may ask us to ignore this
173-
* restriction because they're always able to safely ignore the fact that
174-
* SIREAD locks do not work with parallelism.
175-
*/
176-
if (IsolationIsSerializable() && !serializable_okay)
177-
nworkers = 0;
178-
179171
/* We might be running in a short-lived memory context. */
180172
oldcontext = MemoryContextSwitchTo(TopTransactionContext);
181173

@@ -327,6 +319,7 @@ InitializeParallelDSM(ParallelContext *pcxt)
327319
fps->parallel_master_backend_id = MyBackendId;
328320
fps->xact_ts = GetCurrentTransactionStartTimestamp();
329321
fps->stmt_ts = GetCurrentStatementStartTimestamp();
322+
fps->serializable_xact_handle = ShareSerializableXact();
330323
SpinLockInit(&fps->mutex);
331324
fps->last_xlog_end = 0;
332325
shm_toc_insert(pcxt->toc, PARALLEL_KEY_FIXED, fps);
@@ -1422,6 +1415,9 @@ ParallelWorkerMain(Datum main_arg)
14221415
false);
14231416
RestoreEnumBlacklist(enumblacklistspace);
14241417

1418+
/* Attach to the leader's serializable transaction, if SERIALIZABLE. */
1419+
AttachSerializableXact(fps->serializable_xact_handle);
1420+
14251421
/*
14261422
* We've initialized all of our state now; nothing should change
14271423
* hereafter.

‎src/backend/access/transam/xact.c

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2024,9 +2024,12 @@ CommitTransaction(void)
20242024
/*
20252025
* Mark serializable transaction as complete for predicate locking
20262026
* purposes. This should be done as late as we can put it and still allow
2027-
* errors to be raised for failure patterns found at commit.
2027+
* errors to be raised for failure patterns found at commit. This is not
2028+
* appropriate in a parallel worker however, because we aren't committing
2029+
* the leader's transaction and its serializable state will live on.
20282030
*/
2029-
PreCommit_CheckForSerializationFailure();
2031+
if (!is_parallel_worker)
2032+
PreCommit_CheckForSerializationFailure();
20302033

20312034
/*
20322035
* Insert notifications sent by NOTIFY commands into the queue. This

‎src/backend/executor/execParallel.c

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -604,7 +604,7 @@ ExecInitParallelPlan(PlanState *planstate, EState *estate,
604604
pstmt_data = ExecSerializePlan(planstate->plan, estate);
605605

606606
/* Create a parallel context. */
607-
pcxt = CreateParallelContext("postgres", "ParallelQueryMain", nworkers, false);
607+
pcxt = CreateParallelContext("postgres", "ParallelQueryMain", nworkers);
608608
pei->pcxt = pcxt;
609609

610610
/*

‎src/backend/optimizer/plan/planner.c

Lines changed: 1 addition & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -337,22 +337,13 @@ standard_planner(Query *parse, int cursorOptions, ParamListInfo boundParams)
337337
* parallel worker. We might eventually be able to relax this
338338
* restriction, but for now it seems best not to have parallel workers
339339
* trying to create their own parallel workers.
340-
*
341-
* We can't use parallelism in serializable mode because the predicate
342-
* locking code is not parallel-aware. It's not catastrophic if someone
343-
* tries to run a parallel plan in serializable mode; it just won't get
344-
* any workers and will run serially. But it seems like a good heuristic
345-
* to assume that the same serialization level will be in effect at plan
346-
* time and execution time, so don't generate a parallel plan if we're in
347-
* serializable mode.
348340
*/
349341
if ((cursorOptions & CURSOR_OPT_PARALLEL_OK) != 0 &&
350342
IsUnderPostmaster &&
351343
parse->commandType == CMD_SELECT &&
352344
!parse->hasModifyingCTE &&
353345
max_parallel_workers_per_gather > 0 &&
354-
!IsParallelWorker() &&
355-
!IsolationIsSerializable())
346+
!IsParallelWorker())
356347
{
357348
/* all the cheap tests pass, so scan the query tree */
358349
glob->maxParallelHazard = max_parallel_hazard(parse);

‎src/backend/storage/lmgr/lwlock.c

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -521,6 +521,7 @@ RegisterLWLockTranches(void)
521521
LWLockRegisterTranche(LWTRANCHE_TBM, "tbm");
522522
LWLockRegisterTranche(LWTRANCHE_PARALLEL_APPEND, "parallel_append");
523523
LWLockRegisterTranche(LWTRANCHE_PARALLEL_HASH_JOIN, "parallel_hash_join");
524+
LWLockRegisterTranche(LWTRANCHE_SXACT, "serializable_xact");
524525

525526
/* Register named tranches. */
526527
for (i = 0; i < NamedLWLockTrancheRequests; i++)

0 commit comments

Comments
 (0)