Skip to content

Commit cbc1279

Browse files
amitlanamitlan
committed
Track unpruned relids to avoid processing pruned relations
This commit introduces changes to track unpruned relations explicitly, making it possible for top-level plan nodes, such as ModifyTable and LockRows, to avoid processing partitions pruned during initial pruning. Scan-level nodes, such as Append and MergeAppend, already avoid the unnecessary processing by accessing partition pruning results directly via part_prune_index. In contrast, top-level nodes cannot access pruning results directly and need to determine which partitions remain unpruned. To address this, this commit introduces a new bitmapset field, es_unpruned_relids, which the executor uses to track the set of unpruned relations. This field is referenced during plan initialization to skip initializing certain nodes for pruned partitions. It is initialized with PlannedStmt.unprunableRelids, a new field that the planner populates with RT indexes of relations that cannot be pruned during runtime pruning. These include relations not subject to partition pruning and those required for execution regardless of pruning. PlannedStmt.unprunableRelids is computed during set_plan_refs() by removing the RT indexes of runtime-prunable relations, identified from PartitionPruneInfos, from the full set of relation RT indexes. ExecDoInitialPruning() then updates es_unpruned_relids by adding partitions that survive initial pruning. To support this, PartitionedRelPruneInfo and PartitionedRelPruningData now include a leafpart_rti_map[] array that maps partition indexes to their corresponding RT indexes. The former is used in set_plan_refs() when constructing unprunableRelids, while the latter is used in ExecDoInitialPruning() to convert partition indexes returned by get_matching_partitions() into RT indexes, which are then added to es_unpruned_relids. These changes make it possible for ModifyTable and LockRows nodes to process only relations that remain unpruned after initial pruning. ExecInitModifyTable() trims lists, such as resultRelations, withCheckOptionLists, returningLists, and updateColnosLists, to consider only unpruned partitions. It also creates ResultRelInfo structs only for these partitions. Similarly, child RowMarks for pruned relations are skipped. By avoiding unnecessary initialization of structures for pruned partitions, these changes improve the performance of updates and deletes on partitioned tables during initial runtime pruning. Due to ExecInitModifyTable() changes as described above, EXPLAIN on a plan for UPDATE and DELETE that uses runtime initial pruning no longer lists partitions pruned during initial pruning. Reviewed-by: Robert Haas <robertmhaas@gmail.com> (earlier versions) Reviewed-by: Tomas Vondra <tomas@vondra.me> Discussion: https://postgr.es/m/CA+HiwqFGkMSge6TgC9KQzde0ohpAycLQuV7ooitEEpbKB0O_mg@mail.gmail.com
1 parent 926c7fc commit cbc1279

File tree

21 files changed

+340
-51
lines changed

21 files changed

+340
-51
lines changed

‎src/backend/commands/copyfrom.c

+2-1
Original file line numberDiff line numberDiff line change
@@ -774,7 +774,8 @@ CopyFrom(CopyFromState cstate)
774774
* index-entry-making machinery. (There used to be a huge amount of code
775775
* here that basically duplicated execUtils.c ...)
776776
*/
777-
ExecInitRangeTable(estate, cstate->range_table, cstate->rteperminfos);
777+
ExecInitRangeTable(estate, cstate->range_table, cstate->rteperminfos,
778+
bms_make_singleton(1));
778779
resultRelInfo = target_resultRelInfo = makeNode(ResultRelInfo);
779780
ExecInitResultRelation(estate, resultRelInfo, 1);
780781

‎src/backend/executor/execMain.c

+16-3
Original file line numberDiff line numberDiff line change
@@ -851,7 +851,8 @@ InitPlan(QueryDesc *queryDesc, int eflags)
851851
/*
852852
* initialize the node's execution state
853853
*/
854-
ExecInitRangeTable(estate, rangeTable, plannedstmt->permInfos);
854+
ExecInitRangeTable(estate, rangeTable, plannedstmt->permInfos,
855+
bms_copy(plannedstmt->unprunableRelids));
855856

856857
estate->es_plannedstmt = plannedstmt;
857858
estate->es_part_prune_infos = plannedstmt->partPruneInfos;
@@ -881,8 +882,13 @@ InitPlan(QueryDesc *queryDesc, int eflags)
881882
Relation relation;
882883
ExecRowMark *erm;
883884

884-
/* ignore "parent" rowmarks; they are irrelevant at runtime */
885-
if (rc->isParent)
885+
/*
886+
* Ignore "parent" rowmarks, because they are irrelevant at
887+
* runtime. Also ignore the rowmarks belonging to child tables
888+
* that have been pruned in ExecDoInitialPruning().
889+
*/
890+
if (rc->isParent ||
891+
!bms_is_member(rc->rti, estate->es_unpruned_relids))
886892
continue;
887893

888894
/* get relation's OID (will produce InvalidOid if subquery) */
@@ -2933,6 +2939,13 @@ EvalPlanQualStart(EPQState *epqstate, Plan *planTree)
29332939
}
29342940
}
29352941

2942+
/*
2943+
* Copy es_unpruned_relids so that pruned relations are ignored by
2944+
* ExecInitLockRows() and ExecInitModifyTable() when initializing the plan
2945+
* trees below.
2946+
*/
2947+
rcestate->es_unpruned_relids = parentestate->es_unpruned_relids;
2948+
29362949
/*
29372950
* Initialize private state information for each SubPlan. We must do this
29382951
* before running ExecInitNode on the main query tree, since

‎src/backend/executor/execParallel.c

+1
Original file line numberDiff line numberDiff line change
@@ -183,6 +183,7 @@ ExecSerializePlan(Plan *plan, EState *estate)
183183
pstmt->planTree = plan;
184184
pstmt->partPruneInfos = estate->es_part_prune_infos;
185185
pstmt->rtable = estate->es_range_table;
186+
pstmt->unprunableRelids = estate->es_unpruned_relids;
186187
pstmt->permInfos = estate->es_rteperminfos;
187188
pstmt->resultRelations = NIL;
188189
pstmt->appendRelations = NIL;

‎src/backend/executor/execPartition.c

+71-12
Original file line numberDiff line numberDiff line change
@@ -182,7 +182,8 @@ static char *ExecBuildSlotPartitionKeyDescription(Relation rel,
182182
static List *adjust_partition_colnos(List *colnos, ResultRelInfo *leaf_part_rri);
183183
static List *adjust_partition_colnos_using_map(List *colnos, AttrMap *attrMap);
184184
static PartitionPruneState *CreatePartitionPruneState(EState *estate,
185-
PartitionPruneInfo *pruneinfo);
185+
PartitionPruneInfo *pruneinfo,
186+
Bitmapset **all_leafpart_rtis);
186187
static void InitPartitionPruneContext(PartitionPruneContext *context,
187188
List *pruning_steps,
188189
PartitionDesc partdesc,
@@ -196,7 +197,8 @@ static void InitExecPartitionPruneContexts(PartitionPruneState *prunstate,
196197
static void find_matching_subplans_recurse(PartitionPruningData *prunedata,
197198
PartitionedRelPruningData *pprune,
198199
bool initial_prune,
199-
Bitmapset **validsubplans);
200+
Bitmapset **validsubplans,
201+
Bitmapset **validsubplan_rtis);
200202

201203

202204
/*
@@ -1820,9 +1822,12 @@ ExecDoInitialPruning(EState *estate)
18201822
PartitionPruneInfo *pruneinfo = lfirst_node(PartitionPruneInfo, lc);
18211823
PartitionPruneState *prunestate;
18221824
Bitmapset *validsubplans = NULL;
1825+
Bitmapset *all_leafpart_rtis = NULL;
1826+
Bitmapset *validsubplan_rtis = NULL;
18231827

18241828
/* Create and save the PartitionPruneState. */
1825-
prunestate = CreatePartitionPruneState(estate, pruneinfo);
1829+
prunestate = CreatePartitionPruneState(estate, pruneinfo,
1830+
&all_leafpart_rtis);
18261831
estate->es_part_prune_states = lappend(estate->es_part_prune_states,
18271832
prunestate);
18281833

@@ -1831,7 +1836,13 @@ ExecDoInitialPruning(EState *estate)
18311836
* bitmapset or NULL as described in the header comment.
18321837
*/
18331838
if (prunestate->do_initial_prune)
1834-
validsubplans = ExecFindMatchingSubPlans(prunestate, true);
1839+
validsubplans = ExecFindMatchingSubPlans(prunestate, true,
1840+
&validsubplan_rtis);
1841+
else
1842+
validsubplan_rtis = all_leafpart_rtis;
1843+
1844+
estate->es_unpruned_relids = bms_add_members(estate->es_unpruned_relids,
1845+
validsubplan_rtis);
18351846
estate->es_part_prune_results = lappend(estate->es_part_prune_results,
18361847
validsubplans);
18371848
}
@@ -1944,9 +1955,16 @@ ExecInitPartitionExecPruning(PlanState *planstate,
19441955
* initialized here. Those required for exec pruning are initialized later in
19451956
* ExecInitPartitionExecPruning(), as they depend on the availability of the
19461957
* parent plan node's PlanState.
1958+
*
1959+
* If initial pruning steps are to be skipped (e.g., during EXPLAIN
1960+
* (GENERIC_PLAN)), *all_leafpart_rtis will be populated with the RT indexes of
1961+
* all leaf partitions whose scanning subnode is included in the parent plan
1962+
* node's list of child plans. The caller must add these RT indexes to
1963+
* estate->es_unpruned_relids.
19471964
*/
19481965
static PartitionPruneState *
1949-
CreatePartitionPruneState(EState *estate, PartitionPruneInfo *pruneinfo)
1966+
CreatePartitionPruneState(EState *estate, PartitionPruneInfo *pruneinfo,
1967+
Bitmapset **all_leafpart_rtis)
19501968
{
19511969
PartitionPruneState *prunestate;
19521970
int n_part_hierarchies;
@@ -2039,8 +2057,8 @@ CreatePartitionPruneState(EState *estate, PartitionPruneInfo *pruneinfo)
20392057
* The set of partitions that exist now might not be the same that
20402058
* existed when the plan was made. The normal case is that it is;
20412059
* optimize for that case with a quick comparison, and just copy
2042-
* the subplan_map and make subpart_map point to the one in
2043-
* PruneInfo.
2060+
* the subplan_map and make subpart_map, leafpart_rti_map point to
2061+
* the ones in PruneInfo.
20442062
*
20452063
* For the case where they aren't identical, we could have more
20462064
* partitions on either side; or even exactly the same number of
@@ -2059,6 +2077,7 @@ CreatePartitionPruneState(EState *estate, PartitionPruneInfo *pruneinfo)
20592077
sizeof(int) * partdesc->nparts) == 0)
20602078
{
20612079
pprune->subpart_map = pinfo->subpart_map;
2080+
pprune->leafpart_rti_map = pinfo->leafpart_rti_map;
20622081
memcpy(pprune->subplan_map, pinfo->subplan_map,
20632082
sizeof(int) * pinfo->nparts);
20642083
}
@@ -2079,6 +2098,7 @@ CreatePartitionPruneState(EState *estate, PartitionPruneInfo *pruneinfo)
20792098
* mismatches.
20802099
*/
20812100
pprune->subpart_map = palloc(sizeof(int) * partdesc->nparts);
2101+
pprune->leafpart_rti_map = palloc(sizeof(int) * partdesc->nparts);
20822102

20832103
for (pp_idx = 0; pp_idx < partdesc->nparts; pp_idx++)
20842104
{
@@ -2096,6 +2116,8 @@ CreatePartitionPruneState(EState *estate, PartitionPruneInfo *pruneinfo)
20962116
pinfo->subplan_map[pd_idx];
20972117
pprune->subpart_map[pp_idx] =
20982118
pinfo->subpart_map[pd_idx];
2119+
pprune->leafpart_rti_map[pp_idx] =
2120+
pinfo->leafpart_rti_map[pd_idx];
20992121
pd_idx++;
21002122
continue;
21012123
}
@@ -2133,6 +2155,7 @@ CreatePartitionPruneState(EState *estate, PartitionPruneInfo *pruneinfo)
21332155

21342156
pprune->subpart_map[pp_idx] = -1;
21352157
pprune->subplan_map[pp_idx] = -1;
2158+
pprune->leafpart_rti_map[pp_idx] = 0;
21362159
}
21372160
}
21382161

@@ -2174,6 +2197,25 @@ CreatePartitionPruneState(EState *estate, PartitionPruneInfo *pruneinfo)
21742197
prunestate->execparamids = bms_add_members(prunestate->execparamids,
21752198
pinfo->execparamids);
21762199

2200+
/*
2201+
* Return all leaf partition indexes if we're skipping pruning in
2202+
* the EXPLAIN (GENERIC_PLAN) case.
2203+
*/
2204+
if (pinfo->initial_pruning_steps && !prunestate->do_initial_prune)
2205+
{
2206+
int part_index = -1;
2207+
2208+
while ((part_index = bms_next_member(pprune->present_parts,
2209+
part_index)) >= 0)
2210+
{
2211+
Index rtindex = pprune->leafpart_rti_map[part_index];
2212+
2213+
if (rtindex)
2214+
*all_leafpart_rtis = bms_add_member(*all_leafpart_rtis,
2215+
rtindex);
2216+
}
2217+
}
2218+
21772219
j++;
21782220
}
21792221
i++;
@@ -2439,10 +2481,15 @@ InitExecPartitionPruneContexts(PartitionPruneState *prunestate,
24392481
* Pass initial_prune if PARAM_EXEC Params cannot yet be evaluated. This
24402482
* differentiates the initial executor-time pruning step from later
24412483
* runtime pruning.
2484+
*
2485+
* The caller must pass a non-NULL validsubplan_rtis during initial pruning
2486+
* to collect the RT indexes of leaf partitions whose subnodes will be
2487+
* executed. These RT indexes are later added to EState.es_unpruned_relids.
24422488
*/
24432489
Bitmapset *
24442490
ExecFindMatchingSubPlans(PartitionPruneState *prunestate,
2445-
bool initial_prune)
2491+
bool initial_prune,
2492+
Bitmapset **validsubplan_rtis)
24462493
{
24472494
Bitmapset *result = NULL;
24482495
MemoryContext oldcontext;
@@ -2454,6 +2501,7 @@ ExecFindMatchingSubPlans(PartitionPruneState *prunestate,
24542501
* evaluated *and* there are steps in which to do so.
24552502
*/
24562503
Assert(initial_prune || prunestate->do_exec_prune);
2504+
Assert(validsubplan_rtis != NULL || !initial_prune);
24572505

24582506
/*
24592507
* Switch to a temp context to avoid leaking memory in the executor's
@@ -2477,7 +2525,7 @@ ExecFindMatchingSubPlans(PartitionPruneState *prunestate,
24772525
*/
24782526
pprune = &prunedata->partrelprunedata[0];
24792527
find_matching_subplans_recurse(prunedata, pprune, initial_prune,
2480-
&result);
2528+
&result, validsubplan_rtis);
24812529

24822530
/*
24832531
* Expression eval may have used space in ExprContext too. Avoid
@@ -2495,6 +2543,8 @@ ExecFindMatchingSubPlans(PartitionPruneState *prunestate,
24952543

24962544
/* Copy result out of the temp context before we reset it */
24972545
result = bms_copy(result);
2546+
if (validsubplan_rtis)
2547+
*validsubplan_rtis = bms_copy(*validsubplan_rtis);
24982548

24992549
MemoryContextReset(prunestate->prune_context);
25002550

@@ -2505,13 +2555,16 @@ ExecFindMatchingSubPlans(PartitionPruneState *prunestate,
25052555
* find_matching_subplans_recurse
25062556
* Recursive worker function for ExecFindMatchingSubPlans
25072557
*
2508-
* Adds valid (non-prunable) subplan IDs to *validsubplans
2558+
* Adds valid (non-prunable) subplan IDs to *validsubplans and the RT indexes
2559+
* of their corresponding leaf partitions to *validsubplan_rtis if
2560+
* it's non-NULL.
25092561
*/
25102562
static void
25112563
find_matching_subplans_recurse(PartitionPruningData *prunedata,
25122564
PartitionedRelPruningData *pprune,
25132565
bool initial_prune,
2514-
Bitmapset **validsubplans)
2566+
Bitmapset **validsubplans,
2567+
Bitmapset **validsubplan_rtis)
25152568
{
25162569
Bitmapset *partset;
25172570
int i;
@@ -2538,16 +2591,22 @@ find_matching_subplans_recurse(PartitionPruningData *prunedata,
25382591
while ((i = bms_next_member(partset, i)) >= 0)
25392592
{
25402593
if (pprune->subplan_map[i] >= 0)
2594+
{
25412595
*validsubplans = bms_add_member(*validsubplans,
25422596
pprune->subplan_map[i]);
2597+
if (validsubplan_rtis)
2598+
*validsubplan_rtis = bms_add_member(*validsubplan_rtis,
2599+
pprune->leafpart_rti_map[i]);
2600+
}
25432601
else
25442602
{
25452603
int partidx = pprune->subpart_map[i];
25462604

25472605
if (partidx >= 0)
25482606
find_matching_subplans_recurse(prunedata,
25492607
&prunedata->partrelprunedata[partidx],
2550-
initial_prune, validsubplans);
2608+
initial_prune, validsubplans,
2609+
validsubplan_rtis);
25512610
else
25522611
{
25532612
/*

‎src/backend/executor/execUtils.c

+11-1
Original file line numberDiff line numberDiff line change
@@ -771,7 +771,8 @@ ExecOpenScanRelation(EState *estate, Index scanrelid, int eflags)
771771
* indexed by rangetable index.
772772
*/
773773
void
774-
ExecInitRangeTable(EState *estate, List *rangeTable, List *permInfos)
774+
ExecInitRangeTable(EState *estate, List *rangeTable, List *permInfos,
775+
Bitmapset *unpruned_relids)
775776
{
776777
/* Remember the range table List as-is */
777778
estate->es_range_table = rangeTable;
@@ -782,6 +783,15 @@ ExecInitRangeTable(EState *estate, List *rangeTable, List *permInfos)
782783
/* Set size of associated arrays */
783784
estate->es_range_table_size = list_length(rangeTable);
784785

786+
/*
787+
* Initialize the bitmapset of RT indexes (es_unpruned_relids)
788+
* representing relations that will be scanned during execution. This set
789+
* is initially populated by the caller and may be extended later by
790+
* ExecDoInitialPruning() to include RT indexes of unpruned leaf
791+
* partitions.
792+
*/
793+
estate->es_unpruned_relids = unpruned_relids;
794+
785795
/*
786796
* Allocate an array to store an open Relation corresponding to each
787797
* rangetable entry, and initialize entries to NULL. Relations are opened

‎src/backend/executor/nodeAppend.c

+4-4
Original file line numberDiff line numberDiff line change
@@ -595,7 +595,7 @@ choose_next_subplan_locally(AppendState *node)
595595
else if (!node->as_valid_subplans_identified)
596596
{
597597
node->as_valid_subplans =
598-
ExecFindMatchingSubPlans(node->as_prune_state, false);
598+
ExecFindMatchingSubPlans(node->as_prune_state, false, NULL);
599599
node->as_valid_subplans_identified = true;
600600
}
601601

@@ -662,7 +662,7 @@ choose_next_subplan_for_leader(AppendState *node)
662662
if (!node->as_valid_subplans_identified)
663663
{
664664
node->as_valid_subplans =
665-
ExecFindMatchingSubPlans(node->as_prune_state, false);
665+
ExecFindMatchingSubPlans(node->as_prune_state, false, NULL);
666666
node->as_valid_subplans_identified = true;
667667

668668
/*
@@ -738,7 +738,7 @@ choose_next_subplan_for_worker(AppendState *node)
738738
else if (!node->as_valid_subplans_identified)
739739
{
740740
node->as_valid_subplans =
741-
ExecFindMatchingSubPlans(node->as_prune_state, false);
741+
ExecFindMatchingSubPlans(node->as_prune_state, false, NULL);
742742
node->as_valid_subplans_identified = true;
743743

744744
mark_invalid_subplans_as_finished(node);
@@ -891,7 +891,7 @@ ExecAppendAsyncBegin(AppendState *node)
891891
if (!node->as_valid_subplans_identified)
892892
{
893893
node->as_valid_subplans =
894-
ExecFindMatchingSubPlans(node->as_prune_state, false);
894+
ExecFindMatchingSubPlans(node->as_prune_state, false, NULL);
895895
node->as_valid_subplans_identified = true;
896896

897897
classify_matching_subplans(node);

‎src/backend/executor/nodeLockRows.c

+7-2
Original file line numberDiff line numberDiff line change
@@ -347,8 +347,13 @@ ExecInitLockRows(LockRows *node, EState *estate, int eflags)
347347
ExecRowMark *erm;
348348
ExecAuxRowMark *aerm;
349349

350-
/* ignore "parent" rowmarks; they are irrelevant at runtime */
351-
if (rc->isParent)
350+
/*
351+
* Ignore "parent" rowmarks, because they are irrelevant at runtime.
352+
* Also ignore the rowmarks belonging to child tables that have been
353+
* pruned in ExecDoInitialPruning().
354+
*/
355+
if (rc->isParent ||
356+
!bms_is_member(rc->rti, estate->es_unpruned_relids))
352357
continue;
353358

354359
/* find ExecRowMark and build ExecAuxRowMark */

‎src/backend/executor/nodeMergeAppend.c

+1-1
Original file line numberDiff line numberDiff line change
@@ -233,7 +233,7 @@ ExecMergeAppend(PlanState *pstate)
233233
*/
234234
if (node->ms_valid_subplans == NULL)
235235
node->ms_valid_subplans =
236-
ExecFindMatchingSubPlans(node->ms_prune_state, false);
236+
ExecFindMatchingSubPlans(node->ms_prune_state, false, NULL);
237237

238238
/*
239239
* First time through: pull the first tuple from each valid subplan,

0 commit comments

Comments
 (0)