Skip to content

Commit 7ad6960

Browse files
committed
Still more fixes for planner's handling of LATERAL references.
More fuzz testing by Andreas Seltenreich exposed that the planner did not cope well with chains of lateral references. If relation X references Y laterally, and Y references Z laterally, then we will have to scan X on the inside of a nestloop with Z, so for all intents and purposes X is laterally dependent on Z too. The planner did not understand this and would generate intermediate joins that could not be used. While that was usually harmless except for wasting some planning cycles, under the right circumstances it would lead to "failed to build any N-way joins" or "could not devise a query plan" planner failures. To fix that, convert the existing per-relation lateral_relids and lateral_referencers relid sets into their transitive closures; that is, they now show all relations on which a rel is directly or indirectly laterally dependent. This not only fixes the chained-reference problem but allows some of the relevant tests to be made substantially simpler and faster, since they can be reduced to simple bitmap manipulations instead of searches of the LateralJoinInfo list. Also, when a PlaceHolderVar that is due to be evaluated at a join contains lateral references, we should treat those references as indirect lateral dependencies of each of the join's base relations. This prevents us from trying to join any individual base relations to the lateral reference source before the join is formed, which again cannot work. Andreas' testing also exposed another oversight in the "dangerous PlaceHolderVar" test added in commit 85e5e22. Simply rejecting unsafe join paths in joinpath.c is insufficient, because in some cases we will end up rejecting *all* possible paths for a particular join, again leading to "could not devise a query plan" failures. The restriction has to be known also to join_is_legal and its cohort functions, so that they will not select a join for which that will happen. I chose to move the supporting logic into joinrels.c where the latter functions are. Back-patch to 9.3 where LATERAL support was introduced.
1 parent c6a67bb commit 7ad6960

File tree

9 files changed

+531
-188
lines changed

9 files changed

+531
-188
lines changed

src/backend/optimizer/path/joinpath.c

+6-73
Original file line numberDiff line numberDiff line change
@@ -247,54 +247,6 @@ allow_star_schema_join(PlannerInfo *root,
247247
bms_nonempty_difference(innerparams, outerrelids));
248248
}
249249

250-
/*
251-
* There's a pitfall for creating parameterized nestloops: suppose the inner
252-
* rel (call it A) has a parameter that is a PlaceHolderVar, and that PHV's
253-
* minimum eval_at set includes the outer rel (B) and some third rel (C).
254-
* We might think we could create a B/A nestloop join that's parameterized by
255-
* C. But we would end up with a plan in which the PHV's expression has to be
256-
* evaluated as a nestloop parameter at the B/A join; and the executor is only
257-
* set up to handle simple Vars as NestLoopParams. Rather than add complexity
258-
* and overhead to the executor for such corner cases, it seems better to
259-
* forbid the join. (Note that existence of such a PHV probably means there
260-
* is a join order constraint that will cause us to consider joining B and C
261-
* directly; so we can still make use of A's parameterized path with B+C.)
262-
* So we check whether any PHVs used in the query could pose such a hazard.
263-
* We don't have any simple way of checking whether a risky PHV would actually
264-
* be used in the inner plan, and the case is so unusual that it doesn't seem
265-
* worth working very hard on it.
266-
*
267-
* This case can occur whether or not the join's remaining parameterization
268-
* overlaps param_source_rels, so we have to check for it separately from
269-
* allow_star_schema_join, even though it looks much like a star-schema case.
270-
*/
271-
static inline bool
272-
check_hazardous_phv(PlannerInfo *root,
273-
Path *outer_path,
274-
Path *inner_path)
275-
{
276-
Relids innerparams = PATH_REQ_OUTER(inner_path);
277-
Relids outerrelids = outer_path->parent->relids;
278-
ListCell *lc;
279-
280-
foreach(lc, root->placeholder_list)
281-
{
282-
PlaceHolderInfo *phinfo = (PlaceHolderInfo *) lfirst(lc);
283-
284-
if (!bms_is_subset(phinfo->ph_eval_at, innerparams))
285-
continue; /* ignore, could not be a nestloop param */
286-
if (!bms_overlap(phinfo->ph_eval_at, outerrelids))
287-
continue; /* ignore, not relevant to this join */
288-
if (bms_is_subset(phinfo->ph_eval_at, outerrelids))
289-
continue; /* safe, it can be eval'd within outerrel */
290-
/* Otherwise, it's potentially unsafe, so reject the join */
291-
return false;
292-
}
293-
294-
/* OK to perform the join */
295-
return true;
296-
}
297-
298250
/*
299251
* try_nestloop_path
300252
* Consider a nestloop join path; if it appears useful, push it into
@@ -318,31 +270,24 @@ try_nestloop_path(PlannerInfo *root,
318270
/*
319271
* Check to see if proposed path is still parameterized, and reject if the
320272
* parameterization wouldn't be sensible --- unless allow_star_schema_join
321-
* says to allow it anyway. Also, we must reject if check_hazardous_phv
322-
* doesn't like the look of it.
273+
* says to allow it anyway. Also, we must reject if have_dangerous_phv
274+
* doesn't like the look of it, which could only happen if the nestloop is
275+
* still parameterized.
323276
*/
324277
required_outer = calc_nestloop_required_outer(outer_path,
325278
inner_path);
326279
if (required_outer &&
327280
((!bms_overlap(required_outer, param_source_rels) &&
328281
!allow_star_schema_join(root, outer_path, inner_path)) ||
329-
!check_hazardous_phv(root, outer_path, inner_path)))
282+
have_dangerous_phv(root,
283+
outer_path->parent->relids,
284+
PATH_REQ_OUTER(inner_path))))
330285
{
331286
/* Waste no memory when we reject a path here */
332287
bms_free(required_outer);
333288
return;
334289
}
335290

336-
/*
337-
* Independently of that, add parameterization needed for any
338-
* PlaceHolderVars that need to be computed at the join. We can handle
339-
* that just by adding joinrel->lateral_relids; that might include some
340-
* rels that are already in required_outer, but no harm done. (Note that
341-
* lateral_relids is exactly NULL if empty, so this will not break the
342-
* property that required_outer is too.)
343-
*/
344-
required_outer = bms_add_members(required_outer, joinrel->lateral_relids);
345-
346291
/*
347292
* Do a precheck to quickly eliminate obviously-inferior paths. We
348293
* calculate a cheap lower bound on the path's cost and then use
@@ -416,12 +361,6 @@ try_mergejoin_path(PlannerInfo *root,
416361
return;
417362
}
418363

419-
/*
420-
* Independently of that, add parameterization needed for any
421-
* PlaceHolderVars that need to be computed at the join.
422-
*/
423-
required_outer = bms_add_members(required_outer, joinrel->lateral_relids);
424-
425364
/*
426365
* If the given paths are already well enough ordered, we can skip doing
427366
* an explicit sort.
@@ -501,12 +440,6 @@ try_hashjoin_path(PlannerInfo *root,
501440
return;
502441
}
503442

504-
/*
505-
* Independently of that, add parameterization needed for any
506-
* PlaceHolderVars that need to be computed at the join.
507-
*/
508-
required_outer = bms_add_members(required_outer, joinrel->lateral_relids);
509-
510443
/*
511444
* See comments in try_nestloop_path(). Also note that hashjoin paths
512445
* never have any output pathkeys, per comments in create_hashjoin_path.

0 commit comments

Comments
 (0)