diff options
| author | Tom Lane | 2026-08-28 19:12:26 +0000 |
|---|---|---|
| committer | Tom Lane | 2026-08-28 19:12:26 +0000 |
| commit | 9f25197bf27c4c4a02d754842bc4055d83be735b (patch) | |
| tree | 1f2a9eafb589fbcd9eda43e9a718379d93b7fbaf | |
| parent | b3fa61ab7986ba980f0c03112caa3c01c4f91f0b (diff) | |
Perform join removal by editing the query's jointree.
analyzejoins.c decided which joins could be dropped by consulting the
planner's derived data structures, but then implemented the removal
by updating those structures in-place. That is a lot of fiddly work,
and nothing keeps it in step with the rest of the planner:
remove_leftjoinrel_from_query only bothered to update "parts of the
planner's data structures that will actually be consulted later", with
no good way to know what those are. Bug #19560 is one consequence.
In that report, removing a join leaves an EquivalenceClass that now
gives rise to a base restriction clause, but base restriction clauses
have already been generated and nothing reconsiders them, so the WHERE
condition disappears from the plan and we return wrong answers.
The self-join elimination code has the same design and the same type
of hazard. We have seen many related bugs over the years too, so it's
time to do something drastic.
To fix, do the removals by editing root->parse->jointree (which is a
far simpler and more stable representation than the derived data),
and then have query_planner() discard everything it computed from the
jointree and derive it over again. This requires quite a bit less
code, and doesn't require touching analyzejoins.c every time we change
the data derived by query_planner(). For typical cases it can actually
save a bit of planning time, though in cases where we have to iterate
the derivation loop many times it does add some time.
reduce_unique_semijoins() gets the same treatment: rather than deleting
the semijoin's SpecialJoinInfo and relying on the jointree not being
consulted again, it now changes the JoinExpr's jointype to JOIN_INNER
and recalculates everything.
Some plans change in the join regression test. Qual evaluation order
shifts in a few cases, because the conditions now reach later planning
in jointree order rather than in whatever order the removal code
re-distributed them. A few plans improve, since the rebuilt relation
targetlists no longer carry columns that only a removed join needed.
We also detect a constant-false filter condition whose test used to
carry a FIXME label. One plan gets marginally worse, because the old
code recomputed attr_needed from equivalence classes after a join
removal; that is more accurate than what deconstruct_jointree()
derives from the original clauses, but we no longer do that. Making
that recomputation happen anyway could be worth doing, but it should
be considered independently and perhaps implemented differently.
Back-patch to v16, on the grounds that the introduction of
varnullingrels in v16 made the old approach significantly more complex
and bug-prone; notably, bug #19560 does not manifest before v16.
In released branches, do not remove externally-visible fixup
functions such as remove_join_clause_from_rels, in case any
extensions are relying on them; but they're no longer used by core
code. But we must nonetheless break API/ABI for remove_useless_joins,
reduce_unique_semijoins, and remove_useless_self_joins, as those now
have different outputs and very different behavior than before.
It seems unlikely that any extensions are calling those; but just in
case, make the breakage more obvious by renaming remove_useless_joins
to remove_useless_outer_joins, which is a more sensible name for it
anyway since the addition of remove_useless_self_joins.
Full disclosure: initial drafts of this patch were made with
Claude Opus 4.8.
Bug: #19560
Reported-by: Orestis Markou <orestis@orestis.gr>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Richard Guo <guofenglinux@gmail.com>
Reviewed-by: Thom Brown <thom@linux.com>
Reviewed-by: Jacob Brazeal <jacob.brazeal@gmail.com>
Discussion: https://postgr.es/m/1186816.1784573544@sss.pgh.pa.us
Backpatch-through: 16
| -rw-r--r-- | src/backend/optimizer/plan/analyzejoins.c | 1840 | ||||
| -rw-r--r-- | src/backend/optimizer/plan/planmain.c | 98 | ||||
| -rw-r--r-- | src/backend/optimizer/plan/planner.c | 18 | ||||
| -rw-r--r-- | src/backend/rewrite/rewriteManip.c | 49 | ||||
| -rw-r--r-- | src/include/nodes/primnodes.h | 4 | ||||
| -rw-r--r-- | src/include/optimizer/planmain.h | 6 | ||||
| -rw-r--r-- | src/test/regress/expected/join.out | 156 | ||||
| -rw-r--r-- | src/test/regress/expected/rowsecurity.out | 11 | ||||
| -rw-r--r-- | src/test/regress/sql/join.sql | 58 | ||||
| -rw-r--r-- | src/test/regress/sql/rowsecurity.sql | 4 |
10 files changed, 972 insertions, 1272 deletions
diff --git a/src/backend/optimizer/plan/analyzejoins.c b/src/backend/optimizer/plan/analyzejoins.c index d04d93ac5a8..df7796bbef3 100644 --- a/src/backend/optimizer/plan/analyzejoins.c +++ b/src/backend/optimizer/plan/analyzejoins.c @@ -7,9 +7,14 @@ * certain optimizations cannot be performed at that stage for lack of * detailed information about the query. The routines here are invoked * after initsplan.c has done its work, and can do additional join removal - * and simplification steps based on the information extracted. The penalty - * is that we have to work harder to clean up after ourselves when we modify - * the query, since the derived data structures have to be updated too. + * and simplification steps based on the information extracted. + * + * Although the decisions about what can be removed are made using the + * planner's derived data structures, the removals themselves are implemented + * by editing the query's jointree, which is a far simpler and more stable + * representation. We make no attempt to update the derived data structures + * to match; instead, query_planner() throws them all away and recomputes them + * whenever we report having removed something. * * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group * Portions Copyright (c) 1994, Regents of the University of California @@ -23,13 +28,13 @@ #include "postgres.h" #include "catalog/pg_class.h" +#include "nodes/makefuncs.h" #include "nodes/nodeFuncs.h" -#include "optimizer/joininfo.h" #include "optimizer/optimizer.h" #include "optimizer/pathnode.h" #include "optimizer/paths.h" -#include "optimizer/placeholder.h" #include "optimizer/planmain.h" +#include "optimizer/prep.h" #include "optimizer/restrictinfo.h" #include "rewrite/rewriteManip.h" #include "utils/lsyscache.h" @@ -68,18 +73,11 @@ bool enable_self_join_elimination; /* local functions */ static bool join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo); -static void remove_leftjoinrel_from_query(PlannerInfo *root, int relid, - SpecialJoinInfo *sjinfo); -static void remove_rel_from_restrictinfo(RestrictInfo *rinfo, - int relid, int ojrelid); -static void remove_rel_from_eclass(PlannerInfo *root, EquivalenceClass *ec, - SpecialJoinInfo *sjinfo, - int relid, int subst); -static void remove_rel_from_restrictinfo_phvs(RestrictInfo *rinfo, - int relid, int ojrelid); -static Node *remove_rel_from_phvs(Node *node, int relid, int ojrelid); -static Node *remove_rel_from_phvs_mutator(Node *node, Relids removable); -static List *remove_rel_from_joinlist(List *joinlist, int relid, int *nremoved); +static Node *remove_join_from_jointree(Node *jtnode, int ojrelid, + int *nremoved); +static void remove_rels_from_query_tree(PlannerInfo *root, + Relids removed_relids); +static bool reduce_semijoin_in_jointree(Node *jtnode, Relids syn_righthand); static bool rel_supports_distinctness(PlannerInfo *root, RelOptInfo *rel); static bool rel_is_distinct_for(PlannerInfo *root, RelOptInfo *rel, List *clause_list, List **extra_clauses); @@ -93,71 +91,100 @@ static bool is_innerrel_unique_for(PlannerInfo *root, JoinType jointype, List *restrictlist, List **extra_clauses); +static Node *remove_rel_from_jointree(Node *jtnode, int relid, + Node **orphan_quals, int *nremoved); +static Node *merge_quals(Node *quals1, Node *quals2); +static void fixup_selfjoin_jointree(PlannerInfo *root, Node *jtnode, int relid, + Node **hoist_quals, bool *found_relid); +static List *fixup_selfjoin_quals(PlannerInfo *root, List *quals, int relid); +static Node *replace_selfjoin_qual(Node *qual); static int self_join_candidates_cmp(const void *a, const void *b); -static bool replace_relid_callback(Node *node, - ChangeVarNodes_context *context); /* - * remove_useless_joins + * remove_useless_outer_joins * Check for relations that don't actually need to be joined at all, - * and remove them from the query. + * and remove them from the query's jointree. * - * We are passed the current joinlist and return the updated list. Other - * data structures that have to be updated are accessible via "root". + * Returns true if we removed anything. In that case the caller must discard + * everything it has derived from the jointree and compute it over again, + * since we don't try to update any of that here. */ -List * -remove_useless_joins(PlannerInfo *root, List *joinlist) +bool +remove_useless_outer_joins(PlannerInfo *root) { + Relids removed_relids = NULL; ListCell *lc; /* * We are only interested in relations that are left-joined to, so we can * scan the join_info_list to find them easily. */ -restart: foreach(lc, root->join_info_list) { SpecialJoinInfo *sjinfo = (SpecialJoinInfo *) lfirst(lc); int innerrelid; int nremoved; + RangeTblEntry *rte; /* Skip if not removable */ if (!join_is_removable(root, sjinfo)) continue; /* - * Currently, join_is_removable can only succeed when the sjinfo's - * righthand is a single baserel. Remove that rel from the query and - * joinlist. + * join_is_removable insists that the join's syntactic righthand side + * be a single baserel, so we can implement the removal by dropping + * the JoinExpr and everything below its righthand side. */ - innerrelid = bms_singleton_member(sjinfo->min_righthand); - - remove_leftjoinrel_from_query(root, innerrelid, sjinfo); + innerrelid = bms_singleton_member(sjinfo->syn_righthand); - /* We verify that exactly one reference gets removed from joinlist */ + /* We verify that exactly one JoinExpr gets removed */ nremoved = 0; - joinlist = remove_rel_from_joinlist(joinlist, innerrelid, &nremoved); + root->parse->jointree = (FromExpr *) + remove_join_from_jointree((Node *) root->parse->jointree, + sjinfo->ojrelid, &nremoved); if (nremoved != 1) - elog(ERROR, "failed to find relation %d in joinlist", innerrelid); + elog(ERROR, "failed to find join %d in jointree", sjinfo->ojrelid); + + /* Track all the relids we've removed, for use below */ + removed_relids = bms_add_member(removed_relids, innerrelid); + removed_relids = bms_add_member(removed_relids, sjinfo->ojrelid); /* - * We can delete this SpecialJoinInfo from the list too, since it's no - * longer of interest. (Since we'll restart the foreach loop - * immediately, we don't bother with foreach_delete_current.) + * As in pull_up_simple_subquery, discard no-longer-needed subqueries. + * This is not just an optimization, but is necessary to prevent + * subsequent processing from descending into stale subtrees and + * seeing inconsistent data. Likewise discard any securityQuals of + * the removed rel. (Although simple_rte_array[] will be rebuilt + * shortly, we can still use it to find the RTE in the parse tree.) */ - root->join_info_list = list_delete_cell(root->join_info_list, lc); + rte = root->simple_rte_array[innerrelid]; + if (rte->rtekind == RTE_SUBQUERY) + rte->subquery = NULL; + rte->securityQuals = NIL; /* - * Restart the scan. This is necessary to ensure we find all - * removable joins independently of ordering of the join_info_list - * (note that removal of attr_needed bits may make a join appear - * removable that did not before). + * It's okay to keep scanning join_info_list for more removable joins, + * even though the data that join_is_removable consults is now + * slightly out of date. Removing a join can only delete attr_needed + * bits and join clauses, and any attr_needed bit or join clause that + * mentions the removed rel above its own join level would have + * prevented that rel from being removable. So what remains to be + * examined is unchanged by what we just did. + * + * The converse doesn't hold: dropping a join can make some other join + * removable that didn't look so before. That's why our caller loops + * until we report finding nothing more to remove. */ - goto restart; } - return joinlist; + if (bms_is_empty(removed_relids)) + return false; + + /* Clean up the traces that the removed rels have left elsewhere */ + remove_rels_from_query_tree(root, removed_relids); + + return true; } /* @@ -189,8 +216,15 @@ join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo) if (sjinfo->jointype != JOIN_LEFT) return false; - if (!bms_get_singleton_member(sjinfo->min_righthand, &innerrelid)) + /* + * We test the syntactic righthand side, not min_righthand, because the + * removal is done by deleting the whole righthand subtree of the join. + * (min_righthand can be a singleton when syn_righthand is not, but in + * such a case the attr_needed tests below would reject the join anyway.) + */ + if (!bms_get_singleton_member(sjinfo->syn_righthand, &innerrelid)) return false; + Assert(bms_equal(sjinfo->min_righthand, sjinfo->syn_righthand)); /* * Never try to eliminate a left join to the query result rel. Although @@ -331,702 +365,91 @@ join_is_removable(PlannerInfo *root, SpecialJoinInfo *sjinfo) return false; } - /* - * Remove the target rel->relid and references to the target join from the - * planner's data structures, having determined that there is no need - * to include them in the query. Optionally replace them with subst if subst - * is non-negative. + * remove_join_from_jointree + * Delete the JoinExpr with the given RT index, along with everything + * below its righthand side, from the query's jointree. * - * This function updates only parts needed for both left-join removal and - * self-join removal. - */ -static void -remove_rel_from_query(PlannerInfo *root, RelOptInfo *rel, - int subst, SpecialJoinInfo *sjinfo, - Relids joinrelids) -{ - int relid = rel->relid; - Index rti; - ListCell *l; - Bitmapset *seen_serials = NULL; - - /* - * Update all_baserels and related relid sets. - */ - root->all_baserels = adjust_relid_set(root->all_baserels, relid, subst); - root->all_query_rels = adjust_relid_set(root->all_query_rels, relid, subst); - - if (sjinfo != NULL) - { - root->outer_join_rels = bms_del_member(root->outer_join_rels, - sjinfo->ojrelid); - root->all_query_rels = bms_del_member(root->all_query_rels, - sjinfo->ojrelid); - } - - /* - * Likewise remove references from SpecialJoinInfo data structures. - * - * This is relevant in case the outer join we're deleting is nested inside - * other outer joins: the upper joins' relid sets have to be adjusted. The - * RHS of the target outer join will be made empty here, but that's OK - * since caller will delete that SpecialJoinInfo entirely. - */ - foreach(l, root->join_info_list) - { - SpecialJoinInfo *sjinf = (SpecialJoinInfo *) lfirst(l); - - /* - * initsplan.c is fairly cavalier about allowing SpecialJoinInfos' - * lefthand/righthand relid sets to be shared with other data - * structures. Ensure that we don't modify the original relid sets. - * (The commute_xxx sets are always per-SpecialJoinInfo though.) - */ - sjinf->min_lefthand = bms_copy(sjinf->min_lefthand); - sjinf->min_righthand = bms_copy(sjinf->min_righthand); - sjinf->syn_lefthand = bms_copy(sjinf->syn_lefthand); - sjinf->syn_righthand = bms_copy(sjinf->syn_righthand); - /* Now remove relid from the sets: */ - sjinf->min_lefthand = adjust_relid_set(sjinf->min_lefthand, relid, subst); - sjinf->min_righthand = adjust_relid_set(sjinf->min_righthand, relid, subst); - sjinf->syn_lefthand = adjust_relid_set(sjinf->syn_lefthand, relid, subst); - sjinf->syn_righthand = adjust_relid_set(sjinf->syn_righthand, relid, subst); - - if (sjinfo != NULL) - { - Assert(subst <= 0); - - /* Remove sjinfo->ojrelid bits from the sets: */ - sjinf->min_lefthand = bms_del_member(sjinf->min_lefthand, - sjinfo->ojrelid); - sjinf->min_righthand = bms_del_member(sjinf->min_righthand, - sjinfo->ojrelid); - sjinf->syn_lefthand = bms_del_member(sjinf->syn_lefthand, - sjinfo->ojrelid); - sjinf->syn_righthand = bms_del_member(sjinf->syn_righthand, - sjinfo->ojrelid); - /* relid cannot appear in these fields, but ojrelid can: */ - sjinf->commute_above_l = bms_del_member(sjinf->commute_above_l, - sjinfo->ojrelid); - sjinf->commute_above_r = bms_del_member(sjinf->commute_above_r, - sjinfo->ojrelid); - sjinf->commute_below_l = bms_del_member(sjinf->commute_below_l, - sjinfo->ojrelid); - sjinf->commute_below_r = bms_del_member(sjinf->commute_below_r, - sjinfo->ojrelid); - } - else - { - Assert(subst > 0); - - ChangeVarNodesExtended((Node *) sjinf->semi_rhs_exprs, relid, subst, - 0, replace_relid_callback); - } - } - - /* - * Likewise remove references from PlaceHolderVar data structures, - * removing any no-longer-needed placeholders entirely. We remove PHV - * only for left-join removal. With self-join elimination, PHVs already - * get moved to the remaining relation, where they might still be needed. - * It might also happen that we skip the removal of some PHVs that could - * be removed. However, the overhead of extra PHVs is small compared to - * the complexity of analysis needed to remove them. - * - * Removal is a bit trickier than it might seem: we can remove PHVs that - * are used at the target rel and/or in the join qual, but not those that - * are used at join partner rels or above the join. It's not that easy to - * distinguish PHVs used at partner rels from those used in the join qual, - * since they will both have ph_needed sets that are subsets of - * joinrelids. However, a PHV used at a partner rel could not have the - * target rel in ph_eval_at, so we check that while deciding whether to - * remove or just update the PHV. There is no corresponding test in - * join_is_removable because it doesn't need to distinguish those cases. - */ - foreach(l, root->placeholder_list) - { - PlaceHolderInfo *phinfo = (PlaceHolderInfo *) lfirst(l); - - Assert(sjinfo == NULL || !bms_is_member(relid, phinfo->ph_lateral)); - if (sjinfo != NULL && - bms_is_subset(phinfo->ph_needed, joinrelids) && - bms_is_member(relid, phinfo->ph_eval_at) && - !bms_is_member(sjinfo->ojrelid, phinfo->ph_eval_at)) - { - /* - * This code shouldn't be executed if one relation is substituted - * with another: in this case, the placeholder may be employed in - * a filter inside the scan node the SJE removes. - */ - root->placeholder_list = foreach_delete_current(root->placeholder_list, - l); - root->placeholder_array[phinfo->phid] = NULL; - } - else - { - PlaceHolderVar *phv = phinfo->ph_var; - - phinfo->ph_eval_at = adjust_relid_set(phinfo->ph_eval_at, relid, subst); - if (sjinfo != NULL) - phinfo->ph_eval_at = adjust_relid_set(phinfo->ph_eval_at, - sjinfo->ojrelid, subst); - Assert(!bms_is_empty(phinfo->ph_eval_at)); /* checked previously */ - /* Reduce ph_needed to contain only "relation 0"; see below */ - if (bms_is_member(0, phinfo->ph_needed)) - phinfo->ph_needed = bms_make_singleton(0); - else - phinfo->ph_needed = NULL; - - phinfo->ph_lateral = adjust_relid_set(phinfo->ph_lateral, relid, subst); - - /* - * ph_lateral might contain rels mentioned in ph_eval_at after the - * replacement, remove them. - */ - phinfo->ph_lateral = bms_difference(phinfo->ph_lateral, phinfo->ph_eval_at); - /* ph_lateral might or might not be empty */ - - phv->phrels = adjust_relid_set(phv->phrels, relid, subst); - if (sjinfo != NULL) - phv->phrels = adjust_relid_set(phv->phrels, - sjinfo->ojrelid, subst); - Assert(!bms_is_empty(phv->phrels)); - - ChangeVarNodesExtended((Node *) phv->phexpr, relid, subst, 0, - replace_relid_callback); - - Assert(phv->phnullingrels == NULL); /* no need to adjust */ - } - } - - /* - * Likewise remove references from EquivalenceClasses. - */ - foreach(l, root->eq_classes) - { - EquivalenceClass *ec = (EquivalenceClass *) lfirst(l); - - remove_rel_from_eclass(root, ec, sjinfo, relid, subst); - } - - /* - * Finally, we must recompute per-Var attr_needed and per-PlaceHolderVar - * ph_needed relid sets. These have to be known accurately, else we may - * fail to remove other now-removable outer joins. And our removal of the - * join clause(s) for this outer join may mean that Vars that were - * formerly needed no longer are. So we have to do this honestly by - * repeating the construction of those relid sets. We can cheat to one - * small extent: we can avoid re-examining the targetlist and HAVING qual - * by preserving "relation 0" bits from the existing relid sets. This is - * safe because we'd never remove such references. - * - * So, start by removing all other bits from attr_needed sets and - * lateral_vars lists. (We already did this above for ph_needed.) - * - * Also, for left-join removal, we strip the removed rel and join from any - * PlaceHolderVar embedded in the surviving rels' restriction clauses and - * join clauses; we needn't bother with the rel being removed, nor when - * the query has no PlaceHolderVars. - */ - for (rti = 1; rti < root->simple_rel_array_size; rti++) - { - RelOptInfo *otherrel = root->simple_rel_array[rti]; - int attroff; - - /* there may be empty slots corresponding to non-baserel RTEs */ - if (otherrel == NULL) - continue; - - Assert(otherrel->relid == rti); /* sanity check on array */ - - for (attroff = otherrel->max_attr - otherrel->min_attr; - attroff >= 0; - attroff--) - { - if (bms_is_member(0, otherrel->attr_needed[attroff])) - otherrel->attr_needed[attroff] = bms_make_singleton(0); - else - otherrel->attr_needed[attroff] = NULL; - } - - if (subst > 0) - ChangeVarNodesExtended((Node *) otherrel->lateral_vars, relid, - subst, 0, replace_relid_callback); - - if (sjinfo != NULL && rti != relid && root->glob->lastPHId != 0) - { - foreach_node(RestrictInfo, rinfo, otherrel->baserestrictinfo) - remove_rel_from_restrictinfo_phvs(rinfo, relid, sjinfo->ojrelid); - - /* - * Join clauses need the same treatment, but there's no value in - * processing any join clause more than once. So it's slightly - * annoying that we have to find them via the per-base-relation - * joininfo lists. Avoid duplicate processing by tracking the - * rinfo_serial numbers of join clauses we've already seen. (This - * doesn't work for is_clone clauses, so we must waste effort on - * them.) - */ - foreach_node(RestrictInfo, rinfo, otherrel->joininfo) - { - if (!rinfo->is_clone) /* else serial number is not unique */ - { - if (bms_is_member(rinfo->rinfo_serial, seen_serials)) - continue; /* saw it already */ - seen_serials = bms_add_member(seen_serials, - rinfo->rinfo_serial); - } - remove_rel_from_restrictinfo_phvs(rinfo, relid, sjinfo->ojrelid); - } - } - } -} - -/* - * Remove the target relid and references to the target join from the - * planner's data structures, having determined that there is no need - * to include them in the query. + * The JoinExpr is replaced by its lefthand input. Its ON conditions can just + * be dropped: since this is a left join, they could only have determined + * which righthand rows join to a given lefthand row, and there are no + * righthand rows anymore. * - * We are not terribly thorough here. We only bother to update parts of - * the planner's data structures that will actually be consulted later. + * *nremoved is incremented by the number of JoinExprs removed (there should + * be exactly one, but the caller checks that). */ -static void -remove_leftjoinrel_from_query(PlannerInfo *root, int relid, - SpecialJoinInfo *sjinfo) +static Node * +remove_join_from_jointree(Node *jtnode, int ojrelid, int *nremoved) { - RelOptInfo *rel = find_base_rel(root, relid); - int ojrelid = sjinfo->ojrelid; - Relids joinrelids; - Relids join_plus_commute; - List *joininfos; - ListCell *l; - - /* Compute the relid set for the join we are considering */ - joinrelids = bms_union(sjinfo->min_lefthand, sjinfo->min_righthand); - Assert(ojrelid != 0); - joinrelids = bms_add_member(joinrelids, ojrelid); - - remove_rel_from_query(root, rel, -1, sjinfo, joinrelids); - - /* - * Remove any joinquals referencing the rel from the joininfo lists. - * - * In some cases, a joinqual has to be put back after deleting its - * reference to the target rel. This can occur for pseudoconstant and - * outerjoin-delayed quals, which can get marked as requiring the rel in - * order to force them to be evaluated at or above the join. We can't - * just discard them, though. Only quals that logically belonged to the - * outer join being discarded should be removed from the query. - * - * We might encounter a qual that is a clone of a deletable qual with some - * outer-join relids added (see deconstruct_distribute_oj_quals). To - * ensure we get rid of such clones as well, add the relids of all OJs - * commutable with this one to the set we test against for - * pushed-down-ness. - */ - join_plus_commute = bms_union(joinrelids, - sjinfo->commute_above_r); - join_plus_commute = bms_add_members(join_plus_commute, - sjinfo->commute_below_l); - - /* - * We must make a copy of the rel's old joininfo list before starting the - * loop, because otherwise remove_join_clause_from_rels would destroy the - * list while we're scanning it. - */ - joininfos = list_copy(rel->joininfo); - foreach(l, joininfos) + if (jtnode == NULL) + return NULL; + if (IsA(jtnode, RangeTblRef)) { - RestrictInfo *rinfo = (RestrictInfo *) lfirst(l); - - remove_join_clause_from_rels(root, rinfo, rinfo->required_relids); - - if (RINFO_IS_PUSHED_DOWN(rinfo, join_plus_commute)) - { - /* - * There might be references to relid or ojrelid in the - * RestrictInfo's relid sets, as a consequence of PHVs having had - * ph_eval_at sets that include those. We already checked above - * that any such PHV is safe (and updated its ph_eval_at), so we - * can just drop those references. - */ - remove_rel_from_restrictinfo(rinfo, relid, ojrelid); - - /* - * Cross-check that the clause itself does not reference the - * target rel or join. - */ -#ifdef USE_ASSERT_CHECKING - { - Relids clause_varnos = pull_varnos(root, - (Node *) rinfo->clause); - - Assert(!bms_is_member(relid, clause_varnos)); - Assert(!bms_is_member(ojrelid, clause_varnos)); - } -#endif - /* Now throw it back into the joininfo lists */ - distribute_restrictinfo_to_rels(root, rinfo); - } + /* nothing to do here */ } - - /* - * There may be references to the rel in root->fkey_list, but if so, - * match_foreign_keys_to_quals() will get rid of them. - */ - - /* - * Now remove the rel from the baserel array to prevent it from being - * referenced again. (We can't do this earlier because - * remove_join_clause_from_rels will touch it.) - */ - root->simple_rel_array[relid] = NULL; - root->simple_rte_array[relid] = NULL; - - /* And nuke the RelOptInfo, just in case there's another access path */ - pfree(rel); - - /* - * Now repeat construction of attr_needed bits coming from all other - * sources. - */ - rebuild_placeholder_attr_needed(root); - rebuild_joinclause_attr_needed(root); - rebuild_eclass_attr_needed(root); - rebuild_lateral_attr_needed(root); -} - -/* - * Remove any references to relid or ojrelid from the RestrictInfo. - * - * We only bother to clean out bits in the RestrictInfo's various relid sets, - * not nullingrel bits in contained Vars and PHVs. (This might have to be - * improved sometime.) However, if the RestrictInfo contains an OR clause - * we have to also clean up the sub-clauses. - */ -static void -remove_rel_from_restrictinfo(RestrictInfo *rinfo, int relid, int ojrelid) -{ - /* - * initsplan.c is fairly cavalier about allowing RestrictInfos to share - * relid sets with other RestrictInfos, and SpecialJoinInfos too. Make - * sure this RestrictInfo has its own relid sets before we modify them. - * (In present usage, clause_relids is probably not shared, but - * required_relids could be; let's not assume anything.) - */ - rinfo->clause_relids = bms_copy(rinfo->clause_relids); - rinfo->clause_relids = bms_del_member(rinfo->clause_relids, relid); - rinfo->clause_relids = bms_del_member(rinfo->clause_relids, ojrelid); - /* Likewise for required_relids */ - rinfo->required_relids = bms_copy(rinfo->required_relids); - rinfo->required_relids = bms_del_member(rinfo->required_relids, relid); - rinfo->required_relids = bms_del_member(rinfo->required_relids, ojrelid); - /* Likewise for incompatible_relids */ - rinfo->incompatible_relids = bms_copy(rinfo->incompatible_relids); - rinfo->incompatible_relids = bms_del_member(rinfo->incompatible_relids, relid); - rinfo->incompatible_relids = bms_del_member(rinfo->incompatible_relids, ojrelid); - /* Likewise for outer_relids */ - rinfo->outer_relids = bms_copy(rinfo->outer_relids); - rinfo->outer_relids = bms_del_member(rinfo->outer_relids, relid); - rinfo->outer_relids = bms_del_member(rinfo->outer_relids, ojrelid); - /* Likewise for left_relids */ - rinfo->left_relids = bms_copy(rinfo->left_relids); - rinfo->left_relids = bms_del_member(rinfo->left_relids, relid); - rinfo->left_relids = bms_del_member(rinfo->left_relids, ojrelid); - /* Likewise for right_relids */ - rinfo->right_relids = bms_copy(rinfo->right_relids); - rinfo->right_relids = bms_del_member(rinfo->right_relids, relid); - rinfo->right_relids = bms_del_member(rinfo->right_relids, ojrelid); - - /* If it's an OR, recurse to clean up sub-clauses */ - if (restriction_is_or_clause(rinfo)) + else if (IsA(jtnode, FromExpr)) { - ListCell *lc; - - Assert(is_orclause(rinfo->orclause)); - foreach(lc, ((BoolExpr *) rinfo->orclause)->args) - { - Node *orarg = (Node *) lfirst(lc); - - /* OR arguments should be ANDs or sub-RestrictInfos */ - if (is_andclause(orarg)) - { - List *andargs = ((BoolExpr *) orarg)->args; - ListCell *lc2; - - foreach(lc2, andargs) - { - RestrictInfo *rinfo2 = lfirst_node(RestrictInfo, lc2); - - remove_rel_from_restrictinfo(rinfo2, relid, ojrelid); - } - } - else - { - RestrictInfo *rinfo2 = castNode(RestrictInfo, orarg); - - remove_rel_from_restrictinfo(rinfo2, relid, ojrelid); - } - } - } -} - -/* - * Remove any references to relid or sjinfo->ojrelid (if sjinfo != NULL) - * from the EquivalenceClass. - * - * We fix the EC and EM relid sets to ensure that implied join equalities will - * be generated at the appropriate join level(s). We also strip the removed - * rel from PlaceHolderVars embedded in member expressions; a member's - * em_relids reflects ph_eval_at rather than the PHV's phrels, so the latter - * can still mention the removed rel even when em_relids does not. Like - * remove_rel_from_restrictinfo, we don't bother with nullingrel bits in - * contained plain Vars. - */ -static void -remove_rel_from_eclass(PlannerInfo *root, EquivalenceClass *ec, - SpecialJoinInfo *sjinfo, - int relid, int subst) -{ - ListCell *lc; + FromExpr *f = (FromExpr *) jtnode; + ListCell *l; - /* - * Strip the removed rel/join from PlaceHolderVars in member expressions. - * This is needed even when the EC's relids don't mention the removed rel. - * Plain Vars and Consts can't contain a PlaceHolderVar, so skip them. - */ - if (sjinfo != NULL && root->glob->lastPHId != 0) - { - foreach_node(EquivalenceMember, em, ec->ec_members) - { - if (!IsA(em->em_expr, Var) && !IsA(em->em_expr, Const)) - em->em_expr = (Expr *) - remove_rel_from_phvs((Node *) em->em_expr, relid, - sjinfo->ojrelid); - } + foreach(l, f->fromlist) + lfirst(l) = remove_join_from_jointree((Node *) lfirst(l), + ojrelid, nremoved); } - - if (!bms_is_member(relid, ec->ec_relids) && - (sjinfo == NULL || !bms_is_member(sjinfo->ojrelid, ec->ec_relids))) - return; - - /* Fix up the EC's overall relids */ - ec->ec_relids = adjust_relid_set(ec->ec_relids, relid, subst); - if (sjinfo != NULL) - ec->ec_relids = adjust_relid_set(ec->ec_relids, - sjinfo->ojrelid, subst); - - /* - * We don't expect any EC child members to exist at this point. Ensure - * that's the case, otherwise, we might be getting asked to do something - * this function hasn't been coded for. - */ - Assert(ec->ec_childmembers == NULL); - - /* - * Fix up the member expressions. Any non-const member that ends with - * empty em_relids must be a Var or PHV of the removed relation. We don't - * need it anymore, so we can drop it. - */ - foreach(lc, ec->ec_members) + else if (IsA(jtnode, JoinExpr)) { - EquivalenceMember *cur_em = (EquivalenceMember *) lfirst(lc); + JoinExpr *j = (JoinExpr *) jtnode; - if (bms_is_member(relid, cur_em->em_relids) || - (sjinfo != NULL && bms_is_member(sjinfo->ojrelid, - cur_em->em_relids))) + if (j->rtindex == ojrelid) { - Assert(!cur_em->em_is_const); - cur_em->em_relids = adjust_relid_set(cur_em->em_relids, relid, subst); - if (sjinfo != NULL) - cur_em->em_relids = adjust_relid_set(cur_em->em_relids, - sjinfo->ojrelid, subst); - if (bms_is_empty(cur_em->em_relids)) - ec->ec_members = foreach_delete_current(ec->ec_members, lc); + (*nremoved)++; + return j->larg; } + j->larg = remove_join_from_jointree(j->larg, ojrelid, nremoved); + j->rarg = remove_join_from_jointree(j->rarg, ojrelid, nremoved); } + else + elog(ERROR, "unrecognized jointree node type: %d", + (int) nodeTag(jtnode)); - /* Fix up the source clauses, in case we can re-use them later */ - foreach(lc, ec->ec_sources) - { - RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc); - - if (sjinfo == NULL) - ChangeVarNodesExtended((Node *) rinfo, relid, subst, 0, - replace_relid_callback); - else - remove_rel_from_restrictinfo(rinfo, relid, sjinfo->ojrelid); - } - - /* - * Rather than expend code on fixing up any already-derived clauses, just - * drop them. (At this point, any such clauses would be base restriction - * clauses, which we'd not need anymore anyway.) - */ - ec_clear_derived_clauses(ec); + return jtnode; } /* - * Remove any references to relid or ojrelid from the PlaceHolderVars embedded - * in a RestrictInfo's clause. + * remove_rels_from_query_tree + * Delete all remaining references to the given relids from the query. * - * If it's an OR clause, we must also fix up the orclause, which is a parallel - * representation built from its own sub-RestrictInfos. We recurse into the - * sub-clauses for that, mirroring remove_rel_from_restrictinfo. + * Having removed some relations and outer joins from the jointree, we must + * get rid of any references to them that are left behind elsewhere. There + * should be no ordinary Vars of a removed relation left, but OJ relids can + * still appear in the nullingrels sets of surviving Vars and PlaceHolderVars, + * and both regular and OJ relids can appear in the phrels sets of + * PlaceHolderVars. ChangeVarNodes knows how to strip a relid out of all of + * those. */ static void -remove_rel_from_restrictinfo_phvs(RestrictInfo *rinfo, int relid, int ojrelid) +remove_rels_from_query_tree(PlannerInfo *root, Relids removed_relids) { - rinfo->clause = (Expr *) - remove_rel_from_phvs((Node *) rinfo->clause, relid, ojrelid); - - /* If it's an OR, recurse to clean up sub-clauses */ - if (restriction_is_or_clause(rinfo)) - { - ListCell *lc; - - Assert(is_orclause(rinfo->orclause)); - foreach(lc, ((BoolExpr *) rinfo->orclause)->args) - { - Node *orarg = (Node *) lfirst(lc); - - /* OR arguments should be ANDs or sub-RestrictInfos */ - if (is_andclause(orarg)) - { - List *andargs = ((BoolExpr *) orarg)->args; - ListCell *lc2; - - foreach(lc2, andargs) - { - RestrictInfo *rinfo2 = lfirst_node(RestrictInfo, lc2); - - remove_rel_from_restrictinfo_phvs(rinfo2, relid, ojrelid); - } - } - else - { - RestrictInfo *rinfo2 = castNode(RestrictInfo, orarg); - - remove_rel_from_restrictinfo_phvs(rinfo2, relid, ojrelid); - } - } - } -} - -/* - * Remove any references to the specified RT index(es) from the phrels (and - * phnullingrels) of every PlaceHolderVar in the given expression. - * - * remove_rel_from_query() fixes up the relid sets of RestrictInfos and - * EquivalenceMembers, but not the PlaceHolderVars embedded in their - * expressions. That's normally fine, but such an expression may later be - * translated for an appendrel child and have its relids recomputed by - * pull_varnos(). A leftover removed relid in phrels would then make - * pull_varnos() reference a nonexistent rel, so we strip it here to match the - * canonical PlaceHolderVar. - */ -static Node * -remove_rel_from_phvs(Node *node, int relid, int ojrelid) -{ - Relids removable = bms_add_member(bms_make_singleton(relid), ojrelid); - - return remove_rel_from_phvs_mutator(node, removable); -} + int relid = -1; -static Node * -remove_rel_from_phvs_mutator(Node *node, Relids removable) -{ - if (node == NULL) - return NULL; - if (IsA(node, PlaceHolderVar)) + while ((relid = bms_next_member(removed_relids, relid)) >= 0) { - PlaceHolderVar *phv = (PlaceHolderVar *) node; - Relids newphrels; - - /* Upper-level PlaceHolderVars should be long gone at this point */ - Assert(phv->phlevelsup == 0); - - /* Copy the PlaceHolderVar and mutate what's below ... */ - phv = (PlaceHolderVar *) - expression_tree_mutator(node, - remove_rel_from_phvs_mutator, - removable); + ChangeVarNodes((Node *) root->parse, relid, INVALID_VAR, 0); /* - * ... then strip the removed rels from its relid sets. - * - * If stripping would empty phrels, the PHV is evaluated only at the - * removed relation(s); it then belongs to an EquivalenceMember that - * the caller drops immediately afterwards. Leave such a PHV - * untouched rather than build one with empty phrels, which the rest - * of the planner assumes never occurs. + * processed_tlist shares some but not all of its nodes with + * parse->targetList, so it has to be processed separately. (That's + * harmless: ChangeVarNodes works in-place, and removing a relid that + * isn't there is idempotent.) */ - newphrels = bms_difference(phv->phrels, removable); - if (!bms_is_empty(newphrels)) - { - phv->phrels = newphrels; - phv->phnullingrels = bms_difference(phv->phnullingrels, - removable); - } - - return (Node *) phv; - } - return expression_tree_mutator(node, - remove_rel_from_phvs_mutator, - removable); -} - -/* - * Remove any occurrences of the target relid from a joinlist structure. - * - * It's easiest to build a whole new list structure, so we handle it that - * way. Efficiency is not a big deal here. - * - * *nremoved is incremented by the number of occurrences removed (there - * should be exactly one, but the caller checks that). - */ -static List * -remove_rel_from_joinlist(List *joinlist, int relid, int *nremoved) -{ - List *result = NIL; - ListCell *jl; - - foreach(jl, joinlist) - { - Node *jlnode = (Node *) lfirst(jl); - - if (IsA(jlnode, RangeTblRef)) - { - int varno = ((RangeTblRef *) jlnode)->rtindex; + ChangeVarNodes((Node *) root->processed_tlist, relid, INVALID_VAR, 0); - if (varno == relid) - (*nremoved)++; - else - result = lappend(result, jlnode); - } - else if (IsA(jlnode, List)) - { - /* Recurse to handle subproblem */ - List *sublist; - - sublist = remove_rel_from_joinlist((List *) jlnode, - relid, nremoved); - /* Avoid including empty sub-lists in the result */ - if (sublist) - result = lappend(result, sublist); - } - else - { - elog(ERROR, "unrecognized joinlist node type: %d", - (int) nodeTag(jlnode)); - } + /* There could be references in the append_rel_list, too */ + if (root->append_rel_list != NIL) + ChangeVarNodes((Node *) root->append_rel_list, relid, INVALID_VAR, 0); } - - return result; } - /* * reduce_unique_semijoins * Check for semijoins that can be simplified to plain inner joins @@ -1035,14 +458,13 @@ remove_rel_from_joinlist(List *joinlist, int relid, int *nremoved) * Ideally this would happen during reduce_outer_joins, but we don't have * enough information at that point. * - * To perform the strength reduction when applicable, we need only delete - * the semijoin's SpecialJoinInfo from root->join_info_list. (We don't - * bother fixing the join type attributed to it in the query jointree, - * since that won't be consulted again.) + * Like the join removal cases, we do this on the query's jointree, so + * returning true means the caller must recompute the derived data. */ -void +bool reduce_unique_semijoins(PlannerInfo *root) { + bool changed = false; ListCell *lc; /* @@ -1063,8 +485,13 @@ reduce_unique_semijoins(PlannerInfo *root) if (sjinfo->jointype != JOIN_SEMI) continue; - if (!bms_get_singleton_member(sjinfo->min_righthand, &innerrelid)) + /* + * We test the syntactic righthand side, since that's what identifies + * the JoinExpr we'll modify. + */ + if (!bms_get_singleton_member(sjinfo->syn_righthand, &innerrelid)) continue; + Assert(bms_equal(sjinfo->min_righthand, sjinfo->syn_righthand)); innerrel = find_base_rel(root, innerrelid); @@ -1099,9 +526,65 @@ reduce_unique_semijoins(PlannerInfo *root) JOIN_SEMI, restrictlist, true)) continue; - /* OK, remove the SpecialJoinInfo from the list. */ - root->join_info_list = foreach_delete_current(root->join_info_list, lc); + /* OK, reduce the join to a plain inner join in the jointree. */ + if (!reduce_semijoin_in_jointree((Node *) root->parse->jointree, + sjinfo->syn_righthand)) + elog(ERROR, "failed to find semijoin in jointree"); + changed = true; } + + return changed; +} + +/* + * reduce_semijoin_in_jointree + * Find the JoinExpr for the semijoin with the given syntactic righthand + * side, and turn it into an inner join. + * + * Semijoins have no RT index of their own, so we have to identify the one + * we want by the set of relids on its righthand side. + */ +static bool +reduce_semijoin_in_jointree(Node *jtnode, Relids syn_righthand) +{ + if (jtnode == NULL) + return false; + if (IsA(jtnode, RangeTblRef)) + { + /* nothing to do here */ + } + else if (IsA(jtnode, FromExpr)) + { + FromExpr *f = (FromExpr *) jtnode; + ListCell *l; + + foreach(l, f->fromlist) + { + if (reduce_semijoin_in_jointree((Node *) lfirst(l), syn_righthand)) + return true; + } + } + else if (IsA(jtnode, JoinExpr)) + { + JoinExpr *j = (JoinExpr *) jtnode; + + if (j->jointype == JOIN_SEMI && + bms_equal(get_relids_in_jointree(j->rarg, true, false), + syn_righthand)) + { + j->jointype = JOIN_INNER; + return true; + } + if (reduce_semijoin_in_jointree(j->larg, syn_righthand)) + return true; + if (reduce_semijoin_in_jointree(j->rarg, syn_righthand)) + return true; + } + else + elog(ERROR, "unrecognized jointree node type: %d", + (int) nodeTag(jtnode)); + + return false; } @@ -1762,493 +1245,430 @@ is_innerrel_unique_for(PlannerInfo *root, } /* - * Update EC members to point to the remaining relation instead of the removed - * one, removing duplicates. - * - * Restriction clauses for base relations are already distributed to - * the respective baserestrictinfo lists (see - * generate_implied_equalities_for_column). The above code has already processed - * this list and updated these clauses to reference the remaining - * relation, so that we can skip them here based on their relids. + * Remove the toRemove relation after we have proven that it participates only + * in an unneeded unique self-join with toKeep. * - * Likewise, we have already processed the join clauses that join the - * removed relation to the remaining one. + * The removal is done by deleting the relation's RangeTblRef from the + * jointree and then pointing everything that referenced it at the relation we + * are keeping. All the conditions that were attached to the removed relation + * thereby become conditions on the remaining one, which is what we want: + * we've proven that the two relations select the same rows. Note that + * this change requires us to hoist those conditions up to someplace + * syntactically enclosing toKeep. * - * Finally, there might be join clauses tying the removed relation to - * some third relation. We can't just delete the source clauses and - * regenerate them from the EC because the corresponding equality - * operators might be missing (see the handling of ec_broken). - * Therefore, we will update the references in the source clauses. - * - * Derived clauses can be generated again, so it is simpler just to - * delete them. + * kmark and rmark are the PlanRowMarks (if any) for the kept and removed + * relations. We could re-locate those, but the caller already found them. */ static void -update_eclasses(EquivalenceClass *ec, int from, int to) +remove_self_join_rel(PlannerInfo *root, + RelOptInfo *toKeep, RelOptInfo *toRemove, + PlanRowMark *kmark, PlanRowMark *rmark) { - List *new_members = NIL; - List *new_sources = NIL; + Node *orphan_quals = NULL; + int nremoved = 0; + Node *hoist_quals = NULL; + bool found_relid = false; + + Assert(toKeep->relid > 0); + Assert(toRemove->relid > 0); + + /* We verify that exactly one reference gets removed from the jointree */ + root->parse->jointree = (FromExpr *) + remove_rel_from_jointree((Node *) root->parse->jointree, + toRemove->relid, + &orphan_quals, &nremoved); + if (nremoved != 1) + elog(ERROR, "failed to find relation %d in jointree", toRemove->relid); + /* The topmost FromExpr can't have gone away, so nothing can be orphaned */ + Assert(root->parse->jointree != NULL); + Assert(orphan_quals == NULL); /* - * We don't expect any EC child members to exist at this point. Ensure - * that's the case, otherwise, we might be getting asked to do something - * this function hasn't been coded for. + * Replace all references to the removed relation. Note that this must + * happen after the jointree surgery, else we'd not be able to tell the + * two relations' RangeTblRefs apart. */ - Assert(ec->ec_childmembers == NULL); + ChangeVarNodes((Node *) root->parse, toRemove->relid, toKeep->relid, 0); - foreach_node(EquivalenceMember, em, ec->ec_members) - { - bool is_redundant = false; + /* + * processed_tlist shares some but not all of its nodes with + * parse->targetList, so it has to be processed separately. (That's + * harmless: ChangeVarNodes works in-place, and the second visit to a + * shared node finds nothing to change.) + */ + ChangeVarNodes((Node *) root->processed_tlist, toRemove->relid, + toKeep->relid, 0); + + /* There could be references in the append_rel_list, too */ + if (root->append_rel_list != NIL) + ChangeVarNodes((Node *) root->append_rel_list, toRemove->relid, + toKeep->relid, 0); + + /* Clean up the quals that the substitution has messed with */ + fixup_selfjoin_jointree(root, (Node *) root->parse->jointree, + toKeep->relid, + &hoist_quals, &found_relid); + /* We shouldn't have any leftover quals, and we must have found toKeep */ + Assert(hoist_quals == NULL); + Assert(found_relid); - if (!bms_is_member(from, em->em_relids)) + /* + * If the removed relation has a row mark, transfer it to the remaining + * one. + * + * If both rels have row marks, just keep the one corresponding to the + * remaining relation because we verified earlier that they have the same + * strength. + */ + if (rmark) + { + if (kmark) { - new_members = lappend(new_members, em); - continue; - } - - em->em_relids = adjust_relid_set(em->em_relids, from, to); - em->em_jdomain->jd_relids = adjust_relid_set(em->em_jdomain->jd_relids, from, to); - - /* We only process inner joins */ - ChangeVarNodesExtended((Node *) em->em_expr, from, to, 0, - replace_relid_callback); + Assert(kmark->markType == rmark->markType); - foreach_node(EquivalenceMember, other, new_members) + root->rowMarks = list_delete_ptr(root->rowMarks, rmark); + } + else { - if (!equal(em->em_relids, other->em_relids)) - continue; + /* Shouldn't have inheritance children yet. */ + Assert(rmark->rti == rmark->prti); - if (equal(em->em_expr, other->em_expr)) - { - is_redundant = true; - break; - } + rmark->rti = rmark->prti = toKeep->relid; } - - if (!is_redundant) - new_members = lappend(new_members, em); } +} - list_free(ec->ec_members); - ec->ec_members = new_members; - - ec_clear_derived_clauses(ec); - - /* Update EC source expressions */ - foreach_node(RestrictInfo, rinfo, ec->ec_sources) +/* + * remove_rel_from_jointree + * Delete the RangeTblRef for the given relation from the query's + * jointree. + * + * This is used for self-join elimination, where the removed relation's + * qual conditions must all be preserved (they will be transposed onto the + * remaining relation afterwards). Hence, if dropping the RangeTblRef leaves + * a JoinExpr or FromExpr with nothing under it, we can't simply drop that + * node; we hand its quals back to the caller in *orphan_quals, to be merged + * into the nearest enclosing node that still has some content. That's a + * valid transformation only for inner joins, but a jointree node can't become + * empty at an outer join here: remove_self_joins_one_group() insists that the + * two relations be on the same side of every outer join, so the relation we + * are keeping would have to be in the emptied subtree too. + * + * *nremoved is incremented by the number of RangeTblRefs removed (there + * should be exactly one, but the caller checks that). + */ +static Node * +remove_rel_from_jointree(Node *jtnode, int relid, + Node **orphan_quals, int *nremoved) +{ + if (jtnode == NULL) + return NULL; + if (IsA(jtnode, RangeTblRef)) { - bool is_redundant = false; + RangeTblRef *rtr = (RangeTblRef *) jtnode; - if (!bms_is_member(from, rinfo->required_relids)) + if (rtr->rtindex == relid) { - new_sources = lappend(new_sources, rinfo); - continue; + (*nremoved)++; + return NULL; } + } + else if (IsA(jtnode, FromExpr)) + { + FromExpr *f = (FromExpr *) jtnode; + List *newfromlist = NIL; + Node *sub_orphans = NULL; + ListCell *l; - ChangeVarNodesExtended((Node *) rinfo, from, to, 0, - replace_relid_callback); - - /* - * After switching the clause to the remaining relation, check it for - * redundancy with existing ones. We don't have to check for - * redundancy with derived clauses, because we've just deleted them. - */ - foreach_node(RestrictInfo, other, new_sources) + foreach(l, f->fromlist) { - if (!equal(rinfo->clause_relids, other->clause_relids)) - continue; + Node *newchild; - if (equal(rinfo->clause, other->clause)) - { - is_redundant = true; - break; - } + newchild = remove_rel_from_jointree((Node *) lfirst(l), relid, + &sub_orphans, nremoved); + if (newchild != NULL) + newfromlist = lappend(newfromlist, newchild); + } + f->fromlist = newfromlist; + f->quals = merge_quals(sub_orphans, f->quals); + if (newfromlist == NIL) + { + /* Nothing left here, so pass our quals up to the parent */ + *orphan_quals = merge_quals(f->quals, *orphan_quals); + return NULL; } + } + else if (IsA(jtnode, JoinExpr)) + { + JoinExpr *j = (JoinExpr *) jtnode; + Node *sub_orphans = NULL; + + j->larg = remove_rel_from_jointree(j->larg, relid, + &sub_orphans, nremoved); + j->rarg = remove_rel_from_jointree(j->rarg, relid, + &sub_orphans, nremoved); + if (j->larg == NULL || j->rarg == NULL) + { + Node *surviving = (j->larg != NULL) ? j->larg : j->rarg; + Node *quals = merge_quals(sub_orphans, j->quals); - if (!is_redundant) - new_sources = lappend(new_sources, rinfo); + /* As explained above, this can only happen for an inner join */ + Assert(j->jointype == JOIN_INNER); + /* We can't have removed both children */ + Assert(surviving != NULL); + + /* + * Replace the join by a FromExpr, so that the surviving side's + * rows are still filtered by the join's conditions. + */ + return (Node *) makeFromExpr(list_make1(surviving), quals); + } + /* A subtree that survives never hands any quals back to us */ + Assert(sub_orphans == NULL); } + else + elog(ERROR, "unrecognized jointree node type: %d", + (int) nodeTag(jtnode)); - list_free(ec->ec_sources); - ec->ec_sources = new_sources; - ec->ec_relids = adjust_relid_set(ec->ec_relids, from, to); + return jtnode; } /* - * "Logically" compares two RestrictInfo's ignoring the 'rinfo_serial' field, - * which makes almost every RestrictInfo unique. This type of comparison is - * useful when removing duplicates while moving RestrictInfo's from removed - * relation to remaining relation during self-join elimination. + * merge_quals + * Combine two jointree qual conditions. + * + * quals1 should be the quals from the lower of the two jointree levels, + * so that those quals get applied first. * - * XXX: In the future, we might remove the 'rinfo_serial' field completely and - * get rid of this function. + * Jointree quals have been through preprocess_expression() by now, so each + * one is either NULL or an implicitly-ANDed List. */ -static bool -restrict_infos_logically_equal(RestrictInfo *a, RestrictInfo *b) +static Node * +merge_quals(Node *quals1, Node *quals2) { - int saved_rinfo_serial = a->rinfo_serial; - bool result; - - a->rinfo_serial = b->rinfo_serial; - result = equal(a, b); - a->rinfo_serial = saved_rinfo_serial; - - return result; + if (quals1 == NULL) + return quals2; + if (quals2 == NULL) + return quals1; + return (Node *) list_concat(castNode(List, quals1), + castNode(List, quals2)); } /* - * This function adds all non-redundant clauses to the keeping relation - * during self-join elimination. That is a contradictory operation. On the - * one hand, we reduce the length of the `restrict` lists, which can - * impact planning or executing time. Additionally, we improve the - * accuracy of cardinality estimation. On the other hand, it is one more - * place that can make planning time much longer in specific cases. It - * would have been better to avoid calling the equal() function here, but - * it's the only way to detect duplicated inequality expressions. + * fixup_selfjoin_jointree + * Clean up the query's jointree quals after self-join elimination has + * merged one relation into another. (relid is the kept relation.) * - * (*keep_rinfo_list) is given by pointer because it might be altered by - * distribute_restrictinfo_to_rels(). + * See fixup_selfjoin_quals() for what needs fixing locally to each qual list. + * In addition, we need to check quals to see if they refer to relid, and if + * so make sure they get hoisted to someplace syntactically above relid. + * Do that using a "hoist_quals" in/out parameter similar to "orphan_quals" + * in remove_rel_from_jointree. (We can't readily merge these concerns into + * a single pass, since remove_rel_from_jointree must run before we relabel + * the removed rel's Vars.) In addition, *found_relid is set true if + * the subtree rooted at jtnode is found to contain relid's RangeTblRef, + * so that we can tell when to stop hoisting quals. + * If a qual gets hoisted up, we apply fixup_selfjoin_quals() to it only + * after it reaches its final level. This rule improves the odds of + * detecting duplicate quals. */ static void -add_non_redundant_clauses(PlannerInfo *root, - List *rinfo_candidates, - List **keep_rinfo_list, - Index removed_relid) +fixup_selfjoin_jointree(PlannerInfo *root, Node *jtnode, int relid, + Node **hoist_quals, bool *found_relid) { - foreach_node(RestrictInfo, rinfo, rinfo_candidates) + if (jtnode == NULL) + return; + if (IsA(jtnode, RangeTblRef)) { - bool is_redundant = false; + RangeTblRef *rtr = (RangeTblRef *) jtnode; - Assert(!bms_is_member(removed_relid, rinfo->required_relids)); + if (rtr->rtindex == relid) + { + Assert(!*found_relid); + *found_relid = true; + } + } + else if (IsA(jtnode, FromExpr)) + { + FromExpr *f = (FromExpr *) jtnode; + Node *sub_hoist_quals = NULL; + bool sub_found_relid = false; + ListCell *l; - foreach_node(RestrictInfo, src, (*keep_rinfo_list)) + foreach(l, f->fromlist) + fixup_selfjoin_jointree(root, (Node *) lfirst(l), relid, + &sub_hoist_quals, &sub_found_relid); + if (sub_found_relid) { - if (!bms_equal(src->clause_relids, rinfo->clause_relids)) - /* Can't compare trivially different clauses */ - continue; + /* This FromExpr covers relid, so OK to stop hoisting quals here */ + f->quals = merge_quals(sub_hoist_quals, f->quals); + Assert(!*found_relid); + *found_relid = true; + } + else + { + /* We might need to hoist some of our own quals too */ + List *hoistable = NIL; + List *keepable = NIL; - if (src == rinfo || - (rinfo->parent_ec != NULL && - src->parent_ec == rinfo->parent_ec) || - restrict_infos_logically_equal(rinfo, src)) + foreach_ptr(Node, qual, castNode(List, f->quals)) { - is_redundant = true; - break; + if (bms_is_member(relid, pull_varnos(root, qual))) + hoistable = lappend(hoistable, qual); + else + keepable = lappend(keepable, qual); } + f->quals = (Node *) keepable; + sub_hoist_quals = merge_quals(sub_hoist_quals, (Node *) hoistable); + *hoist_quals = merge_quals(sub_hoist_quals, *hoist_quals); } - if (!is_redundant) - distribute_restrictinfo_to_rels(root, rinfo); - } -} - -/* - * A custom callback for ChangeVarNodesExtended() providing Self-join - * elimination (SJE) related functionality - * - * SJE needs to skip the RangeTblRef node type. During SJE's last - * step, remove_rel_from_joinlist() removes remaining RangeTblRefs - * with target relid. If ChangeVarNodes() replaces the target relid - * before, remove_rel_from_joinlist() would fail to identify the nodes - * to delete. - * - * SJE also needs to change the relids within RestrictInfo's. - */ -static bool -replace_relid_callback(Node *node, ChangeVarNodes_context *context) -{ - if (IsA(node, RangeTblRef)) - { - return true; + f->quals = (Node *) fixup_selfjoin_quals(root, + castNode(List, f->quals), + relid); } - else if (IsA(node, RestrictInfo)) + else if (IsA(jtnode, JoinExpr)) { - RestrictInfo *rinfo = (RestrictInfo *) node; - int relid = -1; - bool is_req_equal = - (rinfo->required_relids == rinfo->clause_relids); - bool clause_relids_is_multiple = - (bms_membership(rinfo->clause_relids) == BMS_MULTIPLE); - - /* - * Recurse down into clauses if the target relation is present in - * clause_relids or required_relids. We must check required_relids - * because the relation not present in clause_relids might still be - * present somewhere in orclause. - */ - if (bms_is_member(context->rt_index, rinfo->clause_relids) || - bms_is_member(context->rt_index, rinfo->required_relids)) + JoinExpr *j = (JoinExpr *) jtnode; + Node *sub_hoist_quals = NULL; + bool sub_found_relid = false; + + fixup_selfjoin_jointree(root, j->larg, relid, + &sub_hoist_quals, &sub_found_relid); + fixup_selfjoin_jointree(root, j->rarg, relid, + &sub_hoist_quals, &sub_found_relid); + if (sub_found_relid) { - Relids new_clause_relids; - - ChangeVarNodesWalkExpression((Node *) rinfo->clause, context); - ChangeVarNodesWalkExpression((Node *) rinfo->orclause, context); - - new_clause_relids = adjust_relid_set(rinfo->clause_relids, - context->rt_index, - context->new_index); - - /* - * Incrementally adjust num_base_rels based on the change of - * clause_relids, which could contain both base relids and - * outer-join relids. This operation is legal until we remove - * only baserels. - */ - rinfo->num_base_rels -= bms_num_members(rinfo->clause_relids) - - bms_num_members(new_clause_relids); - - rinfo->clause_relids = new_clause_relids; - rinfo->left_relids = - adjust_relid_set(rinfo->left_relids, context->rt_index, context->new_index); - rinfo->right_relids = - adjust_relid_set(rinfo->right_relids, context->rt_index, context->new_index); + /* This JoinExpr covers relid, so OK to stop hoisting quals here */ + j->quals = merge_quals(sub_hoist_quals, j->quals); + Assert(!*found_relid); + *found_relid = true; } - - if (is_req_equal) - rinfo->required_relids = rinfo->clause_relids; else - rinfo->required_relids = - adjust_relid_set(rinfo->required_relids, context->rt_index, context->new_index); - - rinfo->outer_relids = - adjust_relid_set(rinfo->outer_relids, context->rt_index, context->new_index); - rinfo->incompatible_relids = - adjust_relid_set(rinfo->incompatible_relids, context->rt_index, context->new_index); - - if (rinfo->mergeopfamilies && - bms_get_singleton_member(rinfo->clause_relids, &relid) && - clause_relids_is_multiple && - relid == context->new_index && IsA(rinfo->clause, OpExpr)) { - Expr *leftOp; - Expr *rightOp; - - leftOp = (Expr *) get_leftop(rinfo->clause); - rightOp = (Expr *) get_rightop(rinfo->clause); + /* We might need to hoist some of our own quals too */ + List *hoistable = NIL; + List *keepable = NIL; - /* - * For self-join elimination, changing varnos could transform - * "t1.a = t2.a" into "t1.a = t1.a". That is always true as long - * as "t1.a" is not null. We use equal() to check for such a - * case, and then we replace the qual with a check for not null - * (NullTest). - */ - if (leftOp != NULL && equal(leftOp, rightOp)) + foreach_ptr(Node, qual, castNode(List, j->quals)) { - NullTest *ntest = makeNode(NullTest); - - ntest->arg = leftOp; - ntest->nulltesttype = IS_NOT_NULL; - ntest->argisrow = false; - ntest->location = -1; - rinfo->clause = (Expr *) ntest; - rinfo->mergeopfamilies = NIL; - rinfo->left_em = NULL; - rinfo->right_em = NULL; + if (bms_is_member(relid, pull_varnos(root, qual))) + hoistable = lappend(hoistable, qual); + else + keepable = lappend(keepable, qual); } - Assert(rinfo->orclause == NULL); + j->quals = (Node *) keepable; + sub_hoist_quals = merge_quals(sub_hoist_quals, (Node *) hoistable); + /* We should never need to hoist quals above an outer join */ + Assert(sub_hoist_quals == NULL || j->jointype == JOIN_INNER); + *hoist_quals = merge_quals(sub_hoist_quals, *hoist_quals); } - return true; + j->quals = (Node *) fixup_selfjoin_quals(root, + castNode(List, j->quals), + relid); } - - return false; + else + elog(ERROR, "unrecognized jointree node type: %d", + (int) nodeTag(jtnode)); } /* - * Remove a relation after we have proven that it participates only in an - * unneeded unique self-join. + * fixup_selfjoin_quals + * Clean up one qual list after self-join elimination. * - * Replace any links in planner info structures. + * Two things need fixing here. First, a join clause such as "t1.a = t2.a" + * has turned into "t1.a = t1.a". For a strict mergejoinable operator that + * means "t1.a IS NOT NULL", and we should make the substitution, for two + * reasons: + * 1. It will typically result in better selectivity estimates. + * 2. EquivalenceClass processing is likely to make the substitution + * if we don't. While not directly harmful, we'd then fail to + * recognize it as a duplicate of a user-written "t1.a IS NOT NULL" + * clause, again leading to bad selectivity estimates. + * Second, conditions that were written against the two relations separately + * may now be identical, and we don't want to apply the same condition twice + * (much less double-count its selectivity). * - * Transfer join and restriction clauses from the removed relation to the - * remaining one. We change the Vars of the clause to point to the - * remaining relation instead of the removed one. The clauses that require - * a subset of joinrelids become restriction clauses of the remaining - * relation, and others remain join clauses. We append them to - * baserestrictinfo and joininfo, respectively, trying not to introduce - * duplicates. + * We only touch the top-level conjuncts of the list. There, turning a NULL + * result into FALSE makes no difference, whereas below a NOT it would, + * invalidating the IS NOT NULL substitution. EquivalenceClass processing + * will not be applied to sub-clauses, and cleaning up duplicates in them + * seems like more trouble than it's worth. Also, we only consider clauses + * that mention the relation we merged into, so that we don't change the + * treatment of anything we didn't touch. * - * We also have to process the 'joinclauses' list here, because it - * contains EC-derived join clauses which must become filter clauses. It - * is not enough to just correct the ECs because the EC-derived - * restrictions are generated before join removal (see - * generate_base_implied_equalities). - * - * NOTE: Remember to keep the code in sync with PlannerInfo to be sure all - * cached relids and relid bitmapsets can be correctly cleaned during the - * self-join elimination procedure. + * Since this is not a correctness issue but just an optimization opportunity, + * we likewise don't worry about recognizing duplicates that appear in + * different qual lists. */ -static void -remove_self_join_rel(PlannerInfo *root, PlanRowMark *kmark, PlanRowMark *rmark, - RelOptInfo *toKeep, RelOptInfo *toRemove, - List *restrictlist) +static List * +fixup_selfjoin_quals(PlannerInfo *root, List *quals, int relid) { - List *joininfos; - ListCell *lc; - int i; - List *jinfo_candidates = NIL; - List *binfo_candidates = NIL; - - Assert(toKeep->relid > 0); - Assert(toRemove->relid > 0); - - /* - * Replace the index of the removing table with the keeping one. The - * technique of removing/distributing restrictinfo is used here to attach - * just appeared (for keeping relation) join clauses and avoid adding - * duplicates of those that already exist in the joininfo list. - */ - joininfos = list_copy(toRemove->joininfo); - foreach_node(RestrictInfo, rinfo, joininfos) - { - remove_join_clause_from_rels(root, rinfo, rinfo->required_relids); - ChangeVarNodesExtended((Node *) rinfo, toRemove->relid, toKeep->relid, - 0, replace_relid_callback); - - if (bms_membership(rinfo->required_relids) == BMS_MULTIPLE) - jinfo_candidates = lappend(jinfo_candidates, rinfo); - else - binfo_candidates = lappend(binfo_candidates, rinfo); - } - - /* - * Concatenate restrictlist to the list of base restrictions of the - * removing table just to simplify the replacement procedure: all of them - * weren't connected to any keeping relations and need to be added to some - * rels. - */ - toRemove->baserestrictinfo = list_concat(toRemove->baserestrictinfo, - restrictlist); - foreach_node(RestrictInfo, rinfo, toRemove->baserestrictinfo) - { - ChangeVarNodesExtended((Node *) rinfo, toRemove->relid, toKeep->relid, - 0, replace_relid_callback); - - if (bms_membership(rinfo->required_relids) == BMS_MULTIPLE) - jinfo_candidates = lappend(jinfo_candidates, rinfo); - else - binfo_candidates = lappend(binfo_candidates, rinfo); - } - - /* - * Now, add all non-redundant clauses to the keeping relation. - */ - add_non_redundant_clauses(root, binfo_candidates, - &toKeep->baserestrictinfo, toRemove->relid); - add_non_redundant_clauses(root, jinfo_candidates, - &toKeep->joininfo, toRemove->relid); - - list_free(binfo_candidates); - list_free(jinfo_candidates); - - /* - * Arrange equivalence classes, mentioned removing a table, with the - * keeping one: varno of removing table should be replaced in members and - * sources lists. Also, remove duplicated elements if this replacement - * procedure created them. - */ - i = -1; - while ((i = bms_next_member(toRemove->eclass_indexes, i)) >= 0) - { - EquivalenceClass *ec = (EquivalenceClass *) list_nth(root->eq_classes, i); - - update_eclasses(ec, toRemove->relid, toKeep->relid); - toKeep->eclass_indexes = bms_add_member(toKeep->eclass_indexes, i); - } - - /* - * Transfer the targetlist and attr_needed flags. - */ - - foreach(lc, toRemove->reltarget->exprs) - { - Node *node = lfirst(lc); - - ChangeVarNodesExtended(node, toRemove->relid, toKeep->relid, 0, - replace_relid_callback); - if (!list_member(toKeep->reltarget->exprs, node)) - toKeep->reltarget->exprs = lappend(toKeep->reltarget->exprs, node); - } + List *result = NIL; + ListCell *l; - for (i = toKeep->min_attr; i <= toKeep->max_attr; i++) + foreach(l, quals) { - int attno = i - toKeep->min_attr; - - toRemove->attr_needed[attno] = adjust_relid_set(toRemove->attr_needed[attno], - toRemove->relid, toKeep->relid); - toKeep->attr_needed[attno] = bms_add_members(toKeep->attr_needed[attno], - toRemove->attr_needed[attno]); - } + Node *qual = (Node *) lfirst(l); - /* - * If the removed relation has a row mark, transfer it to the remaining - * one. - * - * If both rels have row marks, just keep the one corresponding to the - * remaining relation because we verified earlier that they have the same - * strength. - */ - if (rmark) - { - if (kmark) - { - Assert(kmark->markType == rmark->markType); - - root->rowMarks = list_delete_ptr(root->rowMarks, rmark); - } - else + if (bms_is_member(relid, pull_varnos(root, qual))) { - /* Shouldn't have inheritance children here. */ - Assert(rmark->rti == rmark->prti); - - rmark->rti = rmark->prti = toKeep->relid; + qual = replace_selfjoin_qual(qual); + /* Drop it if the substitution has made it a duplicate */ + if (list_member(result, qual)) + continue; } + result = lappend(result, qual); } - /* - * Replace varno in all the query structures, except nodes RangeTblRef - * otherwise later remove_rel_from_joinlist will yield errors. - */ - ChangeVarNodesExtended((Node *) root->parse, toRemove->relid, toKeep->relid, - 0, replace_relid_callback); - - /* Replace links in the planner info */ - remove_rel_from_query(root, toRemove, toKeep->relid, NULL, NULL); - - /* At last, replace varno in root targetlist and HAVING clause */ - ChangeVarNodesExtended((Node *) root->processed_tlist, toRemove->relid, - toKeep->relid, 0, replace_relid_callback); - ChangeVarNodesExtended((Node *) root->processed_groupClause, - toRemove->relid, toKeep->relid, 0, - replace_relid_callback); - - adjust_relid_set(root->all_result_relids, toRemove->relid, toKeep->relid); - adjust_relid_set(root->leaf_result_relids, toRemove->relid, toKeep->relid); - - /* - * There may be references to the rel in root->fkey_list, but if so, - * match_foreign_keys_to_quals() will get rid of them. - */ - - /* - * Finally, remove the rel from the baserel array to prevent it from being - * referenced again. (We can't do this earlier because - * remove_join_clause_from_rels will touch it.) - */ - root->simple_rel_array[toRemove->relid] = NULL; - root->simple_rte_array[toRemove->relid] = NULL; - - /* And nuke the RelOptInfo, just in case there's another access path. */ - pfree(toRemove); + return result; +} +/* + * replace_selfjoin_qual + * Replace one "X = X" qual by "X IS NOT NULL", if it is one. + */ +static Node * +replace_selfjoin_qual(Node *qual) +{ + OpExpr *opexpr; + Node *leftop; + Node *rightop; + NullTest *ntest; + + /* See if it looks like "X op X" */ + if (!is_opclause(qual)) + return qual; + opexpr = (OpExpr *) qual; + if (list_length(opexpr->args) != 2) + return qual; + leftop = get_leftop((Expr *) opexpr); + rightop = get_rightop((Expr *) opexpr); + if (!equal(leftop, rightop)) + return qual; /* - * Now repeat construction of attr_needed bits coming from all other - * sources. + * The operator must be strict and behave like btree equality, else we + * can't conclude that it yields true for any non-null input. And the + * input had better not be volatile, else the two evaluations might not + * agree. If either condition doesn't hold, the clause is not a candidate + * to be an equivalence, so we needn't worry about it getting replaced by + * equivclass.c. */ - rebuild_placeholder_attr_needed(root); - rebuild_joinclause_attr_needed(root); - rebuild_eclass_attr_needed(root); - rebuild_lateral_attr_needed(root); + set_opfuncid(opexpr); + if (!func_strict(opexpr->opfuncid)) + return qual; + if (!op_mergejoinable(opexpr->opno, exprType(leftop))) + return qual; + if (contain_volatile_functions(leftop)) + return qual; + + /* OK, replace it */ + ntest = makeNode(NullTest); + ntest->arg = (Expr *) leftop; + ntest->nulltesttype = IS_NOT_NULL; + ntest->argisrow = false; /* correct even if composite arg */ + ntest->location = -1; + return (Node *) ntest; } /* @@ -2273,7 +1693,12 @@ split_selfjoin_quals(PlannerInfo *root, List *joinquals, List **selfjoinquals, Node *leftexpr; Node *rightexpr; - /* In general, clause looks like F(arg1) = G(arg2) */ + /* + * Since the given joinquals all came from + * generate_join_implied_equalities, they ought to look like equality + * operators on single-relation expressions. But let's check that. + * Anything that doesn't look like that can be dumped into ojoinquals. + */ if (!rinfo->mergeopfamilies || bms_num_members(rinfo->clause_relids) != 2 || bms_membership(rinfo->left_relids) != BMS_SINGLETON || @@ -2304,10 +1729,9 @@ split_selfjoin_quals(PlannerInfo *root, List *joinquals, List **selfjoinquals, * when we have cast of the same var to different (but compatible) * types. */ - ChangeVarNodesExtended(rightexpr, - bms_singleton_member(rinfo->right_relids), - bms_singleton_member(rinfo->left_relids), 0, - replace_relid_callback); + ChangeVarNodes(rightexpr, + bms_singleton_member(rinfo->right_relids), + bms_singleton_member(rinfo->left_relids), 0); if (equal(leftexpr, rightexpr)) sjoinquals = lappend(sjoinquals, rinfo); @@ -2343,8 +1767,7 @@ match_unique_clauses(PlannerInfo *root, RelOptInfo *outer, List *uclauses, bms_is_empty(rinfo->right_relids)); clause = (Expr *) copyObject(rinfo->clause); - ChangeVarNodesExtended((Node *) clause, relid, outer->relid, 0, - replace_relid_callback); + ChangeVarNodes((Node *) clause, relid, outer->relid, 0); iclause = bms_is_empty(rinfo->left_relids) ? get_rightop(clause) : get_leftop(clause); @@ -2389,12 +1812,23 @@ match_unique_clauses(PlannerInfo *root, RelOptInfo *outer, List *uclauses, * Find and remove unique self-joins in a group of base relations that have * the same Oid. * - * Returns a set of relids that were removed. + * Return true if we removed any joins. + * + * After a removal, we continue searching for more removals, even though the + * tests will be using derived data that is now partially stale. That is safe + * because we are trying to prove that a candidate pair of relations must + * match the same row, and the stale data can only omit quals, never invent + * them. The removed relation's quals are moved onto the kept relation in + * the jointree but not into its baserestrictinfo, and no other derived data + * changes. A proof made from a subset of the applicable quals remains valid + * when the rest are added, since extra quals can only remove rows from the + * join. So a pass may miss a removal that a later pass will find, but it + * cannot make one that isn't justified. */ -static Relids +static bool remove_self_joins_one_group(PlannerInfo *root, Relids relids) { - Relids result = NULL; + bool removed = false; int k; /* Index of kept relation */ int r = -1; /* Index of removed relation */ @@ -2402,8 +1836,8 @@ remove_self_joins_one_group(PlannerInfo *root, Relids relids) { RelOptInfo *rrel = root->simple_rel_array[r]; + /* k iterates over the relids after r */ k = r; - while ((k = bms_next_member(relids, k)) > 0) { Relids joinrelids = NULL; @@ -2423,8 +1857,8 @@ remove_self_joins_one_group(PlannerInfo *root, Relids relids) /* * It is impossible to eliminate the join of two relations if they - * belong to different rules of order. Otherwise, the planner - * can't find any variants of the correct query plan. + * are not on the same side of every outer join. Otherwise, the + * planner can't find any variants of the correct query plan. */ foreach(lc, root->join_info_list) { @@ -2538,32 +1972,35 @@ remove_self_joins_one_group(PlannerInfo *root, Relids relids) if (!match_unique_clauses(root, rrel, uclauses, krel->relid)) continue; + /* OK, remove rrel from the query */ + remove_self_join_rel(root, krel, rrel, kmark, rmark); + removed = true; + /* - * Remove rrel ReloptInfo from the planner structures and the - * corresponding row mark. + * Since relation r is now gone, we mustn't keep looking for + * matches to it. But we can keep scanning later relids members + * for additional join pairs. */ - remove_self_join_rel(root, kmark, rmark, krel, rrel, restrictlist); - - result = bms_add_member(result, r); - - /* We have removed the outer relation, try the next one. */ break; } } - return result; + return removed; } /* - * Gather indexes of base relations from the joinlist and try to eliminate self - * joins. + * Gather indexes of base relations from the joinlist and try to eliminate + * self-joins. + * + * Return true if we removed any joins. */ -static Relids -remove_self_joins_recurse(PlannerInfo *root, List *joinlist, Relids toRemove) +static bool +remove_self_joins_recurse(PlannerInfo *root, List *joinlist) { + bool removed = false; ListCell *jl; Relids relids = NULL; - SelfJoinCandidate *candidates = NULL; + SelfJoinCandidate *candidates; int i; int j; int numRels; @@ -2582,7 +2019,7 @@ remove_self_joins_recurse(PlannerInfo *root, List *joinlist, Relids toRemove) * We only consider ordinary relations as candidates to be * removed, and these relations should not have TABLESAMPLE * clauses specified. Removing a relation with TABLESAMPLE clause - * could potentially change the syntax of the query. Because of + * could potentially change the semantics of the query. Because of * UPDATE/DELETE EPQ mechanism, currently Query->resultRelation or * Query->mergeTargetRelation associated rel cannot be eliminated. */ @@ -2598,9 +2035,8 @@ remove_self_joins_recurse(PlannerInfo *root, List *joinlist, Relids toRemove) } else if (IsA(jlnode, List)) { - /* Recursively go inside the sub-joinlist */ - toRemove = remove_self_joins_recurse(root, (List *) jlnode, - toRemove); + /* Recursively perform SJE within the sub-joinlist */ + removed |= remove_self_joins_recurse(root, (List *) jlnode); } else elog(ERROR, "unrecognized joinlist node type: %d", @@ -2609,9 +2045,9 @@ remove_self_joins_recurse(PlannerInfo *root, List *joinlist, Relids toRemove) numRels = bms_num_members(relids); - /* Need at least two relations for the join */ + /* No work if not at least two relations at this level */ if (numRels < 2) - return toRemove; + return removed; /* ... but don't fail to report sub-removals */ /* * In order to find relations with the same oid we first build an array of @@ -2633,15 +2069,13 @@ remove_self_joins_recurse(PlannerInfo *root, List *joinlist, Relids toRemove) /* * Iteratively form a group of relation indexes with the same oid and - * launch the routine that detects self-joins in this group and removes - * excessive range table entries. + * launch the routine that detects self-joins in this group. * - * At the end of the iteration, exclude the group from the overall relids - * list. So each next iteration of the cycle will involve less and less - * value of relids. + * We remove considered relations from relids as we scan, so that that set + * should be empty at the end. */ i = 0; - for (j = 1; j < numRels + 1; j++) + for (j = 1; j <= numRels; j++) { if (j == numRels || candidates[j].reloid != candidates[i].reloid) { @@ -2649,7 +2083,6 @@ remove_self_joins_recurse(PlannerInfo *root, List *joinlist, Relids toRemove) { /* Create a group of relation indexes with the same oid */ Relids group = NULL; - Relids removed; while (i < j) { @@ -2658,35 +2091,25 @@ remove_self_joins_recurse(PlannerInfo *root, List *joinlist, Relids toRemove) } relids = bms_del_members(relids, group); - /* - * Try to remove self-joins from a group of identical entries. - * Make the next attempt iteratively - if something is deleted - * from a group, changes in clauses and equivalence classes - * can give us a chance to find more candidates. - */ - do - { - Assert(!bms_overlap(group, toRemove)); - removed = remove_self_joins_one_group(root, group); - toRemove = bms_add_members(toRemove, removed); - group = bms_del_members(group, removed); - } while (!bms_is_empty(removed) && - bms_membership(group) == BMS_MULTIPLE); - bms_free(removed); + /* Try to remove self-joins from the group */ + removed |= remove_self_joins_one_group(root, group); bms_free(group); } else { - /* Single relation, just remove it from the set */ - relids = bms_del_member(relids, candidates[i].relid); - i = j; + /* Nothing to do with this group, just drop it from the set */ + while (i < j) + { + relids = bms_del_member(relids, candidates[i].relid); + i++; + } } } } Assert(bms_is_empty(relids)); - return toRemove; + return removed; } /* @@ -2728,45 +2151,26 @@ self_join_candidates_cmp(const void *a, const void *b) * go over each set with the same Oid, and consider each pair of relations * in this set. * - * To remove the join, we mark one of the participating relations as dead - * and rewrite all references to it to point to the remaining relation. - * This includes modifying RestrictInfos, EquivalenceClasses, and - * EquivalenceMembers. We also have to modify the row marks. The join clauses - * of the removed relation become either restriction or join clauses, based on - * whether they reference any relations not participating in the removed join. + * To remove the join, we delete one of the participating relations from the + * query's jointree and rewrite all references to it to point to the remaining + * relation. We also have to modify their row marks. * - * 'joinlist' is the top-level joinlist of the query. If it has any - * references to the removed relations, we update them to point to the - * remaining ones. + * 'joinlist' is the top-level joinlist of the query; we use it to identify + * groups of relations that could be joined to each other. + * + * We return true if we removed any self-joins. If so, the caller must + * recompute everything that was derived from the jointree, and should then + * try join simplifications again since we might have exposed opportunities + * for additional simplifications. */ -List * +bool remove_useless_self_joins(PlannerInfo *root, List *joinlist) { - Relids toRemove = NULL; - int relid = -1; - + /* Skip if SJE is disabled, or if the joinlist has less than 2 members. */ if (!enable_self_join_elimination || joinlist == NIL || (list_length(joinlist) == 1 && !IsA(linitial(joinlist), List))) - return joinlist; - - /* - * Merge pairs of relations participated in self-join. Remove unnecessary - * range table entries. - */ - toRemove = remove_self_joins_recurse(root, joinlist, toRemove); - - if (unlikely(toRemove != NULL)) - { - /* At the end, remove orphaned relation links */ - while ((relid = bms_next_member(toRemove, relid)) >= 0) - { - int nremoved = 0; - - joinlist = remove_rel_from_joinlist(joinlist, relid, &nremoved); - if (nremoved != 1) - elog(ERROR, "failed to find relation %d in joinlist", relid); - } - } + return false; - return joinlist; + /* Try to merge pairs of self-joined relations. */ + return remove_self_joins_recurse(root, joinlist); } diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c index 5467e094ca7..f9831870142 100644 --- a/src/backend/optimizer/plan/planmain.c +++ b/src/backend/optimizer/plan/planmain.c @@ -54,30 +54,68 @@ RelOptInfo * query_planner(PlannerInfo *root, query_pathkeys_callback qp_callback, void *qp_extra) { - Query *parse = root->parse; + Query *parse; List *joinlist; RelOptInfo *final_rel; /* - * Init planner lists to empty. + * The join simplification steps below work by modifying parse->jointree, + * and they make no attempt to update the information we derive from it. + * So whenever one of them succeeds, we must throw away all that derived + * information and recompute it from scratch, which we do by looping back + * to "restart". We cannot loop indefinitely, because each successful + * simplification either deletes a base relation from the jointree or + * turns a semijoin into an inner join, and neither of those can be undone + * by a later pass. * - * NOTE: append_rel_list was set up by subquery_planner, so do not touch - * here. + * These initial Asserts check that the state at entry is not too complex + * for the code below to restore. There mustn't be any EquivalenceClasses + * yet, and we should have only the top-level JoinDomain. */ + Assert(root->eq_classes == NIL); + Assert(list_length(root->join_domains) == 1); + +restart: + parse = root->parse; + + /* + * Initialize information derived from the jointree to empty. + * + * It's critical that this reset every field that the steps below will + * fill in, since we may be going around this loop more than once. + * + * NOTE: append_rel_list was created earlier, so do not clear it here; + * rowMarks ditto. Join simplification must update those if necessary. + */ + root->all_baserels = NULL; + root->outer_join_rels = NULL; + root->all_query_rels = NULL; root->join_rel_list = NIL; root->join_rel_hash = NULL; root->join_rel_level = NULL; root->join_cur_level = 0; + root->eq_classes = NIL; + root->ec_merging_done = false; root->canon_pathkeys = NIL; root->left_join_clauses = NIL; root->right_join_clauses = NIL; root->full_join_clauses = NIL; root->join_info_list = NIL; + root->last_rinfo_serial = 0; root->placeholder_list = NIL; root->placeholder_array = NULL; root->placeholder_array_size = 0; + root->placeholdersFrozen = false; root->fkey_list = NIL; root->initial_rels = NIL; + root->hasPseudoConstantQuals = false; + + /* + * We don't want to delete the top-level join domain, but get rid of other + * ones so as to reset the list to initial state. deconstruct_jointree + * will take care of (re)computing the top level's jd_relids. + */ + root->join_domains = list_truncate(root->join_domains, 1); /* * Set up arrays for accessing base relations and AppendRelInfos. @@ -142,6 +180,21 @@ query_planner(PlannerInfo *root, set_cheapest(final_rel); /* + * Fill in all_result_relids and leaf_result_relids, just in + * case something looks at them (at this writing, the core + * code won't). This must match the similar stanza below. + */ + if (parse->resultRelation) + { + int rti = parse->resultRelation; + RangeTblEntry *res_rte = root->simple_rte_array[rti]; + + root->all_result_relids = bms_make_singleton(rti); + if (!res_rte->inh) + root->leaf_result_relids = bms_make_singleton(rti); + } + + /* * We don't need to run generate_base_implied_equalities, but * we do need to pretend that EC merging is complete. */ @@ -224,19 +277,30 @@ query_planner(PlannerInfo *root, * Remove any useless outer joins. Ideally this would be done during * jointree preprocessing, but the necessary information isn't available * until we've built baserel data structures and classified qual clauses. + * If we remove a join, loop back to the top and redo what we did so far. */ - joinlist = remove_useless_joins(root, joinlist); + if (remove_useless_outer_joins(root)) + goto restart; /* * Also, reduce any semijoins with unique inner rels to plain inner joins. - * Likewise, this can't be done until now for lack of needed info. + * Likewise, this can't be done until now for lack of needed info, and we + * must loop around if we find any simplifications. + */ + if (reduce_unique_semijoins(root)) + goto restart; + + /* + * Remove self joins on a unique column. Again, this couldn't be done any + * earlier, and we must loop around if we find anything to remove. */ - reduce_unique_semijoins(root); + if (remove_useless_self_joins(root, joinlist)) + goto restart; /* - * Remove self joins on a unique column. + * No more join simplifications apply, so we're done looping. Code below + * this point does not need to be able to restart. */ - joinlist = remove_useless_self_joins(root, joinlist); /* * Now distribute "placeholders" to base rels as needed. This has to be @@ -266,6 +330,22 @@ query_planner(PlannerInfo *root, extract_restriction_or_clauses(root); /* + * If there's a result relation, initialize all_result_relids to include + * it; and if we've verified that it is non-inheriting, mark it as a leaf + * target. add_other_rels_to_query() will expand these sets if the result + * relation has children. + */ + if (parse->resultRelation) + { + int rti = parse->resultRelation; + RangeTblEntry *rte = root->simple_rte_array[rti]; + + root->all_result_relids = bms_make_singleton(rti); + if (!rte->inh) + root->leaf_result_relids = bms_make_singleton(rti); + } + + /* * Now expand appendrels by adding "otherrels" for their children. We * delay this to the end so that we have as much information as possible * available for each baserel, including all restriction clauses. That diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c index cf49f8604fa..28abddea637 100644 --- a/src/backend/optimizer/plan/planner.c +++ b/src/backend/optimizer/plan/planner.c @@ -691,9 +691,8 @@ subquery_planner(PlannerGlobal *glob, Query *parse, PlannerInfo *parent_root, root->eq_classes = NIL; root->ec_merging_done = false; root->last_rinfo_serial = 0; - root->all_result_relids = - parse->resultRelation ? bms_make_singleton(parse->resultRelation) : NULL; - root->leaf_result_relids = NULL; /* we'll find out leaf-ness later */ + root->all_result_relids = NULL; + root->leaf_result_relids = NULL; root->append_rel_list = NIL; root->row_identity_vars = NIL; root->rowMarks = NIL; @@ -852,19 +851,6 @@ subquery_planner(PlannerGlobal *glob, Query *parse, PlannerInfo *parent_root, } /* - * If we have now verified that the query target relation is - * non-inheriting, mark it as a leaf target. - */ - if (parse->resultRelation) - { - RangeTblEntry *rte = rt_fetch(parse->resultRelation, parse->rtable); - - if (!rte->inh) - root->leaf_result_relids = - bms_make_singleton(parse->resultRelation); - } - - /* * This would be a convenient time to check access permissions for all * relations mentioned in the query, since it would be better to fail now, * before doing any detailed planning. However, for historical reasons, diff --git a/src/backend/rewrite/rewriteManip.c b/src/backend/rewrite/rewriteManip.c index 2e75f286b8b..bad153e5a6e 100644 --- a/src/backend/rewrite/rewriteManip.c +++ b/src/backend/rewrite/rewriteManip.c @@ -540,8 +540,14 @@ offset_relid_set(Relids relids, int offset) * * Find all Var nodes in the given tree belonging to a specific relation * (identified by sublevels_up and rt_index), and change their varno fields - * to 'new_index'. The varnosyn fields are changed too. Also, adjust other - * nodes that contain rangetable indexes, such as RangeTblRef and JoinExpr. + * to 'new_index', and update varnosyn and varnullingrels fields similarly. + * Also adjust other nodes that contain rangetable indexes, such as + * RangeTblRef and JoinExpr. + * + * Also, new_index can be INVALID_VAR to indicate that we are deleting the + * given relid from the tree. In this case we expect to find rt_index only + * in Relids fields (varnullingrels, phnullingrels, phrels), never in any + * field that identifies a single relation. * * NOTE: although this has the form of a walker, we cheat and modify the * nodes in-place. The given expression tree should have been copied @@ -564,12 +570,18 @@ ChangeVarNodes_walker(Node *node, ChangeVarNodes_context *context) if (var->varlevelsup == context->sublevels_up) { if (var->varno == context->rt_index) + { + Assert(context->new_index != INVALID_VAR); var->varno = context->new_index; + } var->varnullingrels = adjust_relid_set(var->varnullingrels, context->rt_index, context->new_index); if (var->varnosyn == context->rt_index) + { + Assert(context->new_index != INVALID_VAR); var->varnosyn = context->new_index; + } } return false; } @@ -579,7 +591,10 @@ ChangeVarNodes_walker(Node *node, ChangeVarNodes_context *context) if (context->sublevels_up == 0 && cexpr->cvarno == context->rt_index) + { + Assert(context->new_index != INVALID_VAR); cexpr->cvarno = context->new_index; + } return false; } if (IsA(node, RangeTblRef)) @@ -588,7 +603,10 @@ ChangeVarNodes_walker(Node *node, ChangeVarNodes_context *context) if (context->sublevels_up == 0 && rtr->rtindex == context->rt_index) + { + Assert(context->new_index != INVALID_VAR); rtr->rtindex = context->new_index; + } /* the subquery itself is visited separately */ return false; } @@ -598,7 +616,10 @@ ChangeVarNodes_walker(Node *node, ChangeVarNodes_context *context) if (context->sublevels_up == 0 && j->rtindex == context->rt_index) + { + Assert(context->new_index != INVALID_VAR); j->rtindex = context->new_index; + } /* fall through to examine children */ } if (IsA(node, PlaceHolderVar)) @@ -623,9 +644,15 @@ ChangeVarNodes_walker(Node *node, ChangeVarNodes_context *context) if (context->sublevels_up == 0) { if (rowmark->rti == context->rt_index) + { + Assert(context->new_index != INVALID_VAR); rowmark->rti = context->new_index; + } if (rowmark->prti == context->rt_index) + { + Assert(context->new_index != INVALID_VAR); rowmark->prti = context->new_index; + } } return false; } @@ -636,9 +663,15 @@ ChangeVarNodes_walker(Node *node, ChangeVarNodes_context *context) if (context->sublevels_up == 0) { if (appinfo->parent_relid == context->rt_index) + { + Assert(context->new_index != INVALID_VAR); appinfo->parent_relid = context->new_index; + } if (appinfo->child_relid == context->rt_index) + { + Assert(context->new_index != INVALID_VAR); appinfo->child_relid = context->new_index; + } } /* fall through to examine children */ } @@ -706,21 +739,33 @@ ChangeVarNodesExtended(Node *node, int rt_index, int new_index, ListCell *l; if (qry->resultRelation == rt_index) + { + Assert(new_index != INVALID_VAR); qry->resultRelation = new_index; + } if (qry->mergeTargetRelation == rt_index) + { + Assert(new_index != INVALID_VAR); qry->mergeTargetRelation = new_index; + } /* this is unlikely to ever be used, but ... */ if (qry->onConflict && qry->onConflict->exclRelIndex == rt_index) + { + Assert(new_index != INVALID_VAR); qry->onConflict->exclRelIndex = new_index; + } foreach(l, qry->rowMarks) { RowMarkClause *rc = (RowMarkClause *) lfirst(l); if (rc->rti == rt_index) + { + Assert(new_index != INVALID_VAR); rc->rti = new_index; + } } } query_tree_walker(qry, ChangeVarNodes_walker, &context, 0); diff --git a/src/include/nodes/primnodes.h b/src/include/nodes/primnodes.h index 6dfca3cb35b..92f7a477e34 100644 --- a/src/include/nodes/primnodes.h +++ b/src/include/nodes/primnodes.h @@ -216,6 +216,9 @@ typedef struct Expr * row identity information during UPDATE/DELETE/MERGE. This value should * never be seen outside the planner. * + * INVALID_VAR should never appear as anything's varno. We use it in a + * few APIs to denote removal of an RTE. + * * varnullingrels is the set of RT indexes of outer joins that can force * the Var's value to null (at the point where it appears in the query). * See optimizer/README for discussion of that. @@ -243,6 +246,7 @@ typedef struct Expr #define OUTER_VAR (-2) /* reference to outer subplan */ #define INDEX_VAR (-3) /* reference to index column */ #define ROWID_VAR (-4) /* row identity column during planning */ +#define INVALID_VAR (-5) /* this is not a valid varno! */ #define IS_SPECIAL_VARNO(varno) ((int) (varno) < 0) diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h index 9d3debcab28..10a23e21a5f 100644 --- a/src/include/optimizer/planmain.h +++ b/src/include/optimizer/planmain.h @@ -107,8 +107,8 @@ extern void match_foreign_keys_to_quals(PlannerInfo *root); /* * prototypes for plan/analyzejoins.c */ -extern List *remove_useless_joins(PlannerInfo *root, List *joinlist); -extern void reduce_unique_semijoins(PlannerInfo *root); +extern bool remove_useless_outer_joins(PlannerInfo *root); +extern bool reduce_unique_semijoins(PlannerInfo *root); extern bool query_supports_distinctness(Query *query); extern bool query_is_distinct_for(Query *query, List *colnos, List *opids); extern bool innerrel_is_unique(PlannerInfo *root, @@ -118,7 +118,7 @@ extern bool innerrel_is_unique_ext(PlannerInfo *root, Relids joinrelids, Relids outerrelids, RelOptInfo *innerrel, JoinType jointype, List *restrictlist, bool force_cache, List **extra_clauses); -extern List *remove_useless_self_joins(PlannerInfo *root, List *joinlist); +extern bool remove_useless_self_joins(PlannerInfo *root, List *joinlist); /* * prototypes for plan/setrefs.c diff --git a/src/test/regress/expected/join.out b/src/test/regress/expected/join.out index 884995efecf..b367d0e59fd 100644 --- a/src/test/regress/expected/join.out +++ b/src/test/regress/expected/join.out @@ -6727,6 +6727,38 @@ where t1.a = s.c; (0 rows) rollback; +-- join removal bug #19560: removing a join can leave an EquivalenceClass that +-- now yields a base restriction clause, so we must redo equivalence +-- processing from scratch +begin; +create temp table items (id text, owner text); +create temp table follows (item_id text, user_id text, + unique (user_id, item_id)); +insert into items values ('item1', 'alice'); +explain (costs off) +with viewer as (select 'bob' as id) +select count(*) from items + left join follows on follows.item_id = items.id and follows.user_id = 'bob' + left join viewer on true +where items.owner = viewer.id; + QUERY PLAN +--------------------------------------- + Aggregate + -> Seq Scan on items + Filter: (owner = 'bob'::text) +(3 rows) + +with viewer as (select 'bob' as id) +select count(*) from items + left join follows on follows.item_id = items.id and follows.user_id = 'bob' + left join viewer on true +where items.owner = viewer.id; + count +------- + 0 +(1 row) + +rollback; -- check handling of semijoins after join removal: we must suppress -- unique-ification of known-constant values begin; @@ -6751,17 +6783,17 @@ where exists (select 1 from t t4 Output: t1.a Index Cond: (t1.a = 1) -> HashAggregate - Output: t5.a + Output: t4.a, t5.a Group Key: t5.a -> Hash Join - Output: t5.a + Output: t4.a, t5.a Hash Cond: (t6.b = t4.b) -> Seq Scan on pg_temp.t t6 Output: t6.a, t6.b -> Hash - Output: t4.b, t5.b, t5.a + Output: t4.b, t4.a, t5.b, t5.a -> Hash Join - Output: t4.b, t5.b, t5.a + Output: t4.b, t4.a, t5.b, t5.a Inner Unique: true Hash Cond: (t5.b = t4.b) -> Seq Scan on pg_temp.t t5 @@ -7192,7 +7224,7 @@ on q1.ax = q2.a; Nested Loop Left Join Join Filter: (t2.a = t4.a) -> Seq Scan on sj t2 - Filter: ((b IS NULL) AND (a IS NOT NULL) AND ((c * c) = (c + 2))) + Filter: ((a IS NOT NULL) AND (b IS NULL) AND ((c * c) = (c + 2))) -> Seq Scan on sj t4 Filter: (c IS NOT NULL) (6 rows) @@ -7275,12 +7307,70 @@ select t1.a from sj t1 where t1.b in ( -> Seq Scan on public.sj t1 Output: t1.a, t1.b, t1.c -> Materialize - Output: t3.c, t3.b + Output: t3.b -> Seq Scan on public.sj t3 - Output: t3.c, t3.b + Output: t3.b Filter: (t3.c IS NOT NULL) (10 rows) +-- Check that quals get hoisted to the appropriate join level after SJE removal +explain (verbose, costs off) +select a2.a +from sj b1 + join sj a1 on b1.b = a1.a + join sj a2 on a2.a = a1.a and a2.b = a1.b; + QUERY PLAN +------------------------------------------------------------- + Nested Loop + Output: a2.a + Join Filter: (b1.b = a2.a) + -> Seq Scan on public.sj a2 + Output: a2.a, a2.b, a2.c + Filter: ((a2.a IS NOT NULL) AND (a2.b IS NOT NULL)) + -> Seq Scan on public.sj b1 + Output: b1.a, b1.b, b1.c +(8 rows) + +-- Same, when a semijoin removal happens first +explain (verbose, costs off) +select a1.a from sj b1 join sj a1 on a1.a = b1.b + where exists (select 1 from sj s where s.a = a1.a); + QUERY PLAN +----------------------------------------- + Nested Loop + Output: s.a + Inner Unique: true + Join Filter: (b1.b = s.a) + -> Seq Scan on public.sj b1 + Output: b1.a, b1.b, b1.c + -> Materialize + Output: s.a + -> Seq Scan on public.sj s + Output: s.a + Filter: (s.a IS NOT NULL) +(11 rows) + +-- A different case, where modified qual is on a lower join level +explain (verbose, costs off) +select a2.a +from ((sj b1 join sj a1 on true) join sj c1 on c1.b = a1.a) + join sj a2 on a2.a = a1.a and a2.b = a1.b; + QUERY PLAN +------------------------------------------------------------------- + Nested Loop + Output: a2.a + -> Nested Loop + Output: a2.a + Join Filter: (c1.b = a2.a) + -> Seq Scan on public.sj a2 + Output: a2.a, a2.b, a2.c + Filter: ((a2.a IS NOT NULL) AND (a2.b IS NOT NULL)) + -> Seq Scan on public.sj c1 + Output: c1.a, c1.b, c1.c + -> Seq Scan on public.sj b1 + Output: b1.a, b1.b, b1.c +(12 rows) + -- -- SJE corner case: uniqueness of an inner is [partially] derived from -- baserestrictinfo clauses. @@ -7611,14 +7701,13 @@ explain (costs off) select * from sj p join sj q on p.a = q.a -> Seq Scan on sj r (6 rows) --- FIXME this constant false filter doesn't look good. Should we merge --- equivalence classes? +-- Check that we detect constant-false condition after merging ECs. explain (costs off) select * from sj p, sj q where p.a = q.a and p.b = 1 and q.b = 2; - QUERY PLAN ------------------------------------------------------ - Seq Scan on sj q - Filter: ((a IS NOT NULL) AND (b = 2) AND (b = 1)) + QUERY PLAN +-------------------------- + Result + One-Time Filter: false (2 rows) -- Check that attr_needed is updated correctly after self-join removal. In this @@ -7755,11 +7844,9 @@ where s1.x = 1; -> Seq Scan on public.emp1 t1 Output: t1.id, t1.code -> Materialize - Output: t3.id -> Seq Scan on public.emp1 t3 - Output: t3.id Filter: (1 = 1) -(9 rows) +(7 rows) -- Check that PHVs do not impose any constraints on removing self joins explain (verbose, costs off) @@ -7809,14 +7896,14 @@ SELECT 1 FROM tbl_phv t1 LEFT JOIN (SELECT y FROM tbl_phv tr) t4 ON t4.y = t3.y ON true WHERE t3.extra IS NOT NULL AND t3.x = t1.x % 2; - QUERY PLAN ---------------------------------------------------------- + QUERY PLAN +-------------------------------------------------------------- Nested Loop Output: 1 -> Seq Scan on public.tbl_phv t1 Output: t1.x, t1.y - -> Index Scan using tbl_phv_idx on public.tbl_phv tr - Output: tr.x, tr.y + -> Index Only Scan using tbl_phv_idx on public.tbl_phv tr + Output: tr.x Index Cond: (tr.x = (t1.x % 2)) Filter: (1 IS NOT NULL) (8 rows) @@ -7996,7 +8083,7 @@ where t1.b = t2.b and t2.a = 3 and t1.a = 3 --------------------------------------------------------------------------------------------- Seq Scan on public.sl t2 Output: t2.a, t2.b, t2.c, t2.a, t2.b, t2.c - Filter: ((t2.c IS NOT NULL) AND (t2.b IS NOT NULL) AND (t2.a IS NOT NULL) AND (t2.a = 3)) + Filter: ((t2.b IS NOT NULL) AND (t2.c IS NOT NULL) AND (t2.a IS NOT NULL) AND (t2.a = 3)) (3 rows) -- Join qual isn't mergejoinable, but inner is unique. @@ -8038,10 +8125,35 @@ SELECT 1 AS c1 FROM sl sl1 LEFT JOIN (sl AS sl2 NATURAL JOIN sl AS sl3) -> Nested Loop Left Join Join Filter: sl3.bool_col -> Seq Scan on sl sl3 - Filter: (bool_col AND (a IS NOT NULL) AND (b IS NOT NULL) AND (c IS NOT NULL) AND (bool_col IS NOT NULL)) + Filter: ((a IS NOT NULL) AND (b IS NOT NULL) AND (c IS NOT NULL) AND (bool_col IS NOT NULL) AND bool_col) -> Seq Scan on sl sl4 (7 rows) +-- Check that quals of a jointree node that becomes empty when the self-join +-- is removed are not lost, and that they don't migrate above an outer join +EXPLAIN (COSTS OFF) +SELECT s.a FROM (SELECT * FROM sl WHERE c IS NOT NULL) s, sl t +WHERE t.a = s.a AND t.b = s.b; + QUERY PLAN +--------------------------------------------------------------------- + Seq Scan on sl + Filter: ((c IS NOT NULL) AND (a IS NOT NULL) AND (b IS NOT NULL)) +(2 rows) + +EXPLAIN (COSTS OFF) +SELECT t1.a, ss.a FROM sl t1 + LEFT JOIN (SELECT s.a FROM (SELECT * FROM sl WHERE c IS NOT NULL) s + JOIN sl t ON t.a = s.a AND t.b = s.b) ss + ON ss.a = t1.a; + QUERY PLAN +--------------------------------------------------------------------------- + Nested Loop Left Join + Join Filter: (sl.a = t1.a) + -> Seq Scan on sl t1 + -> Seq Scan on sl + Filter: ((c IS NOT NULL) AND (a IS NOT NULL) AND (b IS NOT NULL)) +(5 rows) + -- Check optimization disabling if it will violate special join conditions. -- Two identical joined relations satisfies self join removal conditions but -- stay in different special join infos. diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out index 8c879509313..732a779ac88 100644 --- a/src/test/regress/expected/rowsecurity.out +++ b/src/test/regress/expected/rowsecurity.out @@ -195,6 +195,17 @@ NOTICE: f_leak => awesome science fiction 9 | 22 | 1 | regress_rls_dave | awesome science fiction (4 rows) +-- a rel with RLS quals can still be removed by outer-join removal +EXPLAIN (COSTS OFF) +SELECT c.cid FROM category c LEFT JOIN document d ON c.cid = d.did; + QUERY PLAN +---------------------------------------------------- + Seq Scan on category c + InitPlan 1 + -> Index Scan using uaccount_pkey on uaccount + Index Cond: (pguser = CURRENT_USER) +(4 rows) + -- viewpoint from regress_rls_carol SET SESSION AUTHORIZATION regress_rls_carol; SELECT * FROM document WHERE f_leak(dtitle) ORDER BY did; diff --git a/src/test/regress/sql/join.sql b/src/test/regress/sql/join.sql index 56244c2577e..19677aa7eed 100644 --- a/src/test/regress/sql/join.sql +++ b/src/test/regress/sql/join.sql @@ -2535,6 +2535,31 @@ where t1.a = s.c; rollback; +-- join removal bug #19560: removing a join can leave an EquivalenceClass that +-- now yields a base restriction clause, so we must redo equivalence +-- processing from scratch +begin; + +create temp table items (id text, owner text); +create temp table follows (item_id text, user_id text, + unique (user_id, item_id)); +insert into items values ('item1', 'alice'); + +explain (costs off) +with viewer as (select 'bob' as id) +select count(*) from items + left join follows on follows.item_id = items.id and follows.user_id = 'bob' + left join viewer on true +where items.owner = viewer.id; + +with viewer as (select 'bob' as id) +select count(*) from items + left join follows on follows.item_id = items.id and follows.user_id = 'bob' + left join viewer on true +where items.owner = viewer.id; + +rollback; + -- check handling of semijoins after join removal: we must suppress -- unique-ification of known-constant values begin; @@ -2801,6 +2826,24 @@ explain (verbose, costs off) select t1.a from sj t1 where t1.b in ( select t2.b from sj t2 join sj t3 on t2.c=t3.c); +-- Check that quals get hoisted to the appropriate join level after SJE removal +explain (verbose, costs off) +select a2.a +from sj b1 + join sj a1 on b1.b = a1.a + join sj a2 on a2.a = a1.a and a2.b = a1.b; + +-- Same, when a semijoin removal happens first +explain (verbose, costs off) +select a1.a from sj b1 join sj a1 on a1.a = b1.b + where exists (select 1 from sj s where s.a = a1.a); + +-- A different case, where modified qual is on a lower join level +explain (verbose, costs off) +select a2.a +from ((sj b1 join sj a1 on true) join sj c1 on c1.b = a1.a) + join sj a2 on a2.a = a1.a and a2.b = a1.b; + -- -- SJE corner case: uniqueness of an inner is [partially] derived from -- baserestrictinfo clauses. @@ -2949,8 +2992,7 @@ select 1 from (select y.* from sj x, sj y where x.a = y.a) q, explain (costs off) select * from sj p join sj q on p.a = q.a left join sj r on p.a + q.a = r.a; --- FIXME this constant false filter doesn't look good. Should we merge --- equivalence classes? +-- Check that we detect constant-false condition after merging ECs. explain (costs off) select * from sj p, sj q where p.a = q.a and p.b = 1 and q.b = 2; @@ -3148,6 +3190,18 @@ EXPLAIN (COSTS OFF) SELECT 1 AS c1 FROM sl sl1 LEFT JOIN (sl AS sl2 NATURAL JOIN sl AS sl3) ON sl2.bool_col LEFT JOIN sl AS sl4 ON sl2.bool_col; +-- Check that quals of a jointree node that becomes empty when the self-join +-- is removed are not lost, and that they don't migrate above an outer join +EXPLAIN (COSTS OFF) +SELECT s.a FROM (SELECT * FROM sl WHERE c IS NOT NULL) s, sl t +WHERE t.a = s.a AND t.b = s.b; + +EXPLAIN (COSTS OFF) +SELECT t1.a, ss.a FROM sl t1 + LEFT JOIN (SELECT s.a FROM (SELECT * FROM sl WHERE c IS NOT NULL) s + JOIN sl t ON t.a = s.a AND t.b = s.b) ss + ON ss.a = t1.a; + -- Check optimization disabling if it will violate special join conditions. -- Two identical joined relations satisfies self join removal conditions but -- stay in different special join infos. diff --git a/src/test/regress/sql/rowsecurity.sql b/src/test/regress/sql/rowsecurity.sql index c08b56bdace..ab96b05b4f4 100644 --- a/src/test/regress/sql/rowsecurity.sql +++ b/src/test/regress/sql/rowsecurity.sql @@ -121,6 +121,10 @@ SELECT * FROM document NATURAL JOIN category WHERE f_leak(dtitle) ORDER BY did; SELECT * FROM document TABLESAMPLE BERNOULLI(50) REPEATABLE(0) WHERE f_leak(dtitle) ORDER BY did; +-- a rel with RLS quals can still be removed by outer-join removal +EXPLAIN (COSTS OFF) +SELECT c.cid FROM category c LEFT JOIN document d ON c.cid = d.did; + -- viewpoint from regress_rls_carol SET SESSION AUTHORIZATION regress_rls_carol; SELECT * FROM document WHERE f_leak(dtitle) ORDER BY did; |
