diff options
Diffstat (limited to 'doc/src')
| -rw-r--r-- | doc/src/sgml/custom-scan.sgml | 134 | ||||
| -rw-r--r-- | doc/src/sgml/fdwhandler.sgml | 152 |
2 files changed, 177 insertions, 109 deletions
diff --git a/doc/src/sgml/custom-scan.sgml b/doc/src/sgml/custom-scan.sgml index 9fd1db6fde4..62a8a3305bb 100644 --- a/doc/src/sgml/custom-scan.sgml +++ b/doc/src/sgml/custom-scan.sgml @@ -32,12 +32,13 @@ </para> <sect1 id="custom-scan-path"> - <title>Implementing Custom Paths</title> + <title>Creating Custom Scan Paths</title> <para> - A custom scan provider will typically add paths by setting the following - hook, which is called after the core code has generated what it believes - to be the complete and correct set of access paths for the relation. + A custom scan provider will typically add paths for a base relation by + setting the following hook, which is called after the core code has + generated what it believes to be the complete and correct set of access + paths for the relation. <programlisting> typedef void (*set_rel_pathlist_hook_type) (PlannerInfo *root, RelOptInfo *rel, @@ -74,7 +75,7 @@ typedef struct CustomPath can support mark and restore. Both capabilities are optional. <structfield>custom_private</> can be used to store the custom path's private data. Private data should be stored in a form that can be handled - by <literal>nodeToString</>, so that debugging routines which attempt to + by <literal>nodeToString</>, so that debugging routines that attempt to print the custom path will work as designed. <structfield>methods</> must point to a (usually statically allocated) object implementing the required custom path methods, of which there are currently only two, as further @@ -82,29 +83,28 @@ typedef struct CustomPath </para> <para> - A custom scan provider can also add join paths; in this case, the scan - must produce the same output as would normally be produced by the join - it replaces. To do this, the join provider should set the following hook. - This hook may be invoked repeatedly for the same pair of relations, with - different combinations of inner and outer relations; it is the - responsibility of the hook to minimize duplicated work. + A custom scan provider can also provide join paths. Just as for base + relations, such a path must produce the same output as would normally be + produced by the join it replaces. To do this, the join provider should + set the following hook, and then within the hook function, + create <structname>CustomPath</> path(s) for the join relation. <programlisting> typedef void (*set_join_pathlist_hook_type) (PlannerInfo *root, RelOptInfo *joinrel, RelOptInfo *outerrel, RelOptInfo *innerrel, - List *restrictlist, JoinType jointype, - SpecialJoinInfo *sjinfo, - SemiAntiJoinFactors *semifactors, - Relids param_source_rels, - Relids extra_lateral_rels); + JoinPathExtraData *extra); extern PGDLLIMPORT set_join_pathlist_hook_type set_join_pathlist_hook; </programlisting> + + This hook will be invoked repeatedly for the same join relation, with + different combinations of inner and outer relations; it is the + responsibility of the hook to minimize duplicated work. </para> <sect2 id="custom-scan-path-callbacks"> - <title>Custom Path Callbacks</title> + <title>Custom Scan Path Callbacks</title> <para> <programlisting> @@ -125,7 +125,7 @@ void (*TextOutCustomPath) (StringInfo str, const CustomPath *node); </programlisting> Generate additional output when <function>nodeToString</> is invoked on - this custom path. This callback is optional. Since + this custom path. This callback is optional. Since <function>nodeToString</> will automatically dump all fields in the structure that it can see, including <structfield>custom_private</>, this is only useful if the <structname>CustomPath</> is actually embedded in a @@ -135,7 +135,7 @@ void (*TextOutCustomPath) (StringInfo str, </sect1> <sect1 id="custom-scan-plan"> - <title>Implementing Custom Plans</title> + <title>Creating Custom Scan Plans</title> <para> A custom scan is represented in a finished plan tree using the following @@ -146,9 +146,9 @@ typedef struct CustomScan Scan scan; uint32 flags; List *custom_exprs; - List *custom_ps_tlist; List *custom_private; - List *custom_relids; + List *custom_scan_tlist; + Bitmapset *custom_relids; const CustomScanMethods *methods; } CustomScan; </programlisting> @@ -158,16 +158,21 @@ typedef struct CustomScan <structfield>scan</> must be initialized as for any other scan, including estimated costs, target lists, qualifications, and so on. <structfield>flags</> is a bitmask with the same meaning as in - <structname>CustomPath</>. <structfield>custom_exprs</> should be used to + <structname>CustomPath</>. + <structfield>custom_exprs</> should be used to store expression trees that will need to be fixed up by <filename>setrefs.c</> and <filename>subselect.c</>, while - <literal>custom_private</> should be used to store other private data that - is only used by the custom scan provider itself. Plan trees must be able - to be duplicated using <function>copyObject</>, so all the data stored - within these two fields must consist of nodes that function can handle. - <literal>custom_relids</> is set by the core code to the set of relations - which this scan node must handle; except when this scan is replacing a - join, it will have only one member. + <structfield>custom_private</> should be used to store other private data + that is only used by the custom scan provider itself. + <structfield>custom_scan_tlist</> can be NIL when scanning a base + relation, indicating that the custom scan returns scan tuples that match + the base relation's rowtype. Otherwise it is a targetlist describing + the actual scan tuples. <structfield>custom_scan_tlist</> must be + provided for joins, and could be provided for scans if the custom scan + provider can compute some non-Var expressions. + <structfield>custom_relids</> is set by the core code to the set of + relations (rangetable indexes) that this scan node handles; except when + this scan is replacing a join, it will have only one member. <structfield>methods</> must point to a (usually statically allocated) object implementing the required custom scan methods, which are further detailed below. @@ -175,19 +180,22 @@ typedef struct CustomScan <para> When a <structname>CustomScan</> scans a single relation, - <structfield>scan.scanrelid</> should be the range table index of the table - to be scanned, and <structfield>custom_ps_tlist</> should be - <literal>NULL</>. When it replaces a join, <structfield>scan.scanrelid</> - should be zero, and <structfield>custom_ps_tlist</> should be a list of - <structname>TargetEntry</> nodes. This is necessary because, when a join - is replaced, the target list cannot be constructed from the table - definition. At execution time, this list will be used to initialize the - tuple descriptor of the <structname>TupleTableSlot</>. It will also be - used by <command>EXPLAIN</>, when deparsing. + <structfield>scan.scanrelid</> must be the range table index of the table + to be scanned. When it replaces a join, <structfield>scan.scanrelid</> + should be zero. + </para> + + <para> + Plan trees must be able to be duplicated using <function>copyObject</>, + so all the data stored within the <quote>custom</> fields must consist of + nodes that that function can handle. Furthermore, custom scan providers + cannot substitute a larger structure that embeds + a <structname>CustomScan</> for the structure itself, as would be possible + for a <structname>CustomPath</> or <structname>CustomScanState</>. </para> <sect2 id="custom-scan-plan-callbacks"> - <title>Custom Scan Callbacks</title> + <title>Custom Scan Plan Callbacks</title> <para> <programlisting> Node *(*CreateCustomScanState) (CustomScan *cscan); @@ -195,12 +203,12 @@ Node *(*CreateCustomScanState) (CustomScan *cscan); Allocate a <structname>CustomScanState</> for this <structname>CustomScan</>. The actual allocation will often be larger than required for an ordinary <structname>CustomScanState</>, because many - scan types will wish to embed that as the first field of a large structure. + providers will wish to embed that as the first field of a larger structure. The value returned must have the node tag and <structfield>methods</> - set appropriately, but the other fields need not be initialized at this + set appropriately, but other fields should be left as zeroes at this stage; after <function>ExecInitCustomScan</> performs basic initialization, the <function>BeginCustomScan</> callback will be invoked to give the - custom scan state a chance to do whatever else is needed. + custom scan provider a chance to do whatever else is needed. </para> <para> @@ -209,23 +217,21 @@ void (*TextOutCustomScan) (StringInfo str, const CustomScan *node); </programlisting> Generate additional output when <function>nodeToString</> is invoked on - this custom plan. This callback is optional. Since a - <structname>CustomScan</> must be copyable by <function>copyObject</>, - custom scan providers cannot substitute a larger structure that embeds a - <structname>CustomScan</> for the structure itself, as would be possible - for a <structname>CustomPath</> or <structname>CustomScanState</>. - Therefore, providing this callback is unlikely to be useful. + this custom plan node. This callback is optional. Since + <function>nodeToString</> will automatically dump all fields in the + structure, including the substructure of the <quote>custom</> fields, + there is usually not much need for this callback. </para> </sect2> </sect1> - <sect1 id="custom-scan-scan"> - <title>Implementing Custom Scans</title> + <sect1 id="custom-scan-execution"> + <title>Executing Custom Scans</title> <para> When a <structfield>CustomScan</> is executed, its execution state is represented by a <structfield>CustomScanState</>, which is declared as - follows. + follows: <programlisting> typedef struct CustomScanState { @@ -237,7 +243,9 @@ typedef struct CustomScanState </para> <para> - <structfield>ss</> must be initialized as for any other scanstate; + <structfield>ss</> is initialized as for any other scanstate, + except that if the scan is for a join rather than a base relation, + <literal>ss.ss_currentRelation</> is left NULL. <structfield>flags</> is a bitmask with the same meaning as in <structname>CustomPath</> and <structname>CustomScan</>. <structfield>methods</> must point to a (usually statically allocated) @@ -247,8 +255,8 @@ typedef struct CustomScanState structure embedding the above as its first member. </para> - <sect2 id="custom-scan-scan-callbacks"> - <title>Custom Execution-Time Callbacks</title> + <sect2 id="custom-scan-execution-callbacks"> + <title>Custom Scan Execution Callbacks</title> <para> <programlisting> @@ -257,8 +265,8 @@ void (*BeginCustomScan) (CustomScanState *node, int eflags); </programlisting> Complete initialization of the supplied <structname>CustomScanState</>. - Some initialization is performed by <function>ExecInitCustomScan</>, but - any private fields should be initialized here. + Standard fields have been initialized by <function>ExecInitCustomScan</>, + but any private fields should be initialized here. </para> <para> @@ -276,8 +284,8 @@ TupleTableSlot *(*ExecCustomScan) (CustomScanState *node); void (*EndCustomScan) (CustomScanState *node); </programlisting> Clean up any private data associated with the <literal>CustomScanState</>. - This method is required, but may not need to do anything if the associated - data does not exist or will be cleaned up automatically. + This method is required, but it does not need to do anything if there is + no associated data or it will be cleaned up automatically. </para> <para> @@ -293,8 +301,8 @@ void (*ReScanCustomScan) (CustomScanState *node); void (*MarkPosCustomScan) (CustomScanState *node); </programlisting> Save the current scan position so that it can subsequently be restored - by the <function>RestrPosCustomScan</> callback. This calback is optional, - and need only be supplied if + by the <function>RestrPosCustomScan</> callback. This callback is + optional, and need only be supplied if the <literal>CUSTOMPATH_SUPPORT_MARK_RESTORE</> flag is set. </para> @@ -304,7 +312,7 @@ void (*RestrPosCustomScan) (CustomScanState *node); </programlisting> Restore the previous scan position as saved by the <function>MarkPosCustomScan</> callback. This callback is optional, - and need only be supplied if + and need only be supplied if the <literal>CUSTOMPATH_SUPPORT_MARK_RESTORE</> flag is set. </para> @@ -314,8 +322,8 @@ void (*ExplainCustomScan) (CustomScanState *node, List *ancestors, ExplainState *es); </programlisting> - Output additional information on <command>EXPLAIN</> that involves - custom-scan node. This callback is optional. Common data stored in the + Output additional information for <command>EXPLAIN</> of a custom-scan + plan node. This callback is optional. Common data stored in the <structname>ScanState</>, such as the target list and scan relation, will be shown even without this callback, but the callback allows the display of additional, private state. diff --git a/doc/src/sgml/fdwhandler.sgml b/doc/src/sgml/fdwhandler.sgml index bc06d2cbb26..33863f04f82 100644 --- a/doc/src/sgml/fdwhandler.sgml +++ b/doc/src/sgml/fdwhandler.sgml @@ -175,8 +175,11 @@ GetForeignPlan (PlannerInfo *root, access path. This is called at the end of query planning. The parameters are as for <function>GetForeignRelSize</>, plus the selected <structname>ForeignPath</> (previously produced by - <function>GetForeignPaths</>), the target list to be emitted by the - plan node, and the restriction clauses to be enforced by the plan node. + <function>GetForeignPaths</> or <function>GetForeignJoinPaths</>), + the target list to be emitted by the plan node, + and the restriction clauses to be enforced by the plan node. + (If the path is for a join rather than a base + relation, <literal>foreigntableid</> is <literal>InvalidOid</>.) </para> <para> @@ -235,9 +238,12 @@ IterateForeignScan (ForeignScanState *node); </para> <para> - The rows returned must match the column signature of the foreign table - being scanned. If you choose to optimize away fetching columns that - are not needed, you should insert nulls in those column positions. + The rows returned must match the <structfield>fdw_scan_tlist</> target + list if one was supplied, otherwise they must match the rowtype of the + foreign table being scanned. If you choose to optimize away fetching + columns that are not needed, you should insert nulls in those column + positions, or else generate a <structfield>fdw_scan_tlist</> list with + those columns omitted. </para> <para> @@ -275,6 +281,67 @@ EndForeignScan (ForeignScanState *node); </sect2> + <sect2 id="fdw-callbacks-join-scan"> + <title>FDW Routines For Scanning Foreign Joins</title> + + <para> + If an FDW supports performing foreign joins remotely (rather than + by fetching both tables' data and doing the join locally), it should + provide this callback function: + </para> + + <para> +<programlisting> +void +GetForeignJoinPaths (PlannerInfo *root, + RelOptInfo *joinrel, + RelOptInfo *outerrel, + RelOptInfo *innerrel, + JoinType jointype, + JoinPathExtraData *extra); +</programlisting> + Create possible access paths for a join of two (or more) foreign tables + that all belong to the same foreign server. This optional + function is called during query planning. As + with <function>GetForeignPaths</>, this function should + generate <structname>ForeignPath</> path(s) for the + supplied <literal>joinrel</>, and call <function>add_path</> to add these + paths to the set of paths considered for the join. But unlike + <function>GetForeignPaths</>, it is not necessary that this function + succeed in creating at least one path, since paths involving local + joining are always possible. + </para> + + <para> + Note that this function will be invoked repeatedly for the same join + relation, with different combinations of inner and outer relations; it is + the responsibility of the FDW to minimize duplicated work. + </para> + + <para> + If a <structname>ForeignPath</> path is chosen for the join, it will + represent the entire join process; paths generated for the component + tables and subsidiary joins will not be used. Subsequent processing of + the join path proceeds much as it does for a path scanning a single + foreign table. One difference is that the <structfield>scanrelid</> of + the resulting <structname>ForeignScan</> plan node should be set to zero, + since there is no single relation that it represents; instead, + the <structfield>fs_relids</> field of the <structname>ForeignScan</> + node represents the set of relations that were joined. (The latter field + is set up automatically by the core planner code, and need not be filled + by the FDW.) Another difference is that, because the column list for a + remote join cannot be found from the system catalogs, the FDW must + fill <structfield>fdw_scan_tlist</> with an appropriate list + of <structfield>TargetEntry</> nodes, representing the set of columns + it will supply at runtime in the tuples it returns. + </para> + + <para> + See <xref linkend="fdw-planning"> for additional information. + </para> + + </sect2> + <sect2 id="fdw-callbacks-update"> <title>FDW Routines For Updating Foreign Tables</title> @@ -598,42 +665,6 @@ IsForeignRelUpdatable (Relation rel); </sect2> - <sect2> - <title>FDW Routines For Remote Joins</title> - <para> -<programlisting> -void -GetForeignJoinPaths(PlannerInfo *root, - RelOptInfo *joinrel, - RelOptInfo *outerrel, - RelOptInfo *innerrel, - List *restrictlist, - JoinType jointype, - SpecialJoinInfo *sjinfo, - SemiAntiJoinFactors *semifactors, - Relids param_source_rels, - Relids extra_lateral_rels); -</programlisting> - Create possible access paths for a join of two foreign tables managed - by the same foreign data wrapper. - This optional function is called during query planning. - </para> - <para> - This function the FDW to add <structname>ForeignScan</> paths for the - supplied <literal>joinrel</>. Typically, the FDW will send the whole - join to the remote server as a single query, as performing the join - remotely rather than locally is typically much more efficient. - </para> - <para> - Since we cannot construct the slot descriptor for a remote join from - the catalogs, the FDW should set the <structfield>scanrelid</> of the - <structname>ForeignScan</> to zero and <structfield>fdw_ps_tlist</> - to an appropriate list of <structfield>TargetEntry</> nodes. - Junk entries will be ignored, but can be present for the benefit of - deparsing performed by <command>EXPLAIN</>. - </para> - </sect2> - <sect2 id="fdw-callbacks-explain"> <title>FDW Routines for <command>EXPLAIN</></title> @@ -904,10 +935,10 @@ GetForeignServerByName(const char *name, bool missing_ok); <para> The FDW callback functions <function>GetForeignRelSize</>, - <function>GetForeignPaths</>, <function>GetForeignPlan</>, and - <function>PlanForeignModify</> must fit into the workings of the - <productname>PostgreSQL</> planner. Here are some notes about what - they must do. + <function>GetForeignPaths</>, <function>GetForeignPlan</>, + <function>PlanForeignModify</>, and <function>GetForeignJoinPaths</> + must fit into the workings of the <productname>PostgreSQL</> planner. + Here are some notes about what they must do. </para> <para> @@ -934,7 +965,7 @@ GetForeignServerByName(const char *name, bool missing_ok); <literal>baserel->fdw_private</> is a <type>void</> pointer that is available for FDW planning functions to store information relevant to the particular foreign table. The core planner does not touch it except - to initialize it to NULL when the <literal>baserel</> node is created. + to initialize it to NULL when the <literal>RelOptInfo</> node is created. It is useful for passing information forward from <function>GetForeignRelSize</> to <function>GetForeignPaths</> and/or <function>GetForeignPaths</> to <function>GetForeignPlan</>, thereby @@ -1003,6 +1034,23 @@ GetForeignServerByName(const char *name, bool missing_ok); </para> <para> + Another <structname>ForeignScan</> field that can be filled by FDWs + is <structfield>fdw_scan_tlist</>, which describes the tuples returned by + the FDW for this plan node. For simple foreign table scans this can be + set to <literal>NIL</>, implying that the returned tuples have the + rowtype declared for the foreign table. A non-NIL value must be a + targetlist (list of <structname>TargetEntry</>s) containing Vars and/or + expressions representing the returned columns. This might be used, for + example, to show that the FDW has omitted some columns that it noticed + won't be needed for the query. Also, if the FDW can compute expressions + used by the query more cheaply than can be done locally, it could add + those expressions to <structfield>fdw_scan_tlist</>. Note that join + plans (created from paths made by <function>GetForeignJoinPaths</>) must + always supply <structfield>fdw_scan_tlist</> to describe the set of + columns they will return. + </para> + + <para> The FDW should always construct at least one path that depends only on the table's restriction clauses. In join queries, it might also choose to construct path(s) that depend on join clauses, for example @@ -1020,6 +1068,18 @@ GetForeignServerByName(const char *name, bool missing_ok); </para> <para> + If an FDW supports remote joins, <function>GetForeignJoinPaths</> should + produce <structname>ForeignPath</>s for potential remote joins in much + the same way as <function>GetForeignPaths</> works for base tables. + Information about the intended join can be passed forward + to <function>GetForeignPlan</> in the same ways described above. + However, <structfield>baserestrictinfo</> is not relevant for join + relations; instead, the relevant join clauses for a particular join are + passed to <function>GetForeignJoinPaths</> as a separate parameter + (<literal>extra->restrictlist</>). + </para> + + <para> When planning an <command>UPDATE</> or <command>DELETE</>, <function>PlanForeignModify</> can look up the <structname>RelOptInfo</> struct for the foreign table and make use of the |