Parallel index build for BRIN

Hi,

Commit b4375717 introduced parallel CREATE INDEX for BRIN. I've noticed
that a couple of documentation pages need to be updated accordingly. A
small patch is attached.

Thanks,
Egor

Hi everyone,

Thisthread doesn'tseem to have attractedattention, so let me try again.
Two documentation pages claim that B-tree is the only access method that
supports parallel building, which is no longer true. I propose to fix it
in a way like this:

diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index d54f9049569..b5b1580dee7 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -2835,7 +2835,7 @@ include_dir 'conf.d'
          Sets the maximum number of parallel workers that can be
          started by a single utility command.  Currently, the parallel
          utility commands that support the use of parallel workers are
-         <command>CREATE INDEX</command> only when building a B-tree index,
+         <command>CREATE INDEX</command> when building a B-tree or BRIN
index,
          and <command>VACUUM</command> without <literal>FULL</literal>
          option.  Parallel workers are taken from the pool of processes
          established by <xref linkend="guc-max-worker-processes"/>,
limited
diff --git a/doc/src/sgml/ref/create_index.sgml
b/doc/src/sgml/ref/create_index.sgml
index 621bc0e253c..208389e8006 100644
--- a/doc/src/sgml/ref/create_index.sgml
+++ b/doc/src/sgml/ref/create_index.sgml
@@ -808,7 +808,7 @@ Indexes:
    leveraging multiple CPUs in order to process the table rows faster.
    This feature is known as <firstterm>parallel index
    build</firstterm>.  For index methods that support building indexes
-   in parallel (currently, only B-tree),
+   in parallel (currently, B-tree and BRIN),
    <varname>maintenance_work_mem</varname> specifies the maximum
    amount of memory that can be used by each index build operation as
    a whole, regardless of how many worker processes were started.

Thanks,
Egor

On 05.11.2024 12:12, Egor Rogov wrote:
> Hi,
>
> Commit b4375717 introduced parallel CREATE INDEX for BRIN. I've
> noticed that a couple of documentation pages need to be updated
> accordingly. A small patch is attached.
>
> Thanks,
> Egor

Hi,

On 17.11.2024 11:28, Egor Rogov wrote:
> Hi everyone,
>
> This thread doesn't seem to have attracted attention, so let me try
> again. Two documentation pages claim that B-tree is the only access
> method that supports parallel building, which is no longer true. I
> propose to fix it in a way like this:
>
> diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
> index d54f9049569..b5b1580dee7 100644
> --- a/doc/src/sgml/config.sgml
> +++ b/doc/src/sgml/config.sgml
> @@ -2835,7 +2835,7 @@ include_dir 'conf.d'
>           Sets the maximum number of parallel workers that can be
>           started by a single utility command.  Currently, the parallel
>           utility commands that support the use of parallel workers are
> -         <command>CREATE INDEX</command> only when building a B-tree
> index,
> +         <command>CREATE INDEX</command> when building a B-tree or
> BRIN index,
>           and <command>VACUUM</command> without <literal>FULL</literal>
>           option.  Parallel workers are taken from the pool of processes
>           established by <xref linkend="guc-max-worker-processes"/>,
> limited
> diff --git a/doc/src/sgml/ref/create_index.sgml
> b/doc/src/sgml/ref/create_index.sgml
> index 621bc0e253c..208389e8006 100644
> --- a/doc/src/sgml/ref/create_index.sgml
> +++ b/doc/src/sgml/ref/create_index.sgml
> @@ -808,7 +808,7 @@ Indexes:
>     leveraging multiple CPUs in order to process the table rows faster.
>     This feature is known as <firstterm>parallel index
>     build</firstterm>.  For index methods that support building indexes
> -   in parallel (currently, only B-tree),
> +   in parallel (currently, B-tree and BRIN),
>     <varname>maintenance_work_mem</varname> specifies the maximum
>     amount of memory that can be used by each index build operation as
>     a whole, regardless of how many worker processes were started.

I've spotted another mention of B-tree being the only AM that supports
parallel builds: comment in src/backend/catalog/index.c. As this mention
is not visible to the users, I'd propose removing it altogether rather
than fixing it. Updated patch is attached.

>
> Thanks,
> Egor
>
>
> On 05.11.2024 12:12, Egor Rogov wrote:
>> Hi,
>>
>> Commit b4375717 introduced parallel CREATE INDEX for BRIN. I've
>> noticed that a couple of documentation pages need to be updated
>> accordingly. A small patch is attached.
>>
>> Thanks,
>> Egor

On 12/8/24 16:00, Egor Rogov wrote:
> Hi,
>
>
> On 17.11.2024 11:28, Egor Rogov wrote:
>> Hi everyone,
>>
>> This thread doesn't seem to have attracted attention, so let me try
>> again. Two documentation pages claim that B-tree is the only access
>> method that supports parallel building, which is no longer true. I
>> propose to fix it in a way like this:
>>
>> diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
>> index d54f9049569..b5b1580dee7 100644
>> --- a/doc/src/sgml/config.sgml
>> +++ b/doc/src/sgml/config.sgml
>> @@ -2835,7 +2835,7 @@ include_dir 'conf.d'
>>           Sets the maximum number of parallel workers that can be
>>           started by a single utility command.  Currently, the parallel
>>           utility commands that support the use of parallel workers are
>> -         <command>CREATE INDEX</command> only when building a B-tree
>> index,
>> +         <command>CREATE INDEX</command> when building a B-tree or
>> BRIN index,
>>           and <command>VACUUM</command> without <literal>FULL</literal>
>>           option.  Parallel workers are taken from the pool of processes
>>           established by <xref linkend="guc-max-worker-processes"/>,
>> limited
>> diff --git a/doc/src/sgml/ref/create_index.sgml b/doc/src/sgml/ref/
>> create_index.sgml
>> index 621bc0e253c..208389e8006 100644
>> --- a/doc/src/sgml/ref/create_index.sgml
>> +++ b/doc/src/sgml/ref/create_index.sgml
>> @@ -808,7 +808,7 @@ Indexes:
>>     leveraging multiple CPUs in order to process the table rows faster.
>>     This feature is known as <firstterm>parallel index
>>     build</firstterm>.  For index methods that support building indexes
>> -   in parallel (currently, only B-tree),
>> +   in parallel (currently, B-tree and BRIN),
>>     <varname>maintenance_work_mem</varname> specifies the maximum
>>     amount of memory that can be used by each index build operation as
>>     a whole, regardless of how many worker processes were started.
>
>
> I've spotted another mention of B-tree being the only AM that supports
> parallel builds: comment in src/backend/catalog/index.c. As this mention
> is not visible to the users, I'd propose removing it altogether rather
> than fixing it. Updated patch is attached.
>

Thanks for noticing this and the patches. You're right, this should have
been updated with the BRIN parallel builds. I'll get this committed
sometime the week.

regards

--
Tomas Vondra

On 12/9/24 19:54, Tomas Vondra wrote:
> On 12/8/24 16:00, Egor Rogov wrote:
>> Hi,
>>
>> ...
>>
>> I've spotted another mention of B-tree being the only AM that supports
>> parallel builds: comment in src/backend/catalog/index.c. As this mention
>> is not visible to the users, I'd propose removing it altogether rather
>> than fixing it. Updated patch is attached.
>>
>
> Thanks for noticing this and the patches. You're right, this should have
> been updated with the BRIN parallel builds. I'll get this committed
> sometime the week.
>

I've pushed the doc fix, and backpatched it to PG 17.

Thanks for the patience!

--
Tomas Vondra

On 16.12.2024 21:24, Tomas Vondra wrote:

> On 12/9/24 19:54, Tomas Vondra wrote:
>> On 12/8/24 16:00, Egor Rogov wrote:
>>> Hi,
>>>
>>> ...
>>>
>>> I've spotted another mention of B-tree being the only AM that supports
>>> parallel builds: comment in src/backend/catalog/index.c. As this mention
>>> is not visible to the users, I'd propose removing it altogether rather
>>> than fixing it. Updated patch is attached.
>>>
>> Thanks for noticing this and the patches. You're right, this should have
>> been updated with the BRIN parallel builds. I'll get this committed
>> sometime the week.
>>
> I've pushed the doc fix, and backpatched it to PG 17.

Thanks so much, Tomas!

Please note that the comment in src/backend/catalog/index.c remains
unchanged:

--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -2988,8 +2988,7 @@ index_build(Relation heapRelation,
Assert(PointerIsValid(indexRelation->rd_indam->ambuildempty));

    /*
-    * Determine worker process details for parallel CREATE INDEX. 
Currently,
-    * only btree has support for parallel builds.
+    * Determine worker process details for parallel CREATE INDEX.
     *
     * Note that planner considers parallel safety for us.
     */

On 12/17/24 07:40, Egor Rogov wrote:
> On 16.12.2024 21:24, Tomas Vondra wrote:
>
>> On 12/9/24 19:54, Tomas Vondra wrote:
>>> On 12/8/24 16:00, Egor Rogov wrote:
>>>> Hi,
>>>>
>>>> ...
>>>>
>>>> I've spotted another mention of B-tree being the only AM that supports
>>>> parallel builds: comment in src/backend/catalog/index.c. As this
>>>> mention
>>>> is not visible to the users, I'd propose removing it altogether rather
>>>> than fixing it. Updated patch is attached.
>>>>
>>> Thanks for noticing this and the patches. You're right, this should have
>>> been updated with the BRIN parallel builds. I'll get this committed
>>> sometime the week.
>>>
>> I've pushed the doc fix, and backpatched it to PG 17.
>
>
> Thanks so much, Tomas!
>
> Please note that the comment in src/backend/catalog/index.c remains
> unchanged:
>
>
> --- a/src/backend/catalog/index.c
> +++ b/src/backend/catalog/index.c
> @@ -2988,8 +2988,7 @@ index_build(Relation heapRelation,
> Assert(PointerIsValid(indexRelation->rd_indam->ambuildempty));
>
>     /*
> -    * Determine worker process details for parallel CREATE INDEX. 
> Currently,
> -    * only btree has support for parallel builds.
> +    * Determine worker process details for parallel CREATE INDEX.
>      *
>      * Note that planner considers parallel safety for us.
>      */
>

Oh, thanks for noticing that. I'm not sure why I missed that, I must
have used the first patch by mistake.

Anyway, I found another obsolete comment in planner.c, so I fixed that
too and pushed.

Thanks

--
Tomas Vondra