| Lists: | pgsql-generalpgsql-sql |
|---|
| From: | Shaozhong SHI <shishaozhong(at)gmail(dot)com> |
|---|---|
| To: | pgsql-general <pgsql-general(at)lists(dot)postgresql(dot)org> |
| Subject: | How long does iteration over 4-5 million rows usually take? |
| Date: | 2022-04-02 01:34:26 |
| Message-ID: | CA+i5JwYtma8t7OQGmLwS_fmk7iHQgm8AfuM520mSS-dYcP8oQA@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Lists: | pgsql-general pgsql-sql |
I have a script running to iterate over 4-5 million rows. It keeps showing
up in red in PgAdmin. It remains active.
How long does iteration over 4-5 million rows usually take?
Regards,
David
| From: | Shaozhong SHI <shishaozhong(at)gmail(dot)com> |
|---|---|
| To: | pgsql-sql <pgsql-sql(at)lists(dot)postgresql(dot)org> |
| Subject: | How long does iteration over 4-5 million rows usually take? |
| Date: | 2022-04-02 01:35:04 |
| Message-ID: | CA+i5JwaNx645eZM0MWh=phhsY9KZem8af6f8pEMRgDqrG5W2kQ@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Lists: | pgsql-general pgsql-sql |
I have a script running to iterate over 4-5 million rows. It keeps showing
up in red in PgAdmin. It remains active.
How long does iteration over 4-5 million rows usually take?
Regards,
David
| From: | Adrian Klaver <adrian(dot)klaver(at)aklaver(dot)com> |
|---|---|
| To: | Shaozhong SHI <shishaozhong(at)gmail(dot)com>, pgsql-general <pgsql-general(at)lists(dot)postgresql(dot)org> |
| Subject: | Re: How long does iteration over 4-5 million rows usually take? |
| Date: | 2022-04-02 01:47:42 |
| Message-ID: | [email protected] |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Lists: | pgsql-general pgsql-sql |
On 4/1/22 18:34, Shaozhong SHI wrote:
>
> I have a script running to iterate over 4-5 million rows. It keeps
> showing up in red in PgAdmin. It remains active.
>
> How long does iteration over 4-5 million rows usually take?
Given that there is no real information provided in the problem
description there is no chance for an answer more detailed then; as long
as it takes.
>
> Regards,
>
> David
--
Adrian Klaver
adrian(dot)klaver(at)aklaver(dot)com
| From: | Ron <ronljohnsonjr(at)gmail(dot)com> |
|---|---|
| To: | pgsql-general(at)lists(dot)postgresql(dot)org |
| Subject: | Re: How long does iteration over 4-5 million rows usually take? |
| Date: | 2022-04-02 02:18:45 |
| Message-ID: | [email protected] |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Lists: | pgsql-general pgsql-sql |
On 4/1/22 20:34, Shaozhong SHI wrote:
>
> I have a script running to iterate over 4-5 million rows. It keeps
> showing up in red in PgAdmin. It remains active.
>
> How long does iteration over 4-5 million rows usually take?
What /*exactly*/ are you doing?
--
Angular momentum makes the world go 'round.
| From: | Rob Sargent <robjsargent(at)gmail(dot)com> |
|---|---|
| To: | Ron <ronljohnsonjr(at)gmail(dot)com> |
| Cc: | pgsql-general(at)lists(dot)postgresql(dot)org |
| Subject: | Re: How long does iteration over 4-5 million rows usually take? |
| Date: | 2022-04-02 13:21:00 |
| Message-ID: | [email protected] |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Lists: | pgsql-general pgsql-sql |
> On Apr 1, 2022, at 10:18 PM, Ron <ronljohnsonjr(at)gmail(dot)com> wrote:
>
> On 4/1/22 20:34, Shaozhong SHI wrote:
>>
>> I have a script running to iterate over 4-5 million rows. It keeps showing up in red in PgAdmin. It remains active.
>>
>> How long does iteration over 4-5 million rows usually take?
4-5 million times as long as it takes to do one iteration ( if you’re doing it correctly)
| From: | Karsten Hilbert <Karsten(dot)Hilbert(at)gmx(dot)net> |
|---|---|
| To: | Rob Sargent <robjsargent(at)gmail(dot)com> |
| Cc: | Ron <ronljohnsonjr(at)gmail(dot)com>, pgsql-general(at)lists(dot)postgresql(dot)org |
| Subject: | Aw: Re: How long does iteration over 4-5 million rows usually take? |
| Date: | 2022-04-02 13:46:55 |
| Message-ID: | trinity-3d817328-9ee8-4c97-a95c-99c034f30558-1648907215445@3c-app-gmx-bap24 |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Lists: | pgsql-general pgsql-sql |
> > On Apr 1, 2022, at 10:18 PM, Ron <ronljohnsonjr(at)gmail(dot)com> wrote:
> >
> > On 4/1/22 20:34, Shaozhong SHI wrote:
> >>
> >> I have a script running to iterate over 4-5 million rows. It keeps showing up in red in PgAdmin. It remains active.
> >>
> >> How long does iteration over 4-5 million rows usually take?
>
> 4-5 million times as long as it takes to do one iteration ( if you’re doing it correctly)
I may not take quite that long because setup/teardown times might not be needed for each iteration.
Best,
Karsten
| From: | Shaozhong SHI <shishaozhong(at)gmail(dot)com> |
|---|---|
| To: | Karsten Hilbert <Karsten(dot)Hilbert(at)gmx(dot)net> |
| Cc: | Rob Sargent <robjsargent(at)gmail(dot)com>, Ron <ronljohnsonjr(at)gmail(dot)com>, pgsql-general <pgsql-general(at)lists(dot)postgresql(dot)org> |
| Subject: | Re: Re: How long does iteration over 4-5 million rows usually take? |
| Date: | 2022-04-02 14:16:14 |
| Message-ID: | CA+i5JwaUOJ+r6H+3VfNaoZXTykAyhpy8q8j5JjHw-=_AWTtLfQ@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Lists: | pgsql-general pgsql-sql |
Thanks, Karsten,
I would like the information to work planning purpose.
Regards,
David
On Sat, 2 Apr 2022 at 14:47, Karsten Hilbert <Karsten(dot)Hilbert(at)gmx(dot)net>
wrote:
> > > On Apr 1, 2022, at 10:18 PM, Ron <ronljohnsonjr(at)gmail(dot)com> wrote:
> > >
> > > On 4/1/22 20:34, Shaozhong SHI wrote:
> > >>
> > >> I have a script running to iterate over 4-5 million rows. It keeps
> showing up in red in PgAdmin. It remains active.
> > >>
> > >> How long does iteration over 4-5 million rows usually take?
> >
> > 4-5 million times as long as it takes to do one iteration ( if you’re
> doing it correctly)
>
> I may not take quite that long because setup/teardown times might not be
> needed for each iteration.
>
> Best,
> Karsten
>
>
>
| From: | Benedict Holland <benedict(dot)m(dot)holland(at)gmail(dot)com> |
|---|---|
| To: | Shaozhong SHI <shishaozhong(at)gmail(dot)com> |
| Cc: | Karsten Hilbert <Karsten(dot)Hilbert(at)gmx(dot)net>, Rob Sargent <robjsargent(at)gmail(dot)com>, Ron <ronljohnsonjr(at)gmail(dot)com>, pgsql-general <pgsql-general(at)lists(dot)postgresql(dot)org> |
| Subject: | Re: Re: How long does iteration over 4-5 million rows usually take? |
| Date: | 2022-04-02 14:34:34 |
| Message-ID: | CAD+mzozmywUViFppKBixdD-iTndVDKsDZndq9fvG9=WWOf0x5A@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Lists: | pgsql-general pgsql-sql |
For work planning, create a temporary table, copy from a generated data
set, and test it out. It really depends. Normally, you don't iterate in SQL
since rows are assumed to be independent. You think about SQL in terms of
sets and let application code handle the data transfers. It also really
depends on the size of your tables, indexing, joins, and more. Basically,
there isn't really an answer but it isnt hard to test.
Thanks,
Ben
On Sat, Apr 2, 2022, 10:16 AM Shaozhong SHI <shishaozhong(at)gmail(dot)com> wrote:
> Thanks, Karsten,
>
> I would like the information to work planning purpose.
>
> Regards,
>
> David
>
> On Sat, 2 Apr 2022 at 14:47, Karsten Hilbert <Karsten(dot)Hilbert(at)gmx(dot)net>
> wrote:
>
>> > > On Apr 1, 2022, at 10:18 PM, Ron <ronljohnsonjr(at)gmail(dot)com> wrote:
>> > >
>> > > On 4/1/22 20:34, Shaozhong SHI wrote:
>> > >>
>> > >> I have a script running to iterate over 4-5 million rows. It keeps
>> showing up in red in PgAdmin. It remains active.
>> > >>
>> > >> How long does iteration over 4-5 million rows usually take?
>> >
>> > 4-5 million times as long as it takes to do one iteration ( if you’re
>> doing it correctly)
>>
>> I may not take quite that long because setup/teardown times might not be
>> needed for each iteration.
>>
>> Best,
>> Karsten
>>
>>
>>
| From: | Steve Midgley <science(at)misuse(dot)org> |
|---|---|
| To: | Shaozhong SHI <shishaozhong(at)gmail(dot)com> |
| Cc: | pgsql-sql <pgsql-sql(at)lists(dot)postgresql(dot)org> |
| Subject: | Re: How long does iteration over 4-5 million rows usually take? |
| Date: | 2022-04-02 16:38:41 |
| Message-ID: | CAJexoSJ5AjwXzxErAL93D+TLHT2EewtS9bk4iokHao9opa6XCw@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Lists: | pgsql-general pgsql-sql |
On Fri, Apr 1, 2022 at 6:35 PM Shaozhong SHI <shishaozhong(at)gmail(dot)com> wrote:
>
>
> I have a script running to iterate over 4-5 million rows. It keeps
> showing up in red in PgAdmin. It remains active.
>
> How long does iteration over 4-5 million rows usually take?
>
>
Please share the SQL query, schema, and sample data. If you are using a
third party script with a db connector, then the answer is much harder to
answer.