Skip to content

Commit 564a5d2

Browse files
author
Commitfest Bot
committed
[CF 5355] v28 - Enhance memory context statistics reporting for all PostgreSQL processes.
This branch was automatically generated by a robot using patches from an email thread registered at: https://commitfest.postgresql.org/patch/5355 The branch will be overwritten each time a new patch version is posted to the thread, and also periodically to check for bitrot caused by changes on the master branch. Patch(es): https://www.postgresql.org/message-id/CAH2L28tp8RMa0CrCgdCJw20vFzeGQMuHkXAoPgYC5JZuXY8_+g@mail.gmail.com Author(s): Rahila Syed
2 parents 105b2cb + 7d03af3 commit 564a5d2

File tree

26 files changed

+1383
-45
lines changed

26 files changed

+1383
-45
lines changed

‎doc/src/sgml/func.sgml

+171
Original file line numberDiff line numberDiff line change
@@ -28663,6 +28663,144 @@ acl | {postgres=arwdDxtm/postgres,foo=r/postgres}
2866328663
</para></entry>
2866428664
</row>
2866528665

28666+
<row>
28667+
<entry role="func_table_entry"><para role="func_signature">
28668+
<indexterm>
28669+
<primary>pg_get_process_memory_contexts</primary>
28670+
</indexterm>
28671+
<function>pg_get_process_memory_contexts</function> ( <parameter>pid</parameter> <type>integer</type>, <parameter>summary</parameter> <type>boolean</type>, <parameter>timeout</parameter> <type>float</type> )
28672+
<returnvalue>setof record</returnvalue>
28673+
( <parameter>name</parameter> <type>text</type>,
28674+
<parameter>ident</parameter> <type>text</type>,
28675+
<parameter>type</parameter> <type>text</type>,
28676+
<parameter>path</parameter> <type>integer[]</type>,
28677+
<parameter>level</parameter> <type>integer</type>,
28678+
<parameter>total_bytes</parameter> <type>bigint</type>,
28679+
<parameter>total_nblocks</parameter> <type>bigint</type>,
28680+
<parameter>free_bytes</parameter> <type>bigint</type>,
28681+
<parameter>free_chunks</parameter> <type>bigint</type>,
28682+
<parameter>used_bytes</parameter> <type>bigint</type>,
28683+
<parameter>num_agg_contexts</parameter> <type>integer</type>,
28684+
<parameter>stats_timestamp</parameter> <type>timestamptz</type> )
28685+
</para>
28686+
<para>
28687+
This function handles requests to display the memory contexts of a
28688+
<productname>PostgreSQL</productname> process with the specified
28689+
process ID. The function can be used to send requests to backends as
28690+
well as <glossterm linkend="glossary-auxiliary-proc">auxiliary processes</glossterm>.
28691+
</para>
28692+
<para>
28693+
The returned record contains extended statistics per each memory
28694+
context:
28695+
<itemizedlist spacing="compact">
28696+
<listitem>
28697+
<para>
28698+
<parameter>name</parameter> - The name of the memory context.
28699+
</para>
28700+
</listitem>
28701+
<listitem>
28702+
<para>
28703+
<parameter>ident</parameter> - Memory context ID (if any).
28704+
</para>
28705+
</listitem>
28706+
<listitem>
28707+
<para>
28708+
<parameter>type</parameter> - The type of memory context, possible
28709+
values are: AllocSet, Generation, Slab and Bump.
28710+
</para>
28711+
</listitem>
28712+
<listitem>
28713+
<para>
28714+
<parameter>path</parameter> - Memory contexts are organized in a
28715+
tree model with TopMemoryContext as the root, and all other memory
28716+
contexts as nodes in the tree. The <parameter>path</parameter>
28717+
displays the path from the root to the current memory context. The
28718+
path is limited to 100 children per node, which each node limited
28719+
to a max depth of 100, to preserve memory during reporting. The
28720+
printed path will also be limited to 100 nodes counting from the
28721+
TopMemoryContext.
28722+
</para>
28723+
</listitem>
28724+
<listitem>
28725+
<para>
28726+
<parameter>level</parameter> - The level in the tree of the current
28727+
memory context.
28728+
</para>
28729+
</listitem>
28730+
<listitem>
28731+
<para>
28732+
<parameter>total_bytes</parameter> - The total number of bytes
28733+
allocated to this memory context.
28734+
</para>
28735+
</listitem>
28736+
<listitem>
28737+
<para>
28738+
<parameter>total_nblocks</parameter> - The total number of blocks
28739+
used for the allocated memory.
28740+
</para>
28741+
</listitem>
28742+
<listitem>
28743+
<para>
28744+
<parameter>free_bytes</parameter> - The amount of free memory in
28745+
this memory context.
28746+
</para>
28747+
</listitem>
28748+
<listitem>
28749+
<para>
28750+
<parameter>free_chunks</parameter> - The number of chunks that
28751+
<parameter>free_bytes</parameter> corresponds to.
28752+
</para>
28753+
</listitem>
28754+
<listitem>
28755+
<para>
28756+
<parameter>used_bytes</parameter> - The total number of bytes
28757+
currently occupied.
28758+
</para>
28759+
</listitem>
28760+
<listitem>
28761+
<para>
28762+
<parameter>num_agg_contexts</parameter> - The number of memory
28763+
contexts aggregated in the displayed statistics.
28764+
</para>
28765+
</listitem>
28766+
<listitem>
28767+
<para>
28768+
<parameter>stats_timestamp</parameter> - When the statistics were
28769+
extracted from the process.
28770+
</para>
28771+
</listitem>
28772+
</itemizedlist>
28773+
</para>
28774+
<para>
28775+
When <parameter>summary</parameter> is <literal>true</literal>, statistics
28776+
for memory contexts at levels 1 and 2 are displayed, with level 1
28777+
representing the root node (i.e., <literal>TopMemoryContext</literal>).
28778+
Statistics for contexts on level 2 and below are aggregates of all
28779+
child contexts' statistics, where <literal>num_agg_contexts</literal>
28780+
indicate the number aggregated child contexts. When
28781+
<parameter>summary</parameter> is <literal>false</literal>,
28782+
<literal>the num_agg_contexts</literal> value is <literal>1</literal>,
28783+
indicating that individual statistics are being displayed. The levels
28784+
are limited to the first 100 contexts.
28785+
</para>
28786+
<para>
28787+
Busy processes can delay reporting memory context statistics,
28788+
<parameter>timeout</parameter> specifies the number of seconds
28789+
to wait for updated statistics. <parameter>timeout</parameter> can be
28790+
specified in fractions of a second.
28791+
</para>
28792+
<para>
28793+
After receiving memory context statistics from the target process, it
28794+
returns the results as one row per context. If all the contexts don't
28795+
fit within the pre-determined size limit, the remaining context
28796+
statistics are aggregated and a cumulative total is displayed. The
28797+
<literal>num_agg_contexts</literal> column indicates the number of
28798+
contexts aggregated in the displayed statistics. When
28799+
<literal>num_agg_contexts</literal> is <literal>1</literal> is means
28800+
that the context statistics are displayed separately.
28801+
</para></entry>
28802+
</row>
28803+
2866628804
<row>
2866728805
<entry role="func_table_entry"><para role="func_signature">
2866828806
<indexterm>
@@ -28802,6 +28940,39 @@ LOG: Grand total: 1651920 bytes in 201 blocks; 622360 free (88 chunks); 1029560
2880228940
because it may generate a large number of log messages.
2880328941
</para>
2880428942

28943+
<para>
28944+
<function>pg_get_process_memory_contexts</function> can be used
28945+
to request the memory contexts statistics of any postgres process. For example:
28946+
<programlisting>
28947+
postgres=# SELECT * FROM pg_get_process_memory_contexts(
28948+
(SELECT pid FROM pg_stat_activity
28949+
WHERE backend_type = 'checkpointer'),
28950+
false, 0.5) LIMIT 1;
28951+
-[ RECORD 1 ]----+------------------------------
28952+
name | TopMemoryContext
28953+
ident |
28954+
type | AllocSet
28955+
path | {1}
28956+
level | 1
28957+
total_bytes | 90304
28958+
total_nblocks | 3
28959+
free_bytes | 2880
28960+
free_chunks | 1
28961+
used_bytes | 87424
28962+
num_agg_contexts | 1
28963+
stats_timestamp | 2025-03-24 13:55:47.796698+01
28964+
</programlisting>
28965+
<note>
28966+
<para>
28967+
While <function>pg_get_process_memory_contexts</function> can be used to
28968+
query memory contexts of the local backend,
28969+
<structname>pg_backend_memory_contexts</structname>
28970+
(see <xref linkend="view-pg-backend-memory-contexts"/> for more details)
28971+
will be less resource intensive when only the local backend is of interest.
28972+
</para>
28973+
</note>
28974+
</para>
28975+
2880528976
</sect2>
2880628977

2880728978
<sect2 id="functions-admin-backup">

‎src/backend/catalog/system_views.sql

+5
Original file line numberDiff line numberDiff line change
@@ -674,6 +674,11 @@ GRANT SELECT ON pg_backend_memory_contexts TO pg_read_all_stats;
674674
REVOKE EXECUTE ON FUNCTION pg_get_backend_memory_contexts() FROM PUBLIC;
675675
GRANT EXECUTE ON FUNCTION pg_get_backend_memory_contexts() TO pg_read_all_stats;
676676

677+
REVOKE EXECUTE ON FUNCTION
678+
pg_get_process_memory_contexts(integer, boolean, float) FROM PUBLIC;
679+
GRANT EXECUTE ON FUNCTION
680+
pg_get_process_memory_contexts(integer, boolean, float) TO pg_read_all_stats;
681+
677682
-- Statistics views
678683

679684
CREATE VIEW pg_stat_all_tables AS

‎src/backend/postmaster/autovacuum.c

+4
Original file line numberDiff line numberDiff line change
@@ -781,6 +781,10 @@ ProcessAutoVacLauncherInterrupts(void)
781781
if (LogMemoryContextPending)
782782
ProcessLogMemoryContextInterrupt();
783783

784+
/* Publish memory contexts of this process */
785+
if (PublishMemoryContextPending)
786+
ProcessGetMemoryContextInterrupt();
787+
784788
/* Process sinval catchup interrupts that happened while sleeping */
785789
ProcessCatchupInterrupt();
786790
}

‎src/backend/postmaster/checkpointer.c

+4
Original file line numberDiff line numberDiff line change
@@ -663,6 +663,10 @@ ProcessCheckpointerInterrupts(void)
663663
/* Perform logging of memory contexts of this process */
664664
if (LogMemoryContextPending)
665665
ProcessLogMemoryContextInterrupt();
666+
667+
/* Publish memory contexts of this process */
668+
if (PublishMemoryContextPending)
669+
ProcessGetMemoryContextInterrupt();
666670
}
667671

668672
/*

‎src/backend/postmaster/interrupt.c

+4
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,10 @@ ProcessMainLoopInterrupts(void)
4848
/* Perform logging of memory contexts of this process */
4949
if (LogMemoryContextPending)
5050
ProcessLogMemoryContextInterrupt();
51+
52+
/* Publish memory contexts of this process */
53+
if (PublishMemoryContextPending)
54+
ProcessGetMemoryContextInterrupt();
5155
}
5256

5357
/*

‎src/backend/postmaster/pgarch.c

+4
Original file line numberDiff line numberDiff line change
@@ -867,6 +867,10 @@ ProcessPgArchInterrupts(void)
867867
if (LogMemoryContextPending)
868868
ProcessLogMemoryContextInterrupt();
869869

870+
/* Publish memory contexts of this process */
871+
if (PublishMemoryContextPending)
872+
ProcessGetMemoryContextInterrupt();
873+
870874
if (ConfigReloadPending)
871875
{
872876
char *archiveLib = pstrdup(XLogArchiveLibrary);

‎src/backend/postmaster/startup.c

+4
Original file line numberDiff line numberDiff line change
@@ -192,6 +192,10 @@ ProcessStartupProcInterrupts(void)
192192
/* Perform logging of memory contexts of this process */
193193
if (LogMemoryContextPending)
194194
ProcessLogMemoryContextInterrupt();
195+
196+
/* Publish memory contexts of this process */
197+
if (PublishMemoryContextPending)
198+
ProcessGetMemoryContextInterrupt();
195199
}
196200

197201

‎src/backend/postmaster/walsummarizer.c

+4
Original file line numberDiff line numberDiff line change
@@ -879,6 +879,10 @@ ProcessWalSummarizerInterrupts(void)
879879
/* Perform logging of memory contexts of this process */
880880
if (LogMemoryContextPending)
881881
ProcessLogMemoryContextInterrupt();
882+
883+
/* Publish memory contexts of this process */
884+
if (PublishMemoryContextPending)
885+
ProcessGetMemoryContextInterrupt();
882886
}
883887

884888
/*

‎src/backend/storage/ipc/ipci.c

+3
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,7 @@
5151
#include "storage/sinvaladt.h"
5252
#include "utils/guc.h"
5353
#include "utils/injection_point.h"
54+
#include "utils/memutils.h"
5455

5556
/* GUCs */
5657
int shared_memory_type = DEFAULT_SHARED_MEMORY_TYPE;
@@ -150,6 +151,7 @@ CalculateShmemSize(int *num_semaphores)
150151
size = add_size(size, InjectionPointShmemSize());
151152
size = add_size(size, SlotSyncShmemSize());
152153
size = add_size(size, AioShmemSize());
154+
size = add_size(size, MemoryContextReportingShmemSize());
153155

154156
/* include additional requested shmem from preload libraries */
155157
size = add_size(size, total_addin_request);
@@ -343,6 +345,7 @@ CreateOrAttachShmemStructs(void)
343345
WaitEventCustomShmemInit();
344346
InjectionPointShmemInit();
345347
AioShmemInit();
348+
MemoryContextReportingShmemInit();
346349
}
347350

348351
/*

‎src/backend/storage/ipc/procsignal.c

+3
Original file line numberDiff line numberDiff line change
@@ -690,6 +690,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
690690
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
691691
HandleLogMemoryContextInterrupt();
692692

693+
if (CheckProcSignal(PROCSIG_GET_MEMORY_CONTEXT))
694+
HandleGetMemoryContextInterrupt();
695+
693696
if (CheckProcSignal(PROCSIG_PARALLEL_APPLY_MESSAGE))
694697
HandleParallelApplyMessageInterrupt();
695698

‎src/backend/storage/lmgr/lwlock.c

+2
Original file line numberDiff line numberDiff line change
@@ -178,6 +178,8 @@ static const char *const BuiltinTrancheNames[] = {
178178
[LWTRANCHE_XACT_SLRU] = "XactSLRU",
179179
[LWTRANCHE_PARALLEL_VACUUM_DSA] = "ParallelVacuumDSA",
180180
[LWTRANCHE_AIO_URING_COMPLETION] = "AioUringCompletion",
181+
[LWTRANCHE_MEMORY_CONTEXT_REPORTING_STATE] = "MemoryContextReportingState",
182+
[LWTRANCHE_MEMORY_CONTEXT_REPORTING_PROC] = "MemoryContextReportingPerProcess",
181183
};
182184

183185
StaticAssertDecl(lengthof(BuiltinTrancheNames) ==

‎src/backend/storage/lmgr/proc.c

+1
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,7 @@
5050
#include "storage/procsignal.h"
5151
#include "storage/spin.h"
5252
#include "storage/standby.h"
53+
#include "utils/memutils.h"
5354
#include "utils/timeout.h"
5455
#include "utils/timestamp.h"
5556

‎src/backend/tcop/postgres.c

+3
Original file line numberDiff line numberDiff line change
@@ -3535,6 +3535,9 @@ ProcessInterrupts(void)
35353535
if (LogMemoryContextPending)
35363536
ProcessLogMemoryContextInterrupt();
35373537

3538+
if (PublishMemoryContextPending)
3539+
ProcessGetMemoryContextInterrupt();
3540+
35383541
if (ParallelApplyMessagePending)
35393542
ProcessParallelApplyMessages();
35403543
}

‎src/backend/utils/activity/wait_event_names.txt

+1
Original file line numberDiff line numberDiff line change
@@ -161,6 +161,7 @@ WAL_RECEIVER_EXIT "Waiting for the WAL receiver to exit."
161161
WAL_RECEIVER_WAIT_START "Waiting for startup process to send initial data for streaming replication."
162162
WAL_SUMMARY_READY "Waiting for a new WAL summary to be generated."
163163
XACT_GROUP_UPDATE "Waiting for the group leader to update transaction status at transaction end."
164+
MEM_CXT_PUBLISH "Waiting for a process to publish memory information."
164165

165166
ABI_compatibility:
166167

0 commit comments

Comments
 (0)