Skip to content

Commit 7949d95

Browse files
committed
Introduce pluggable APIs for Cumulative Statistics
This commit adds support in the backend for $subject, allowing out-of-core extensions to plug their own custom kinds of cumulative statistics. This feature has come up a few times into the lists, and the first, original, suggestion came from Andres Freund, about pg_stat_statements to use the cumulative statistics APIs in shared memory rather than its own less efficient internals. The advantage of this implementation is that this can be extended to any kind of statistics. The stats kinds are divided into two parts: - The in-core "builtin" stats kinds, with designated initializers, able to use IDs up to 128. - The "custom" stats kinds, able to use a range of IDs from 128 to 256 (128 slots available as of this patch), with information saved in TopMemoryContext. This can be made larger, if necessary. There are two types of cumulative statistics in the backend: - For fixed-numbered objects (like WAL, archiver, etc.). These are attached to the snapshot and pgstats shmem control structures for efficiency, and built-in stats kinds still do that to avoid any redirection penalty. The data of custom kinds is stored in a first array in snapshot structure and a second array in the shmem control structure, both indexed by their ID, acting as an equivalent of the builtin stats. - For variable-numbered objects (like tables, functions, etc.). These are stored in a dshash using the stats kind ID in the hash lookup key. Internally, the handling of the builtin stats is unchanged, and both fixed and variabled-numbered objects are supported. Structure definitions for builtin stats kinds are renamed to reflect better the differences with custom kinds. Like custom RMGRs, custom cumulative statistics can only be loaded with shared_preload_libraries at startup, and must allocate a unique ID shared across all the PostgreSQL extension ecosystem with the following wiki page to avoid conflicts: https://wiki.postgresql.org/wiki/CustomCumulativeStats This makes the detection of the stats kinds and their handling when reading and writing stats much easier than, say, allocating IDs for stats kinds from a shared memory counter, that may change the ID used by a stats kind across restarts. When under development, extensions can use PGSTAT_KIND_EXPERIMENTAL. Two examples that can be used as templates for fixed-numbered and variable-numbered stats kinds will be added in some follow-up commits, with tests to provide coverage. Some documentation is added to explain how to use this plugin facility. Author: Michael Paquier Reviewed-by: Dmitry Dolgov, Bertrand Drouvot Discussion: https://postgr.es/m/Zmqm9j5EO0I4W8dx@paquier.xyz
1 parent 365b5a3 commit 7949d95

File tree

6 files changed

+348
-33
lines changed

6 files changed

+348
-33
lines changed

doc/src/sgml/xfunc.sgml

+57
Original file line numberDiff line numberDiff line change
@@ -3864,6 +3864,63 @@ extern bool InjectionPointDetach(const char *name);
38643864
</para>
38653865
</sect2>
38663866

3867+
<sect2 id="xfunc-addin-custom-cumulative-statistics">
3868+
<title>Custom Cumulative Statistics</title>
3869+
3870+
<para>
3871+
Is is possible for add-ins written in C-language to use custom types
3872+
of cumulative statistics registered in the
3873+
<link linkend="monitoring-stats-setup">Cumulative Statistics System</link>.
3874+
</para>
3875+
3876+
<para>
3877+
First, define a <literal>PgStat_KindInfo</literal> that includes all
3878+
the information related to the custom type registered. For example:
3879+
<programlisting>
3880+
static const PgStat_KindInfo custom_stats = {
3881+
.name = "custom_stats",
3882+
.fixed_amount = false,
3883+
.shared_size = sizeof(PgStatShared_Custom),
3884+
.shared_data_off = offsetof(PgStatShared_Custom, stats),
3885+
.shared_data_len = sizeof(((PgStatShared_Custom *) 0)->stats),
3886+
.pending_size = sizeof(PgStat_StatCustomEntry),
3887+
}
3888+
</programlisting>
3889+
3890+
Then, each backend that needs to use this custom type needs to register
3891+
it with <literal>pgstat_register_kind</literal> and a unique ID used to
3892+
store the entries related to this type of statistics:
3893+
<programlisting>
3894+
extern PgStat_Kind pgstat_add_kind(PgStat_Kind kind,
3895+
const PgStat_KindInfo *kind_info);
3896+
</programlisting>
3897+
While developing a new extension, use
3898+
<literal>PGSTAT_KIND_EXPERIMENTAL</literal> for
3899+
<parameter>kind</parameter>. When you are ready to release the extension
3900+
to users, reserve a kind ID at the
3901+
<ulink url="https://wiki.postgresql.org/wiki/CustomCumulativeStats">
3902+
Custom Cumulative Statistics</ulink> page.
3903+
</para>
3904+
3905+
<para>
3906+
The details of the API for <literal>PgStat_KindInfo</literal> can
3907+
be found in <filename>src/include/utils/pgstat_internal.h</filename>.
3908+
</para>
3909+
3910+
<para>
3911+
The type of statistics registered is associated with a name and a unique
3912+
ID shared across the server in shared memory. Each backend using a
3913+
custom type of statistics maintains a local cache storing the information
3914+
of each custom <literal>PgStat_KindInfo</literal>.
3915+
</para>
3916+
3917+
<para>
3918+
Place the extension module implementing the custom cumulative statistics
3919+
type in <xref linkend="guc-shared-preload-libraries"/> so that it will
3920+
be loaded early during <productname>PostgreSQL</productname> startup.
3921+
</para>
3922+
</sect2>
3923+
38673924
<sect2 id="extend-cpp">
38683925
<title>Using C++ for Extensibility</title>
38693926

0 commit comments

Comments
 (0)