Skip to content

Commit f1bebef

Browse files
macdicemacdice
committed
Add shared_memory_type GUC.
Since 9.3 we have used anonymous shared mmap for our main shared memory region, except in EXEC_BACKEND builds. Provide a GUC so that users can opt for System V shared memory once again, like in 9.2 and earlier. A later patch proposes to add huge/large page support for AIX, which requires System V shared memory and provided the motivation to revive this possibility. It may also be useful on some BSDs. Author: Andres Freund (revived and documented by Thomas Munro) Discussion: https://postgr.es/m/HE1PR0202MB28126DB4E0B6621CC6A1A91286D90%40HE1PR0202MB2812.eurprd02.prod.outlook.com Discussion: https://postgr.es/m/2AE143D2-87D3-4AD1-AC78-CE2258230C05%40FreeBSD.org
1 parent 0d1fe9f commit f1bebef

File tree

7 files changed

+101
-33
lines changed

7 files changed

+101
-33
lines changed

‎doc/src/sgml/config.sgml

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1694,6 +1694,31 @@ include_dir 'conf.d'
16941694
</listitem>
16951695
</varlistentry>
16961696

1697+
<varlistentry id="guc-shared-memory-type" xreflabel="shared_memory_type">
1698+
<term><varname>shared_memory_type</varname> (<type>enum</type>)
1699+
<indexterm>
1700+
<primary><varname>shared_memory_type</varname> configuration parameter</primary>
1701+
</indexterm>
1702+
</term>
1703+
<listitem>
1704+
<para>
1705+
Specifies the shared memory implementation that the server
1706+
should use for the main shared memory region that holds
1707+
<productname>PostgreSQL</productname>'s shared buffers and other
1708+
shared data. Possible values are <literal>mmap</literal> (for
1709+
anonymous shared memory allocated using <function>mmap</function>),
1710+
<literal>sysv</literal> (for System V shared memory allocated via
1711+
<function>shmget</function>) and <literal>windows</literal> (for Windows
1712+
shared memory). Not all values are supported on all platforms; the
1713+
first supported option is the default for that platform. The use of
1714+
the <literal>sysv</literal> option, which is not the default on any
1715+
platform, is generally discouraged because it typically requires
1716+
non-default kernel settings to allow for large allocations (see <xref
1717+
linkend="sysvipc"/>).
1718+
</para>
1719+
</listitem>
1720+
</varlistentry>
1721+
16971722
<varlistentry id="guc-dynamic-shared-memory-type" xreflabel="dynamic_shared_memory_type">
16981723
<term><varname>dynamic_shared_memory_type</varname> (<type>enum</type>)
16991724
<indexterm>

‎doc/src/sgml/runtime.sgml

Lines changed: 11 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -638,9 +638,12 @@ psql: could not connect to server: No such file or directory
638638
</para>
639639

640640
<para>
641-
Upon starting the server, <productname>PostgreSQL</productname> normally allocates
641+
By default, <productname>PostgreSQL</productname> allocates
642642
a very small amount of System V shared memory, as well as a much larger
643-
amount of POSIX (<function>mmap</function>) shared memory.
643+
amount of anonymous <function>mmap</function> shared memory.
644+
Alternatively, a single large System V shared memory region can be used
645+
(see <xref linkend="guc-shared-memory-type"/>).
646+
644647
In addition a significant number of semaphores, which can be either
645648
System V or POSIX style, are created at server startup. Currently,
646649
POSIX semaphores are used on Linux and FreeBSD systems while other
@@ -752,8 +755,10 @@ psql: could not connect to server: No such file or directory
752755
<productname>PostgreSQL</productname> requires a few bytes of System V shared memory
753756
(typically 48 bytes, on 64-bit platforms) for each copy of the server.
754757
On most modern operating systems, this amount can easily be allocated.
755-
However, if you are running many copies of the server, or if other
756-
applications are also using System V shared memory, it may be necessary to
758+
However, if you are running many copies of the server or you explicitly
759+
configure the server to use large amounts of System V shared memory (see
760+
<xref linkend="guc-shared-memory-type"/> and <xref
761+
linkend="guc-dynamic-shared-memory-type"/>), it may be necessary to
757762
increase <varname>SHMALL</varname>, which is the total amount of System V shared
758763
memory system-wide. Note that <varname>SHMALL</varname> is measured in pages
759764
rather than bytes on many systems.
@@ -879,7 +884,7 @@ kern.ipc.semmns=512
879884
</para>
880885

881886
<para>
882-
You might also want to configure your kernel to lock shared
887+
You might also want to configure your kernel to lock System V shared
883888
memory into RAM and prevent it from being paged out to swap.
884889
This can be accomplished using the <command>sysctl</command>
885890
setting <literal>kern.ipc.shm_use_phys</literal>.
@@ -928,7 +933,7 @@ kern.ipc.semmns=512
928933
</para>
929934

930935
<para>
931-
You might also want to configure your kernel to lock shared
936+
You might also want to configure your kernel to lock System V shared
932937
memory into RAM and prevent it from being paged out to swap.
933938
This can be accomplished using the <command>sysctl</command>
934939
setting <literal>kern.ipc.shm_use_phys</literal>.

‎src/backend/port/sysv_shmem.c

Lines changed: 16 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -62,10 +62,11 @@
6262
* to a process after exec(). Since EXEC_BACKEND is intended only for
6363
* developer use, this shouldn't be a big problem. Because of this, we do
6464
* not worry about supporting anonymous shmem in the EXEC_BACKEND cases below.
65+
*
66+
* As of PostgreSQL 12, we regained the ability to use a large System V shared
67+
* memory region even in non-EXEC_BACKEND builds, if shared_memory_type is set
68+
* to sysv (though this is not the default).
6569
*/
66-
#ifndef EXEC_BACKEND
67-
#define USE_ANONYMOUS_SHMEM
68-
#endif
6970

7071

7172
typedef key_t IpcMemoryKey; /* shared memory key passed to shmget(2) */
@@ -75,10 +76,8 @@ typedef int IpcMemoryId; /* shared memory ID returned by shmget(2) */
7576
unsigned long UsedShmemSegID = 0;
7677
void *UsedShmemSegAddr = NULL;
7778

78-
#ifdef USE_ANONYMOUS_SHMEM
7979
static Size AnonymousShmemSize;
8080
static void *AnonymousShmem = NULL;
81-
#endif
8281

8382
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
8483
static void IpcMemoryDetach(int status, Datum shmaddr);
@@ -370,8 +369,6 @@ PGSharedMemoryIsInUse(unsigned long id1, unsigned long id2)
370369
return true;
371370
}
372371

373-
#ifdef USE_ANONYMOUS_SHMEM
374-
375372
#ifdef MAP_HUGETLB
376373

377374
/*
@@ -534,8 +531,6 @@ AnonymousShmemDetach(int status, Datum arg)
534531
}
535532
}
536533

537-
#endif /* USE_ANONYMOUS_SHMEM */
538-
539534
/*
540535
* PGSharedMemoryCreate
541536
*
@@ -566,7 +561,7 @@ PGSharedMemoryCreate(Size size, bool makePrivate, int port,
566561
Size sysvsize;
567562

568563
/* Complain if hugepages demanded but we can't possibly support them */
569-
#if !defined(USE_ANONYMOUS_SHMEM) || !defined(MAP_HUGETLB)
564+
#if !defined(MAP_HUGETLB)
570565
if (huge_pages == HUGE_PAGES_ON)
571566
ereport(ERROR,
572567
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
@@ -576,18 +571,19 @@ PGSharedMemoryCreate(Size size, bool makePrivate, int port,
576571
/* Room for a header? */
577572
Assert(size > MAXALIGN(sizeof(PGShmemHeader)));
578573

579-
#ifdef USE_ANONYMOUS_SHMEM
580-
AnonymousShmem = CreateAnonymousSegment(&size);
581-
AnonymousShmemSize = size;
574+
if (shared_memory_type == SHMEM_TYPE_MMAP)
575+
{
576+
AnonymousShmem = CreateAnonymousSegment(&size);
577+
AnonymousShmemSize = size;
582578

583-
/* Register on-exit routine to unmap the anonymous segment */
584-
on_shmem_exit(AnonymousShmemDetach, (Datum) 0);
579+
/* Register on-exit routine to unmap the anonymous segment */
580+
on_shmem_exit(AnonymousShmemDetach, (Datum) 0);
585581

586-
/* Now we need only allocate a minimal-sized SysV shmem block. */
587-
sysvsize = sizeof(PGShmemHeader);
588-
#else
589-
sysvsize = size;
590-
#endif
582+
/* Now we need only allocate a minimal-sized SysV shmem block. */
583+
sysvsize = sizeof(PGShmemHeader);
584+
}
585+
else
586+
sysvsize = size;
591587

592588
/* Make sure PGSharedMemoryAttach doesn't fail without need */
593589
UsedShmemSegAddr = NULL;
@@ -687,14 +683,10 @@ PGSharedMemoryCreate(Size size, bool makePrivate, int port,
687683
* block. Otherwise, the System V shared memory block is only a shim, and
688684
* we must return a pointer to the real block.
689685
*/
690-
#ifdef USE_ANONYMOUS_SHMEM
691686
if (AnonymousShmem == NULL)
692687
return hdr;
693688
memcpy(AnonymousShmem, hdr, sizeof(PGShmemHeader));
694689
return (PGShmemHeader *) AnonymousShmem;
695-
#else
696-
return hdr;
697-
#endif
698690
}
699691

700692
#ifdef EXEC_BACKEND
@@ -801,15 +793,13 @@ PGSharedMemoryDetach(void)
801793
UsedShmemSegAddr = NULL;
802794
}
803795

804-
#ifdef USE_ANONYMOUS_SHMEM
805796
if (AnonymousShmem != NULL)
806797
{
807798
if (munmap(AnonymousShmem, AnonymousShmemSize) < 0)
808799
elog(LOG, "munmap(%p, %zu) failed: %m",
809800
AnonymousShmem, AnonymousShmemSize);
810801
AnonymousShmem = NULL;
811802
}
812-
#endif
813803
}
814804

815805

‎src/backend/storage/ipc/ipci.c

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,8 @@
4646
#include "storage/spin.h"
4747
#include "utils/snapmgr.h"
4848

49+
/* GUCs */
50+
int shared_memory_type = DEFAULT_SHARED_MEMORY_TYPE;
4951

5052
shmem_startup_hook_type shmem_startup_hook = NULL;
5153

‎src/backend/utils/misc/guc.c

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -453,6 +453,19 @@ const struct config_enum_entry ssl_protocol_versions_info[] = {
453453
{NULL, 0, false}
454454
};
455455

456+
static struct config_enum_entry shared_memory_options[] = {
457+
#ifndef WIN32
458+
{ "sysv", SHMEM_TYPE_SYSV, false},
459+
#endif
460+
#ifndef EXEC_BACKEND
461+
{ "mmap", SHMEM_TYPE_MMAP, false},
462+
#endif
463+
#ifdef WIN32
464+
{ "windows", SHMEM_TYPE_WINDOWS, false},
465+
#endif
466+
{NULL, 0, false}
467+
};
468+
456469
/*
457470
* Options for enum values stored in other modules
458471
*/
@@ -4327,6 +4340,16 @@ static struct config_enum ConfigureNamesEnum[] =
43274340
NULL, NULL, NULL
43284341
},
43294342

4343+
{
4344+
{"shared_memory_type", PGC_POSTMASTER, RESOURCES_MEM,
4345+
gettext_noop("Selects the shared memory implementation used for the main shared memory region."),
4346+
NULL
4347+
},
4348+
&shared_memory_type,
4349+
DEFAULT_SHARED_MEMORY_TYPE, shared_memory_options,
4350+
NULL, NULL, NULL
4351+
},
4352+
43304353
{
43314354
{"wal_sync_method", PGC_SIGHUP, WAL_SETTINGS,
43324355
gettext_noop("Selects the method used for forcing WAL updates to disk."),

‎src/backend/utils/misc/postgresql.conf.sample

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -129,6 +129,12 @@
129129
#maintenance_work_mem = 64MB # min 1MB
130130
#autovacuum_work_mem = -1 # min 1MB, or -1 to use maintenance_work_mem
131131
#max_stack_depth = 2MB # min 100kB
132+
#shared_memory_type = mmap # the default is the first option
133+
# supported by the operating system:
134+
# mmap
135+
# sysv
136+
# windows
137+
# (change requires restart)
132138
#dynamic_shared_memory_type = posix # the default is the first option
133139
# supported by the operating system:
134140
# posix

‎src/include/storage/pg_shmem.h

Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,8 @@ typedef struct PGShmemHeader /* standard header for all Postgres shmem */
4141
#endif
4242
} PGShmemHeader;
4343

44-
/* GUC variable */
44+
/* GUC variables */
45+
extern int shared_memory_type;
4546
extern int huge_pages;
4647

4748
/* Possible values for huge_pages */
@@ -52,13 +53,29 @@ typedef enum
5253
HUGE_PAGES_TRY
5354
} HugePagesType;
5455

56+
/* Possible values for shared_memory_type */
57+
typedef enum
58+
{
59+
SHMEM_TYPE_WINDOWS,
60+
SHMEM_TYPE_SYSV,
61+
SHMEM_TYPE_MMAP
62+
} PGShmemType;
63+
5564
#ifndef WIN32
5665
extern unsigned long UsedShmemSegID;
5766
#else
5867
extern HANDLE UsedShmemSegID;
5968
#endif
6069
extern void *UsedShmemSegAddr;
6170

71+
#if !defined(WIN32) && !defined(EXEC_BACKEND)
72+
#define DEFAULT_SHARED_MEMORY_TYPE SHMEM_TYPE_MMAP
73+
#elif !defined(WIN32)
74+
#define DEFAULT_SHARED_MEMORY_TYPE SHMEM_TYPE_SYSV
75+
#else
76+
#define DEFAULT_SHARED_MEMORY_TYPE SHMEM_TYPE_WINDOWS
77+
#endif
78+
6279
#ifdef EXEC_BACKEND
6380
extern void PGSharedMemoryReAttach(void);
6481
extern void PGSharedMemoryNoReAttach(void);

0 commit comments

Comments
 (0)