Skip to content

[3.14] gh-134160: Use multi-phase init in documentation examples (GH-134296) #134753

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 26, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Doc/c-api/allocation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,6 @@ Allocating Objects on the Heap

.. seealso::

:c:func:`PyModule_Create`
:ref:`moduleobjects`
To allocate and create extension modules.

4 changes: 2 additions & 2 deletions Doc/c-api/intro.rst
Original file line number Diff line number Diff line change
Expand Up @@ -127,15 +127,15 @@ complete listing.
item defined in the module file. Example::

static struct PyModuleDef spam_module = {
PyModuleDef_HEAD_INIT,
.m_base = PyModuleDef_HEAD_INIT,
.m_name = "spam",
...
};

PyMODINIT_FUNC
PyInit_spam(void)
{
return PyModule_Create(&spam_module);
return PyModuleDef_Init(&spam_module);
}


Expand Down
6 changes: 3 additions & 3 deletions Doc/extending/building.rst
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,10 @@ instance. See :ref:`initializing-modules` for details.
.. highlight:: python

For modules with ASCII-only names, the function must be named
``PyInit_<modulename>``, with ``<modulename>`` replaced by the name of the
module. When using :ref:`multi-phase-initialization`, non-ASCII module names
:samp:`PyInit_{<name>}`, with ``<name>`` replaced by the name of the module.
When using :ref:`multi-phase-initialization`, non-ASCII module names
are allowed. In this case, the initialization function name is
``PyInitU_<modulename>``, with ``<modulename>`` encoded using Python's
:samp:`PyInitU_{<name>}`, with ``<name>`` encoded using Python's
*punycode* encoding with hyphens replaced by underscores. In Python::

def initfunc_name(name):
Expand Down
12 changes: 7 additions & 5 deletions Doc/extending/embedding.rst
Original file line number Diff line number Diff line change
Expand Up @@ -245,21 +245,23 @@ Python extension. For example::
return PyLong_FromLong(numargs);
}

static PyMethodDef EmbMethods[] = {
static PyMethodDef emb_module_methods[] = {
{"numargs", emb_numargs, METH_VARARGS,
"Return the number of arguments received by the process."},
{NULL, NULL, 0, NULL}
};

static PyModuleDef EmbModule = {
PyModuleDef_HEAD_INIT, "emb", NULL, -1, EmbMethods,
NULL, NULL, NULL, NULL
static struct PyModuleDef emb_module = {
.m_base = PyModuleDef_HEAD_INIT,
.m_name = "emb",
.m_size = 0,
.m_methods = emb_module_methods,
};

static PyObject*
PyInit_emb(void)
{
return PyModule_Create(&EmbModule);
return PyModuleDef_Init(&emb_module);
}

Insert the above code just above the :c:func:`main` function. Also, insert the
Expand Down
150 changes: 69 additions & 81 deletions Doc/extending/extending.rst
Original file line number Diff line number Diff line change
Expand Up @@ -203,31 +203,42 @@ function usually raises :c:data:`PyExc_TypeError`. If you have an argument whos
value must be in a particular range or must satisfy other conditions,
:c:data:`PyExc_ValueError` is appropriate.

You can also define a new exception that is unique to your module. For this, you
usually declare a static object variable at the beginning of your file::
You can also define a new exception that is unique to your module.
For this, you can declare a static global object variable at the beginning
of the file::

static PyObject *SpamError;

and initialize it in your module's initialization function (:c:func:`!PyInit_spam`)
with an exception object::
and initialize it with an exception object in the module's
:c:data:`Py_mod_exec` function (:c:func:`!spam_module_exec`)::

PyMODINIT_FUNC
PyInit_spam(void)
static int
spam_module_exec(PyObject *m)
{
PyObject *m;

m = PyModule_Create(&spammodule);
if (m == NULL)
return NULL;

SpamError = PyErr_NewException("spam.error", NULL, NULL);
if (PyModule_AddObjectRef(m, "error", SpamError) < 0) {
Py_CLEAR(SpamError);
Py_DECREF(m);
return NULL;
if (PyModule_AddObjectRef(m, "SpamError", SpamError) < 0) {
return -1;
}

return m;
return 0;
}

static PyModuleDef_Slot spam_module_slots[] = {
{Py_mod_exec, spam_module_exec},
{0, NULL}
};

static struct PyModuleDef spam_module = {
.m_base = PyModuleDef_HEAD_INIT,
.m_name = "spam",
.m_size = 0, // non-negative
.m_slots = spam_module_slots,
};

PyMODINIT_FUNC
PyInit_spam(void)
{
return PyModuleDef_Init(&spam_module);
}

Note that the Python name for the exception object is :exc:`!spam.error`. The
Expand Down Expand Up @@ -318,7 +329,7 @@ The Module's Method Table and Initialization Function
I promised to show how :c:func:`!spam_system` is called from Python programs.
First, we need to list its name and address in a "method table"::

static PyMethodDef SpamMethods[] = {
static PyMethodDef spam_methods[] = {
...
{"system", spam_system, METH_VARARGS,
"Execute a shell command."},
Expand All @@ -343,13 +354,10 @@ function.

The method table must be referenced in the module definition structure::

static struct PyModuleDef spammodule = {
PyModuleDef_HEAD_INIT,
"spam", /* name of module */
spam_doc, /* module documentation, may be NULL */
-1, /* size of per-interpreter state of the module,
or -1 if the module keeps state in global variables. */
SpamMethods
static struct PyModuleDef spam_module = {
...
.m_methods = spam_methods,
...
};

This structure, in turn, must be passed to the interpreter in the module's
Expand All @@ -360,23 +368,17 @@ only non-\ ``static`` item defined in the module file::
PyMODINIT_FUNC
PyInit_spam(void)
{
return PyModule_Create(&spammodule);
return PyModuleDef_Init(&spam_module);
}

Note that :c:macro:`PyMODINIT_FUNC` declares the function as ``PyObject *`` return type,
declares any special linkage declarations required by the platform, and for C++
declares the function as ``extern "C"``.

When the Python program imports module :mod:`!spam` for the first time,
:c:func:`!PyInit_spam` is called. (See below for comments about embedding Python.)
It calls :c:func:`PyModule_Create`, which returns a module object, and
inserts built-in function objects into the newly created module based upon the
table (an array of :c:type:`PyMethodDef` structures) found in the module definition.
:c:func:`PyModule_Create` returns a pointer to the module object
that it creates. It may abort with a fatal error for
certain errors, or return ``NULL`` if the module could not be initialized
satisfactorily. The init function must return the module object to its caller,
so that it then gets inserted into ``sys.modules``.
:c:func:`!PyInit_spam` is called when each interpreter imports its module
:mod:`!spam` for the first time. (See below for comments about embedding Python.)
A pointer to the module definition must be returned via :c:func:`PyModuleDef_Init`,
so that the import machinery can create the module and store it in ``sys.modules``.

When embedding Python, the :c:func:`!PyInit_spam` function is not called
automatically unless there's an entry in the :c:data:`PyImport_Inittab` table.
Expand Down Expand Up @@ -433,23 +435,19 @@ optionally followed by an import of the module::

.. note::

Removing entries from ``sys.modules`` or importing compiled modules into
multiple interpreters within a process (or following a :c:func:`fork` without an
intervening :c:func:`exec`) can create problems for some extension modules.
Extension module authors should exercise caution when initializing internal data
structures.
If you declare a global variable or a local static one, the module may
experience unintended side-effects on re-initialisation, for example when
removing entries from ``sys.modules`` or importing compiled modules into
multiple interpreters within a process
(or following a :c:func:`fork` without an intervening :c:func:`exec`).
If module state is not yet fully :ref:`isolated <isolating-extensions-howto>`,
authors should consider marking the module as having no support for subinterpreters
(via :c:macro:`Py_MOD_MULTIPLE_INTERPRETERS_NOT_SUPPORTED`).

A more substantial example module is included in the Python source distribution
as :file:`Modules/xxmodule.c`. This file may be used as a template or simply
as :file:`Modules/xxlimited.c`. This file may be used as a template or simply
read as an example.

.. note::

Unlike our ``spam`` example, ``xxmodule`` uses *multi-phase initialization*
(new in Python 3.5), where a PyModuleDef structure is returned from
``PyInit_spam``, and creation of the module is left to the import machinery.
For details on multi-phase initialization, see :PEP:`489`.


.. _compilation:

Expand Down Expand Up @@ -790,18 +788,17 @@ Philbrick (philbrick@hks.com)::
{NULL, NULL, 0, NULL} /* sentinel */
};

static struct PyModuleDef keywdargmodule = {
PyModuleDef_HEAD_INIT,
"keywdarg",
NULL,
-1,
keywdarg_methods
static struct PyModuleDef keywdarg_module = {
.m_base = PyModuleDef_HEAD_INIT,
.m_name = "keywdarg",
.m_size = 0,
.m_methods = keywdarg_methods,
};

PyMODINIT_FUNC
PyInit_keywdarg(void)
{
return PyModule_Create(&keywdargmodule);
return PyModuleDef_Init(&keywdarg_module);
}


Expand Down Expand Up @@ -1072,8 +1069,9 @@ why his :meth:`!__del__` methods would fail...

The second case of problems with a borrowed reference is a variant involving
threads. Normally, multiple threads in the Python interpreter can't get in each
other's way, because there is a global lock protecting Python's entire object
space. However, it is possible to temporarily release this lock using the macro
other's way, because there is a :term:`global lock <global interpreter lock>`
protecting Python's entire object space.
However, it is possible to temporarily release this lock using the macro
:c:macro:`Py_BEGIN_ALLOW_THREADS`, and to re-acquire it using
:c:macro:`Py_END_ALLOW_THREADS`. This is common around blocking I/O calls, to
let other threads use the processor while waiting for the I/O to complete.
Expand Down Expand Up @@ -1259,32 +1257,26 @@ two more lines must be added::
#include "spammodule.h"

The ``#define`` is used to tell the header file that it is being included in the
exporting module, not a client module. Finally, the module's initialization
function must take care of initializing the C API pointer array::
exporting module, not a client module. Finally, the module's :c:data:`mod_exec
<Py_mod_exec>` function must take care of initializing the C API pointer array::

PyMODINIT_FUNC
PyInit_spam(void)
static int
spam_module_exec(PyObject *m)
{
PyObject *m;
static void *PySpam_API[PySpam_API_pointers];
PyObject *c_api_object;

m = PyModule_Create(&spammodule);
if (m == NULL)
return NULL;

/* Initialize the C API pointer array */
PySpam_API[PySpam_System_NUM] = (void *)PySpam_System;

/* Create a Capsule containing the API pointer array's address */
c_api_object = PyCapsule_New((void *)PySpam_API, "spam._C_API", NULL);

if (PyModule_Add(m, "_C_API", c_api_object) < 0) {
Py_DECREF(m);
return NULL;
return -1;
}

return m;
return 0;
}

Note that ``PySpam_API`` is declared ``static``; otherwise the pointer
Expand Down Expand Up @@ -1343,20 +1335,16 @@ like this::

All that a client module must do in order to have access to the function
:c:func:`!PySpam_System` is to call the function (or rather macro)
:c:func:`!import_spam` in its initialization function::
:c:func:`!import_spam` in its :c:data:`mod_exec <Py_mod_exec>` function::

PyMODINIT_FUNC
PyInit_client(void)
static int
client_module_exec(PyObject *m)
{
PyObject *m;

m = PyModule_Create(&clientmodule);
if (m == NULL)
return NULL;
if (import_spam() < 0)
return NULL;
if (import_spam() < 0) {
return -1;
}
/* additional initialization can happen here */
return m;
return 0;
}

The main disadvantage of this approach is that the file :file:`spammodule.h` is
Expand Down
4 changes: 4 additions & 0 deletions Doc/extending/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,10 @@ assistance from third party tools. It is intended primarily for creators
of those tools, rather than being a recommended way to create your own
C extensions.

.. seealso::

:pep:`489` -- Multi-phase extension module initialization

.. toctree::
:maxdepth: 2
:numbered:
Expand Down
Loading
Loading