-
-
Notifications
You must be signed in to change notification settings - Fork 32.5k
gh-137210: Add a struct, slot & function for checking an extension's ABI #137212
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
894ef3d
72349ba
3b6aa96
c015606
c5740af
da293e3
44df416
feb428d
aa1eb4a
b628d7d
4800a5d
4c9e918
1b18761
814ac60
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -2,9 +2,9 @@ | |
|
||
.. _stable: | ||
|
||
*************** | ||
C API Stability | ||
*************** | ||
*********************** | ||
C API and ABI Stability | ||
*********************** | ||
|
||
Unless documented otherwise, Python's C API is covered by the Backwards | ||
Compatibility Policy, :pep:`387`. | ||
|
@@ -199,6 +199,151 @@ | |
third-party distributors. | ||
|
||
|
||
ABI Checking | ||
============ | ||
|
||
.. versionadded:: next | ||
|
||
Python includes a rudimentary check for ABI compatibility. | ||
|
||
This check is not comprehensive. | ||
It only guards against common cases of incompatible modules being | ||
installed for the wrong interpreter. | ||
It also does not take :ref:`platform incompatibilities <stable-abi-platform>` | ||
into account. | ||
It can only be done after an extension is successfully loaded. | ||
|
||
Despite these limitations, it is recommended that extension modules use this | ||
mechanism, so that detectable incompatibilities raise exceptions rather than | ||
crash. | ||
|
||
Most modules can use this check via the :c:data:`Py_mod_abi` | ||
slot and the :c:macro:`PyABIInfo_VAR` macro. | ||
The full API is described below for advanced use cases. | ||
|
||
.. c:function:: int PyABIInfo_Check(PyABIInfo *info, const char *module_name) | ||
|
||
Verify that the given *info* is compatible with the currently running | ||
interpreter. | ||
|
||
Return 0 on success. On failure, raise an exception and return -1. | ||
|
||
If the ABI is incompatible, the raised exception will be :py:exc:`ImportError`. | ||
|
||
The *module_name* argument can be ``NULL``, or point to a NUL-terminated | ||
UTF-8-encoded string used for error messages. | ||
|
||
Note that if *info* describes the ABI that the current code uses (as defined | ||
by :c:macro:`PyABIInfo_VAR`, for example), using any other Python C API | ||
may lead to crashes. | ||
In particular, it is not safe to examine the raised exception. | ||
|
||
.. versionadded:: next | ||
|
||
.. c:macro:: PyABIInfo_VAR(NAME) | ||
|
||
Define a static :c:struct:`PyABIInfo` variable with the given *NAME* that | ||
describes the ABI that the current code will use. | ||
This macro expands to: | ||
|
||
.. code-block:: c | ||
|
||
static PyABIInfo NAME = { | ||
1, 0, | ||
PyABIInfo_DEFAULT_FLAGS, | ||
PY_VERSION_HEX, | ||
PyABIInfo_DEFAULT_ABI_VERSION | ||
} | ||
|
||
.. versionadded:: next | ||
|
||
.. c:type:: PyABIInfo | ||
|
||
.. c:member:: uint8_t abiinfo_major_version | ||
|
||
The major version of :c:struct:`PyABIInfo`. Can be set to: | ||
|
||
* ``0`` to skip all checking, or | ||
* ``1`` to specify this version of :c:struct:`!PyABIInfo`. | ||
|
||
.. c:member:: uint8_t abiinfo_minor_version | ||
|
||
The major version of :c:struct:`PyABIInfo`. | ||
Must be set to ``0``; larger values are reserved for backwards-compatible | ||
future versions of :c:struct:`!PyABIInfo`. | ||
|
||
.. c:member:: uint16_t flags | ||
|
||
.. c:namespace:: NULL | ||
|
||
This field is usually set to the following macro: | ||
|
||
.. c:macro:: PyABIInfo_DEFAULT_FLAGS | ||
|
||
Default flags, based on current values of macros such as | ||
:c:macro:`Py_LIMITED_API` and :c:macro:`Py_GIL_DISABLED`. | ||
|
||
Alternately, the field can be set to to the following flags, combined | ||
by bitwise OR. | ||
Unused bits must be set to zero. | ||
|
||
ABI variant -- one of: | ||
|
||
.. c:macro:: PyABIInfo_STABLE | ||
|
||
Specifies that the stable ABI is used. | ||
|
||
.. c:macro:: PyABIInfo_INTERNAL | ||
|
||
Specifies ABI specific to a particular build of CPython. | ||
Internal use only. | ||
|
||
Free-threading compatibility -- one of: | ||
|
||
.. c:macro:: PyABIInfo_FREETHREADED | ||
|
||
Specifies ABI compatible with free-threading builds of CPython. | ||
(That is, ones compiled with :option:`--disable-gil`; with ``t`` | ||
in :py:data:`sys.abiflags`) | ||
|
||
.. c:macro:: PyABIInfo_GIL | ||
|
||
Specifies ABI compatible with non-free-threading builds of CPython | ||
(ones compiled *without* :option:`--disable-gil`). | ||
|
||
.. c:member:: uint32_t build_version | ||
|
||
The version of the Python headers used to build the code, in the format | ||
used by :c:macro:`PY_VERSION_HEX`. | ||
|
||
This can be set to ``0`` to skip any checks related to this field. | ||
This option is meant mainly for projects that do not use the CPython | ||
headers directly, and do not emulate a specific version of them. | ||
|
||
.. c:member:: uint32_t abi_version | ||
|
||
The ABI version. | ||
|
||
For the Stable ABI, this field should be the value of | ||
:c:macro:`Py_LIMITED_API` | ||
(except if :c:macro:`Py_LIMITED_API` is ``3``; use | ||
:c:expr:`Py_PACK_VERSION(3, 2)` in that case). | ||
Comment on lines
+329
to
+330
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Seems easier for us to just check this on our side than to expect users to get it right? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. And isn't the legacy value There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Well, users should use The legacy value is 3. (Not that it matters that much; it's always compared to a Python version.) |
||
|
||
Otherwise, it should be set to :c:macro:`PY_VERSION_HEX`. | ||
|
||
It can also be set to ``0`` to skip any checks related to this field. | ||
|
||
.. c:namespace:: NULL | ||
|
||
.. c:macro:: PyABIInfo_DEFAULT_ABI_VERSION | ||
|
||
The value that should be used for this field, based on current | ||
values of macros such as :c:macro:`Py_LIMITED_API`, | ||
:c:macro:`PY_VERSION_HEX` and :c:macro:`Py_GIL_DISABLED`. | ||
|
||
.. versionadded:: next | ||
|
||
|
||
.. _limited-api-list: | ||
|
||
Contents of Limited API | ||
|
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -56,55 +56,14 @@ PyAPI_FUNC(int) PyModule_ExecDef(PyObject *module, PyModuleDef *def); | |
|
||
#define Py_CLEANUP_SUPPORTED 0x20000 | ||
|
||
/* The API and ABI versions are left for backwards compatibility. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can't we just... update them? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. To what? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't think it particularly matters, it could be inferred from It's just an arbitrary value with a purpose that isn't really going away, so deprecating them seems unnecessary when we could just keep using them for their intended purposes (and ignore that they didn't get updated for a large number of ABI changes). There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Oh, do you want to use an I don't think reusing these would buy us much. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Or just use it as one of the values that goes into the struct. We can include more information if needed, it just seems like a waste to declare these names as irrelevant when we could just as easily make them relevant. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We can make them relevant, but what relevance should they have? If we can find a use case in the future, that's what the versioning in |
||
They've not been updated since 2006 and 2010, respectively. | ||
API/ABI versioning is now tied to the CPython version. | ||
The *_VERSION and *_STRING symbols should define the same value; as | ||
number and string literal respectively. Make sure the definitions match. | ||
*/ | ||
#define PYTHON_API_VERSION 1013 | ||
#define PYTHON_API_STRING "1013" | ||
/* The API version is maintained (independently from the Python version) | ||
so we can detect mismatches between the interpreter and dynamically | ||
loaded modules. These are diagnosed by an error message but | ||
the module is still loaded (because the mismatch can only be tested | ||
after loading the module). The error message is intended to | ||
explain the core dump a few seconds later. | ||
|
||
The symbol PYTHON_API_STRING defines the same value as a string | ||
literal. *** PLEASE MAKE SURE THE DEFINITIONS MATCH. *** | ||
|
||
Please add a line or two to the top of this log for each API | ||
version change: | ||
|
||
22-Feb-2006 MvL 1013 PEP 353 - long indices for sequence lengths | ||
|
||
19-Aug-2002 GvR 1012 Changes to string object struct for | ||
interning changes, saving 3 bytes. | ||
|
||
17-Jul-2001 GvR 1011 Descr-branch, just to be on the safe side | ||
|
||
25-Jan-2001 FLD 1010 Parameters added to PyCode_New() and | ||
PyFrame_New(); Python 2.1a2 | ||
|
||
14-Mar-2000 GvR 1009 Unicode API added | ||
|
||
3-Jan-1999 GvR 1007 Decided to change back! (Don't reuse 1008!) | ||
|
||
3-Dec-1998 GvR 1008 Python 1.5.2b1 | ||
|
||
18-Jan-1997 GvR 1007 string interning and other speedups | ||
|
||
11-Oct-1996 GvR renamed Py_Ellipses to Py_Ellipsis :-( | ||
|
||
30-Jul-1996 GvR Slice and ellipses syntax added | ||
|
||
23-Jul-1996 GvR For 1.4 -- better safe than sorry this time :-) | ||
|
||
7-Nov-1995 GvR Keyword arguments (should've been done at 1.3 :-( ) | ||
|
||
10-Jan-1995 GvR Renamed globals to new naming scheme | ||
|
||
9-Jan-1995 GvR Initial version (incompatible with older API) | ||
*/ | ||
|
||
/* The PYTHON_ABI_VERSION is introduced in PEP 384. For the lifetime of | ||
Python 3, it will stay at the value of 3; changes to the limited API | ||
must be performed in a strictly backwards-compatible manner. */ | ||
#define PYTHON_ABI_VERSION 3 | ||
#define PYTHON_ABI_STRING "3" | ||
|
||
|
@@ -134,6 +93,72 @@ PyAPI_FUNC(PyObject *) PyModule_FromDefAndSpec2(PyModuleDef *def, | |
|
||
#endif /* New in 3.5 */ | ||
|
||
/* ABI info & checking (new in 3.15) */ | ||
#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x030f0000 | ||
typedef struct PyABIInfo { | ||
uint8_t abiinfo_major_version; | ||
uint8_t abiinfo_minor_version; | ||
uint16_t flags; | ||
uint32_t build_version; | ||
uint32_t abi_version; | ||
} PyABIInfo; | ||
#define PyABIInfo_STABLE 0x0001 | ||
#define PyABIInfo_GIL 0x0002 | ||
#define PyABIInfo_FREETHREADED 0x0004 | ||
#define PyABIInfo_INTERNAL 0x0008 | ||
|
||
#define PyABIInfo_FREETHREADING_AGNOSTIC (PyABIInfo_GIL|PyABIInfo_FREETHREADED) | ||
|
||
PyAPI_FUNC(int) PyABIInfo_Check(PyABIInfo *info, const char *module_name); | ||
|
||
// Define the defaults | ||
#ifdef Py_LIMITED_API | ||
#define _PyABIInfo_DEFAULT_FLAG_STABLE PyABIInfo_STABLE | ||
#if Py_LIMITED_API == 3 | ||
#define PyABIInfo_DEFAULT_ABI_VERSION _Py_PACK_VERSION(3, 2) | ||
#else | ||
#define PyABIInfo_DEFAULT_ABI_VERSION Py_LIMITED_API | ||
#endif | ||
#else | ||
#define _PyABIInfo_DEFAULT_FLAG_STABLE 0 | ||
#define PyABIInfo_DEFAULT_ABI_VERSION PY_VERSION_HEX | ||
#endif | ||
#if defined(Py_LIMITED_API) && defined(_Py_OPAQUE_PYOBJECT) | ||
#define _PyABIInfo_DEFAULT_FLAG_FT PyABIInfo_FREETHREADING_AGNOSTIC | ||
#elif defined(Py_GIL_DISABLED) | ||
#define _PyABIInfo_DEFAULT_FLAG_FT PyABIInfo_FREETHREADED | ||
#else | ||
#define _PyABIInfo_DEFAULT_FLAG_FT PyABIInfo_GIL | ||
#endif | ||
#if defined(Py_BUILD_CORE) | ||
#define _PyABIInfo_DEFAULT_FLAG_INTERNAL PyABIInfo_INTERNAL | ||
#else | ||
#define _PyABIInfo_DEFAULT_FLAG_INTERNAL 0 | ||
#endif | ||
|
||
#define PyABIInfo_DEFAULT_FLAGS ( \ | ||
_PyABIInfo_DEFAULT_FLAG_STABLE \ | ||
| _PyABIInfo_DEFAULT_FLAG_FT \ | ||
| _PyABIInfo_DEFAULT_FLAG_INTERNAL \ | ||
) \ | ||
///////////////////////////////////////////////////////// | ||
|
||
#define _PyABIInfo_DEFAULT() { \ | ||
1, 0, \ | ||
PyABIInfo_DEFAULT_FLAGS, \ | ||
PY_VERSION_HEX, \ | ||
PyABIInfo_DEFAULT_ABI_VERSION } \ | ||
///////////////////////////////////////////////////////// | ||
|
||
#define PyABIInfo_VAR(NAME) \ | ||
static PyABIInfo NAME = _PyABIInfo_DEFAULT; | ||
|
||
#undef _PyABIInfo_DEFAULT_STABLE | ||
#undef _PyABIInfo_DEFAULT_FT | ||
#undef _PyABIInfo_DEFAULT_INTERNAL | ||
|
||
#endif /* ABI info (new in 3.15) */ | ||
|
||
#ifndef Py_LIMITED_API | ||
# define Py_CPYTHON_MODSUPPORT_H | ||
# include "cpython/modsupport.h" | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We're still planning to get rid of these options once FT is merged, right? Maybe we ought to use
abiflags
here (i.e. a string) instead, for future extensibility? This isn't going to be checked anywhere that it needs to be bitflags.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How would you encode being compatible with both GIL and FT?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IMO, we can remove them when there are no GIL-only extensions around any more. That would be a lot of time after FT is the only option.