Skip to content

gh-132629: Deprecate acception out of range values for unsigned integers in PyArg_Parse #132630

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
+649 −145
Open

gh-132629: Deprecate acception out of range values for unsigned integers in PyArg_Parse #132630

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
fbc7873
gh-132629: Deprecate acception out of range values for unsigned integ…
serhiy-storchaka Apr 17, 2025
ddd9ee5
Fix compilation.
serhiy-storchaka Apr 17, 2025
ffa69cd
Merge branch 'main' into PyArg_Parse-unsigned-ints
serhiy-storchaka Apr 23, 2025
478d396
Use the limited C API again.
serhiy-storchaka Apr 23, 2025
dc1b80b
Merge branch 'main' into PyArg_Parse-unsigned-ints
serhiy-storchaka Apr 29, 2025
be4f26b
Fix merge error.
serhiy-storchaka Apr 29, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 17 additions & 10 deletions Doc/c-api/arg.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -241,9 +241,11 @@ the Python object to the required type.

For signed integer formats, :exc:`OverflowError` is raised if the value
is out of range for the C type.
For unsigned integer formats, no range checking is done --- the
For unsigned integer formats, the
most significant bits are silently truncated when the receiving field is too
small to receive the value.
small to receive the value, and :exc:`DeprecationWarning` is emitted when
the value is larger than the maximal value the C type or less than the
minimal value for the corresponding signed integer type.

``b`` (:class:`int`) [unsigned char]
Convert a nonnegative Python integer to an unsigned tiny integer, stored in a C
Expand All @@ -252,27 +254,25 @@ small to receive the value.
``B`` (:class:`int`) [unsigned char]
Convert a Python integer to a tiny integer without overflow checking, stored in a C
:c:expr:`unsigned char`.
Convert a Python integer to a C :c:expr:`unsigned char`.

``h`` (:class:`int`) [short int]
Convert a Python integer to a C :c:expr:`short int`.

``H`` (:class:`int`) [unsigned short int]
Convert a Python integer to a C :c:expr:`unsigned short int`, without overflow
checking.
Convert a Python integer to a C :c:expr:`unsigned short int`.

``i`` (:class:`int`) [int]
Convert a Python integer to a plain C :c:expr:`int`.

``I`` (:class:`int`) [unsigned int]
Convert a Python integer to a C :c:expr:`unsigned int`, without overflow
checking.
Convert a Python integer to a C :c:expr:`unsigned int`.

``l`` (:class:`int`) [long int]
Convert a Python integer to a C :c:expr:`long int`.

``k`` (:class:`int`) [unsigned long]
Convert a Python integer to a C :c:expr:`unsigned long` without
overflow checking.
Convert a Python integer to a C :c:expr:`unsigned long`.

.. versionchanged:: next
Use :meth:`~object.__index__` if available.
Expand All @@ -281,8 +281,7 @@ small to receive the value.
Convert a Python integer to a C :c:expr:`long long`.

``K`` (:class:`int`) [unsigned long long]
Convert a Python integer to a C :c:expr:`unsigned long long`
without overflow checking.
Convert a Python integer to a C :c:expr:`unsigned long long`.

.. versionchanged:: next
Use :meth:`~object.__index__` if available.
Expand Down Expand Up @@ -310,6 +309,14 @@ small to receive the value.
``D`` (:class:`complex`) [Py_complex]
Convert a Python complex number to a C :c:type:`Py_complex` structure.

.. deprecated:: next

For unsigned integer formats ``B``, ``H``, ``I``, ``k`` and ``K``,
:exc:`DeprecationWarning` is emitted when the value is larger than
the maximal value the C type or less than the minimal value for
the corresponding signed integer type.


Other objects
-------------

Expand Down
7 changes: 7 additions & 0 deletions Doc/whatsnew/3.14.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2211,6 +2211,13 @@ Deprecated
or a :term:`borrowed reference`.
(Contributed by Serhiy Storchaka in :gh:`50333`.)

* For unsigned integer formats in :c:func:`PyArg_ParseTuple`,
accepting Python integers with value that is larger than
the maximal value the corresponding C type or less than
the minimal value for the corresponding signed integer type
is now deprecated.
(Contributed by Serhiy Storchaka in :gh:`132629`.)

* The previously undocumented function :c:func:`PySequence_In` is :term:`soft deprecated`.
Use :c:func:`PySequence_Contains` instead.
(Contributed by Yuki Kobayashi in :gh:`127896`.)
Expand Down
95 changes: 78 additions & 17 deletions Lib/test/clinic.test.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1020,12 +1020,19 @@ test_unsigned_char_converter(PyObject *module, PyObject *const *args, Py_ssize_t
goto skip_optional;
}
{
unsigned long ival = PyLong_AsUnsignedLongMask(args[2]);
if (ival == (unsigned long)-1 && PyErr_Occurred()) {
Py_ssize_t _bytes = PyLong_AsNativeBytes(args[2], &c, sizeof(unsigned char),
Py_ASNATIVEBYTES_NATIVE_ENDIAN |
Py_ASNATIVEBYTES_ALLOW_INDEX |
Py_ASNATIVEBYTES_UNSIGNED_BUFFER);
if (_bytes < 0) {
goto exit;
}
else {
c = (unsigned char) ival;
if ((size_t)_bytes > sizeof(unsigned char)) {
if (PyErr_WarnEx(PyExc_DeprecationWarning,
"integer value out of range", 1) < 0)
{
goto exit;
}
}
}
skip_optional:
Expand All @@ -1038,7 +1045,7 @@ test_unsigned_char_converter(PyObject *module, PyObject *const *args, Py_ssize_t
static PyObject *
test_unsigned_char_converter_impl(PyObject *module, unsigned char a,
unsigned char b, unsigned char c)
/*[clinic end generated code: output=45920dbedc22eb55 input=021414060993e289]*/
/*[clinic end generated code: output=49eda9faaf53372a input=021414060993e289]*/


/*[clinic input]
Expand Down Expand Up @@ -1151,9 +1158,21 @@ test_unsigned_short_converter(PyObject *module, PyObject *const *args, Py_ssize_
if (nargs < 3) {
goto skip_optional;
}
c = (unsigned short)PyLong_AsUnsignedLongMask(args[2]);
if (c == (unsigned short)-1 && PyErr_Occurred()) {
goto exit;
{
Py_ssize_t _bytes = PyLong_AsNativeBytes(args[2], &c, sizeof(unsigned short),
Py_ASNATIVEBYTES_NATIVE_ENDIAN |
Py_ASNATIVEBYTES_ALLOW_INDEX |
Py_ASNATIVEBYTES_UNSIGNED_BUFFER);
if (_bytes < 0) {
goto exit;
}
if ((size_t)_bytes > sizeof(unsigned short)) {
if (PyErr_WarnEx(PyExc_DeprecationWarning,
"integer value out of range", 1) < 0)
{
goto exit;
}
}
}
skip_optional:
return_value = test_unsigned_short_converter_impl(module, a, b, c);
Expand All @@ -1165,7 +1184,7 @@ test_unsigned_short_converter(PyObject *module, PyObject *const *args, Py_ssize_
static PyObject *
test_unsigned_short_converter_impl(PyObject *module, unsigned short a,
unsigned short b, unsigned short c)
/*[clinic end generated code: output=e6e990df729114fc input=cdfd8eff3d9176b4]*/
/*[clinic end generated code: output=f591c7797e150f49 input=cdfd8eff3d9176b4]*/


/*[clinic input]
Expand Down Expand Up @@ -1298,9 +1317,21 @@ test_unsigned_int_converter(PyObject *module, PyObject *const *args, Py_ssize_t
if (nargs < 3) {
goto skip_optional;
}
c = (unsigned int)PyLong_AsUnsignedLongMask(args[2]);
if (c == (unsigned int)-1 && PyErr_Occurred()) {
goto exit;
{
Py_ssize_t _bytes = PyLong_AsNativeBytes(args[2], &c, sizeof(unsigned int),
Py_ASNATIVEBYTES_NATIVE_ENDIAN |
Py_ASNATIVEBYTES_ALLOW_INDEX |
Py_ASNATIVEBYTES_UNSIGNED_BUFFER);
if (_bytes < 0) {
goto exit;
}
if ((size_t)_bytes > sizeof(unsigned int)) {
if (PyErr_WarnEx(PyExc_DeprecationWarning,
"integer value out of range", 1) < 0)
{
goto exit;
}
}
}
skip_optional:
return_value = test_unsigned_int_converter_impl(module, a, b, c);
Expand All @@ -1312,7 +1343,7 @@ test_unsigned_int_converter(PyObject *module, PyObject *const *args, Py_ssize_t
static PyObject *
test_unsigned_int_converter_impl(PyObject *module, unsigned int a,
unsigned int b, unsigned int c)
/*[clinic end generated code: output=f9cdbe410ccc98a3 input=5533534828b62fc0]*/
/*[clinic end generated code: output=50a413f1cc82dc11 input=5533534828b62fc0]*/


/*[clinic input]
Expand Down Expand Up @@ -1414,7 +1445,22 @@ test_unsigned_long_converter(PyObject *module, PyObject *const *args, Py_ssize_t
_PyArg_BadArgument("test_unsigned_long_converter", "argument 3", "int", args[2]);
goto exit;
}
c = PyLong_AsUnsignedLongMask(args[2]);
{
Py_ssize_t _bytes = PyLong_AsNativeBytes(args[2], &c, sizeof(unsigned long),
Py_ASNATIVEBYTES_NATIVE_ENDIAN |
Py_ASNATIVEBYTES_ALLOW_INDEX |
Py_ASNATIVEBYTES_UNSIGNED_BUFFER);
if (_bytes < 0) {
goto exit;
}
if ((size_t)_bytes > sizeof(unsigned long)) {
if (PyErr_WarnEx(PyExc_DeprecationWarning,
"integer value out of range", 1) < 0)
{
goto exit;
}
}
}
skip_optional:
return_value = test_unsigned_long_converter_impl(module, a, b, c);

Expand All @@ -1425,7 +1471,7 @@ test_unsigned_long_converter(PyObject *module, PyObject *const *args, Py_ssize_t
static PyObject *
test_unsigned_long_converter_impl(PyObject *module, unsigned long a,
unsigned long b, unsigned long c)
/*[clinic end generated code: output=d74eed227d77a31b input=f450d94cae1ef73b]*/
/*[clinic end generated code: output=1bbf5620093cc914 input=f450d94cae1ef73b]*/


/*[clinic input]
Expand Down Expand Up @@ -1529,7 +1575,22 @@ test_unsigned_long_long_converter(PyObject *module, PyObject *const *args, Py_ss
_PyArg_BadArgument("test_unsigned_long_long_converter", "argument 3", "int", args[2]);
goto exit;
}
c = PyLong_AsUnsignedLongLongMask(args[2]);
{
Py_ssize_t _bytes = PyLong_AsNativeBytes(args[2], &c, sizeof(unsigned long long),
Py_ASNATIVEBYTES_NATIVE_ENDIAN |
Py_ASNATIVEBYTES_ALLOW_INDEX |
Py_ASNATIVEBYTES_UNSIGNED_BUFFER);
if (_bytes < 0) {
goto exit;
}
if ((size_t)_bytes > sizeof(unsigned long long)) {
if (PyErr_WarnEx(PyExc_DeprecationWarning,
"integer value out of range", 1) < 0)
{
goto exit;
}
}
}
skip_optional:
return_value = test_unsigned_long_long_converter_impl(module, a, b, c);

Expand All @@ -1542,7 +1603,7 @@ test_unsigned_long_long_converter_impl(PyObject *module,
unsigned long long a,
unsigned long long b,
unsigned long long c)
/*[clinic end generated code: output=5ca4e4dfb3db644b input=a15115dc41866ff4]*/
/*[clinic end generated code: output=582a6623dc845824 input=a15115dc41866ff4]*/


/*[clinic input]
Expand Down
70 changes: 55 additions & 15 deletions Lib/test/test_capi/test_getargs.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,18 +48,17 @@
LARGE = 0x7FFFFFFF
VERY_LARGE = 0xFF0000121212121212121242

from _testcapi import UCHAR_MAX, USHRT_MAX, UINT_MAX, ULONG_MAX, INT_MAX, \
INT_MIN, LONG_MIN, LONG_MAX, PY_SSIZE_T_MIN, PY_SSIZE_T_MAX, \
from _testcapi import UCHAR_MAX, USHRT_MAX, UINT_MAX, ULONG_MAX, ULLONG_MAX, INT_MAX, \
INT_MIN, LONG_MIN, LONG_MAX, LLONG_MIN, LLONG_MAX, PY_SSIZE_T_MIN, PY_SSIZE_T_MAX, \
SHRT_MIN, SHRT_MAX, FLT_MIN, FLT_MAX, DBL_MIN, DBL_MAX

DBL_MAX_EXP = sys.float_info.max_exp
INF = float('inf')
NAN = float('nan')

# fake, they are not defined in Python's header files
LLONG_MAX = 2**63-1
LLONG_MIN = -2**63
ULLONG_MAX = 2**64-1
SCHAR_MAX = UCHAR_MAX // 2
SCHAR_MIN = SCHAR_MAX - UCHAR_MAX

NULL = None

Expand Down Expand Up @@ -209,10 +208,23 @@ def test_B(self):
self.assertEqual(UCHAR_MAX, getargs_B(-1))
self.assertEqual(0, getargs_B(0))
self.assertEqual(UCHAR_MAX, getargs_B(UCHAR_MAX))
self.assertEqual(0, getargs_B(UCHAR_MAX+1))
with self.assertWarns(DeprecationWarning):
self.assertEqual(0, getargs_B(UCHAR_MAX+1))
with self.assertWarns(DeprecationWarning):
self.assertEqual(1, getargs_B(-UCHAR_MAX))
self.assertEqual(SCHAR_MAX+1, getargs_B(SCHAR_MIN))
with self.assertWarns(DeprecationWarning):
self.assertEqual(SCHAR_MAX, getargs_B(SCHAR_MIN-1))

self.assertEqual(128, getargs_B(-2**7))
with self.assertWarns(DeprecationWarning):
self.assertEqual(127, getargs_B(-2**7-1))

self.assertEqual(42, getargs_B(42))
self.assertEqual(UCHAR_MAX & VERY_LARGE, getargs_B(VERY_LARGE))
with self.assertWarns(DeprecationWarning):
self.assertEqual(UCHAR_MAX & VERY_LARGE, getargs_B(VERY_LARGE))
with self.assertWarns(DeprecationWarning):
self.assertEqual(UCHAR_MAX & -VERY_LARGE, getargs_B(-VERY_LARGE))

def test_H(self):
from _testcapi import getargs_H
Expand All @@ -233,11 +245,18 @@ def test_H(self):
self.assertEqual(USHRT_MAX, getargs_H(-1))
self.assertEqual(0, getargs_H(0))
self.assertEqual(USHRT_MAX, getargs_H(USHRT_MAX))
self.assertEqual(0, getargs_H(USHRT_MAX+1))
with self.assertWarns(DeprecationWarning):
self.assertEqual(0, getargs_H(USHRT_MAX+1))
self.assertEqual(SHRT_MAX+1, getargs_H(SHRT_MIN))
with self.assertWarns(DeprecationWarning):
self.assertEqual(SHRT_MAX, getargs_H(SHRT_MIN-1))

self.assertEqual(42, getargs_H(42))

self.assertEqual(VERY_LARGE & USHRT_MAX, getargs_H(VERY_LARGE))
with self.assertWarns(DeprecationWarning):
self.assertEqual(USHRT_MAX & VERY_LARGE, getargs_H(VERY_LARGE))
with self.assertWarns(DeprecationWarning):
self.assertEqual(USHRT_MAX & -VERY_LARGE, getargs_H(-VERY_LARGE))

def test_I(self):
from _testcapi import getargs_I
Expand All @@ -258,11 +277,18 @@ def test_I(self):
self.assertEqual(UINT_MAX, getargs_I(-1))
self.assertEqual(0, getargs_I(0))
self.assertEqual(UINT_MAX, getargs_I(UINT_MAX))
self.assertEqual(0, getargs_I(UINT_MAX+1))
with self.assertWarns(DeprecationWarning):
self.assertEqual(0, getargs_I(UINT_MAX+1))
self.assertEqual(INT_MAX+1, getargs_I(INT_MIN))
with self.assertWarns(DeprecationWarning):
self.assertEqual(INT_MAX, getargs_I(INT_MIN-1))

self.assertEqual(42, getargs_I(42))

self.assertEqual(VERY_LARGE & UINT_MAX, getargs_I(VERY_LARGE))
with self.assertWarns(DeprecationWarning):
self.assertEqual(UINT_MAX & VERY_LARGE, getargs_I(VERY_LARGE))
with self.assertWarns(DeprecationWarning):
self.assertEqual(UINT_MAX & -VERY_LARGE, getargs_I(-VERY_LARGE))

def test_k(self):
from _testcapi import getargs_k
Expand All @@ -283,11 +309,18 @@ def test_k(self):
self.assertEqual(ULONG_MAX, getargs_k(-1))
self.assertEqual(0, getargs_k(0))
self.assertEqual(ULONG_MAX, getargs_k(ULONG_MAX))
self.assertEqual(0, getargs_k(ULONG_MAX+1))
with self.assertWarns(DeprecationWarning):
self.assertEqual(0, getargs_k(ULONG_MAX+1))
self.assertEqual(LONG_MAX+1, getargs_k(LONG_MIN))
with self.assertWarns(DeprecationWarning):
self.assertEqual(LONG_MAX, getargs_k(LONG_MIN-1))

self.assertEqual(42, getargs_k(42))

self.assertEqual(VERY_LARGE & ULONG_MAX, getargs_k(VERY_LARGE))
with self.assertWarns(DeprecationWarning):
self.assertEqual(ULONG_MAX & VERY_LARGE, getargs_k(VERY_LARGE))
with self.assertWarns(DeprecationWarning):
self.assertEqual(ULONG_MAX & -VERY_LARGE, getargs_k(-VERY_LARGE))

class Signed_TestCase(unittest.TestCase):
def test_h(self):
Expand Down Expand Up @@ -434,11 +467,18 @@ def test_K(self):
self.assertEqual(ULLONG_MAX, getargs_K(ULLONG_MAX))
self.assertEqual(0, getargs_K(0))
self.assertEqual(ULLONG_MAX, getargs_K(ULLONG_MAX))
self.assertEqual(0, getargs_K(ULLONG_MAX+1))
with self.assertWarns(DeprecationWarning):
self.assertEqual(0, getargs_K(ULLONG_MAX+1))
self.assertEqual(LLONG_MAX+1, getargs_K(LLONG_MIN))
with self.assertWarns(DeprecationWarning):
self.assertEqual(LLONG_MAX, getargs_K(LLONG_MIN-1))

self.assertEqual(42, getargs_K(42))

self.assertEqual(VERY_LARGE & ULLONG_MAX, getargs_K(VERY_LARGE))
with self.assertWarns(DeprecationWarning):
self.assertEqual(ULLONG_MAX & VERY_LARGE, getargs_K(VERY_LARGE))
with self.assertWarns(DeprecationWarning):
self.assertEqual(ULLONG_MAX & -VERY_LARGE, getargs_K(-VERY_LARGE))


class Float_TestCase(unittest.TestCase, FloatsAreIdenticalMixin):
Expand Down
Loading
Loading