Skip to content

gh-133164: Add PyUnstable_Object_IsUniqueReferencedTemporary C API #133170

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
32 changes: 32 additions & 0 deletions Doc/c-api/object.rst
Original file line number Diff line number Diff line change
Expand Up @@ -613,6 +613,38 @@ Object Protocol

.. versionadded:: 3.14

.. c:function:: int PyUnstable_Object_IsUniqueReferencedTemporary(PyObject *obj)

Check if *obj* is a unique temporary object.
Returns ``1`` if *obj* is known to be a unique temporary object,
and ``0`` otherwise. This function cannot fail, but the check is
conservative, and may return ``0`` in some cases even if *obj* is a unique
temporary object.

If an object is a unique temporary, it is guaranteed that the current code
has the only reference to the object. For arguments to C functions, this
should be used instead of checking if the reference count is ``1``. Starting
with Python 3.14, the interpreter internally avoids some reference count
modifications when loading objects onto the operands stack by
:term:`borrowing <borrowed reference>` references when possible, which means
that a reference count of ``1`` by itself does not guarantee that a function
argument uniquely referenced.

In the example below, ``my_func`` is called with a unique temporary object
as its argument::

my_func([1, 2, 3])

In the example below, ``my_func`` is **not** called with a unique temporary
object as its argument, even if its refcount is ``1``::

my_list = [1, 2, 3]
my_func(my_list)

See also the function :c:func:`Py_REFCNT`.

.. versionadded:: 3.14

.. c:function:: int PyUnstable_IsImmortal(PyObject *obj)

This function returns non-zero if *obj* is :term:`immortal`, and zero
Expand Down
2 changes: 2 additions & 0 deletions Doc/c-api/refcounting.rst
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ of Python objects.

Use the :c:func:`Py_SET_REFCNT()` function to set an object reference count.

See also the function :c:func:`PyUnstable_Object_IsUniqueReferencedTemporary()`.

.. versionchanged:: 3.10
:c:func:`Py_REFCNT()` is changed to the inline static function.

Expand Down
20 changes: 20 additions & 0 deletions Doc/whatsnew/3.14.rst
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,10 @@ If you encounter :exc:`NameError`\s or pickling errors coming out of
:mod:`multiprocessing` or :mod:`concurrent.futures`, see the
:ref:`forkserver restrictions <multiprocessing-programming-forkserver>`.

The interpreter avoids some reference count modifications internally when
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👌

it's safe to do so. This can lead to different values returned from
:func:`sys.getrefcount` and :c:func:`Py_REFCNT` compared to previous versions
of Python. See :ref:`below <whatsnew314-refcount>` for details.

New features
============
Expand Down Expand Up @@ -2210,6 +2214,11 @@ New features
take a C integer and produce a Python :class:`bool` object. (Contributed by
Pablo Galindo in :issue:`45325`.)

* Add :c:func:`PyUnstable_Object_IsUniqueReferencedTemporary` to determine if an object
is a unique temporary object on the interpreter's operand stack. This can
be used in some cases as a replacement for checking if :c:func:`Py_REFCNT`
is ``1`` for Python objects passed as arguments to C API functions.


Limited C API changes
---------------------
Expand Down Expand Up @@ -2244,6 +2253,17 @@ Porting to Python 3.14
a :exc:`UnicodeError` object.
(Contributed by Bénédikt Tran in :gh:`127691`.)

.. _whatsnew314-refcount:

* The interpreter internally avoids some reference count modifications when
loading objects onto the operands stack by :term:`borrowing <borrowed reference>`
references when possible. This can lead to smaller reference count values
compared to previous Python versions. C API extensions that checked
:c:func:`Py_REFCNT` of ``1`` to determine if an function argument is not
referenced by any other code should instead use
:c:func:`PyUnstable_Object_IsUniqueReferencedTemporary` as a safer replacement.


* Private functions promoted to public C APIs:

* ``_PyBytes_Join()``: :c:func:`PyBytes_Join`.
Expand Down
5 changes: 5 additions & 0 deletions Include/cpython/object.h
Original file line number Diff line number Diff line change
Expand Up @@ -476,6 +476,11 @@ PyAPI_FUNC(PyRefTracer) PyRefTracer_GetTracer(void**);
*/
PyAPI_FUNC(int) PyUnstable_Object_EnableDeferredRefcount(PyObject *);

/* Determine if the object exists as a unique temporary variable on the
* topmost frame of the interpreter.
*/
PyAPI_FUNC(int) PyUnstable_Object_IsUniqueReferencedTemporary(PyObject *);

/* Check whether the object is immortal. This cannot fail. */
PyAPI_FUNC(int) PyUnstable_IsImmortal(PyObject *);

Expand Down
13 changes: 13 additions & 0 deletions Lib/test/test_capi/test_object.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import enum
import sys
import textwrap
import unittest
from test import support
Expand Down Expand Up @@ -223,5 +224,17 @@ def __del__(self):
obj = MyObj()
_testinternalcapi.incref_decref_delayed(obj)

def test_is_unique_temporary(self):
self.assertTrue(_testcapi.pyobject_is_unique_temporary(object()))
obj = object()
self.assertFalse(_testcapi.pyobject_is_unique_temporary(obj))

def func(x):
# This relies on the LOAD_FAST_BORROW optimization (gh-130704)
self.assertEqual(sys.getrefcount(x), 1)
self.assertFalse(_testcapi.pyobject_is_unique_temporary(x))

func(object())

if __name__ == "__main__":
unittest.main()
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
Add :c:func:`PyUnstable_Object_IsUniqueReferencedTemporary` function for
determining if an object exists as a unique temporary variable on the
interpreter's stack. This is a replacement for some cases where checking
that :c:func:`Py_REFCNT` is one is no longer sufficient to determine if it's
safe to modify a Python object in-place with no visible side effects.
8 changes: 8 additions & 0 deletions Modules/_testcapi/object.c
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,13 @@ pyobject_enable_deferred_refcount(PyObject *self, PyObject *obj)
return PyLong_FromLong(result);
}

static PyObject *
pyobject_is_unique_temporary(PyObject *self, PyObject *obj)
{
int result = PyUnstable_Object_IsUniqueReferencedTemporary(obj);
return PyLong_FromLong(result);
}

static int MyObject_dealloc_called = 0;

static void
Expand Down Expand Up @@ -478,6 +485,7 @@ static PyMethodDef test_methods[] = {
{"pyobject_print_os_error", pyobject_print_os_error, METH_VARARGS},
{"pyobject_clear_weakrefs_no_callbacks", pyobject_clear_weakrefs_no_callbacks, METH_O},
{"pyobject_enable_deferred_refcount", pyobject_enable_deferred_refcount, METH_O},
{"pyobject_is_unique_temporary", pyobject_is_unique_temporary, METH_O},
{"test_py_try_inc_ref", test_py_try_inc_ref, METH_NOARGS},
{"test_xincref_doesnt_leak",test_xincref_doesnt_leak, METH_NOARGS},
{"test_incref_doesnt_leak", test_incref_doesnt_leak, METH_NOARGS},
Expand Down
24 changes: 24 additions & 0 deletions Objects/object.c
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
#include "pycore_hamt.h" // _PyHamtItems_Type
#include "pycore_initconfig.h" // _PyStatus_OK()
#include "pycore_instruction_sequence.h" // _PyInstructionSequence_Type
#include "pycore_interpframe.h" // _PyFrame_Stackbase()
#include "pycore_interpolation.h" // _PyInterpolation_Type
#include "pycore_list.h" // _PyList_DebugMallocStats()
#include "pycore_long.h" // _PyLong_GetZero()
Expand Down Expand Up @@ -2621,6 +2622,29 @@ PyUnstable_Object_EnableDeferredRefcount(PyObject *op)
#endif
}

int
PyUnstable_Object_IsUniqueReferencedTemporary(PyObject *op)
{
if (!_PyObject_IsUniquelyReferenced(op)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this just demonstrates that _PyObject_IsUniquelyReferenced is badly named, otherwise we wouldn't need a new function.

We need to check that the following are true:

  • The object has an explicit (not borrowed) reference count of 1
  • It is mortal
  • It does not support deferred reclamation.

I think _PyObject_IsUniquelyReferenced does that, but it isn't clear from the name.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's discuss the name for _PyObject_IsUniquelyReferenced in #133144

return 0;
}

_PyInterpreterFrame *frame = _PyEval_GetFrame();
if (frame == NULL) {
return 0;
}

_PyStackRef *base = _PyFrame_Stackbase(frame);
_PyStackRef *stackpointer = frame->stackpointer;
while (stackpointer > base) {
stackpointer--;
if (op == PyStackRef_AsPyObjectBorrow(*stackpointer)) {
return PyStackRef_IsHeapSafe(*stackpointer);
}
}
return 0;
}

int
PyUnstable_TryIncRef(PyObject *op)
{
Expand Down
Loading