Skip to content

[3.9] Descriptor HowTo: Improve the fidelity of the member object simulation (GH-23475) #23479

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Nov 23, 2020
Merged

[3.9] Descriptor HowTo: Improve the fidelity of the member object simulation (GH-23475) #23479

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 46 additions & 11 deletions Doc/howto/descriptor.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1079,6 +1079,8 @@ simulation where the actual C structure for slots is emulated by a private
``_slotvalues`` list. Reads and writes to that private structure are managed
by member descriptors::

null = object()

class Member:

def __init__(self, name, clsname, offset):
Expand All @@ -1091,20 +1093,28 @@ by member descriptors::
def __get__(self, obj, objtype=None):
'Emulate member_get() in Objects/descrobject.c'
# Also see PyMember_GetOne() in Python/structmember.c
return obj._slotvalues[self.offset]
value = obj._slotvalues[self.offset]
if value is null:
raise AttributeError(self.name)
return value

def __set__(self, obj, value):
'Emulate member_set() in Objects/descrobject.c'
obj._slotvalues[self.offset] = value

def __delete__(self, obj):
'Emulate member_delete() in Objects/descrobject.c'
value = obj._slotvalues[self.offset]
if value is null:
raise AttributeError(self.name)
obj._slotvalues[self.offset] = null

def __repr__(self):
'Emulate member_repr() in Objects/descrobject.c'
return f'<Member {self.name!r} of {self.clsname!r}>'

The :meth:`type.__new__` method takes care of adding member objects to class
variables. The :meth:`object.__new__` method takes care of creating instances
that have slots instead of an instance dictionary. Here is a rough equivalent
in pure Python::
variables::

class Type(type):
'Simulate how the type metaclass adds member objects for slots'
Expand All @@ -1117,20 +1127,44 @@ in pure Python::
mapping[name] = Member(name, clsname, offset)
return type.__new__(mcls, clsname, bases, mapping)

The :meth:`object.__new__` method takes care of creating instances that have
slots instead of an instance dictionary. Here is a rough simulation in pure
Python::

class Object:
'Simulate how object.__new__() allocates memory for __slots__'

def __new__(cls, *args):
'Emulate object_new() in Objects/typeobject.c'
inst = super().__new__(cls)
if hasattr(cls, 'slot_names'):
inst._slotvalues = [None] * len(cls.slot_names)
empty_slots = [null] * len(cls.slot_names)
object.__setattr__(inst, '_slotvalues', empty_slots)
return inst

def __setattr__(self, name, value):
'Emulate _PyObject_GenericSetAttrWithDict() Objects/object.c'
cls = type(self)
if hasattr(cls, 'slot_names') and name not in cls.slot_names:
raise AttributeError(
f'{type(self).__name__!r} object has no attribute {name!r}'
)
super().__setattr__(name, value)

def __delattr__(self, name):
'Emulate _PyObject_GenericSetAttrWithDict() Objects/object.c'
cls = type(self)
if hasattr(cls, 'slot_names') and name not in cls.slot_names:
raise AttributeError(
f'{type(self).__name__!r} object has no attribute {name!r}'
)
super().__delattr__(name)

To use the simulation in a real class, just inherit from :class:`Object` and
set the :term:`metaclass` to :class:`Type`::

class H(Object, metaclass=Type):
'Instance variables stored in slots'

slot_names = ['x', 'y']

Expand All @@ -1143,11 +1177,11 @@ At this point, the metaclass has loaded member objects for *x* and *y*::
>>> import pprint
>>> pprint.pp(dict(vars(H)))
{'__module__': '__main__',
'__doc__': 'Instance variables stored in slots',
'slot_names': ['x', 'y'],
'__init__': <function H.__init__ at 0x7fb5d302f9d0>,
'x': <Member 'x' of 'H'>,
'y': <Member 'y' of 'H'>,
'__doc__': None}
'y': <Member 'y' of 'H'>}

When instances are created, they have a ``slot_values`` list where the
attributes are stored::
Expand All @@ -1159,8 +1193,9 @@ attributes are stored::
>>> vars(h)
{'_slotvalues': [55, 20]}

Unlike the real ``__slots__``, this simulation does have an instance
dictionary just to hold the ``_slotvalues`` array. So, unlike the real code,
this simulation doesn't block assignments to misspelled attributes::
Misspelled or unassigned attributes will raise an exception::

>>> h.xz = 30 # For actual __slots__ this would raise an AttributeError
>>> h.xz
Traceback (most recent call last):
...
AttributeError: 'H' object has no attribute 'xz'