Description
The bug is localized to invocations ofonly()
and defer()
on models that extend CachingMixin
. It results in deep recursions that occasionally bust the stack, but otherwise create huge flush_lists
class MyModel(CachingMixin):
id = models.CharField(max_length=50, blank=True)
nickname = models.CharField(max_length=50, blank=True)
favorite_color = models.CharField(max_length=50, blank=True)
content_owner = models.ForeignKey(OtherModel)
m = MyModel.objects.only('id').all()
The bug occurs in the following lines(https://github.com/jbalogh/django-cache-machine/blob/f827f05b195ad3fc1b0111131669471d843d631f/caching/base.py#L253-L254). In this case self is a instance of MyModel with a mix of deferred and undeferred attributes:
fks = dict((f, getattr(self, f.attname)) for f in self._meta.fields
if isinstance(f, models.ForeignKey))
The use of only()
in the Django ORM does some meta programming magic that overrides the unfetched attributes with Django's DeferredAttribute
implementation. Under normal circumstances an access to favorite_color
would invoke DeferredAttribute.__get__
(https://github.com/django/django/blob/ad96254af98f961d6691481c27127613afeee840/django/db/models/query_utils.py#L88-L120) and fetch the attribute either from the result cache or the data source.
This is the problem when looping over the foreign keys in the Model and accessing their values, Cache Machine induces a recursion. getattr(self, f.attname)
on an attribute that is deferred induces a fetch of a Model that has the CachingMixin
applied and has deferred attributes. Seen in the line below.
val = getattr(
non_deferred_model._base_manager.only(name).using(
instance._state.db).get(pk=instance.pk),
self.field_name
)
Some testing in our environment shows that even in contrived scenarios flush lists can grow 2, 5, 10x the size of the same query executed without only()
or defer()