You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The bug is localized to invocations ofonly() and defer() on models that extend CachingMixin. It results in deep recursions that occasionally bust the stack, but otherwise create huge flush_lists
This is the problem when looping over the foreign keys in the Model and accessing their values, Cache Machine induces a recursion. getattr(self, f.attname) on an attribute that is deferred induces a fetch of a Model that has the CachingMixin applied and has deferred attributes. Seen in the line below.
val = getattr(
non_deferred_model._base_manager.only(name).using(
instance._state.db).get(pk=instance.pk),
self.field_name
)
Some testing in our environment shows that even in contrived scenarios flush lists can grow 2, 5, 10x the size of the same query executed without only() or defer()
The text was updated successfully, but these errors were encountered:
nsfyn55
changed the title
Located a recursion in the the cache_objects logic in Django 1.7
Located a recursion in the the cache_objects logic in Django 1.7 that causes large flush lists
Jan 30, 2015
@nsfyn55 Thanks for the report! It sounds like you've done a fair bit of debugging around this already. Would you be up for submitting a PR with a test for the issue & potentially a fix if you find one?
Was just looking at the title. It's probably an issue in 1.8+ so if the title can be fixed to "Recursion in cache_objects logic creates large flush lists" that would be good.
The bug is localized to invocations of
only()
anddefer()
on models that extendCachingMixin
. It results in deep recursions that occasionally bust the stack, but otherwise create hugeflush_lists
The bug occurs in the following lines(https://github.com/jbalogh/django-cache-machine/blob/f827f05b195ad3fc1b0111131669471d843d631f/caching/base.py#L253-L254). In this case self is a instance of MyModel with a mix of deferred and undeferred attributes:
The use of
only()
in the Django ORM does some meta programming magic that overrides the unfetched attributes with Django'sDeferredAttribute
implementation. Under normal circumstances an access tofavorite_color
would invokeDeferredAttribute.__get__
(https://github.com/django/django/blob/ad96254af98f961d6691481c27127613afeee840/django/db/models/query_utils.py#L88-L120) and fetch the attribute either from the result cache or the data source.This is the problem when looping over the foreign keys in the Model and accessing their values, Cache Machine induces a recursion.
getattr(self, f.attname)
on an attribute that is deferred induces a fetch of a Model that has theCachingMixin
applied and has deferred attributes. Seen in the line below.Some testing in our environment shows that even in contrived scenarios flush lists can grow 2, 5, 10x the size of the same query executed without
only()
ordefer()
The text was updated successfully, but these errors were encountered: