Skip to content

Remove circular references in parser_context and renderer_context. #5826

Closed
@zubchick

Description

@zubchick

I found a memory leak in APIView code caused by circular references.

Minimal working example :

(tested with python2.7 and rest-framework from master)

app.py

#!/usr/bin/env python
import sys

from django.conf.urls import url

from rest_framework.views import APIView
from rest_framework.response import Response


class AnonymousUser(object):
    pass


class MemoryLeakView(APIView):
    def post(self, request, **kwargs):
        self._20_megabyte = "0123456789abcdef" * 64 * 1024 * 20
        return Response({'a': 'done'})


urlpatterns = [
    url(r'^memory-leak/$', MemoryLeakView.as_view()),
]


if __name__ == "__main__":
    from django.core.management import execute_from_command_line
    execute_from_command_line([sys.argv[0], 'runserver'])

settings.py

DEBUG = True
MIDDLEWARE_CLASSES = []
INSTALLED_APPS = ['rest_framework', 'app']
ROOT_URLCONF = 'app'

REST_FRAMEWORK = {
    'UNAUTHENTICATED_USER': 'app.AnonymousUser',
}
SECRET_KEY = '123'

Expected behavior

The same amount of memory before and after POST request.

Actual behavior

Every request curl -i -XPOST 'http://127.0.0.1:8000/memory-leak/' adds ~20Mb to memory usage of the process.

The cause

The couse of the problem is circular references from APIView to Request and back (and from APIView to Response and back). All these objects (and many others) remain in memory as a garbage, not collected by gc for certain reasons.

Ideas

The first idea I had was to wrap context references with weakref.proxy here, here, and here.
This fixes the issue (I've tested), but because of questionable design decisions, view references are sometimes used after the view destruction (for example response.render() internally calls get_default_renderer that expects view). So it could not be a sufficient solution.

A proper way to fix this is to change get_*context functions and return only objects are needed to request and response, without the whole view.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions