Skip to content

bpo-35053: Enhance tracemalloc to trace free lists #10063

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Oct 25, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions Include/object.h
Original file line number Diff line number Diff line change
Expand Up @@ -776,6 +776,9 @@ PyAPI_FUNC(void) _Py_AddToAllObjects(PyObject *, int force);
* inline.
*/
#define _Py_NewReference(op) ( \
(_Py_tracemalloc_config.tracing \
? _PyTraceMalloc_NewReference(op) \
: 0), \
_Py_INC_TPALLOCS(op) _Py_COUNT_ALLOCS_COMMA \
_Py_INC_REFTOTAL _Py_REF_DEBUG_COMMA \
Py_REFCNT(op) = 1)
Expand Down
38 changes: 38 additions & 0 deletions Include/pymem.h
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,10 @@ PyAPI_FUNC(int) PyTraceMalloc_Track(
uintptr_t ptr,
size_t size);

/* Update the Python traceback of an object.
This function can be used when a memory block is reused from a free list. */
PyAPI_FUNC(int) _PyTraceMalloc_NewReference(PyObject *op);

/* Untrack an allocated memory block in the tracemalloc module.
Do nothing if the block was not tracked.

Expand Down Expand Up @@ -239,6 +243,40 @@ PyAPI_FUNC(int) _PyMem_SetDefaultAllocator(
PyMemAllocatorEx *old_alloc);
#endif


/* bpo-35053: expose _Py_tracemalloc_config for performance:
_Py_NewReference() needs an efficient check to test if tracemalloc is
tracing. */
struct _PyTraceMalloc_Config {
/* Module initialized?
Variable protected by the GIL */
enum {
TRACEMALLOC_NOT_INITIALIZED,
TRACEMALLOC_INITIALIZED,
TRACEMALLOC_FINALIZED
} initialized;

/* Is tracemalloc tracing memory allocations?
Variable protected by the GIL */
int tracing;

/* limit of the number of frames in a traceback, 1 by default.
Variable protected by the GIL. */
int max_nframe;

/* use domain in trace key?
Variable protected by the GIL. */
int use_domain;
};

PyAPI_DATA(struct _PyTraceMalloc_Config) _Py_tracemalloc_config;

#define _PyTraceMalloc_Config_INIT \
{.initialized = TRACEMALLOC_NOT_INITIALIZED, \
.tracing = 0, \
.max_nframe = 1, \
.use_domain = 0}

#ifdef __cplusplus
}
#endif
Expand Down
20 changes: 20 additions & 0 deletions Lib/test/test_tracemalloc.py
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,26 @@ def test_get_object_traceback(self):
traceback = tracemalloc.get_object_traceback(obj)
self.assertEqual(traceback, obj_traceback)

def test_new_reference(self):
tracemalloc.clear_traces()
# gc.collect() indirectly calls PyList_ClearFreeList()
support.gc_collect()

# Create a list and "destroy it": put it in the PyListObject free list
obj = []
obj = None

# Create a list which should reuse the previously created empty list
obj = []

nframe = tracemalloc.get_traceback_limit()
frames = get_frames(nframe, -3)
obj_traceback = tracemalloc.Traceback(frames)

traceback = tracemalloc.get_object_traceback(obj)
self.assertIsNotNone(traceback)
self.assertEqual(traceback, obj_traceback)

def test_set_traceback_limit(self):
obj_size = 10

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
tracemalloc now tries to update the traceback when an object is reused from a
"free list" (optimization for faster object creation, used by the builtin list
type for example).
Loading