Memory leak when attaching event handlers from Python #1972

sean-marten · 2022-10-13T00:07:00Z

Environment

Pythonnet version: 3.0.0
Python version: 3.7.4
Operating System: Windows 10
.NET Framework 4.8.4515.0

Details

There appears to be a memory leak when attaching a method to an event handler from Python. I was able to break this down into a very simple example. Below is my C# Library

using System;

namespace MemoryLeak
{
    public Class MemoryLeak : IDisposable
    {
        public event EventHandler LeakEvent;
        public MemoryLeak()
        {
            GC.Collect();
            Console.WriteLine(GC.GetTotalMemory(true));
        }
        public void Dispose()
        {
            this.LeakEvent = null;
        }
    }
}

(This was the file in my C# class library, just built and took the DLL to use in the below python script)

and the python script that utilizes this library:

import gc
import sys

import clr

def main():
    sys.path.append("./dlls") # the dll from the above class library is stored here
    clr.AddReference("MemoryLeak")

    import System

    from MemoryLeak import MemoryLeak

    def event_handler():
        pass

    for _ in range(200):
        example = MemoryLeak()
        example.LeakEvent += event_handler
        example.LeakEvent -= event_handler
        example.Dispose()
        del example
        gc.collect()
        System.GC.Collect()

if __name__ == "__main__":
    main()

If you run this python script successfully, you will notice that the total memory steadily increases. If you simply remove the following two lines from the python script regarding the addition of the event handler into the C# object, you will notice no memory increase:

        example.LeakEvent += event_handler
        example.LeakEvent -= event_handler

I would expect that between the cleanup of the attached handler, disposal and manual collections from both python and C#, that the event handler + memory leak class would get garbage collected properly, but they do not appear to be removed entirely based on memory tracking.

To note, I did attempt to make a C# console application that performs the same steps as the python script and the memory did not leak in that example.

I am more than happy to help contribute if this is deemed to be an issue with pythonnet, but would need a little guidance on where to start.

sean-marten · 2022-10-13T00:09:55Z

Output from full python script (in bytes):

Output when event handler lines are removed (in bytes):

If you run even more cycles, the problem will exacerbate as well.

lostmsu · 2022-10-13T00:36:10Z

I don't see you calling .NET GC anywhere.

sean-marten · 2022-10-13T00:37:47Z

I don't see you calling .NET GC anywhere.

Do you mean I am not manually calling collect?

sean-marten · 2022-10-13T00:48:31Z

@lostmsu Sorry I was typing over from a different computer, I was actually manually calling GC.Collect(); (and results are the same). If that is not what you meant, could you please clarify?

EDIT: I saw a different issue where you also mentioned someone was not calling .NET GC, and by those comments I assume that you mean that I am not doing some sort of:

import System

# run some code...

System.GC.Collect()

...in my python script. I have tried this out and edited my original python script and the results are still the same.

lostmsu · 2022-10-13T02:29:33Z

@sean-marten please, doublecheck the output with all the GC calls, then if it reproduces, I can take a look at it.

sean-marten · 2022-10-13T15:05:02Z

Okay @lostmsu , so I re-ran with the .NET GC collection calls both at the end of the python loop and in the dispose method of the C# library. Also for posterity I still am running the python gc collection at the end of the python loop. I also added a dummy array object to the MemoryLeak constructor class which exacerbates the memory leak as well. I will paste in the updated scripts and a summary of the results:

import gc
import sys

import clr

def main():
    sys.path.append("./dlls") # the dll from the above class library is stored here
    clr.AddReference("MemoryLeak")

    import System

    from MemoryLeak import MemoryLeak

    def event_handler():
        pass

    for _ in range(200):
        example = MemoryLeak()
        example.LeakEvent += event_handler
        example.LeakEvent -= event_handler
        example.Dispose()
        del example
        gc.collect()
        System.GC.Collect()

if __name__ == "__main__":
    main()

using System;
using System.Linq;

namespace MemoryLeak
{
    public Class MemoryLeak : IDisposable
    {
        public event EventHandler LeakEvent;
        private Array arr;  // dummy array to exacerbate memory leak

        public MemoryLeak()
        {
            this.arr = Enumerable.Range(0, 10000).ToArray();
        }

        public void Dispose()
        {
            this.LeakEvent = null;
            GC.Collect();
            Console.WriteLine(GC.GetTotalMemory(true));
        }
    }
}

When run normally, we see the min and max of the GC.GetTotalMemory method return ~0.6 MB and ~8.6 MB, respectively. If you increase the number of loops in the python script from 200 to let's say 500, that becomes a min and max of ~0.6 MB and ~20.6 MB, respectively. And increasing the loop number only makes it worse, as expected.

If you remove the python lines regarding registering the event handler from python, the output of GC.GetTotalMemory teeters between ~0.6 MB and ~2.1 MB, but never exceeds the larger value.

Please let me know if you would like me to do any further diagnosis!

…re event handlers fixes pythonnet#1972

sean-marten changed the title ~~Memory link when attaching event handlers from Python~~ Memory leak when attaching event handlers from Python Oct 13, 2022

lostmsu closed this as not planned Won't fix, can't repro, duplicate, stale Oct 13, 2022

filmor reopened this Oct 13, 2022

lostmsu added a commit to losttech/pythonnet that referenced this issue Oct 13, 2022

delete target object from event handler collections when it has no mo…

ecc1245

…re event handlers fixes pythonnet#1972

lostmsu added a commit to losttech/pythonnet that referenced this issue Oct 13, 2022

delete target object from event handler collections when it has no mo…

9ace8dc

…re event handlers fixes pythonnet#1972

lostmsu mentioned this issue Oct 13, 2022

Delete target object from event handler collections when it has no more event handlers #1973

Merged

2 tasks

lostmsu added a commit to losttech/pythonnet that referenced this issue Oct 13, 2022

delete target object from event handler collections when it has no mo…

5a28fd4

…re event handlers fixes pythonnet#1972

filmor closed this as completed in #1973 Oct 14, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Memory leak when attaching event handlers from Python #1972

Memory leak when attaching event handlers from Python #1972

sean-marten commented Oct 13, 2022 •

edited

Loading

sean-marten commented Oct 13, 2022 •

edited

Loading

lostmsu commented Oct 13, 2022

sean-marten commented Oct 13, 2022

sean-marten commented Oct 13, 2022 •

edited

Loading

lostmsu commented Oct 13, 2022

sean-marten commented Oct 13, 2022

Memory leak when attaching event handlers from Python #1972

Memory leak when attaching event handlers from Python #1972

Comments

sean-marten commented Oct 13, 2022 • edited Loading

Environment

Details

sean-marten commented Oct 13, 2022 • edited Loading

lostmsu commented Oct 13, 2022

sean-marten commented Oct 13, 2022

sean-marten commented Oct 13, 2022 • edited Loading

lostmsu commented Oct 13, 2022

sean-marten commented Oct 13, 2022

sean-marten commented Oct 13, 2022 •

edited

Loading

sean-marten commented Oct 13, 2022 •

edited

Loading

sean-marten commented Oct 13, 2022 •

edited

Loading