Skip to content

Should Python.Runtime.Py.With use Runtime.None instead of new PyObject(Runtime.PyNone)? #1192

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
IvanAntipov opened this issue Jul 30, 2020 · 2 comments

Comments

@IvanAntipov
Copy link

I'm not sure if I'm right, but I apologize in advance if I'm misleading the community

After long exectution (I train large ML model), I fall into Fatal Python error: deallocating None

Fatal Python error: deallocating None

Current thread 0x00007fc518efd700 (most recent call first):

Thread 0x00007fc4997fe700 (most recent call first):
  File "/usr/local/lib/python3.7/threading.py", line 300 in wait
  File "/usr/local/lib/python3.7/queue.py", line 179 in get
  File "/usr/local/lib/python3.7/site-packages/tensorboard/summary/writer/event_file_writer.py", line 232 in run
  File "/usr/local/lib/python3.7/threading.py", line 926 in _bootstrap_inner
  File "/usr/local/lib/python3.7/threading.py", line 890 in _bootstrap

Sadly, I can't provide full repro, because problem does not occur in small data set, and I work on private project.

In my code I use Should Python.Runtime.Py.With method.

In this method I see PyNone to be used without increment of references count

IntPtr type = Runtime.PyNone;
...
new PyObject(type)

As I undestand from stackoverflow each usage of None should increment Ref counter.

There is Python.Runtime.Runtime.None property

Should Runtime.None be used instead of Runtime.PyNone

Environment

  • Pythonnet version: pythonnet_netstandard (I know, it is a fork of this project, but the code under dicussion is under this project) "Python.Runtime.Mono" Version="3.7.1"
  • Python version: 3.7.7
  • Operating System: Ubuntu 18.04.3 (in docker)
@IvanAntipov
Copy link
Author

IvanAntipov commented Jul 30, 2020

The proble with "delallocating None" has gone, after I have switched to my own implmetation of "Py.With"

F# code:

    let inline pyWith(pyObj: PyObject, func: PyObject -> 'T) =
        let mutable ptr: PyObject option = None // this is F# None, in C# implementation you can use ``null`` instead 
        let mutable ptr2: PyObject option = None // this is F# None
        let mutable ptr3: PyObject option = None // this is F# None
        let mutable ex : PythonException = null;
        let mutable res : 'T option = None // this is F# None

        try 
            let obj2: PyObject = pyObj.InvokeMethod("__enter__");
            
            res <- Some <| func(obj2)
        with
        | :? PythonException as ex2 ->            
            ex <- ex2;
            ptr <- Some <| PyObject ex.PyType
            ptr2 <- Some <| PyObject ex.PyValue;
            ptr3 <- Some <| PyObject ex.PyTB;

        let toPyNoneIfEmpty(o: PyObject option) = o |> Option.defaultWith(fun () -> Runtime.GetPyNone()) // Runtime.GetPyNone() pythonnet_standard alternative of Py.None

        let exitObject : PyObject = pyObj.InvokeMethod("__exit__", ptr |> toPyNoneIfEmpty, ptr |> toPyNoneIfEmpty, ptr2 |> toPyNoneIfEmpty, ptr3 |> toPyNoneIfEmpty);
        
        if (ex |> isNull |> not && not <| exitObject.IsTrue()) then
            raise ex

        res |> Option.get

@lostmsu
Copy link
Member

lostmsu commented Jul 30, 2020

@IvanAntipov this has already been fixed before 2.5.0 was released in #1062. The very link you provided to this repo has XIncref(type) just a few lines above new PyObject(type).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants