Skip to content

Registering RawProxyEncoder Causes Stack Overflow #1427

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
richard-gemmell-42 opened this issue Mar 30, 2021 · 3 comments
Closed

Registering RawProxyEncoder Causes Stack Overflow #1427

richard-gemmell-42 opened this issue Mar 30, 2021 · 3 comments

Comments

@richard-gemmell-42
Copy link

richard-gemmell-42 commented Mar 30, 2021

Environment

  • Pythonnet version: installed from master branch on 30th March 2021
  • Python version: 3.9.2
  • Operating System: Windows 10
  • .NET Runtime: Microsoft.NETCore.App 5.0.4, Microsoft.WindowsDesktop.App 3.1.13

Summary

Registering a RawProxyEncoder with PyObjectConversions.RegisterEncoder() causes the script to fail with a stack overflow.

Thanks

First of all, I'd like to say thanks to everyone who's contributed to this project. It's incredibly useful for us to write one set of code to serve both .NET and Python developers. We've had very few problems with Pythonnet. Thanks very much.

Details

I ran into a problem with Pythonnet converting System.Collections.Generic.List into a native python list as described in issue 1153.

lostmsu recommended using a RawProxyEncoder to disable the automatic conversion. Unfortunately, if I register an encoder and then call a .NET method the system crashes with a stack overflow.

import sys
from pathlib import Path
from clr_loader import get_coreclr
from pythonnet import set_runtime
api_path = Path('..\\PythonStubs\\bin\\Debug\\netcoreapp3.1')
sys.path.append(str(api_path))
rt = get_coreclr(runtime_config=str(api_path / f'PythonStubs.runtimeconfig.json'))
set_runtime(rt)
import clr
import System
from Python.Runtime import PyObjectConversions
from Python.Runtime.Codecs import RawProxyEncoder


class ListAsRawEncoder(RawProxyEncoder):
    __namespace__ = 'Dummy'

    def CanEncode(self, clr_type):
        return clr_type.Name == 'IList`1' and clr_type.Namespace == 'System.Collections.Generic'


if __name__ == '__main__':
    list_encoder = ListAsRawEncoder()
    PyObjectConversions.RegisterEncoder(list_encoder)

    domain = System.AppDomain.CurrentDomain
    print("Found", domain.GetAssemblies().Length, "assemblies in domain")

The output is:

    Stack overflow

Commenting out the call to RegisterEncoder prevents the crash.

@lostmsu
Copy link
Member

lostmsu commented Mar 30, 2021

I investigated this issue. Stack overflow happens when trying access a C# object, that might need codec (but does not have one), and there's at least one encoder implemented and registered from Python (like ListAsProxyEncoder).

What happens is that in order to call CanEncode Python.NET must pass the clr_type to the encoder. That starts search for encoder, that can handle System.Type, which hits ListAsProxyEncoder.CanEncode again with System.Type, that needs to be passed to Python, and so on.

lostmsu added a commit to losttech/pythonnet that referenced this issue Mar 30, 2021
(stack overflow trying to pass `System.Type` to `CanEncode`)
lostmsu added a commit to losttech/pythonnet that referenced this issue Mar 30, 2021
(stack overflow trying to pass `System.Type` to `CanEncode`)
lostmsu added a commit to losttech/pythonnet that referenced this issue Mar 30, 2021
(stack overflow trying to pass `System.Type` to `CanEncode`)
lostmsu added a commit to losttech/pythonnet that referenced this issue Mar 30, 2021
(stack overflow trying to pass `System.Type` to `CanEncode`)
lostmsu added a commit to losttech/pythonnet that referenced this issue Mar 30, 2021
Without this restriction encoders created in Python cause stack overflow due to repeated attempts to pass `System.Type` instance to `CanDecode`, which requires encoding the instance of `System.Type`, et. cetera

fixes pythonnet#1427
lostmsu added a commit to losttech/pythonnet that referenced this issue Mar 31, 2021
(stack overflow trying to pass `System.Type` to `CanEncode`)
lostmsu added a commit to losttech/pythonnet that referenced this issue Mar 31, 2021
Without this restriction encoders created in Python cause stack overflow due to repeated attempts to pass `System.Type` instance to `CanDecode`, which requires encoding the instance of `System.Type`, et. cetera

fixes pythonnet#1427
lostmsu added a commit that referenced this issue Mar 31, 2021
(stack overflow trying to pass `System.Type` to `CanEncode`)
@richard-gemmell-42
Copy link
Author

Thanks for fixing this. I can't believe you did it so quickly.
cheers,
Richard

@lostmsu
Copy link
Member

lostmsu commented Mar 31, 2021

Haha, that's for the lack of time to work on new features :-D @richard-gemmell-42

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants