Skip to content

When I try to use the Chinese namespace in C# #662

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
charSLee013 opened this issue Apr 18, 2018 · 9 comments
Closed

When I try to use the Chinese namespace in C# #662

charSLee013 opened this issue Apr 18, 2018 · 9 comments

Comments

@charSLee013
Copy link

Environment

  • Pythonnet version:2.3.0
  • Python version:3.6.3
  • Operating System:Win 10

Details

Hi,

  • When I try to use the Chinese namespace in C#,Pythonnet will prompt memory corruption.
using System;
using Python.Runtime;

namespace 中文测试
{
    public class 中文
    {
        static void Main(string[] args)
        {
            using (Py.GIL())
            {
                dynamic sys = Py.Import("sys");
                Console.WriteLine(sys.verison);
                Console.ReadKey();
            }
        }
    }
}
  • The visual studio then prompts the following error.

error

  • How should I solve this problem in the Chinese namespace?

THANK YOU SO MUCH!

@den-run-ai
Copy link
Contributor

@charSLee013 this is likely not going to be supported, but let's check with @yagweb if this makes any sense.

@charSLee013
Copy link
Author

This bug is very big, which is also a hindrance to the globalization of Pythonnet. We need to solve this problem urgently and hope to get your help :)

@den-run-ai
Copy link
Contributor

Contributions are welcome!

@charSLee013
Copy link
Author

I tried to troubleshoot the problem, when running to moduleobject.cs 1194 lines , pyname returns null pointer

moduleName = "中文";
IntPtr pyname = Runtime.PyString_FromString(moduleName);
out:>>>pyname  == 0x0000000000000

IntPtr pyfilename = Runtime.PyString_FromString(filename);    //error

I tried to find the problem, and found it inruntime.cs line 1194 , and changed it to the following code.

        internal static IntPtr PyString_FromString(string value)
        {
            byte[] len = Encoding.UTF8.GetBytes(value);
            IntPtr a = PyString_FromStringAndSize(value, value.Length);
            foreach (char t in value)
            {
                if (t >= 0x4e00 && t <= 0x9fbb)
                {
                    a = PyString_FromStringAndSize(value, len.Length);
                    return a;
                }

            }
            return a;
        }

The exciting thing is that it does.

But in pythonengine.cs in line 191 appeared error, unable to load clr._extras module

try
{
IntPtr module = Runtime.PyImport_AddModule("clr._extras");
out:>>>module  == 0x00000000000000

IntPtr module_globals = Runtime.PyModule_GetDict(module);    //error
IntPtr builtins = Runtime.PyEval_GetBuiltins();
Runtime.PyDict_SetItemString(module_globals, "__builtins__", builtins);
  • Looking forward to your reply.

@yagweb
Copy link
Contributor

yagweb commented Apr 19, 2018

@charSLee013 It seems the exist of method PyString_FromString is to be compatible with python2. So for users it should not be used with unicode strings.
From only the point of view of Python3, this method should work with unicode string, so your discovery about the PyString_FromString is a bug of pythonnet.

A recommended fix is call PyUnicode_FromString instead of the faked PyString_FromStringAndSize method of Python3.

public static IntPtr PyString_FromString(string value)
{
#if PYTHON3
            return PyUnicode_FromString(value);
#elif PYTHON2
            return PyString_FromStringAndSize(value, value.Length);
#endif
}

Even with this modification, Chinese namespace is not working now. The problem here is that pythonnet needs to support Python2 and Python3 at the same time. Python3 supports modules with chinese name but Python2 does not. So nobody tests this before.

Since both .net and Python3 support Chinese namespace, pythonnet with Python3 should work with Chinese namespace in theory. It seems one problem is located in the import hooks (maybe not the only one), heavily debugging work is needed.

@charSLee013
Copy link
Author

Looking forward to your good news, this test result is very important @yagweb

@daixinwei2018
Copy link

楼主你好,因为看你给出的示例,我知道你是中国人。我是刚刚接触pythonnet,对这个并不是特别熟悉,我本来想运行一下你的代码(我把中文换为了英文),但是还是报错。
5_eilv 2vn ey1y60 l _ 8
如果可以的话,我想多问问您一些c#调用python相关的问题,谢谢!

@den-run-ai
Copy link
Contributor

Pinvokestackimbalance is fixed in master branch @daixinwei2018

@filmor
Copy link
Member

filmor commented Aug 29, 2018

The fix was this has been merged to master, so I'll close this. Feel free to reopen if the version in the master branch doesn't solve the issue.

@filmor filmor closed this as completed Aug 29, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants