Skip to content

Cannot convert Numpy Float64 array to double[] #1887

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Tarudro opened this issue Jul 25, 2022 · 2 comments
Closed

Cannot convert Numpy Float64 array to double[] #1887

Tarudro opened this issue Jul 25, 2022 · 2 comments

Comments

@Tarudro
Copy link

Tarudro commented Jul 25, 2022

Environment

  • Pythonnet version: 3.0.0-rc4
  • Python version: 3.7 (also tested with 3.10)
  • Operating System: Windows 10 Enterprise (10.0.19044 Build 19044)
  • .NET Runtime: .NET 6.0

Details

Hi,
We want to update our PythonEnvironment (conda env) to 3.10 (previously were using 3.7). Therefore I'm trying to update the our used pythonnet version to the latest prerelease one. (3.0.0-rc4 right now) However, when doing so, our unit tests for converting 1, 2 and 3 dimensional numpy (float) arrays to a CLR (double[] type) are breaking. I debugged it and could nail it down to be reproducible using the following code (which executes using version 3.0.0-preview2021-08-03):

// call PythonEngine.Initialize() and set PythonHome and Paths correctly
using (Py.GIL())
{
    PyModule scope = Py.CreateScope();
    scope.Exec(
        "import numpy as np \n" +
        "arr = np.array([1., 2.])"
    );
    PyObject arr = scope.Get("arr");
    Type dataType = typeof(double[]);
    var array = arr.AsManagedObject(dataType);

    // CollectionAssert.AreEqual(new double[] { 1.0, 2.0 }, (double[])result);
}

This throws an InvalidCastException in AsManagedObject:

System.InvalidCastException: cannot convert object to target type
---> Python.Runtime.PythonException: 'numpy.float64' value cannot be converted to System.Double

Let me know if you need something else or if I made a mistake. We were initially using pythonnet version 2.5.2 but I tried to refactor and change all the code as required for 3.0.0.
Help is highly appreciated. I couldn't find something/someone else that is having problems converting numpy arrays. Which seems a bit weird to me.

Thanks for your efforts developing this great and helpful library, keep it up!

Cheers,
Lukas

@filmor
Copy link
Member

filmor commented Jul 25, 2022

We don't include non-trivial conversions by default anymore. You should use a codec like this for the conversion: https://github.com/fdieulle/pandasnet/blob/main/dotnet/PandasNet/NumpyCodec.cs

@filmor filmor closed this as completed Jul 25, 2022
@Tarudro
Copy link
Author

Tarudro commented Jul 28, 2022

Hi @filmor,
Thanks for your guidance. I've added the pandasnet code from @fdieulle to my project and I can convert numpy arrays and pandas dataframes once again.

However, for the conversion of a numpy.float64 I keep ending up in a conversion error within pythonnet.

I did this and then called my conversion function:

            PyModule scope = Py.CreateScope();
            scope.Exec(
                "import numpy as np \n" +
                "float64_var = np.float64(1.2)"
            );
            PyObject float64_var = scope.Get("float64_var");

Internally in the NumpyCodecs.cs it's then mapped to the pythonnet function PyObject.As<double>:

codecs.Register("<class 'numpy.float64'>", p => p.As<double>());

but when this is executed, I'm getting the mentioned conversion exception within pythonnet.
Refer to my screenshot below for the excact locations and stack trace.
image

I'm not sure how this is supposed to work. I'm also not sure if I should report this issue in here or in the pandasnet github repo tbh.

Any help is highly appreciated. It should be also easy to reproduce.
Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants