Skip to content

Some fixes. #219

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Jul 28, 2016
Merged

Some fixes. #219

merged 6 commits into from
Jul 28, 2016

Conversation

matthid
Copy link
Contributor

@matthid matthid commented May 26, 2016

Fixes a bunch issues I encountered when trying to implement https://github.com/matthid/googleMusicAllAccess.

Note: this includes #194 (because I needed the fix), let me know when you merge #194 so I can rebase and remove the last commit.

@vmuriart
Copy link
Contributor

Do you have tests for the fixes ?

kits_suffix = "bin"
if PLATFORM == "x64":
kits_suffix += r"\x64"
kits_suffix = os.path.join("bin",PLATFORM)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be a separate pull request, since this is unrelated commit from someone else:

https://github.com/pythonnet/pythonnet/pull/194/files

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note that #194 is linked with other pull requests. @tonyroberts is going to merge these changes once appveyor CI gets fixed.

@den-run-ai
Copy link
Contributor

@matthid I left 2 comments in your code, in addition can you please rebase/squash all 4 commits into one logical commit. I agree with @vmuriart that the unit tests would be great to include!

@vmuriart
Copy link
Contributor

From looking at the commit titles, it looks like 3 commits its about right. Seems to be addressing 3 different issues. in-lieu of doing 3 separate pr, i think 1 with three commits is ok. In case I am wrong, and all three issues are actually the same, then yes should be squashed.

@den-run-ai
Copy link
Contributor

@vmuriart thanks for correcting me :) @matthid you pull request looks good if #194 gets merged!

only tests are missing - not sure if you can get equivalent C# code for your failing F# code?

@den-run-ai
Copy link
Contributor

also need to downgrade your syntax to C# 4.0

@matthid
Copy link
Contributor Author

matthid commented May 27, 2016

@denfromufa Yes I agree with all points. I just wanted to quickly show up the problems. It could take me a while to clean this up as I'm trying to work on something else (I just figured it would be a good idea to let people know there are some fixes available). If someone wants to step in until then feel free to do so. If you encounter problems while reproducing feel free to ask. I'm pretty sure everything can be reproduced with C# as well.

Is there a strong reason to stay on C# 4?

@matthid matthid changed the title Some fixes. WIP Some fixes. May 27, 2016
@matthid
Copy link
Contributor Author

matthid commented May 27, 2016

By the way after all these fixes I still see

  clrmodule -> C:\PROJ\googleMusicAllAccess\temp\pythonnet\src\clrmodule\bin\x86\ReleaseWin\clrmodule.dll
C:\PROJ\googleMusicAllAccess\temp\pythonnet\packages\UnmanagedExports.1.2.6\tools\RGiesecke.DllExport.targets(42,5): error : Microsoft.Build.Utilities.ToolLocationHelper could not find ildasm.exe. [C:\PROJ\googleMusicAllAccess\temp\pythonnet\src\clrmodule\clrmodule.csproj]

in the build output. So there is still something wrong. As I used a VS developer prompt everything should be available. But I didn't investigate as the build continues and the package is available in dist/

@den-run-ai
Copy link
Contributor

@matthid we are tracking this issue with Unmanaged Exports here:

#216

@den-run-ai
Copy link
Contributor

@matthid c# 6 syntax should be downgraded to c# 4 due to .NET 4.0 and Mono compatibility.
This can be re-considered if someone is willing to look into this closer.

@matthid
Copy link
Contributor Author

matthid commented May 31, 2016

Let me first say: I have no problem with downgrading (once I have some time... the project looks a bit dead anyway as there are quite some outstanding PRs). But IMHO those are the wrong reasons. You should ensure mono and net40 compat though build and test.

There is nothing stopping you from compiling c#6 to net40. Only a small subset is not working because of missing .net classes but even those you can get working by defining the missing classes yourself (if really needed). Just google for c#6 and net4 to get sources for this.

The travis build is green, which I would say means it works. Btw later mono versions only run with net45 anyway.

@den-run-ai
Copy link
Contributor

@matthid I read about partial compatibility of C# 6 with .NET 4.0, but did not know that Mono defaults to C# 6. However looks like their main documentation is outdated, which is where I looked first: http://www.mono-project.com/docs/about-mono/languages/csharp/

C# 6 is not ready in Mono compiler according to that page.

WRT pythonnet slow development cycle - please note that it is still more alive development than IronPython and the codebase is older than both IronPython and F#!

@matthid
Copy link
Contributor Author

matthid commented May 31, 2016

@denfromufa Yes Thanks, and the code worked great for me (after the bugfixes at least). I tried IronPython first and couldn't make it do anything :)

Yeah the page is really outdated. Mono moved fast after MS open sourced a lot of things... I think they use Roslyn now and can evolve as fast as Visual Studio...

@matthid
Copy link
Contributor Author

matthid commented Jun 4, 2016

I cherry picked the original commit from #194 this should make it possible for you to merge both PRs, and it makes it easy to use my branch for other things. Just to be sure you should merge #194 first (but git should be able to handle that just fine).

Additionally I added test for 2823941. Now I'm looking into test for 35cd5fa. I think cc8b796 is quite hard to test...

@matthid
Copy link
Contributor Author

matthid commented Jun 4, 2016

I'm not using python that much, but as far as I understand it I had to add a module to be able to use relative imports. I hope its fine the way I did it, please review.

Edit: http://stackoverflow.com/questions/16981921/relative-imports-in-python-3

@matthid
Copy link
Contributor Author

matthid commented Jun 4, 2016

Maybe a stupid question: But where are the Python.Embedding tests run? It seems like they are not run as part of the test suite?

@den-run-ai
Copy link
Contributor

den-run-ai commented Jun 4, 2016

@matthid that is a good point! embedding tests are not run on travis or appveyor. Once I did manage to run them locally and passed, but there is something brittle there with importing file paths:

https://github.com/pythonnet/pythonnet/blob/master/src/embed_tests/pyimport.cs#L27

@den-run-ai
Copy link
Contributor

I opened a separate issue for embedding tests: #224

@matthid
Copy link
Contributor Author

matthid commented Jun 4, 2016

Ok I managed to reproduce the problem with the "regular" test suite, but I think it might make more sense to add it to the embedding tests (the code might be a bit simpler there).

I think this PR is now ready to review and merge. What about C#6. Should I still change to C#4?

@matthid matthid changed the title WIP Some fixes. Some fixes. Jun 4, 2016
@matthid
Copy link
Contributor Author

matthid commented Jun 4, 2016

To test that the test cases do indeed reproduce the problems you can use my add_tests branch. Then cherry-pick the fixes to make the tests work one by one.

@den-run-ai
Copy link
Contributor

den-run-ai commented Jun 4, 2016

@matthid travis CI is failing to build your current pull request:

https://travis-ci.org/pythonnet/pythonnet/jobs/135245011

@matthid
Copy link
Contributor Author

matthid commented Jun 4, 2016

Yep investigating, locally its working :(

@den-run-ai
Copy link
Contributor

I think this PR is now ready to review and merge. What about C#6. Should I still change to C#4?

No problem, as long as both travis (looks good) and appveyor (have not seen yet) are able to build this.

@matthid
Copy link
Contributor Author

matthid commented Jun 4, 2016

might be a mono bug, an appveyor run might be cool here to clarify

@matthid
Copy link
Contributor Author

matthid commented Jun 4, 2016

Yep its a mono bug:

    public class TestConvert : System.Dynamic.DynamicObject
    {
        public override bool TryConvert(System.Dynamic.ConvertBinder binder, out object result)
        {
            result = null;
            return true;
        }
    }

    public class Test : System.Dynamic.DynamicObject
    {
        public override bool TryInvokeMember(System.Dynamic.InvokeMemberBinder binder, object[] args, out object result)
        {
            result = new TestConvert();
            return true;
        }

        public static void TestMethod()
        {
            dynamic t = new Test();
            string result = t.SomeMethod();
        }
    }
Test.TestMethod(); // Works on .NET, crashes on mono

@matthid
Copy link
Contributor Author

matthid commented Jun 4, 2016

We could workaround here at pythonnet by checking for Pythons None and returning null immediatly in the TryInvokeMember call instead of wrapping in a PyObject instance. Apparently we cannot convert a PyObject to null on mono. Will report this to mono as well but I think we still need a workaround.

@matthid
Copy link
Contributor Author

matthid commented Jun 4, 2016

matthid added a commit to matthid/pythonnet that referenced this pull request Jun 4, 2016
@matthid
Copy link
Contributor Author

matthid commented Jun 4, 2016

Done.

@vmuriart
Copy link
Contributor

vmuriart commented Jun 4, 2016

You can configure a appveyor account to run off your github account. That
what I did for mine to atleast verify my pr

On Saturday, June 4, 2016, Matthias Dittrich notifications@github.com
wrote:

Done.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#219 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AMr87ASXXjyppOxhZTa8BDcjpDXKAYrKks5qIZaxgaJpZM4In_8l
.

@matthid
Copy link
Contributor Author

matthid commented Jun 4, 2016

@vmuriart Yeah, thanks! But I'm testing on windows already locally. Just mentioned it that it would be nice for others to see (because everything was red above)...

@tonyroberts
Copy link
Contributor

I've cherry picked "Alternative Fix for #182" into master. Thanks!

tonyroberts pushed a commit to tonyroberts/pythonnet that referenced this pull request Jun 23, 2016
{
if (pyObj != null)
{
if (pyObj.obj == Runtime.PyNone)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't this alter the behaviour? Previously a PyObject wrapping None would be returned, but now that's replaced with null? I must be missing something...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah you are right, it is a workaround that you are not able to return null via dynamic conversion (=later) on mono. please see the linked bug report as well. Feel free to ask if you want a code example (could take some time though)

Copy link
Contributor

@tonyroberts tonyroberts Jun 23, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I get that, but here you are returning null where previously an object (PyNone) was returned. Could function just not return pyObj without checking for None? That's the bit I don't understand.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you could and even should imho, but then you are triggering the mono bug :). The problem is later. when you save such a none object in a dynamic and want to resolve it to a string (for example). This will not work as long as the mono bug exists. In fact one of my tests discovered this glitch

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah ok, that was the bit I was missing - that the bug was triggered later when dealing with the None PyObject instance.

@matthid
Copy link
Contributor Author

matthid commented Jun 23, 2016

I've cherry picked "Alternative Fix for #182" into master. Thanks!

Ok, but you could have just told me to split the PR or tell me what the problem was with the whole PR (didn't see that there are conflicts until now, because I didn't saw that on the phone).
Fixed the conflicts by rebasing the branch on latest master

@den-run-ai
Copy link
Contributor

den-run-ai commented Jun 28, 2016

apparently in python 2.7.12 some importing issues have been fixed in C-API:

- ``PyImport_Import`` and ``PyImport_ImportModule`` now always do
  absolute imports. In earlier versions they might have used relative
  imports under some conditions.

https://hg.python.org/cpython/raw-file/v2.7.12/Misc/NEWS

@matthid
Copy link
Contributor Author

matthid commented Jul 1, 2016

@denfromufa I have no idea what you are trying to tell me or what I should do now with this PR :)

@filmor
Copy link
Member

filmor commented Jul 14, 2016

Okay, decision time: Do we allow C#6? Is there any compelling reason not to, as the tests are working?

In my opinion, this PR can be merged.

@den-run-ai
Copy link
Contributor

I thought @tonyroberts cherry-picked this pull request?

C# 6.0 is supported by both CI, so I'm +1 on adopting this!

On Thursday, July 14, 2016, Benedikt Reinartz notifications@github.com
wrote:

Okay, decision time: Do we allow C#6? Is there any compelling reason not
to, as the tests are working?

In my opinion, this PR can be merged.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#219 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AHgZ5YjN_C25nL_rze39HYqLE6NfZrb_ks5qVkfOgaJpZM4In_8l
.

@matthid
Copy link
Contributor Author

matthid commented Jul 15, 2016

@denfromufa Only parts, not sure whats wrong with the rest. I can only improve this when I know whats missing...

@filmor
Copy link
Member

filmor commented Jul 22, 2016

@denfromufa @vmuriart Merge?

@den-run-ai
Copy link
Contributor

@filmor can you give me one day to test this locally? I would like to make
sure that it also works with python 2.7.12 due to the bug fix that I linked
above.

On Fri, Jul 22, 2016 at 3:33 AM, Benedikt Reinartz <notifications@github.com

wrote:

@denfromufa https://github.com/denfromufa @vmuriart
https://github.com/vmuriart Merge?


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#219 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHgZ5QSW5SDEeR1o_E9ce6CsOuzXCBj9ks5qYIB2gaJpZM4In_8l
.

@den-run-ai
Copy link
Contributor

ok, tested locally and it is working! great finding and fix!

C:\Users\denis.akhiyarov>python
Python 2.7.11 |Anaconda 4.0.0 (32-bit)| (default, Mar  4 2016, 15:18:41) [MSC v.
1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import clr
>>> import testimport
>>> from testimport import relative_import

Unhandled Exception: System.ArgumentException: Name must not be empty!
   at Python.Runtime.ModuleObject..ctor(String name)
   at Python.Runtime.ModuleObject.GetAttribute(String name, Boolean guess)
   at Python.Runtime.ImportHook.__import__(IntPtr self, IntPtr args, IntPtr kw)
   at Python.Runtime.Runtime.PyObject_Call(IntPtr pointer, IntPtr args, IntPtr k
w)
   at Python.Runtime.ImportHook.__import__(IntPtr self, IntPtr args, IntPtr kw)

C:\Users\denis.akhiyarov>pip uninstall pythonnet
Uninstalling pythonnet-2.1.0:
  c:\python\python27_32b\lib\site-packages\clr.pyd
  c:\python\python27_32b\lib\site-packages\python.runtime.dll
  c:\python\python27_32b\lib\site-packages\python.runtime.dll.config
  c:\python\python27_32b\lib\site-packages\pythonnet-2.1.0.dist-info\description
.rst
  c:\python\python27_32b\lib\site-packages\pythonnet-2.1.0.dist-info\installer
  c:\python\python27_32b\lib\site-packages\pythonnet-2.1.0.dist-info\metadata
  c:\python\python27_32b\lib\site-packages\pythonnet-2.1.0.dist-info\metadata.js
on
  c:\python\python27_32b\lib\site-packages\pythonnet-2.1.0.dist-info\record
  c:\python\python27_32b\lib\site-packages\pythonnet-2.1.0.dist-info\top_level.t
xt
  c:\python\python27_32b\lib\site-packages\pythonnet-2.1.0.dist-info\wheel
Proceed (y/n)? y
  Successfully uninstalled pythonnet-2.1.0
You are using pip version 8.1.1, however version 8.1.2 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' comm
and.

C:\Users\denis.akhiyarov>pip install git+https://github.com/matthid/pythonnet.gi
t@myfixes
Collecting git+https://github.com/matthid/pythonnet.git@myfixes
  Cloning https://github.com/matthid/pythonnet.git (to myfixes) to c:\users\deni
s~1.akh\appdata\local\temp\pip-inppy5-build
Installing collected packages: pythonnet
  Running setup.py install for pythonnet ... done
Successfully installed pythonnet-2.1.0
You are using pip version 8.1.1, however version 8.1.2 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' comm
and.

C:\Users\denis.akhiyarov>python
Python 2.7.11 |Anaconda 4.0.0 (32-bit)| (default, Mar  4 2016, 15:18:41) [MSC v.
1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import clr
>>> import testimport
>>> from testimport import relative_import
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "testimport\relative_import.py", line 4, in <module>
    from . import _missing_import
  File "testimport\_missing_import.py", line 1, in <module>
    import this_package_should_never_exist_ever
ImportError: No module named this_package_should_never_exist_ever
>>> exit()

C:\Users\denis.akhiyarov>conda update python
Using Anaconda Cloud api site https://api.anaconda.org
Fetching package metadata .........
Solving package specifications: ..........

Package plan for installation in environment C:\Python\Python27_32b:

The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    python-2.7.12              |                0        22.4 MB
    anaconda-custom            |           py27_0          13 KB
    conda-4.1.9                |           py27_0         242 KB
    ------------------------------------------------------------
                                           Total:        22.6 MB

The following packages will be UPDATED:

    anaconda: 4.0.0-np110py27_0 --> custom-py27_0
    conda:    4.1.2-py27_0      --> 4.1.9-py27_0
    python:   2.7.11-4          --> 2.7.12-0

Proceed ([y]/n)? y

Fetching packages ...
python-2.7.12- 100% |###############################| Time: 0:00:14   1.57 MB/s
anaconda-custo 100% |###############################| Time: 0:00:00 293.28 kB/s
conda-4.1.9-py 100% |###############################| Time: 0:00:00 691.15 kB/s
Extracting packages ...
[      COMPLETE      ]|##################################################| 100%
Unlinking packages ...
[      COMPLETE      ]|##################################################| 100%
Linking packages ...
[      COMPLETE      ]|##################################################| 100%

C:\Users\denis.akhiyarov>python
Python 2.7.11 |Anaconda custom (32-bit)| (default, Mar  4 2016, 15:18:41) [MSC v
.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import clr
>>> import testimport
>>> from testimport import relative_import
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "testimport\relative_import.py", line 4, in <module>
    from . import _missing_import
  File "testimport\_missing_import.py", line 1, in <module>
    import this_package_should_never_exist_ever
ImportError: No module named this_package_should_never_exist_ever
>>>

@filmor filmor merged commit 8029ffe into pythonnet:master Jul 28, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants