Skip to content

Constructor argument matching supports simple int to (float|double) types #239

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 16 commits into from
Closed

Constructor argument matching supports simple int to (float|double) types #239

wants to merge 16 commits into from

Conversation

sigbjorn
Copy link

@sigbjorn sigbjorn commented Jul 3, 2016

Refer to Issue #238

This PR aim to improve constructor argument matching so that if you have a .NET constructor accepting a double, it will be selected if you pass it an int.

This corresponds to solution part (a.) in the referenced issue.

The PR involves tests, that with the prior version would fail, but with the fixes, works Ok.
All existing tests, run on python 3.5, x64, windows platform executes Ok.

I have not verified other platforms(mono, 32 bits etc), but could do that on request.

The PR does currently not support raising Type exception in case a falling back to a default constructor due to arguments not matching. However, if anyone contribute with hint's how to get information about the .NET creation object context (super-class, or just plain class), - I would be happy to extend the contents and tests to cover that important aspect as well.

sigbjorn and others added 8 commits June 30, 2016 16:26
…n calling a constructor, prior to the change in the methodbinder.cs, this test failed, since the no-match in ct. was failing silently, now it works. Next step is to give exception when ct args do not match
…g most recent python, so that build and test works without fiddling with settings in vs 2015
… problems, tests runs, but with warning printed out
@sigbjorn
Copy link
Author

sigbjorn commented Jul 4, 2016

Build failing, Ok, I have to fix the build, reverse changes to .sln and .proj files I guess.

@filmor
Copy link
Member

filmor commented Jul 4, 2016

Yes, please.

Also, your test is failing for Python 2.7 as the super() syntax (without arguments) is Python 3. Use super(PySubclass, self).__init__ ... (https://github.com/pythonnet/pythonnet/pull/239/files#diff-cce6857ae7057bf6caf15818e1ad16e0R60).

private bool SimplePromotableType(Type src, Type dst)
{
return (((src == typeof(int) || src == typeof(short)) && (dst == typeof(double) || dst == typeof(float))))
|| ((src == typeof(long)) && (dst == typeof(double)))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This logic is a bit arbitrary, isn't it? A 64bit integer fits in neither double nor float, so why make this distinction here?

Copy link
Author

@sigbjorn sigbjorn Jul 4, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, so it merely answers if its possible to do the conversion.
The real conversion is attempted later, using standard python.net conversion
that hopefully do the best effort, throws, etc. if there are numerical problems representing the one from the other.
We could even drop the test, and just do the python.net conversion, - that would kind of be consistent with what happen, I think, to usual function arguments. If that conversion fail, then it's not promotable.

The reason to be distinct on a numerical type is that we want to be as precise as possible.

And yes, - given that there is both and int and a double constructor, we should select the one with closest match. (Requires rewrite, or redirect logic to some function that already do this ?)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And yes, - given that there is both and int and a double constructor, we should select the one with closest match. (Requires rewrite, or redirect logic to some function that already do this ?)

@sigbjorn as far as I'm aware there is no logic for resolution order in pythonnet for multiple matches due to complexity it brings into code-base:

#217

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My point was that _dis_allowing the conversion long -> float doesn't make sense as the conversion long -> double is also lossy (64 vs 53). So either we allow all of those or none.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, then we could try the approach I suggested in my initial comment, - just use the standard conversion function inside pythonnet. That's wider approach, but there is a hope it's consistently used for argument conversions, and thus we can terminate this discussion, since then the arguments to a constructor undergoes (almost) the same rules as an ordinary pythonnet function.
I am on vacation now, but can try to see if its possible to get in touch with a computer long enough to produce the change.

@filmor
Copy link
Member

filmor commented Jul 4, 2016

@denfromufa Regarding the approach, I'm fine with doing conversions int-type -> float-type. This feels much more natural coming from Python where you'd do the constructor exactly like this (at least in Py3 where int and float behave very similar).

@sigbjorn
Copy link
Author

sigbjorn commented Jul 4, 2016

Struggling a bit with build on arch-linux, mono 4.4.0.40, (before I apply my patch, using master..)
it fail on Target RGieseckeDllExport:
: error : Error initializing task DllExportAppDomainIsolatedTask: Could not load file or assembly 'Microsoft.Build.Utilities, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.

Is this a know error/solution, or should I just try to build on a Ubuntu platform?

@den-run-ai
Copy link
Contributor

Struggling a bit with build on arch-linux, mono 4.4.0.40, (before I apply my patch, using master..)
it fail on Target RGieseckeDllExport:
: error : Error initializing task DllExportAppDomainIsolatedTask: Could not load file or assembly 'Microsoft.Build.Utilities, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.

Is this a know error/solution, or should I just try to build on a Ubuntu platform?

For Mono RGieseckeDllExport cannot be used, since this is Windows-specific feature of reverse pinvoke. Instead the configuration for Mono should be selected, which uses C-API of Mono and Python to bind each other.

@sigbjorn
Copy link
Author

sigbjorn commented Jul 4, 2016

Thanks for the hint, but when using setup.py (as in .travis.yml build),

( Configuration: Release Mono Platform: x64) is printed out during the build
or installing monodevelop, (Ubuntu 15.10 this time, with clang, python-dev etc.)

selecting mono debug/release x64, then build the solution, the clrmodule is the only one failing,
on ubuntu it fail with 'no accesss to the given key' indicating there is some more security related issues .

It also seems that the travis now fail on the RGiesecke related stuff, the build tools 2.0.0.0 not found etc.

Appveyor seems to be happy after 2.7 fixups

@sigbjorn
Copy link
Author

sigbjorn commented Jul 4, 2016

Ok, now all .sln, csproj files are back to original, hopefully that solve the build issues.

I finally did succeed building the mono 4.4.1 version, Ubuntu, python 3.4, python3-dev, libglib2.0-dev added, plus all the mono stuff from standard packages on ubuntu, using virtualenv to setup python part.

But running the tests fail, when Load clr hook executes, "Error: No managed allocator , but we need one for AOT. Are you using non-standard GC options ?"

Sorry to bother you all with this, but maybe someone knows a fix (otherwise, I will just clean, rebuild, select packages specific to the travis.setup etc.., and retry.)

@den-run-ai
Copy link
Contributor

den-run-ai commented Jul 4, 2016

I finally did succeed building the mono 4.4.1 version, Ubuntu, python 3.4, python3-dev, libglib2.0-dev added, plus all the mono stuff from standard packages on ubuntu, using virtualenv to setup python part.

But running the tests fail, when Load clr hook executes, "Error: No managed allocator , but we need one for AOT. Are you using non-standard GC options ?"

This exception in Mono seems to be common lately starting ~4.4.1:

tjanczuk/edge#449

@turbo
Copy link

turbo commented Jul 4, 2016

CC 42169 in Bugzilla for the managed alloc bug. More info there and here for WSL.

@sigbjorn
Copy link
Author

sigbjorn commented Jul 4, 2016

Ok, now the py 2.6 is the only one that need som attention. I will fix that soon, so at least the build is straighten up, then we can see if we can improve this PR to a level where it can be accepted.

Downgrading mono etc. to do local builds work is Ok for now.

@sigbjorn
Copy link
Author

sigbjorn commented Jul 4, 2016

Ok, now it even builds and run on arch-linux, py 3.4, mono 4.4.0.40, that is before the ~4.4.1, so now I can verify the mono version more easily as well. It was my vs2015, changing the .sln and the build that caused the mono build problems, sorry for the mess. Now that I got both win and mono up running it's easier to avoid.

@sigbjorn
Copy link
Author

sigbjorn commented Jul 7, 2016

Is there anything more I can contribute with so that this PR can be merged ?

@den-run-ai
Copy link
Contributor

@sigbjorn the problem I have is with cases when multiple types meet the auto-conversion. Without handling such behavior this PR seems incomplete to me, but resolving this is a lot more effort and will open us to a lot of potential corner cases. Why not just throw exception if exact match is not found? The user should be smart enough to provide the exact type after the error is reported.

@sigbjorn
Copy link
Author

sigbjorn commented Jul 7, 2016

Yes, as mentioned previously, this PR was not originally created to solve all problems related to this issue.

and yes:

A simple fix would be to throw exception if not perfect match when calling a constructor.
But unfortunately, that simple fix will also break the existing sub-class a .NET class feature, that rely on this flexible behaviour. ( My guess is that you would then say NO to this PR, or maybe I am wrong ?)

This problem could be resolved:
If we from the context of calling the constructor, could know that we are in a chain of super().. call-chain, we could make a fix that is aware of the context,
and then: in the context of super().. , allow anything (like it is now)
and in the other contexts (direct call to constructor), enforce exact match.

So if you/or anybody could point me in any direction that allow us to figure out if we are in a super()-call-chain, this is easy to fix. I have briefly checked the code, and found no such info in the call-chain from python to the method-binder in question dealing with constructors.

other alternatives:
Making the constructor-matching more selective, choosing the closest matching constructor can be done, - but requires a lot more changes. -And unfortunately, it does not provide exceptions for non-matching cases.

So my intention was to first provide something that I think resolves a lot of 'practical cases'.

Then, if we could get some more knowledge about the object creation (especially super/sub-class and argument passing). With that knowledge in place, fix selective matching and proper super/sub-classing in addition to throwing exception when there are no matches.

@den-run-ai
Copy link
Contributor

ok, the simple solution right now could be to raise a warning instead of
exception when going the route of ignoring the arguments?

I like the idea to find if we are in the super() chain! Have not figure out
yet.

On Thu, Jul 7, 2016 at 4:11 PM, sigbjorn notifications@github.com wrote:

Yes, as mentioned previously, this PR was not originally created to solve
all problems related to this issue.

and yes:

A simple fix would be to throw exception if not perfect match when calling
a constructor.
But unfortunately, that simple fix will also break the existing
sub-class a .NET class feature, that rely on this flexible behaviour. ( My
guess is that you would then say NO to this PR, or maybe I am wrong ?)

This problem could be resolved:
If we from the context of calling the constructor, could know that we are
in a chain of super().. call-chain, we could make a fix that is aware of
the context,
and then: in the context of super().. , allow anything (like it is now)
and in the other contexts (direct call to constructor), enforce exact
match.

So if you/or anybody could point me in any direction that allow us to
figure out if we are in a super()-call-chain, this is easy to fix. I have
briefly checked the code, and found no such info in the call-chain from
python to the method-binder in question dealing with constructors.

other alternatives:
Making the constructor-matching more selective, choosing the closest
matching constructor can be done, - but requires a lot more changes. -And
unfortunately, it does not provide exceptions for non-matching cases.

So my intention was to first provide something that I think resolves a lot
of 'practical cases'.

Then, if we could get some more knowledge about the object creation
(especially super/sub-class and argument passing). With that knowledge in
place, fix selective matching and proper super/sub-classing in addition to
throwing exception when there are no matches.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#239 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AHgZ5UdBicMXeEZATL0GQ79chMkVFHxLks5qTWudgaJpZM4JD9kR
.

@turbo
Copy link

turbo commented Jul 8, 2016

Downgrading mono etc. to do local builds work is Ok for now.

Update: It should be safe to use Mono embedding again when 4.6.0 is released (or build from git if the tag updates at mono/mono).

@sigbjorn
Copy link
Author

sigbjorn commented Jul 9, 2016

Ok, - the moment we properly detect and figure out the super() call-chain (for all versions of python), its quite straight forward.
It requires a dive into the python type system and mechanism in play during construction of object.
This will require some more effort, - I could have a look into it next week.

…destination type is float or double, leaving overflow checking entirely to the conversion routine
@filmor
Copy link
Member

filmor commented Jul 28, 2016

@sigbjorn If you rebase and squash the PR, I'm okay with merging this. @vmuriart @denfromufa, what's your opinion?

@den-run-ai
Copy link
Contributor

I'm ok with auto-conversion only when a single match is found. This likely
means the number of arguments has to be one and also requires additional
logic to iterate over all possible arguments to set this "single match is
found" flag.

On Thursday, July 28, 2016, Benedikt Reinartz notifications@github.com
wrote:

@sigbjorn https://github.com/sigbjorn If you rebase and squash the PR,
I'm okay with merging this. @vmuriart https://github.com/vmuriart
@denfromufa https://github.com/denfromufa, what's your opinion?


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#239 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHgZ5Zx6UF_A8jA5W7fy529dmKZmF8w1ks5qaFwZgaJpZM4JD9kR
.

@vmuriart
Copy link
Contributor

I'll run some tests tomorrow against my applications and check for no surprises :)

@sigbjorn
Copy link
Author

I could have a look if it's possible to transform the existing ct looping/matching (seems ad-hoc to me), into something that find the closest match.
If I where to invest more hours into this, I think figuring out how to separate sub-class/class initialization and raise exception if no match found would pay off most.

@vmuriart
Copy link
Contributor

good from my side. The only thing I would change is the formatting of the tests, but i can take care of that later.

@sigbjorn
Copy link
Author

sigbjorn commented Aug 5, 2016

Sorry for the delay, I will try to find some time this weekend to dive into class/subclass, and/or also align call to constructor with the 'usual/standard' argument-matching.

@den-run-ai
Copy link
Contributor

@sigbjorn to override .NET constructor we may need to use __new__(), not __init__() method.

@tonyroberts do you recall why you hooked up constructor to __init__(), not __new__() in test_class.py?

The problem with initializer is that it already receives an instance of the class, while new is actually the constructor of the class.

This may be a breaking change, but would enable subclassing immutable types also:

http://stackoverflow.com/a/674369/2230844

@den-run-ai
Copy link
Contributor

@sigbjorn do you agree to MIT license for your contribution once we transition to it?

@filmor
Copy link
Member

filmor commented Mar 23, 2017

@sigbjorn Feel free to reopen when you find the time to address the comments (in particular agreement to the MIT license).

@filmor filmor closed this Mar 23, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants