Skip to content

Operator overloads support #1324

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 25 commits into from
Jan 5, 2021
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
5130aaa
Overload operators except for bit shifts and default args.
christabella Dec 16, 2020
0d0ca87
Remove params array tests since they are unrelated to operator overlo…
christabella Dec 17, 2020
253f9c7
Fix type bug; rename variables pynargs etc.
christabella Dec 17, 2020
8cdb61c
Remove unused variables and add comments.
christabella Dec 17, 2020
c26e589
Add comments and remove unused in operatormethod.cs
christabella Dec 18, 2020
3222a54
Address review by @lostmsu; Add forward operator test.
christabella Dec 20, 2020
550ff31
Add forward operator tests (OpObj, int).
christabella Dec 21, 2020
eab8edc
Address @amos402's comment on raising branch.
christabella Dec 21, 2020
c2be3f1
Add operator overload and rename pynargs, clrnargs
christabella Dec 21, 2020
581a047
Revert variable renames
christabella Dec 22, 2020
e11327f
Update AUTHORS and CHANGELOG
christabella Dec 22, 2020
35be4bc
Revert rename to pynargs
christabella Dec 23, 2020
f19c281
Address @tminka's comments.
christabella Dec 23, 2020
d7f52d2
Remove whitespace
christabella Dec 23, 2020
5855a1b
Fix nits
christabella Dec 24, 2020
e7da0bc
Support reverse binary operations
christabella Dec 28, 2020
a376838
Use reverse instead of forward (semantics)
christabella Dec 29, 2020
6923a78
Address @tminka's comments
christabella Dec 30, 2020
4c992d8
Support unary neg and pos operators
christabella Dec 30, 2020
8cce61d
Remove isOperator from MatchesArgumentCount (simplify), add test.
christabella Jan 4, 2021
09a2047
Revert "Remove isOperator from MatchesArgumentCount (simplify)"
christabella Jan 4, 2021
41bd07f
Properly remove isOperator from MatchesArgumentCount (@tminka comment)
christabella Jan 4, 2021
10ccf1e
Update changelog.
christabella Jan 4, 2021
5682e0c
Add ones complement and modulo tests (@tminka comment)
christabella Jan 4, 2021
5f45c70
Merge branch 'master' into feat/operator-overloads
lostmsu Jan 5, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Support reverse binary operations
  • Loading branch information
christabella committed Dec 29, 2020
commit e7da0bcc451856db03e901d29c64ffb2246cbbe9
36 changes: 36 additions & 0 deletions src/embed_tests/TestOperator.cs
Original file line number Diff line number Diff line change
Expand Up @@ -202,6 +202,42 @@ public void ForwardOperatorOverloads()
assert c.Num == a.Num ^ b
");
}


[Test]
public void ReverseOperatorOverloads()
{
string name = string.Format("{0}.{1}",
typeof(OperableObject).DeclaringType.Name,
typeof(OperableObject).Name);
string module = MethodBase.GetCurrentMethod().DeclaringType.Namespace;

PythonEngine.Exec($@"
from {module} import *
cls = {name}
a = 2
b = cls(10)

c = a + b
assert c.Num == a + b.Num

c = a - b
assert c.Num == a - b.Num

c = a * b
assert c.Num == a * b.Num

c = a & b
assert c.Num == a & b.Num

c = a | b
assert c.Num == a | b.Num

c = a ^ b
assert c.Num == a ^ b.Num
");

}
[Test]
public void ShiftOperatorOverloads()
{
Expand Down
9 changes: 8 additions & 1 deletion src/runtime/classmanager.cs
Original file line number Diff line number Diff line change
Expand Up @@ -472,7 +472,14 @@ private static ClassInfo GetClassInfo(Type type)
ci.members[name] = ob;
if (mlist.Any(OperatorMethod.IsOperatorMethod))
{
ci.members[OperatorMethod.GetPyMethodName(name)] = ob;
string pyName = OperatorMethod.GetPyMethodName(name);
string pyNameReverse = pyName.Insert(2, "r");
MethodInfo[] forwardMethods, reverseMethods;
OperatorMethod.FilterMethods(mlist, out forwardMethods, out reverseMethods);
// Only methods where the left operand is the declaring type.
ci.members[pyName] = new MethodObject(type, name, forwardMethods);
// Only methods where only the right operand is the declaring type.
ci.members[pyNameReverse] = new MethodObject(type, name, reverseMethods);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if there are no reverseMethods?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently it will just create a method that's never used; I can fix that by adding a hasReverse field in the SlotDefinition struct.

Copy link
Contributor Author

@christabella christabella Dec 30, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So if C# reverse operator isn't defined, so the Python reverse name will have an empty method list in MethodObject

So if a user calls b.__radd__(a) by writing a + b where there is no forward +() defined on a so Python resorts to __radd__, AND no reverse +() defined on b, what happens? Wouldn't hurt to add a check for an empty reverseMethods before doing ci.members[pyNameReverse] = new MethodObject(type, name, reverseMethods);

Another unrelated issue is that of doing unnecessary work, if the Python slot doesn't support reverse e.g. unary operators (__rtruediv__ exists even though python docs don't list them)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is also possible to have no "forward" methods.

If no reverse methods are defined, the corresponding slot should not be set. Same for forward.

Python slot doesn't support reverse e.g. __truediv__

Not sure what you mean. There's __rtruediv__

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's true, I've just put a check then to not set the slot method if the method list is empty

}
}

Expand Down
32 changes: 21 additions & 11 deletions src/runtime/methodbinder.cs
Original file line number Diff line number Diff line change
Expand Up @@ -350,9 +350,20 @@ internal Binding Bind(IntPtr inst, IntPtr args, IntPtr kw, MethodBase info, Meth
var outs = 0;
int clrnargs = pi.Length;
isOperator = isOperator && pynargs == clrnargs - 1; // Handle mismatched arg numbers due to Python operator being bound.
// Preprocessing pi to remove either the first or second argument.
bool isForward = isOperator && OperatorMethod.IsForward((MethodInfo)mi); // Only cast if isOperator.
if (isOperator && isForward) {
// The first Python arg is the right operand, while the bound instance is the left.
// We need to skip the first (left operand) CLR argument.
pi = pi.Skip(1).Take(1).ToArray();
}
else if (isOperator && !isForward) {
// The first Python arg is the left operand.
// We need to take the first CLR argument.
pi = pi.Take(1).ToArray();
}
var margs = TryConvertArguments(pi, paramsArray, args, pynargs, kwargDict, defaultArgList,
needsResolution: _methods.Length > 1, // If there's more than one possible match.
isOperator: isOperator,
outs: out outs);
if (margs == null)
{
Expand All @@ -364,7 +375,15 @@ internal Binding Bind(IntPtr inst, IntPtr args, IntPtr kw, MethodBase info, Meth
{
if (ManagedType.GetManagedObject(inst) is CLRObject co)
{
margs[0] = co.inst;
// Postprocessing to extend margs.
var margsTemp = new object[2];
// If forward, the bound instance is the left operand.
int boundOperandIndex = isForward ? 0 : 1;
// If forward, the passed instance is the right operand.
int passedOperandIndex = isForward ? 1 : 0;
margsTemp[boundOperandIndex] = co.inst;
margsTemp[passedOperandIndex] = margs[0];
margs = margsTemp;
}
else { break; }
}
Expand Down Expand Up @@ -488,15 +507,13 @@ static IntPtr HandleParamsArray(IntPtr args, int arrayStart, int pyArgCount, out
/// <param name="kwargDict">Dictionary of keyword argument name to python object pointer</param>
/// <param name="defaultArgList">A list of default values for omitted parameters</param>
/// <param name="needsResolution"><c>true</c>, if overloading resolution is required</param>
/// <param name="isOperator"><c>true</c>, if is operator method</param>
/// <param name="outs">Returns number of output parameters</param>
/// <returns>An array of .NET arguments, that can be passed to a method.</returns>
static object[] TryConvertArguments(ParameterInfo[] pi, bool paramsArray,
IntPtr args, int pyArgCount,
Dictionary<string, IntPtr> kwargDict,
ArrayList defaultArgList,
bool needsResolution,
bool isOperator,
out int outs)
{
outs = 0;
Expand Down Expand Up @@ -535,13 +552,6 @@ static object[] TryConvertArguments(ParameterInfo[] pi, bool paramsArray,
op = Runtime.PyTuple_GetItem(args, paramIndex);
}
}
if (isOperator && paramIndex == 0)
{
// After we've obtained the first argument from Python, we need to skip the first argument of the CLR
// because operator method is a bound method in Python
paramIndex++; // Leave the first .NET param as null (margs).
parameter = pi[paramIndex];
}

bool isOut;
if (!TryConvertArgument(op, parameter.ParameterType, needsResolution, out margs[paramIndex], out isOut))
Expand Down
36 changes: 36 additions & 0 deletions src/runtime/operatormethod.cs
Original file line number Diff line number Diff line change
Expand Up @@ -130,5 +130,41 @@ private static PyObject GetOperatorType()
return locals.GetItem("OperatorMethod");
}
}

public static string ReversePyMethodName(string pyName)
{
return pyName.Insert(2, "r");
}

/// <summary>
/// Check if the method is performing a forward or reverse operation.
/// </summary>
/// <param name="method">The operator method.</param>
/// <returns></returns>
public static bool IsForward(MethodInfo method)
{
Type declaringType = method.DeclaringType;
Type leftOperandType = method.GetParameters()[0].ParameterType;
return leftOperandType == declaringType;
}

public static void FilterMethods(MethodInfo[] methods, out MethodInfo[] forwardMethods, out MethodInfo[] reverseMethods)
{
List<MethodInfo> forwardMethodsList = new List<MethodInfo>();
List<MethodInfo> reverseMethodsList = new List<MethodInfo>();
foreach (var method in methods)
{
if (IsForward(method))
{
forwardMethodsList.Add(method);
} else
{
reverseMethodsList.Add(method);
}

}
forwardMethods = forwardMethodsList.ToArray();
reverseMethods = reverseMethodsList.ToArray();
}
}
}