You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The Python docs on dunder methods with reflected operands says:
These functions are only called if the left operand does not support the corresponding operation and the operands are of different types.
“Does not support” here means that the class has no such method, or the method returns NotImplemented. Do not set the method to None if you want to force fallback to the right operand’s reflected method—that will instead have the opposite effect of explicitly blocking such fallback.
For operands of the same type, it is assumed that if the non-reflected method – such as add() – fails then the overall operation is not supported, which is why the reflected method is not called.
This behaviour seems to be violated when performing an operation like np.ndarray * np.matrix. This code evaluates like np.matrix.__rmul__(np.ndarray).
Confusingly, np.ndarray.__mul__(np.matrix) exists and can be called explicitly, leading to a different result.
AFAIK dispatching to np.matrix rmul is expected, as both left- and right-handed multiplicaitons on np.matrix are evaluated as matrix multiplication. However, according to the standard, calling through mul should also follow this behaviour, or follow the "Does not support" path described above.
I believe this bug happens because a*b bypasses __getattribute("__mul__")__ due to special method lookup rules - x.__mul__ goes through class __getattribute__, type(x).__mul__ goes through metaclass __getattribute__, while x*foo goes directly to x's __mul__ implementation. Unfortunately I do not have enough experience with the numpy codebase to check if this is really the case here.
Side note: np.ndarray *= np.matrix leads to behaviour similar to np.ndarray.__mul__(np.matrix), just with the result being an array - this is confusing, but described in the docs and does not violate the standard. Debugging this is how I found the main issue I'm reporting.
Yes, I know np.matrix use is discouraged.
This might be related to #5844
chedatomasz
changed the title
BUG: <Please write a comprehensive title after the 'BUG: ' prefix>
BUG: A.__mul__(B) returns valid result different than A*B for np.ndarray and np.matrix
Nov 21, 2023
chedatomasz
changed the title
BUG: A.__mul__(B) returns valid result different than A*B for np.ndarray and np.matrix
BUG: A*B promotes to A@B but explicit A.__mul__(B) does not for np.ndarray and np.matrix
Nov 21, 2023
Python behaviour is for a subclass to have precedence. Since matrix is a subclass of ndarray, ndarray * matrix will directly go to matrix.__rmul__(ndarray).
Indeed, closing, the way numpy works is a bit layered, so can be surprising. But this is normal operator precedence behavior.
If anyone has a good argument why we cannot rely on Python preferring subclasses, we can reopen.
Describe the issue:
The Python docs on dunder methods with reflected operands says:
These functions are only called if the left operand does not support the corresponding operation and the operands are of different types.
“Does not support” here means that the class has no such method, or the method returns NotImplemented. Do not set the method to None if you want to force fallback to the right operand’s reflected method—that will instead have the opposite effect of explicitly blocking such fallback.
For operands of the same type, it is assumed that if the non-reflected method – such as add() – fails then the overall operation is not supported, which is why the reflected method is not called.
This behaviour seems to be violated when performing an operation like
np.ndarray * np.matrix
. This code evaluates likenp.matrix.__rmul__(np.ndarray)
.Confusingly,
np.ndarray.__mul__(np.matrix)
exists and can be called explicitly, leading to a different result.AFAIK dispatching to np.matrix rmul is expected, as both left- and right-handed multiplicaitons on np.matrix are evaluated as matrix multiplication. However, according to the standard, calling through mul should also follow this behaviour, or follow the "Does not support" path described above.
I believe this bug happens because
a*b
bypasses__getattribute("__mul__")__
due to special method lookup rules -x.__mul__
goes through class__getattribute__
,type(x).__mul__
goes through metaclass__getattribute__
, whilex*foo
goes directly to x's__mul__
implementation. Unfortunately I do not have enough experience with the numpy codebase to check if this is really the case here.Side note:
np.ndarray *= np.matrix
leads to behaviour similar tonp.ndarray.__mul__(np.matrix)
, just with the result being an array - this is confusing, but described in the docs and does not violate the standard. Debugging this is how I found the main issue I'm reporting.Yes, I know np.matrix use is discouraged.
This might be related to #5844
Reproduce the code example:
Error message:
No response
Runtime information:
1.23.5
3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0]
Context for the issue:
No response
The text was updated successfully, but these errors were encountered: