Skip to content

disable ufunc override for 1.9 release #4920

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 30, 2014
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 0 additions & 9 deletions doc/release/1.9.0-notes.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@ This release supports Python 2.6 - 2.7 and 3.2 - 3.4.

Highlights
==========
* Addition of `__numpy_ufunc__` to allow overriding ufuncs in ndarray
subclasses.
* Numerous performance improvements in various areas, most notably indexing and
operations on small arrays are significantly faster.
Indexing operations now also release the GIL.
Expand Down Expand Up @@ -269,13 +267,6 @@ ufunc reductions do since 1.7. One can now say axis=(index, index) to pick a
list of axes for the reduction. The ``keepdims`` keyword argument was also
added to allow convenient broadcasting to arrays of the original shape.

Ufunc and Dot Overrides
~~~~~~~~~~~~~~~~~~~~~~~
For better compatibility with external objects you can now override
universal functions (ufuncs), ``numpy.core._dotblas.dot``, and
``numpy.core.multiarray.dot`` (the numpy.dot functions). By defining a
``__numpy_ufunc__`` method.

Dtype parameter added to ``np.linspace`` and ``np.logspace``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The returned data type from the ``linspace`` and ``logspace`` functions can
Expand Down
70 changes: 0 additions & 70 deletions doc/source/reference/arrays.classes.rst
Original file line number Diff line number Diff line change
Expand Up @@ -39,76 +39,6 @@ Special attributes and methods

Numpy provides several hooks that classes can customize:

.. function:: class.__numpy_ufunc__(self, ufunc, method, i, inputs, **kwargs)

.. versionadded:: 1.9

Any class (ndarray subclass or not) can define this method to
override behavior of Numpy's ufuncs. This works quite similarly to
Python's ``__mul__`` and other binary operation routines.

- *ufunc* is the ufunc object that was called.
- *method* is a string indicating which Ufunc method was called
(one of ``"__call__"``, ``"reduce"``, ``"reduceat"``,
``"accumulate"``, ``"outer"``, ``"inner"``).
- *i* is the index of *self* in *inputs*.
- *inputs* is a tuple of the input arguments to the ``ufunc``
- *kwargs* is a dictionary containing the optional input arguments
of the ufunc. The ``out`` argument is always contained in
*kwargs*, if given. See the discussion in :ref:`ufuncs` for
details.

The method should return either the result of the operation, or
:obj:`NotImplemented` if the operation requested is not
implemented.

If one of the arguments has a :func:`__numpy_ufunc__` method, it is
executed *instead* of the ufunc. If more than one of the input
arguments implements :func:`__numpy_ufunc__`, they are tried in the
order: subclasses before superclasses, otherwise left to right. The
first routine returning something else than :obj:`NotImplemented`
determines the result. If all of the :func:`__numpy_ufunc__`
operations return :obj:`NotImplemented`, a :exc:`TypeError` is
raised.

If an :class:`ndarray` subclass defines the :func:`__numpy_ufunc__`
method, this disables the :func:`__array_wrap__`,
:func:`__array_prepare__`, :data:`__array_priority__` mechanism
described below.

.. note:: In addition to ufuncs, :func:`__numpy_ufunc__` also
overrides the behavior of :func:`numpy.dot` even though it is
not an Ufunc.

.. note:: If you also define right-hand binary operator override
methods (such as ``__rmul__``) or comparison operations (such as
``__gt__``) in your class, they take precedence over the
:func:`__numpy_ufunc__` mechanism when resolving results of
binary operations (such as ``ndarray_obj * your_obj``).

The technical special case is: ``ndarray.__mul__`` returns
``NotImplemented`` if the other object is *not* a subclass of
:class:`ndarray`, and defines both ``__numpy_ufunc__`` and
``__rmul__``. Similar exception applies for the other operations
than multiplication.

In such a case, when computing a binary operation such as
``ndarray_obj * your_obj``, your ``__numpy_ufunc__`` method
*will not* be called. Instead, the execution passes on to your
right-hand ``__rmul__`` operation, as per standard Python
operator override rules.

Similar special case applies to *in-place operations*: If you
define ``__rmul__``, then ``ndarray_obj *= your_obj`` *will not*
call your ``__numpy_ufunc__`` implementation. Instead, the
default Python behavior ``ndarray_obj = ndarray_obj * your_obj``
occurs.

Note that the above discussion applies only to Python's builtin
binary operation mechanism. ``np.multiply(ndarray_obj,
your_obj)`` always calls only your ``__numpy_ufunc__``, as
expected.

.. function:: class.__array_finalize__(self)

This method is called whenever the system internally allocates a
Expand Down
4 changes: 4 additions & 0 deletions numpy/core/src/private/ufunc_override.h
Original file line number Diff line number Diff line change
Expand Up @@ -188,6 +188,10 @@ PyUFunc_CheckOverride(PyUFuncObject *ufunc, char *method,
/* Pos of each override in args */
int with_override_pos[NPY_MAXARGS];

/* disabled until remaining issues are fixed */
*result = NULL;
return 0;

/*
* Check inputs
*/
Expand Down
1 change: 1 addition & 0 deletions numpy/core/tests/test_blasdot.py
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,7 @@ def test_dot_array_order():
assert_almost_equal(b.dot(c), _dot(b, c), decimal=prec)
assert_almost_equal(c.T.dot(b.T), _dot(c.T, b.T), decimal=prec)

@dec.skipif(True) # ufunc override disabled for 1.9
def test_dot_override():
class A(object):
def __numpy_ufunc__(self, ufunc, method, pos, inputs, **kwargs):
Expand Down
3 changes: 3 additions & 0 deletions numpy/core/tests/test_multiarray.py
Original file line number Diff line number Diff line change
Expand Up @@ -1640,6 +1640,7 @@ def test_dot(self):
a.dot(b=b, out=c)
assert_equal(c, np.dot(a, b))

@dec.skipif(True) # ufunc override disabled for 1.9
def test_dot_override(self):
class A(object):
def __numpy_ufunc__(self, ufunc, method, pos, inputs, **kwargs):
Expand Down Expand Up @@ -1770,6 +1771,7 @@ def test_conjugate(self):


class TestBinop(object):
@dec.skipif(True) # ufunc override disabled for 1.9
def test_ufunc_override_rop_precedence(self):
# Check that __rmul__ and other right-hand operations have
# precedence over __numpy_ufunc__
Expand Down Expand Up @@ -1888,6 +1890,7 @@ def __rop__(self, *other):
yield check, op_name, True
yield check, op_name, False

@dec.skipif(True) # ufunc override disabled for 1.9
def test_ufunc_override_rop_simple(self):
# Check parts of the binary op overriding behavior in an
# explicit test case that is easier to understand.
Expand Down
5 changes: 5 additions & 0 deletions numpy/core/tests/test_umath.py
Original file line number Diff line number Diff line change
Expand Up @@ -962,6 +962,7 @@ def __array__(self):
assert_equal(ncu.maximum(a, B()), 0)
assert_equal(ncu.maximum(a, C()), 0)

@dec.skipif(True) # ufunc override disabled for 1.9
def test_ufunc_override(self):
class A(object):
def __numpy_ufunc__(self, func, method, pos, inputs, **kwargs):
Expand All @@ -988,6 +989,7 @@ def __numpy_ufunc__(self, func, method, pos, inputs, **kwargs):
assert_equal(res0[5], {})
assert_equal(res1[5], {})

@dec.skipif(True) # ufunc override disabled for 1.9
def test_ufunc_override_mro(self):

# Some multi arg functions for testing.
Expand Down Expand Up @@ -1081,6 +1083,7 @@ def __numpy_ufunc__(self, func, method, pos, inputs, **kwargs):
assert_raises(TypeError, four_mul_ufunc, 1, 2, c_sub, c)
assert_raises(TypeError, four_mul_ufunc, 1, c, c_sub, c)

@dec.skipif(True) # ufunc override disabled for 1.9
def test_ufunc_override_methods(self):
class A(object):
def __numpy_ufunc__(self, ufunc, method, pos, inputs, **kwargs):
Expand Down Expand Up @@ -1184,6 +1187,7 @@ def __numpy_ufunc__(self, ufunc, method, pos, inputs, **kwargs):
assert_equal(res[3], 0)
assert_equal(res[4], (a, [4, 2], 'b0'))

@dec.skipif(True) # ufunc override disabled for 1.9
def test_ufunc_override_out(self):
class A(object):
def __numpy_ufunc__(self, ufunc, method, pos, inputs, **kwargs):
Expand Down Expand Up @@ -1218,6 +1222,7 @@ def __numpy_ufunc__(self, ufunc, method, pos, inputs, **kwargs):
assert_equal(res7['out'][0], 'out0')
assert_equal(res7['out'][1], 'out1')

@dec.skipif(True) # ufunc override disabled for 1.9
def test_ufunc_override_exception(self):
class A(object):
def __numpy_ufunc__(self, *a, **kwargs):
Expand Down
22 changes: 22 additions & 0 deletions numpy/lib/tests/test_twodim_base.py
Original file line number Diff line number Diff line change
Expand Up @@ -324,6 +324,28 @@ def test_tril_triu_with_inf():
assert_array_equal(np.tril(arr), out_tril)


def test_tril_triu_dtype():
# Issue 4916
# tril and triu should return the same dtype as input
for c in np.typecodes['All']:
if c == 'V':
continue
arr = np.zeros((3, 3), dtype=c)
assert_equal(np.triu(arr).dtype, arr.dtype)
assert_equal(np.tril(arr).dtype, arr.dtype)

# check special cases
arr = np.array([['2001-01-01T12:00', '2002-02-03T13:56'],
['2004-01-01T12:00', '2003-01-03T13:45']],
dtype='datetime64')
assert_equal(np.triu(arr).dtype, arr.dtype)
assert_equal(np.tril(arr).dtype, arr.dtype)

arr = np.zeros((3,3), dtype='f4,f4')
assert_equal(np.triu(arr).dtype, arr.dtype)
assert_equal(np.tril(arr).dtype, arr.dtype)


def test_mask_indices():
# simple test without offset
iu = mask_indices(3, np.triu)
Expand Down
4 changes: 2 additions & 2 deletions numpy/lib/twodim_base.py
Original file line number Diff line number Diff line change
Expand Up @@ -453,7 +453,7 @@ def tril(m, k=0):
m = asanyarray(m)
mask = tri(*m.shape[-2:], k=k, dtype=bool)

return where(mask, m, 0)
return where(mask, m, zeros(1, m.dtype))


def triu(m, k=0):
Expand Down Expand Up @@ -481,7 +481,7 @@ def triu(m, k=0):
m = asanyarray(m)
mask = tri(*m.shape[-2:], k=k-1, dtype=bool)

return where(mask, 0, m)
return where(mask, zeros(1, m.dtype), m)


# Originally borrowed from John Hunter and matplotlib
Expand Down