Skip to content

BUG: Array ufunc reduce out tuple #9111

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion doc/neps/ufunc-overrides.rst
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,8 @@ Hence, the arguments are normalized: only the required input arguments
passed on as a dict of keyword arguments (``kwargs``). In particular, if
there are output arguments, positional are otherwise, that are not
:obj:`None`, they are passed on as a tuple in the ``out`` keyword
argument.
argument (even for the ``reduce``, ``accumulate``, and ``reduceat`` methods
where in all current cases only a single output makes sense).

The function dispatch proceeds as follows:

Expand Down
8 changes: 8 additions & 0 deletions doc/release/1.13.0-notes.rst
Original file line number Diff line number Diff line change
Expand Up @@ -536,3 +536,11 @@ The ABCPolyBase class, from which the convenience classes are derived, sets
``__array_ufun__ = None`` in order of opt out of ufuncs. If a polynomial
convenience class instance is passed as an argument to a ufunc, a ``TypeError``
will now be raised.

Output arguments to ufuncs can be tuples also for ufunc methods
---------------------------------------------------------------
For calls to ufuncs, it was already possible, and recommended, to use an
``out`` argument with a tuple for ufuncs with multiple outputs. This has now
been extended to output arguments in the ``reduce``, ``accumulate``, and
``reduceat`` methods. This is mostly for compatibility with ``__array_ufunc``;
there are no ufuncs yet that have more than one output.
9 changes: 6 additions & 3 deletions doc/source/reference/ufuncs.rst
Original file line number Diff line number Diff line change
Expand Up @@ -426,8 +426,8 @@ Methods
All ufuncs have four methods. However, these methods only make sense on
ufuncs that take two input arguments and return one output argument.
Attempting to call these methods on other ufuncs will cause a
:exc:`ValueError`. The reduce-like methods all take an *axis* keyword
and a *dtype* keyword, and the arrays must all have dimension >= 1.
:exc:`ValueError`. The reduce-like methods all take an *axis* keyword, a *dtype*
keyword, and an *out* keyword, and the arrays must all have dimension >= 1.
The *axis* keyword specifies the axis of the array over which the reduction
will take place and may be negative, but must be an integer. The
*dtype* keyword allows you to manage a very common problem that arises
Expand All @@ -443,7 +443,10 @@ mostly up to you. There is one exception: if no *dtype* is given for a
reduction on the "add" or "multiply" operations, then if the input type is
an integer (or Boolean) data-type and smaller than the size of the
:class:`int_` data type, it will be internally upcast to the :class:`int_`
(or :class:`uint`) data-type.
(or :class:`uint`) data-type. Finally, the *out* keyword allows you to provide
an output array (for single-output ufuncs, which are currently the only ones
supported; for future extension, however, a tuple with a single argument
can be passed in). If *out* is given, the *dtype* argument is ignored.

Ufuncs also have a fifth method that allows in place operations to be
performed using fancy indexing. No buffering is used on the dimensions where
Expand Down
39 changes: 28 additions & 11 deletions numpy/add_newdocs.py
Original file line number Diff line number Diff line change
Expand Up @@ -5444,9 +5444,11 @@ def luf(lamdaexpr, *args, **kwargs):
----------
*x : array_like
Input arrays.
out : ndarray or tuple of ndarray, optional
out : ndarray, None, or tuple of ndarray and None, optional
Alternate array object(s) in which to put the result; if provided, it
must have a shape that the inputs broadcast to.
must have a shape that the inputs broadcast to. A tuple of arrays
(possible only as a keyword argument) must have length equal to the
number of outputs; use `None` for outputs to be allocated by the ufunc.
where : array_like, optional
Values of True indicate to calculate the ufunc at that position, values
of False indicate to leave the value in the output alone.
Expand Down Expand Up @@ -5667,9 +5669,14 @@ def luf(lamdaexpr, *args, **kwargs):
The type used to represent the intermediate results. Defaults
to the data-type of the output array if this is provided, or
the data-type of the input array if no output array is provided.
out : ndarray, optional
A location into which the result is stored. If not provided, a
freshly-allocated array is returned.
out : ndarray, None, or tuple of ndarray and None, optional
A location into which the result is stored. If not provided or `None`,
a freshly-allocated array is returned. For consistency with
:ref:`ufunc.__call__`, if given as a keyword, this may be wrapped in a
1-element tuple.

.. versionchanged:: 1.13.0
Tuples are allowed for keyword argument.
keepdims : bool, optional
If this is set to True, the axes which are reduced are left
in the result as dimensions with size one. With this option,
Expand Down Expand Up @@ -5741,9 +5748,14 @@ def luf(lamdaexpr, *args, **kwargs):
The data-type used to represent the intermediate results. Defaults
to the data-type of the output array if such is provided, or the
the data-type of the input array if no output array is provided.
out : ndarray, optional
A location into which the result is stored. If not provided a
freshly-allocated array is returned.
out : ndarray, None, or tuple of ndarray and None, optional
A location into which the result is stored. If not provided or `None`,
a freshly-allocated array is returned. For consistency with
:ref:`ufunc.__call__`, if given as a keyword, this may be wrapped in a
1-element tuple.

.. versionchanged:: 1.13.0
Tuples are allowed for keyword argument.
keepdims : bool
Has no effect. Deprecated, and will be removed in future.

Expand Down Expand Up @@ -5820,9 +5832,14 @@ def luf(lamdaexpr, *args, **kwargs):
The type used to represent the intermediate results. Defaults
to the data type of the output array if this is provided, or
the data type of the input array if no output array is provided.
out : ndarray, optional
A location into which the result is stored. If not provided a
freshly-allocated array is returned.
out : ndarray, None, or tuple of ndarray and None, optional
A location into which the result is stored. If not provided or `None`,
a freshly-allocated array is returned. For consistency with
:ref:`ufunc.__call__`, if given as a keyword, this may be wrapped in a
1-element tuple.

.. versionchanged:: 1.13.0
Tuples are allowed for keyword argument.

Returns
-------
Expand Down
8 changes: 5 additions & 3 deletions numpy/core/code_generators/ufunc_docstrings.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,11 @@ def get(name):

# common parameter text to all ufuncs
_params_text = textwrap.dedent("""
out : ndarray or tuple of ndarray, optional
Alternate array object(s) in which to put the result; if provided, it
must have a shape that the inputs broadcast to.
out : ndarray, None, or tuple of ndarray and None, optional
A location into which the result is stored. If provided, it must have
a shape that the inputs broadcast to. If not provided or `None`,
a freshly-allocated array is returned. A tuple (possible only as a
keyword argument) must have length equal to the number of outputs.
where : array_like, optional
Values of True indicate to calculate the ufunc at that position, values
of False indicate to leave the value in the output alone.
Expand Down
52 changes: 29 additions & 23 deletions numpy/core/src/umath/override.c
Original file line number Diff line number Diff line change
Expand Up @@ -144,14 +144,16 @@ normalize_reduce_args(PyUFuncObject *ufunc, PyObject *args,
return -1;
}
obj = PyTuple_GET_ITEM(args, i);
if (obj != Py_None) {
if (i == 3) {
obj = PyTuple_GetSlice(args, 3, 4);
}
PyDict_SetItemString(*normal_kwds, kwlist[i], obj);
if (i == 3) {
Py_DECREF(obj);
if (i == 3) {
/* remove out=None */
if (obj == Py_None) {
continue;
}
obj = PyTuple_GetSlice(args, 3, 4);
}
PyDict_SetItemString(*normal_kwds, kwlist[i], obj);
if (i == 3) {
Py_DECREF(obj);
}
}
return 0;
Expand Down Expand Up @@ -188,14 +190,16 @@ normalize_accumulate_args(PyUFuncObject *ufunc, PyObject *args,
return -1;
}
obj = PyTuple_GET_ITEM(args, i);
if (obj != Py_None) {
if (i == 3) {
obj = PyTuple_GetSlice(args, 3, 4);
}
PyDict_SetItemString(*normal_kwds, kwlist[i], obj);
if (i == 3) {
Py_DECREF(obj);
if (i == 3) {
/* remove out=None */
if (obj == Py_None) {
continue;
}
obj = PyTuple_GetSlice(args, 3, 4);
}
PyDict_SetItemString(*normal_kwds, kwlist[i], obj);
if (i == 3) {
Py_DECREF(obj);
}
}
return 0;
Expand Down Expand Up @@ -234,14 +238,16 @@ normalize_reduceat_args(PyUFuncObject *ufunc, PyObject *args,
return -1;
}
obj = PyTuple_GET_ITEM(args, i);
if (obj != Py_None) {
if (i == 4) {
obj = PyTuple_GetSlice(args, 4, 5);
}
PyDict_SetItemString(*normal_kwds, kwlist[i], obj);
if (i == 4) {
Py_DECREF(obj);
if (i == 4) {
/* remove out=None */
if (obj == Py_None) {
continue;
}
obj = PyTuple_GetSlice(args, 4, 5);
}
PyDict_SetItemString(*normal_kwds, kwlist[i], obj);
if (i == 4) {
Py_DECREF(obj);
}
}
return 0;
Expand Down Expand Up @@ -360,11 +366,11 @@ PyUFunc_CheckOverride(PyUFuncObject *ufunc, char *method,
if (out != NULL) {
int nout = ufunc->nout;

if (PyTuple_Check(out)) {
if (PyTuple_CheckExact(out)) {
int all_none = 1;

if (PyTuple_GET_SIZE(out) != nout) {
PyErr_Format(PyExc_TypeError,
PyErr_Format(PyExc_ValueError,
"The 'out' tuple must have exactly "
"%d entries: one per ufunc output", nout);
goto fail;
Expand Down
18 changes: 16 additions & 2 deletions numpy/core/src/umath/ufunc_object.c
Original file line number Diff line number Diff line change
Expand Up @@ -1084,7 +1084,7 @@ get_ufunc_arguments(PyUFuncObject *ufunc,
"positional and keyword argument");
goto fail;
}
if (PyTuple_Check(value)) {
if (PyTuple_CheckExact(value)) {
if (PyTuple_GET_SIZE(value) != nout) {
PyErr_SetString(PyExc_ValueError,
"The 'out' tuple must have exactly "
Expand Down Expand Up @@ -3894,6 +3894,7 @@ PyUFunc_GenericReduction(PyUFuncObject *ufunc, PyObject *args,
PyObject *obj_ind, *context;
PyArrayObject *indices = NULL;
PyArray_Descr *otype = NULL;
PyObject *out_obj = NULL;
PyArrayObject *out = NULL;
int keepdims = 0;
static char *reduce_kwlist[] = {
Expand Down Expand Up @@ -3927,7 +3928,20 @@ PyUFunc_GenericReduction(PyUFuncObject *ufunc, PyObject *args,
_reduce_type[operation]);
return NULL;
}

/* if there is a tuple of 1 for `out` in kwds, unpack it */
if (kwds != NULL) {
PyObject *out_obj = PyDict_GetItem(kwds, npy_um_str_out);
if (out_obj != NULL && PyTuple_CheckExact(out_obj)) {
if (PyTuple_GET_SIZE(out_obj) != 1) {
PyErr_SetString(PyExc_ValueError,
"The 'out' tuple must have exactly one entry");
return NULL;
}
out_obj = PyTuple_GET_ITEM(out_obj, 0);
PyDict_SetItem(kwds, npy_um_str_out, out_obj);
}
}

if (operation == UFUNC_REDUCEAT) {
PyArray_Descr *indtype;
indtype = PyArray_DescrFromType(NPY_INTP);
Expand Down
2 changes: 1 addition & 1 deletion numpy/core/tests/test_multiarray.py
Original file line number Diff line number Diff line change
Expand Up @@ -3107,7 +3107,7 @@ def __array_ufunc__(self, ufunc, method, *inputs, **kw):
warnings.filterwarnings('always', '', DeprecationWarning)
assert_equal(np.modf(dummy, out=a), (0,))
assert_(w[0].category is DeprecationWarning)
assert_raises(TypeError, np.modf, dummy, out=(a,))
assert_raises(ValueError, np.modf, dummy, out=(a,))

# 2 inputs, 1 output
assert_equal(np.add(a, dummy), 0)
Expand Down
Loading