Skip to content

Add an axis argument to generalized ufuncs? #5197

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
shoyer opened this issue Oct 17, 2014 · 8 comments
Closed

Add an axis argument to generalized ufuncs? #5197

shoyer opened this issue Oct 17, 2014 · 8 comments

Comments

@shoyer
Copy link
Member

shoyer commented Oct 17, 2014

I would like to write generalized ufuncs, using numba, to create fast functions such as nanmean (signature '(n)->()') or rolling_mean (signature '(n),()->(n)') that take the axis along which to aggregate as a keyword argument, e.g., nanmean(x, axis=0) or rolling_mean(x, window=5, axis=0).

Of course, I have the option of writing my own wrapper function to do something like x.swapaxes(axis, -1) before passing the array off to the gufunc. This is not so terrible.

However, it occurs to me that this sort of dimension reordering might be handled more cleanly (and efficiently) within the gufunc machinery itself, somewhat analogously to numpy.ufunc.reduce, except the axis argument (if supplied) would need to be an integer or tuple with length equal to the number of core dimensions on the first gufunc argument.

Thoughts?

@njsmith
Copy link
Member

njsmith commented Oct 17, 2014

I think it's certainly functionality that we want to have. Details of the
design will have to go through the list. One limitation of your initial
straw man proposal is that we'll want to handle gufuncs with multiple
arguments and specify the axes for each. (E.g. think of matrix multiply.)
On 17 Oct 2014 06:40, "Stephan Hoyer" notifications@github.com wrote:

I would like to write generalized ufuncs, using numba
numba/numba#825, to create fast functions
such as nanmean (signature '(n)->()') or rolling_mean (signature
'(n),()->(n)') that take the axis along which to aggregate as a keyword
argument, e.g., nanmean(x, axis=0) or rolling_mean(x, window=5, axis=0).

Of course, I have the option of writing my own wrapper function to do
something like x.swapaxes(axis, -1) before passing the array off to the
gufunc. This is not so terrible.

However, it occurs to me that this sort of dimension reordering might
be handled more cleanly (and efficiently) within the gufunc machinery
itself, somewhat analogously to numpy.ufunc.reduce, except the axis
argument (if supplied) would need to be an integer or tuple with length
equal to the number of core dimensions on the first gufunc argument.

Thoughts?


Reply to this email directly or view it on GitHub
#5197.

@jaimefrio
Copy link
Member

I think it would be a great idea, if we can come up with a syntax that's both general enough and simple enough. The example of matrix_multiply, and its (m,n),(n,p)->(m,p) illustrates this perfectly. You could then specify something like axis=[(-3, -1), (0, 1)], or even axis=[(-3, -1), (0, 1), (-1, -2)] and specify also the axes of the output. As an example already in the API, np.cross has axis selection for multiple operands, although its interface is probably not the best for gufuncs.

There are also plans to add optional dimensions to gufunc signatures, e.g. make matrix_mutiply's signature (m?,n),(n,p?)->(m?,p?) so that it can also handle products with vectors. We also need to figure out how these two would interact.

But yes, it would be great if you could bring this up on the list.

@njsmith
Copy link
Member

njsmith commented Oct 17, 2014

Possibly we should take advantage of the fact that gufunc axes have names
to allow something more symbolic like axis=[{"m": 0, "n": 2}, {"n": 0, "p":
1}].

On Fri, Oct 17, 2014 at 3:16 PM, Jaime notifications@github.com wrote:

I think it would be a great idea, if we can come up with a syntax that's
both general enough and simple enough. The example of matrix_multiply,
and its (m,n),(n,p)->(m,p) illustrates this perfectly. You could then
specify something like axis=[(-3, -1), (0, 1)], or even axis=[(-3, -1),
(0, 1), (-1, -2)] and specify also the axes of the output. As an example
already in the API, np.cross
http://docs.scipy.org/doc/numpy/reference/generated/numpy.cross.html
has axis selection for multiple operands, although its interface is
probably not the best for gufuncs.

There are also plans to add optional dimensions to gufunc signatures, e.g.
make matrix_mutiply's signature (m?,n),(n,p?)->(m?,p?) so that it can
also handle products with vectors. We also need to figure out how these two
would interact.

But yes, it would be great if you could bring this up on the list.


Reply to this email directly or view it on GitHub
#5197 (comment).

Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org

@jaimefrio
Copy link
Member

That would be very nice indeed.

@shoyer
Copy link
Member Author

shoyer commented Oct 18, 2014

@charris
Copy link
Member

charris commented Oct 18, 2014

@ovillellas Thoughts?

@mhvk
Copy link
Contributor

mhvk commented Mar 22, 2017

I just raised issues with the same points as done here, but in addition a bit of a roadmap on how to implement it: see #8810 (and (#8811)

@mattip
Copy link
Member

mattip commented Aug 8, 2018

Implemented.

@mattip mattip closed this as completed Aug 8, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants