-
-
Notifications
You must be signed in to change notification settings - Fork 10.8k
ENH: Create the ability for fused operations (fused ufuncs or map_reduce
) style
#11622
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
"I want to do this chain of operations without intermediate copies" seems to already be a problem solved well by numba - I don't see numpy stepping up in its place any time soon. See also #8528, which is similar. |
I agree with @eric-wieser that this is not something we would want to introduce a new function for. If you don't want the memory copy of the full array |
Numexpr also, although I think they support almost no reductions. This can be done in a bit of (non-pretty) python code, also. If someone has an elegant enough thing, I don't mind adding it to numpy, although it probably might as well be its own sub-package. |
I don't like that idea of adding ufuncs for a small subset of max_abs = np.ufunc.map_reduce(np.abs, np.maximum)
all_equal = np.ufunc.map_reduce(np.equal, np.logical_and)
max_abs(a) # np.maximum.reduce(np.abs(a)), but more efficient
all_equal(a, b) # np.logical_and.reduce(np.equal(a, b)), but more efficient
That seems reasonable to me - anyone is free to create their own gufunc that does these things |
Yeah, we would have to be very careful to be sparingly, so probably a bad idea. Someone could make a package outside numpy if they feel it important ;). Yeah, such a |
Sure, having some It's true that there are a plethora of things that build off NumPy and are useful. Though it's often harder to get a library (especially one you don't own) to use a new dependency. Requiring NumPy is pretty much a given in these cases and downstream libraries tend to encourage features like this one moving into NumPy. For a related discussion (and part of my motivation for thinking about this), please see issue ( scikit-learn/scikit-learn#11681 ). |
Well, I think we just need to make:
I frankly think that none of those things are necessarily a super high bar, at least for individual functions, but my guess is the interest for most of us currently maintaining is not so much to figure those out... Also nothing stops us from moving external well maintained stuff into numpy if everyone feels it is super useful, but at the moment it feels like we are going into a direction of rather a package more then less. But, such a package could be maintained under the numpy umbrella as well, to keep things a bit tighter/more opaque what we depend on? If someone wants an idea of how such things can be partially done using nditer (would be better in the C-api): https://gist.github.com/seberg/30f7bff59b347e71a21571122a5d9245 |
On a more general note as to the value of Side note: A solution like |
How does scikit-learn handle signed integers? |
map_reduce
) style
It would be nice to have something in NumPy, which would give the largest value by magnitude. IOW
abs(a).max()
. However it would be better if it didn't result in creating a second copy of the data asabs(a)
would do.The text was updated successfully, but these errors were encountered: