Skip to content

ENH: Added atleast_nd to numpy and ma #18386

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

madphysicist
Copy link
Contributor

@madphysicist madphysicist commented Feb 10, 2021

As per @charris's request, I am resurrecting #7804.

This likely also closes #12336.

Discussed in the mailing list forever ago: https://mail.python.org/pipermail/numpy-discussion/2016-July/075722.html
Discussed in the mailing list now: https://mail.python.org/pipermail/numpy-discussion/2021-February/081476.html

I did manage to implement np.atleast_3d in terms of np.atleast_nd.

@seberg seberg added 56 - Needs Release Note. Needs an entry in doc/release/upcoming_changes 62 - Python API Changes or additions to the Python API. Mailing list should usually be notified. and removed 56 - Needs Release Note. Needs an entry in doc/release/upcoming_changes labels Feb 10, 2021
@seberg
Copy link
Member

seberg commented Feb 10, 2021

I am pretty neutral on this addition (I honestly don't understand where its useful, but it also doesn't seem like a big feature creep – i.e. this is only useful when you don't know the input). If the old mailing list discussion was super positive, I might have overlooked it.

But I think this has to hit the mailing list again with the concrete current proposal.

I guess the old point is. If I just want to add dimensions on the left (usually not necessary), there are many options like np.broadcast_to(arr, (1,)*ndim). If I know the input dimensions (i.e. it is fixed), I can use np.expand_dims. Of course this function is probably easier on the eye, so maybe worth it even in those cases. Otherwise, I am mainly wondering if the multiple vs. one array choice and the pos kwarg choice need discussion still (could make that kwarg only possibly).

@madphysicist
Copy link
Contributor Author

madphysicist commented Feb 10, 2021

@seberg I am actually in a similar boat. There are a couple of cases where this would seem like a nice-to-have at best. I thought my original PR was dead and was prepared to leave it that way. The original feedback was pretty ambivalent, although I did have a few people ask me about it later. This is here pretty much because @charris asked me to revive it.

@madphysicist
Copy link
Contributor Author

@BvB93 It's pretty clear that the remaining errors are something I did in the typing tests, but I can't seem to figure it out. Could you take a look when you have the chance? I'm especially confused about the error saying that numpy does not have an attribute atleast_nd.

@madphysicist
Copy link
Contributor Author

I've squashed all the commits now that the last bug is fixed. At this point, code is good to go unless someone has something to say on the new mailing list post.

@madphysicist
Copy link
Contributor Author

Looks like the Azure failtures are not related to anything I did.

Base automatically changed from master to main March 4, 2021 02:05


@array_function_dispatch(_atleast_nd_dispatcher)
def atleast_nd(ary, ndim, pos=0):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In numpy.array you can pre-pend ones to the array-shape by giving a 'ndmin' parameter.
Therefore, I think the variable name 'ndim' should be renamed to 'ndmin' to make it consistent with numpy.array.

Copy link
Contributor

@pbrod pbrod Oct 6, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the atleast_nd should allow multiple inputs in the same way as atleast_1d, atleast_2d and atleast_3d do. This will make it easier to use since then the call syntax and output will be the same as the atleast_XXXd functions. Any written code with multiple uses of the function will be shorter like this:

x1, x2, x3 = numpy.atleast_nd(x1,x2,x3, ndmin=3)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pos argument is the axis index to insert the new dimensions. What about renaming 'pos' to 'axis'? That will make it more consistent with the normalize_axis_index function.

@dg-pb
Copy link

dg-pb commented Nov 13, 2023

I hope this will be merged. Otherwise, could anyone provide a simple way to do what this functionality provides?

I.e. if my input is a 2d array - leave it as it is, otherwise if it is a 1d array, append one dimension to the end.

Neither of above suggestions seem to do the trick.

@jonas-eschle
Copy link

Hi, what's the status of this? Glad to help if anything is stoping it from going forward.

If I know the input dimensions (i.e. it is fixed), I can use np.expand_dims

@seberg I think that's the whole point of atleast_1d etc: if you know the shape, you can convert them by hand, but the point is that you don't need to know and check, and all you want is to have a certain shape, no matter what's the input.
The only difference in this PR is that you don't want the additional dimension to be always in the first place, therefore I would argue, whoever needs atleast_nd needs this PR (except if you're lucky and your case coincides with pos=0)

For the use-case, I find the 2D shape of (event, dimensions) very common in many applications and therefore the desire to have one dimensional arrays made to 2D with shape (event, 1)

Looking forward to have this in!

@dg-pb
Copy link

dg-pb commented Apr 18, 2024

I have been using this for a fair while now. Extremely useful. I don't even use the other ones atleast_1d|2d|3d anymore, only this one.

  1. I would like to propose simpler implementation. The one proposed here is elegant, but a bit too clever (at least for my taste) - takes me few seconds to get my head round every time I look at it again.
def atleast_nd(ary, ndim, pos=0):
    ary = array(ary, copy=False, subok=True)
    if ary.ndim > ndim:
        pos = normalize_axis_index(pos, ary.ndim + 1)
        shape = ary.shape
        new_shape = shape[:pos] + (1,) * extra + shape[pos:]
        return reshape(a, new_shape)
    return ary
  1. is operator.index(ndim) needed here? Are there objects that represent number of dimensions in this manner?

  2. What I also have found useful is having one more flag, which raises error if input array number of dimensions is higher than requested. I named it exact (in contrast with at least).

def atleast_nd(ary, ndim, pos=0, exact=False):
    ary = array(ary, copy=False, subok=True)
    extra = ary.ndim > ndim
    if extra > 0:
        pos = normalize_axis_index(pos, ary.ndim + 1)
        shape = ary.shape
        new_shape = shape[:pos] + (1,) * extra + shape[pos:]
        return reshape(a, new_shape)
    elif exact and extra != 0:
        raise ValueError(f'{ndim=} > {nd=}')
    return ary

@madphysicist
Copy link
Contributor Author

madphysicist commented Apr 18, 2024 via email

@dg-pb
Copy link

dg-pb commented Apr 18, 2024

I'm happy to revisit the implementation.

By the way, the version above is 100ns slower than yours.

I chose to use this one for being slightly simpler, but I would be happy with either one merged into numpy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
01 - Enhancement 62 - Python API Changes or additions to the Python API. Mailing list should usually be notified.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

atleast_2d axis argument option
7 participants