This is probably expected, since the implementation has to first ensure that the "elements" being compared are contiguous in memory. In other words it makes "elements" out of the subarrays which are being compared.

This requires a copy of the array in one case but not in the others where the axis is the "slowest in memory". NumPy may not be as fast as it could be (in some cases a copy may not be necessary in general), but memory order matters for speed, and a certain slowdown has to be expected in either case. In most cases, doing the copy is likely the fastest approach we can do... (although the copy could be optimized possible, that would be a pretty large endevour).

That being said, I will not rule out that there is more to this, but it seems unlikely to me. Sorry, if this is a bit too technical. Memory order and speed are not trivial concepts.

thanks for the quick answer. In fact, I just noticed that the title of the issue was wrong, it should be axis=0. Not sure why one needs to copy memory in this case. It seems that at least the case ndim=2 with arr.shape[1]=1 should perform just as fast as the case where arr is a 1d array, since the order of the elements in memory is identical. This is also related to #11136

Ah sorry, yeah... I did not realize you were just creating a single element along the other axis. I am not sure whether this copies, but it mainly adds a large chunk of overhead due to using structured dtype defined comparison/sorting, which does not optimize for the trivial case you got here.

As to gh-11136 there is a reason we have been careful about doing such tricks. They seem nice, but are also dangerous in general.

Thanks for bringing this up @mganahl . I'm not sure there is anything actionable here - do you feel like your question has been sufficiently answered?

I'm wondering if it may be worth adding a unique_interger routine. If I understand correctly, the concerns in #11136 are related to using float dtype.

@mganahl as well as sorting being inconsistent depending on the byte-order.

There would be no reason to add a new routine.

The original optimization was removed in the lines here: https://github.com/numpy/numpy/pull/10588/files#diff-9e06c7b7bcc8e8153f5c2df57283b4cbL238.

We can almost certainly reintroduce the same optimization, but with a stricter whitelist of types than before.

I had a similar use-case where I wanted to find unique 2D points in a list. I found it was much faster to manually sort the array and then find the transitions rather than call np.unique(arr, axis=0) and wrote the following function to speed it up. On my machine, it's faster for NxM arrays so long as M < 10.

def unique_axis0(arr):
    """
    Equivalent to np.unique(arr, axis=0). Based on numpy.lib.arraysetops._unique1d.
    """
    arr = np.asanyarray(arr)
    idxs = np.lexsort(arr.T)
    arr = arr[idxs]
    unique_idxs = np.empty(len(arr), dtype=np.bool_)
    unique_idxs[:1] = True
    unique_idxs[1:] = np.any(arr[:-1, :] != arr[1:, :], axis=-1)
    return arr[unique_idxs]

The function is asymptotically worse than np.unique so it's not a good candidate for a merge request. However, others may also find it a useful speedup in some situations.