I tested this locally and works fine. Also the types are stable:

np.linalg.norm(numpy.array([], dtype=np.float16), ord=np.inf)

has the same output type (np.float16) as the input.

@carlosgmartin Could you add some unit tests for the cases that have been fixed?

@eendebakpt Done.

@eendebakpt Done.

@carlosgmartin Thanks for the updates. Failing tests seem unrelated, so I approved. The PR needs approval of a numpy dev to be merged though.

@charris do you have a very clear picture of what the right thing is?

@carlosgmartin you seem to have just extended this with from the issue for initial=inf without any explanation, please provide one if you have.
Without thinking about it further, the one thing that makes sense to me is to that an empty matrix always has a norm of zero, but I am not 100% sure zero is safe for all norms.

Also ping @melissawm, maybe you have this type of knowledge more available than me. If there are some norms where an empty matrix default value isn't pretty clear, I would be fine with adding a default= kwarg (but I it may be that the need for this is so small that I am not sure we will agree on it being worthwhile).
(NaN might also be a reasonable default for some in that case.)

EDIT: I think this may be an interesting example/reason for why maximum/minimum indeed shouldn't define -inf/inf as a default.

So - all of these norms should indeed be 0 on zero matrices. However, some of these norms are the result of computations e.g. "the sum of singular values". In that case it doesn't seem like assert_equal is the right choice for these tests. I haven't looked that closely at the implementation though!

On the other hand, the initial issue was about empty matrices, not zero-valued matrices. So I'm unsure if this solves that issue.

@melissawm all of the tests work with empty matrices (so zero or empty would be the same)?
As you said, the question is really about what the right thing is for empty matrices, and that is what the PR modifies.

EDIT: Ahhh, zero matrix => zero norm, I suppose also means that empty matrix == empty zero matrix => zero norm. Unless there is some argument to the contrary?

Oh i see, sorry I looked to quickly. Yes I'd say the default should be 0 for all by following what LAPACK does in *LANGE.

Mathematically, raising an error for the norm of an empty matrix makes the most sense. What is the use case for doing otherwise?

@seberg Suppose we are working in a complete lattice $L$.

The identity element of sup (join) $\vee$ is the least element (bottom) $\bot$ of $L$. Thus $\vee \varnothing = \bot$.
The identity element of inf (meet) $\wedge$ is the greatest element (top) $\top$ of $L$. Thus $\wedge \varnothing = \top$.

In the case of norms, $L = [0, \infty] \subset \mathbb{R}$ (the non-negative extended real numbers). Notice the presence of absolute values inside the expressions involving sup and inf. This is why it makes sense, in this context, to define the empty sup as $0$ and empty inf as $\infty$: These are the identity elements of sup and inf, respectively, in that $L$. If $L$ were $[-\infty, \infty]$ instead, then the empty sup would be $-\infty$.

Furthermore, the matrix $p$-norm is defined by

where the last equality holds if $n = 0$ or $d = 0$.

The norm of a zero vector is, by the definition of a norm, zero. (Note that ord < 1 is not strictly a norm, as noted in the docs.)

The fact that the norm of a zero matrix (of which every empty matrix is an example) is zero is also explicitly stated here.

@melissawm

some of these norms are the result of computations e.g. "the sum of singular values". In that case it doesn't seem like assert_equal is the right choice for these tests.

The additional tests are for when $A$ is empty, for which no actual arithmetic is involved, and for which the result is (and should be) always exactly 0 or inf, so exact comparison makes sense in this context.

@charris

Mathematically, raising an error for the norm of an empty matrix makes the most sense.

Hard disagree. Mathematically, the answer is well-defined. That'd be like saying that an empty sum or product should raise an error rather than return 0 or 1 respectively.

What is the use case for doing otherwise?

This is the example that inspired me to create this PR. Other examples might be in the issues I linked to.

That'd be like saying that an empty sum or product should raise an error rather than return 0 or 1 respectively.

Yes, that is a useful convention for sums and product because of how they are used. But a norm, while it may use sums and products, is not just a sum, it has a meaning. Does the height and width of non-existent object make sense? You can return numbers for both, but what purpose would it serve? If your use case is to answer the question "will it fit in my closet", then returning zero for both makes sense. That is why I am asking about the use case.

@charris Geometrically, $\|A\|_p$ measures the longest $p$-radius of the image of the $p$-norm unit ball under $A$. This is used, for example, to compute Lipschitz constants of objective functions for optimization purposes.

@charris I think it's helpful to point out that in many cases NumPy already returns 0 for an empty norm; e.g.

In [1]: import numpy as np

In [2]: np.__version__
Out[2]: '2.2.2'

In [3]: np.linalg.norm(np.zeros((0, 0)))
Out[3]: np.float64(0.0)

In [4]: np.linalg.matrix_norm(np.zeros((10, 0, 0)))
Out[4]: array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])

So the contribution here is not so much about adding new behavior, but rather about making already existing behavior more consistent.

I don't think you can argue with the supremum/infinum being 0 or all inf on the set of all possible values for the norm.

Unless you can argue that 0 is the correct answer for all (sensible) subsets of matrices, however chosen, having a default return is just wrong. That is exactly the same reason why maximum/minimum raise an error on empty arrays (The user might be working only with values within a specific subset, i.e. range, certainly inf is a special value so it isn't actually included in any sensible set to begin with!).
For example:

• The set of matrices of any shape that contains only 0s always has 0 norm. So, IMO, clearly 0 is the only possible choice that might be right.
• But say, the set of a matrix containing a single constant value? If something is defined via a sum([]) then I argue say that is OK, because the sum is const * N and for N=0 that is zero. In general, sum([]) is well defined as 0, so the extension makes sense to me; although I am happy if someone disagrees w.r.t. matrix norm.
  (we ignore matrices containing float inf here, because it is not a mathematical real value.)
• For some norms, other sets of matrices might be interesting, although I suspect just the "constant value" may be enough.

@seberg The mathematical definition of a norm forces the norm of an empty matrix to be zero. That particular fact has nothing to do with subsets one is working in.

How can it be that you have assert_equal(np.linalg.matrix_norm(x, ord=-2), np.inf)?

@seberg ord < 1 aren't norms. They need to be considered separately.

According to the docs, ord=-2 is the smallest singular value. The infimum of an empty set is the greatest possible value, which is infinity.

According to the docs, ord=-2 is the smallest singular value. The infimum of an empty set is the greatest possible value, which is infinity.

This point is similar to the discussion in #5032, where the decision was to not return the supremum/infimum in the case of empty min/max. Given the consensus there, I think we should not add such behavor to norm when ord implies a min or max reduction.

@jakevdp You mean for the cases that aren't norms, right?

If it'll expedite merging, I can restrict this PR to only the actual norms, which are uncontroversial and what I'm currently most interested in.

@carlosgmartin here is the point:

1. IMO (and I think everyone else agrees), returning anything but 0 is definitely incorrect. Clearly we can't merge something we know is wrong.
2. 0 is clearly the only possible choice and it is definitely very often a reasonable and useful choice. But the bullet-proof argument would be that is the correct choice in (almost) all situations (rather than known to be useful in yours).
   E.g. for min/max -inf/inf are very reasonable (numerical) and in many situations correct choices. But this PR is a good example of why it would be a bad default since here 0 rather than -inf is correct.

FWIW, I suspect it is in fine, and probably completely correct, to define it to be 0 for all proper norms. But right now the most helpful argument was Melissa pointing to prior art.

It may be true that this is mathematical convention or drops out of the definition of the norm. But while 0 being the only possible (and definitely often correct) choice, none of the references I found until now explicitly state anything about empty matrices.

@seberg

none of the references I found until now explicitly state anything about empty matrices.

Carl de Boor. An empty exercise. ACM Signum Newsletter 25 (4), 2-6, 1990.

any norm of an empty matrix is zero

I've edited the PR to restrict changes to the proper norms.

We discussed this briefly and we decided to give this a try. But it would be good to have release note maybe "change" just in case it surprises someone. (You can find info in doc/release/upcoming_changes/README.md.

@seberg Done.

Merging. We can continue to improve the release note as needed.

Edit: rerunning a failing 32-bit CI test, it failed to start. If it passes I will merge.

Thanks @carlosgmartin