TYP: Type default values in stubs in `numpy/ma` #29531

MarcoGorelli · 2025-08-07T20:29:20Z

Making some progress towards #28428

Similar PR in pandas: pandas-dev/pandas-stubs#1293

I've done this based on work started in https://gist.github.com/yangdanny97/170f82ee5389584f8b6292bc4ea9c24d, we're looking at open-sourcing a reusable tool to do this automatically where possible:

if a stub file uses = ...
and the corresponding defintion has a simple default
then fill the ... in

The ones in this PR, I checked manually, and they look correct to me

jorenham

I like the libcst approach, and the ones I checked seem to all be correct.

Technically speaking, this has little to do with static typing, but can be very helpful for IDE introspection, which is also one of the main advantages of annotations, so I suppose it's fine to keep the TYP: label.

For most defaults I can see that they can be useful. But in some cases, like out=None, I'm not if the defaults are actually helpful. Because without annotations, just having out=None doesn't give you any additional information about how it can be used. It could even be a bit confusing this way if some parameters use =None and others =np._NoValue (i.e. =...), especially if you consider that the documentation of _NoValue often incorrectly says it defaults to None. That could be confusing because it appears to be inconsistent.

Anyway, it probably doesn't matter much, so I'm fine with keeping those =None. Removing them now would mean we'd have to add them back in again once we add the annotations.

jorenham · 2025-08-11T19:07:58Z

numpy/ma/core.pyi

 get_data = getdata

-def fix_invalid(a, mask=..., copy=..., fill_value=...): ...
+def fix_invalid(a, mask=..., copy=True, fill_value=None): ...


The reasoning behind PYI014 doesn't make much sense to me, and their definition of a "simple" value seems pretty arbitrary.
So I wouldn't mind ignoring it and leaving it up to our own judgement whether we use ... or e.g. a np.False_ like in this case:

Suggested change

def fix_invalid(a, mask=..., copy=True, fill_value=None): ...

def fix_invalid(a, mask=np.False_, copy=True, fill_value=None): ... # noqa: PYI014

but that's just what I think, and I'll leave that decision to you

thanks for your review

I'm OK with using non-simple defaults, but it should be kept in sync, right? because in the .py file there's nomask, not np.False_. I get that nomask is an alias for np.False_, but a static analysis tool doesn't 😄 So, this would be a very manual effort to do it across the codebase, would that be ok? (tbh I think the diff is quite big already, shall we leave that to a separate discussion?)

I get that nomask is an alias for np.False_, but a static analysis tool doesn't 😄

Well, according to ruff's PYI014 docs:

Stub (.pyi) files exist to define type hints, and are not evaluated at runtime. As such, function arguments in stub files should not have default values, as they are ignored by type checkers.

But that's not true, because def f(_: int = ""): ... would be reported as an error in a .pyi. And the same error will be reported when you replace the literal "" with a constant:

from typing import Final, Literal C: Final[Literal[""]] = "" def f(_: int = C) -> None: ...

here, pyright reports

Expression of type "Literal['']" cannot be assigned to parameter of type "int" "Literal['']" is not assignable to "int"

and mypy says

Incompatible default for argument "_" (default has type "Literal['']", argument has type "int")

That means that in case of nomask, if we were to annotate it as e.g. nomask: Final[np.bool[Literal[False]]] = ..., mypy and pyright will treat mask=nomask in exactly the same way as mask=np.False_.

But it's indeed true that tools like pylance will not show mask=nomask when it's actually mask=np.False_, or vice-versa. So it would probably be better to use nomask as default here instead of np.False_.

But then again, I'm also fine with listening to ruff here. We can always reconsider one we actually annotate these functions.

sorry i meant that to a libcst / ast based tool, there's no knowledge that nomask corresponds to np.False_ (unless we start following imports, but currently the tool is just file-per-file, like ruff checks usually are)

So it would probably be better to use nomask as default here instead of np.False_.

nice, this is what I was hoping for 🙌

jorenham · 2025-08-11T19:10:44Z

numpy/ma/core.pyi

-def make_mask(m, copy=..., shrink=..., dtype=...): ...
-def make_mask_none(newshape, dtype=...): ...
-def mask_or(m1, m2, copy=..., shrink=...): ...
+def make_mask(m, copy=False, shrink=True, dtype=...): ...


Suggested change

def make_mask(m, copy=False, shrink=True, dtype=...): ...

def make_mask(m, copy=False, shrink=True, dtype=np.bool): ...

🤷🏻

jorenham · 2025-08-11T19:12:17Z

numpy/ma/core.pyi

+def masked_outside(x, v1, v2, copy=True): ...
+def masked_object(x, value, copy=True, shrink=True): ...
+def masked_values(x, value, rtol=1e-5, atol=1e-8, copy=True, shrink=True): ...
+def masked_invalid(a, copy=True): ...

 class _MaskedPrintOption:


the enable method shrink parameter defaults to 1

ooh, thanks! looks like the script wasn't picking up methods in class functions yangdanny97/docs2types#5

jorenham · 2025-08-11T19:17:17Z

numpy/ma/core.pyi

-def power(a, b, third=...): ...
-def argsort(a, axis=..., kind=..., order=..., endwith=..., fill_value=..., *, stable=...): ...
+def power(a, b, third=None): ...
+def argsort(a, axis=..., kind=None, order=None, endwith=True, fill_value=None, *, stable=...): ...


Doesn't the libcst codemod support keyword-only parameters?

Suggested change

def argsort(a, axis=..., kind=None, order=None, endwith=True, fill_value=None, *, stable=...): ...

def argsort(a, axis=..., kind=None, order=None, endwith=True, fill_value=None, *, stable=None): ...

yup, fixed, thanks! yangdanny97/docs2types#6

jorenham · 2025-08-11T19:22:23Z

numpy/ma/core.pyi

-def transpose(a, axes=...): ...
-def reshape(a, new_shape, order=...): ...
+def transpose(a, axes=None): ...
+def reshape(a, new_shape, order='C'): ...


I realize that there are some existing ' quotes here and there, but " is used way more often. I'm kinda surprised that ruff accepts this though 🤔

Suggested change

def reshape(a, new_shape, order='C'): ...

def reshape(a, new_shape, order="C"): ...

I don't really have a preference, but I think if it's a project preference then it should be automated - I've opened #29548 for this

jorenham · 2025-08-11T19:24:18Z

numpy/ma/core.pyi

 def where(condition, x=..., y=...): ...
-def choose(indices, choices, out=..., mode=...): ...
-def round_(a, decimals=..., out=...): ...
+def choose(indices, choices, out=None, mode='raise'): ...


Suggested change

def choose(indices, choices, out=None, mode='raise'): ...

def choose(indices, choices, out=None, mode="raise"): ...

jorenham · 2025-08-11T19:24:57Z

numpy/ma/core.pyi

+def correlate(a, v, mode='valid', propagate_mask=True): ...
+def convolve(a, v, mode='full', propagate_mask=True): ...


Suggested change

def correlate(a, v, mode='valid', propagate_mask=True): ...

def convolve(a, v, mode='full', propagate_mask=True): ...

def correlate(a, v, mode="valid", propagate_mask=True): ...

def convolve(a, v, mode="full", propagate_mask=True): ...

jorenham · 2025-08-11T19:29:55Z

numpy/ma/extras.pyi

@@ -55,7 +55,7 @@ __all__ = [
    "vstack",
 ]

-def count_masked(arr, axis=...): ...
+def count_masked(arr, axis=None): ...
 def masked_all(shape, dtype=...): ...


Ruff PYI014 wouldn't accept this I think, which is pretty arbitrary if you ask me.

Suggested change

def masked_all(shape, dtype=...): ...

def masked_all(shape, dtype=float): ... # noqa: PYI014

jorenham · 2025-08-11T19:32:58Z

numpy/ma/mrecords.pyi

+    commentchar='#',
+    missingchar='',


Suggested change

commentchar='#',

missingchar='',

commentchar="#",

missingchar="",

TYP: Type default values in stubs in numpy/ma

8c33501

github-actions bot added the 41 - Static typing label Aug 7, 2025

jorenham mentioned this pull request Aug 7, 2025

port the numpy.ma typing improvements from NumPy numpy/numtype#456

Open

52 tasks

MarcoGorelli marked this pull request as ready for review August 8, 2025 07:34

MarcoGorelli added 2 commits August 8, 2025 10:44

argsort and reshape too

ffcd9b3

Merge remote-tracking branch 'upstream/main' into typ-defaults-ma

33d00db

jorenham added the component: numpy.ma masked arrays label Aug 8, 2025

MarcoGorelli added 3 commits August 11, 2025 15:18

Merge remote-tracking branch 'upstream/main' into typ-defaults-ma

78b5f8c

more defaults

b747100

include axis=-1

5cf6a5a

jorenham self-requested a review August 11, 2025 18:50

jorenham approved these changes Aug 11, 2025

View reviewed changes

MarcoGorelli added 2 commits August 12, 2025 10:19

apply class functions too

8edf465

include kwonly too

d446048

This was referenced Aug 12, 2025

support keyword-only params too yangdanny97/docs2types#6

Merged

MAINT: Use double quotes (ruff rule Q) (only on .pyi files) #29548

Open

use float as dtype default in masked_all

fa0d66b

	def fix_invalid(a, mask=..., copy=True, fill_value=None): ...
	def fix_invalid(a, mask=np.False_, copy=True, fill_value=None): ... # noqa: PYI014

	def make_mask(m, copy=False, shrink=True, dtype=...): ...
	def make_mask(m, copy=False, shrink=True, dtype=np.bool): ...

	def argsort(a, axis=..., kind=None, order=None, endwith=True, fill_value=None, *, stable=...): ...
	def argsort(a, axis=..., kind=None, order=None, endwith=True, fill_value=None, *, stable=None): ...

	def reshape(a, new_shape, order='C'): ...
	def reshape(a, new_shape, order="C"): ...

	def choose(indices, choices, out=None, mode='raise'): ...
	def choose(indices, choices, out=None, mode="raise"): ...

		def correlate(a, v, mode='valid', propagate_mask=True): ...
		def convolve(a, v, mode='full', propagate_mask=True): ...

	def masked_all(shape, dtype=...): ...
	def masked_all(shape, dtype=float): ... # noqa: PYI014

Uh oh!

TYP: Type default values in stubs in numpy/ma #29531

Are you sure you want to change the base?

TYP: Type default values in stubs in numpy/ma #29531

Conversation

MarcoGorelli commented Aug 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jorenham left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

TYP: Type default values in stubs in `numpy/ma` #29531

TYP: Type default values in stubs in `numpy/ma` #29531

MarcoGorelli commented Aug 7, 2025 •

edited

Loading