Skip to content

DEP: Deprecate numpy.typing.mypy_plugin: The sequel #28134

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jan 10, 2025

Conversation

jorenham
Copy link
Member

@jorenham jorenham commented Jan 9, 2025

This addresses the suggestions by @seberg in #28129, and adds a corresponding release note.

@seberg seberg merged commit dc78e30 into numpy:main Jan 10, 2025
67 checks passed
@seberg
Copy link
Member

seberg commented Jan 10, 2025

Actually, ping @BvB93, since I think you would be the one with insight into this being a bad idea :).

@jorenham jorenham deleted the typing/mypy-plugin-deprecation-episode-2 branch January 10, 2025 11:27
@BvB93
Copy link
Member

BvB93 commented Jan 10, 2025

So what's the current (runtime) status of platform-dependent bitsizes of the numeric dtypes?

Because while I fully agree that platform specificity is an annoyance, both for typing as well as runtime if I might add, as long as the reported types match their runtime equivalent it feels bit odd curious talk about "bad practice to have platform-dependent type annotations", as the reality of the situation here simply is platform dependent.

@seberg
Copy link
Member

seberg commented Jan 10, 2025

Yeah, the "solution" is to use a tuple union. Which is of course wrong for many use-casees. The other solution could be to have bit-equivalence. In a sense that might even be best (strictly correct!).
But I am not sure we'll make many friends with this either...

(The problem with the union is of course that an intp function says it accepts 32bit ints when it doesn't actually do it.)

I agree that this isn't strictly better, but am OK with it considering that this only works with mypy anyway.

@jorenham
Copy link
Member Author

So what's the current (runtime) status of platform-dependent bitsizes of the numeric dtypes?

Because while I fully agree that platform specificity is an annoyance, both for typing as well as runtime if I might add, as long as the reported types match their runtime equivalent it feels bit odd curious talk about "bad practice to have platform-dependent type annotations", as the reality of the situation here simply is platform dependent.

In case of static typing, platform-dependent types can also lead to platform-dependent typing errors, e.g. when

def f(x: np.intp): ...

is called with f(np.int64(42)), it will be reported as an error on 32-bit systems, but not on 64-bit systems. And even though there might be some rare cases where this could catch a bug (not that I can imagine any), it's very difficult to debug this easy-to-make mistake, leading to classic cases of "but it works on my machine".
There's also the issue that this will intentionally cause different outcomes between different type-checkers, while we're trying to achieve the exact opposite of that.

@BvB93
Copy link
Member

BvB93 commented Jan 10, 2025

Yeah, the "solution" is to use a tuple union. Which is of course wrong for many use-casees. The other solution could be to have bit-equivalence. In a sense that might even be best (strictly correct!).
But I am not sure we'll make many friends with this either...

I'd be a bit careful with unions, as you generally can't freely downcast them to a non-union subtype with something as heavy handed as typing.cast() or something akin to an isinstance() check. This is fine if you can reasonably demand of the user to perform one of aforementioned runtime checks, but if this is not the case (e.g. if the union is there because it's difficult to provide better annotation with the current tools) then this will just add additional busy work for the user in order to satisfy the type checker. The likes of python/typing#566 would help a lot in the latter case, but in the mean time typing.Any is generally better served.

foo: tuple[int | str, ...] = ("hello", "world")

# Incompatible types in assignment (expression has type "tuple[int | str, ...]", variable has type "tuple[str, ...]")
bar: tuple[str, ...] = foo

...is called with f(np.int64(42)), it will be reported as an error on 32-bit systems, but not on 64-bit systems. And even though there might be some rare cases where this could catch a bug (not that I can imagine any), it's very difficult to debug this easy-to-make mistake, leading to classic cases of "but it works on my machine".

Again, I don't really see how any of the provided arguments here are exclusive to static typing. Assuming that the function is annotated with something like np.intp for a good reason (rather than something more general like np.signedinteger), then runtime would yield these very same caveats and "but it works on my machine"-issues as static typing.

There's also the issue that this will intentionally cause different outcomes between different type-checkers, while we're trying to achieve the exact opposite of that.

Yeah, that's fair. While, IMO, this is acceptable for downstream, if this is actively getting in the way of expanding numpy's typing tests suite with more type checkers then that's a different story. Though note that, either way, it's not uncommon to stumble upon tiny type-checker-specific differences in general every now and then.

@jorenham
Copy link
Member Author

Again, I don't really see how any of the provided arguments here are exclusive to static typing.

At runtime you can do things like np.intp is np.int64 and hasattr(np, "float128"). But with static typing, that's not possible. So there's no way to handle platform-specific cases using static typing, and we should therefore remove the need to do so, and make all types platform-agnostic (which they already for those that aren't using the mypy plugin).


Also note that anyone can write a mypy plugin themselves in case they want platform-specific types. I just don't think it's a good idea that we're the ones that provide it.

I also don't think that the mypy plugin is compatible with the typing spec, which only allows for simple platform-specific checks that are limit to sys.version_info and sys.platform:
https://typing.readthedocs.io/en/latest/spec/directives.html#version-and-platform-checking

@jorenham
Copy link
Member Author

jorenham commented Jan 10, 2025

The typing spec also states the following:

If the runtime dynamically adds or removes elements (for example if certain functions are only available on some system configurations), include all possible elements in the stubs.

https://typing.readthedocs.io/en/latest/guides/writing_stubs.html#all

But the mypy plugin effectively removes exports such as np.float80 from the stubs on platforms that don't have it available at runtime.

So the plugin makes it impossible to do things like

import numpy as np
from typing import TypeIs

type FloatEx = np.float80 | np.float96 | np.float128 | np.float256

def is_extended_precision_float(x: np.floating, /) -> TypeIs[FloatEx]: ...

(on Python 3.13)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants