Skip to content

ENH: A subset of "numpy.typing" type hints remain unusable at runtime #22352

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
leycec opened this issue Sep 29, 2022 · 2 comments · Fixed by #22357
Closed

ENH: A subset of "numpy.typing" type hints remain unusable at runtime #22352

leycec opened this issue Sep 29, 2022 · 2 comments · Fixed by #22357
Labels
01 - Enhancement 41 - Static typing

Comments

@leycec
Copy link

leycec commented Sep 29, 2022 •
edited
Loading

Describe the issue:

A few of the public type hints exported by the numpy.typing subpackage are currently unusable at runtime, due to being either non-runtime-checkable protocols (i.e., PEP 544-compliant typing.Protocol subclasses not decorated by @typing.runtime_checkable) or type hints subscripted by one or more non-runtime-checkable protocols. Both prevent runtime-type checkers (like @beartype and typeguard) from supporting those hints.

This includes:

  • The core numpy.typing.ArrayLike type hint, which internally requires either the private non-runtime-checkable numpy.typing._array_like._SupportsArray or numpy._typing._nested_sequence._NestedSequence protocol (depending on NumPy version).
  • The core numpy.typing.DTypeLike type hints, which likewise internally required the private non-runtime-checkable numpy.typing._dtype_like._SupportsDType protocol under a prior NumPy version. I'm kinda unclear whether the current numpy.typing.DTypeLike implementation is similarly encumbered. It might be – or I might just be hitting an obscure edge case in @beartype. More on that below!

The one notable exception is numpy.typing.NDArray[...], which thankfully is usable at runtime. Thanks to this, @beartype has explicitly supported numpy.typing.NDArray[...] for over a year. The @beartype userbase, which is mostly data scientists and machine learning gurus, is grateful. This is probably a useful moment to admit that I maintain @beartype. 😅

@beartype users are currently complaining about both of the above here and here. I'd like to mollify their distress. Also, I'd like to actually use these wonderful things myself. They're awesome!

All Good Things Begin with Arrays

numpy.typing.ArrayLike is unambiguously unusable at runtime, so that seems like a reasonable place to start. Admittedly, the following snippet requires the third-party @beartype runtime type-checker (which is not great). Still, that's probably the simplest way to exhibit this issue (which is great).

Consider this the runtime equivalent of a mypy error, which it kinda is:

>>> from beartype import beartype
>>> from numpy.typing import ArrayLike
>>> @beartype
... def data_science_or_bust(array: ArrayLike) -> int:
...     return len(array)
Traceback (most recent call last):
  File "/home/leycec/py/beartype/beartype/_util/cls/pep/utilpep3119.py", line 124, in die_unless_type_isinstanceable
    isinstance(None, cls)  # type: ignore[arg-type]
  File "/usr/lib/python3.10/typing.py", line 1498, in __instancecheck__
    raise TypeError("Instance and class checks can only be used with"
TypeError: Instance and class checks can only be used with @runtime_checkable protocols

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/leycec/tmp/mopy.py", line 7, in <module>
    def data_science_or_bust(array: ArrayLike) -> int:
  File "/home/leycec/py/beartype/beartype/_decor/_cache/cachedecor.py", line 77, in beartype
    return beartype_object(obj, conf)
  File "/home/leycec/py/beartype/beartype/_decor/decorcore.py", line 239, in beartype_object
    return _beartype_func(  # type: ignore[return-value]
  File "/home/leycec/py/beartype/beartype/_decor/decorcore.py", line 579, in _beartype_func
    func_wrapper_code = generate_code(bear_call)
  File "/home/leycec/py/beartype/beartype/_decor/_wrapper/wrappermain.py", line 215, in generate_code
    code_check_params = _code_check_args(bear_call)
  File "/home/leycec/py/beartype/beartype/_decor/_wrapper/wrappermain.py", line 475, in _code_check_args
    reraise_exception_placeholder(
  File "/home/leycec/py/beartype/beartype/_util/error/utilerror.py", line 212, in reraise_exception_placeholder
    raise exception.with_traceback(exception.__traceback__)
  File "/home/leycec/py/beartype/beartype/_decor/_wrapper/wrappermain.py", line 450, in _code_check_args
    ) = make_func_wrapper_code(hint)
  File "/home/leycec/py/beartype/beartype/_util/cache/utilcachecall.py", line 339, in _callable_cached
    raise exception
  File "/home/leycec/py/beartype/beartype/_util/cache/utilcachecall.py", line 331, in _callable_cached
    return_value = params_flat_to_return_value[params_flat] = func(
  File "/home/leycec/py/beartype/beartype/_decor/_wrapper/_wrappercode.py", line 71, in make_func_wrapper_code
    ) = make_check_expr(hint)
  File "/home/leycec/py/beartype/beartype/_util/cache/utilcachecall.py", line 339, in _callable_cached
    raise exception
  File "/home/leycec/py/beartype/beartype/_util/cache/utilcachecall.py", line 331, in _callable_cached
    return_value = params_flat_to_return_value[params_flat] = func(
  File "/home/leycec/py/beartype/beartype/_check/expr/exprmake.py", line 1712, in make_check_expr
    hint_curr_expr=add_func_scope_type(
  File "/home/leycec/py/beartype/beartype/_check/expr/_exprscope.py", line 196, in add_func_scope_type
    die_unless_type_isinstanceable(cls=cls, exception_prefix=exception_prefix)
  File "/home/leycec/py/beartype/beartype/_util/cls/pep/utilpep3119.py", line 165, in die_unless_type_isinstanceable
    raise exception_cls(exception_message) from exception
beartype.roar.BeartypeDecorHintPep3119Exception: @beartyped __main__.data_science_or_bust()
parameter "array" type hint <class 'numpy.typing._array_like._SupportsArray'> uncheckable at runtime
(i.e., not passable as second parameter to isinstance(), due to raising "Instance and class checks can
only be used with @runtime_checkable protocols" from metaclass __instancecheck__() method).

Here, @beartype is telling us that numpy.typing.ArrayLike internally requires the private non-runtime-checkable numpy.typing._array_like._SupportsArray protocol. In theory, that can be trivially resolved by just decorating numpy.typing._array_like._SupportsArray with @typing.runtime_checkable: e.g.,

# In "numpy.typing._array_like":
#
# Instead of just this...
class _SupportsArray(Protocol[_DType_co]):

# ...do this!
from typing import runtime_checkable
@runtime_checkable  # <-- yes, this is nonsensical boilerplate.
class _SupportsArray(Protocol[_DType_co]):

Yes, @typing.runtime_checkable is nonsensical boilerplate that has no adverse side effects whatsoever and absolutely should have just been applied unconditionally for all protocols. But PEP 544 authors disliked runtime at the time for "reasons" and now we're stuck with it. What you gonna do?

Ideally, similar boilerplate should be applied to all protocols declared throughout the numpy.typing subpackage. I feel your annoyance and raise my fist in solidarity.

Like, It's DTypeLike

numpy.typing.DTypeLike used to be unusable at runtime for similar reasons. But the underlying implementation appears to have significantly changed. I'm unclear exactly what the remaining issue is, but suspect this might be on @beartype's end: e.g.,

>>> from beartype import beartype
>>> from numpy.typing import DTypeLike
>>> @beartype
... def dtype_for_great_justice(dtype: DTypeLike) -> DTypeLike:
...     return dtype
Traceback (most recent call last):
  File "/home/leycec/py/beartype/beartype/_util/cls/pep/utilpep3119.py", line 124, in die_unless_type_isinstanceable
    isinstance(None, cls)  # type: ignore[arg-type]
TypeError: isinstance() argument 2 cannot be a parameterized generic

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/leycec/tmp/mopy.py", line 7, in <module>
    def dtype_for_great_justice(dtype: DTypeLike) -> DTypeLike:
  File "/home/leycec/py/beartype/beartype/_decor/_cache/cachedecor.py", line 77, in beartype
    return beartype_object(obj, conf)
  File "/home/leycec/py/beartype/beartype/_decor/decorcore.py", line 239, in beartype_object
    return _beartype_func(  # type: ignore[return-value]
  File "/home/leycec/py/beartype/beartype/_decor/decorcore.py", line 579, in _beartype_func
    func_wrapper_code = generate_code(bear_call)
  File "/home/leycec/py/beartype/beartype/_decor/_wrapper/wrappermain.py", line 215, in generate_code
    code_check_params = _code_check_args(bear_call)
  File "/home/leycec/py/beartype/beartype/_decor/_wrapper/wrappermain.py", line 475, in _code_check_args
    reraise_exception_placeholder(
  File "/home/leycec/py/beartype/beartype/_util/error/utilerror.py", line 212, in reraise_exception_placeholder
    raise exception.with_traceback(exception.__traceback__)
  File "/home/leycec/py/beartype/beartype/_decor/_wrapper/wrappermain.py", line 450, in _code_check_args
    ) = make_func_wrapper_code(hint)
  File "/home/leycec/py/beartype/beartype/_util/cache/utilcachecall.py", line 339, in _callable_cached
    raise exception
  File "/home/leycec/py/beartype/beartype/_util/cache/utilcachecall.py", line 331, in _callable_cached
    return_value = params_flat_to_return_value[params_flat] = func(
  File "/home/leycec/py/beartype/beartype/_decor/_wrapper/_wrappercode.py", line 71, in make_func_wrapper_code
    ) = make_check_expr(hint)
  File "/home/leycec/py/beartype/beartype/_util/cache/utilcachecall.py", line 339, in _callable_cached
    raise exception
  File "/home/leycec/py/beartype/beartype/_util/cache/utilcachecall.py", line 331, in _callable_cached
    return_value = params_flat_to_return_value[params_flat] = func(
  File "/home/leycec/py/beartype/beartype/_check/expr/exprmake.py", line 1203, in make_check_expr
    hint_curr_expr=add_func_scope_types(
  File "/home/leycec/py/beartype/beartype/_check/expr/_exprscope.py", line 342, in add_func_scope_types
    die_unless_hint_nonpep_type(
  File "/home/leycec/py/beartype/beartype/_util/hint/nonpep/utilnonpeptest.py", line 283, in die_unless_hint_nonpep_type
    die_unless_type_isinstanceable(
  File "/home/leycec/py/beartype/beartype/_util/cls/pep/utilpep3119.py", line 165, in die_unless_type_isinstanceable
    raise exception_cls(exception_message) from exception
beartype.roar.BeartypeDecorHintNonpepException: @beartyped __main__.dtype_for_great_justice()
parameter "dtype" type hint numpy.dtype[typing.Any] uncheckable at runtime (i.e., not passable as
second parameter to isinstance(), due to raising "isinstance() argument 2 cannot be a parameterized
generic" from metaclass __instancecheck__() method).

That is pure unadulterated chaos.

On the one hand, @beartype is fully compliant with PEP 484- and 585-style generics (both subscripted and unsubscripted) as well PEP 544-style protocols (both subscripted and unsubscripted). It's been a few months since we've had an issue submitted against either.

On the other hand, @beartype appears to be implying above that numpy.typing.DTypeLike reduces to numpy.dtype[Any] and that the metaclass of numpy.dtype[Any] defines __isinstancecheck__() to prohibit runtime checks. The exception message "isinstance() argument 2 cannot be a parameterized generic" sounds suspiciously like those raised by the standard PEP 585 superclass types.GenericAlias. If so, this is probably on @beartype – which now needs to additionally support numpy.dtype as a new PEP 585-like thing.

Is that right? If so, would it be sensible for @beartype to just quietly ignore the child type hint subscripting numpy.dtype[...] for the moment?

That's totally fine, of course. We'll happily do all that. Generics and protocols are a wicked darkness that ruthlessly squirm out of your test suite's grasp with every commit. It'd be great to nail that darkness down to the floor for a bit.

Glory Be to NumPy

I debated whether this was a bug or a feature request. I erred on the side of bug, as the unusability of NumPy functionality at runtime that could be trivially usable smells of buggishness. Please relabel this as feature request if I erred on the wrong side.

Thanks so much for all the tremendous volunteerism, breathtaking NumPy devs! You make the burgeoning Big Data world go round. 🥰

Reproduce the code example:

from beartype import beartype
from numpy.typing import ArrayLike
@beartype
def data_science_or_bust(array: ArrayLike) -> int:
    return len(array)

Error message:

Traceback (most recent call last):
  File "/home/leycec/py/beartype/beartype/_util/cls/pep/utilpep3119.py", line 124, in die_unless_type_isinstanceable
    isinstance(None, cls)  # type: ignore[arg-type]
  File "/usr/lib/python3.10/typing.py", line 1498, in __instancecheck__
    raise TypeError("Instance and class checks can only be used with"
TypeError: Instance and class checks can only be used with @runtime_checkable protocols

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/leycec/tmp/mopy.py", line 7, in <module>
    def data_science_or_bust(array: ArrayLike) -> int:
  File "/home/leycec/py/beartype/beartype/_decor/_cache/cachedecor.py", line 77, in beartype
    return beartype_object(obj, conf)
  File "/home/leycec/py/beartype/beartype/_decor/decorcore.py", line 239, in beartype_object
    return _beartype_func(  # type: ignore[return-value]
  File "/home/leycec/py/beartype/beartype/_decor/decorcore.py", line 579, in _beartype_func
    func_wrapper_code = generate_code(bear_call)
  File "/home/leycec/py/beartype/beartype/_decor/_wrapper/wrappermain.py", line 215, in generate_code
    code_check_params = _code_check_args(bear_call)
  File "/home/leycec/py/beartype/beartype/_decor/_wrapper/wrappermain.py", line 475, in _code_check_args
    reraise_exception_placeholder(
  File "/home/leycec/py/beartype/beartype/_util/error/utilerror.py", line 212, in reraise_exception_placeholder
    raise exception.with_traceback(exception.__traceback__)
  File "/home/leycec/py/beartype/beartype/_decor/_wrapper/wrappermain.py", line 450, in _code_check_args
    ) = make_func_wrapper_code(hint)
  File "/home/leycec/py/beartype/beartype/_util/cache/utilcachecall.py", line 339, in _callable_cached
    raise exception
  File "/home/leycec/py/beartype/beartype/_util/cache/utilcachecall.py", line 331, in _callable_cached
    return_value = params_flat_to_return_value[params_flat] = func(
  File "/home/leycec/py/beartype/beartype/_decor/_wrapper/_wrappercode.py", line 71, in make_func_wrapper_code
    ) = make_check_expr(hint)
  File "/home/leycec/py/beartype/beartype/_util/cache/utilcachecall.py", line 339, in _callable_cached
    raise exception
  File "/home/leycec/py/beartype/beartype/_util/cache/utilcachecall.py", line 331, in _callable_cached
    return_value = params_flat_to_return_value[params_flat] = func(
  File "/home/leycec/py/beartype/beartype/_check/expr/exprmake.py", line 1712, in make_check_expr
    hint_curr_expr=add_func_scope_type(
  File "/home/leycec/py/beartype/beartype/_check/expr/_exprscope.py", line 196, in add_func_scope_type
    die_unless_type_isinstanceable(cls=cls, exception_prefix=exception_prefix)
  File "/home/leycec/py/beartype/beartype/_util/cls/pep/utilpep3119.py", line 165, in die_unless_type_isinstanceable
    raise exception_cls(exception_message) from exception
beartype.roar.BeartypeDecorHintPep3119Exception: @beartyped __main__.data_science_or_bust()
parameter "array" type hint <class 'numpy.typing._array_like._SupportsArray'> uncheckable at runtime
(i.e., not passable as second parameter to isinstance(), due to raising "Instance and class checks can
only be used with @runtime_checkable protocols" from metaclass __instancecheck__() method).

NumPy/Python version information:

1.22.4 3.10.6 (main, Sep 6 2022, 17:16:18) [GCC 11.3.0]

Context for the issue:

This issue prevents various NumPy type hints from being used at runtime – especially by runtime type-checkers like @beartype and typeguard. Gah!
@BvB93 BvB93 changed the title BUG: A subset of "numpy.typing" type hints remain unusable at runtime ENH: A subset of "numpy.typing" type hints remain unusable at runtime Sep 30, 2022
@braniii
Copy link

braniii commented Sep 30, 2022

This would be really cool feature :)

@leycec did you try if simply adding @runtime_checkable is working?

@BvB93 BvB93 mentioned this issue Sep 30, 2022
Merged
@BvB93
Copy link
Member

BvB93 commented Sep 30, 2022

Thanks for the suggestion @leycec. I'll be honest: the lack of runtime checkable protocols in the numpy codebase is more of an oversight than anything else, and I agree that this would be a useful feature to have for runtime type checkers.

PR #22357 should implement the requested feature.

@charris charris mentioned this issue Oct 6, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
01 - Enhancement 41 - Static typing
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

TYP,ENH: Mark numpy.typing protocols as runtime checkable BvB93/numpy
4 participants
@leycec @mattip @braniii @BvB93