Skip to content

TypeVar is not resolved correctly with None optional argument #8708

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
yhlam opened this issue Apr 22, 2020 · 7 comments
Closed

TypeVar is not resolved correctly with None optional argument #8708

yhlam opened this issue Apr 22, 2020 · 7 comments

Comments

@yhlam
Copy link

yhlam commented Apr 22, 2020

Running mypy --no-implicit-optional test.py with the code below gives the error Incompatible default for argument "default" (default has type "None", argument has type "D")

from typing import Sequence, TypeVar, Union

D = TypeVar("D")
E = TypeVar("E")


def get_first(values: Sequence[E], default: D = None) -> Union[E, D]:
    if len(values) == 0:
        return default
    else:
        return values[0]

I would expect mypy doesn't emit any error. The type of default should be resolved to None when the function is called without the optional argument, e.g. get_first([]), so that the return type is Union[E, None], which is equivalent to Optional[E].

I am using mypy 0.770 and python 3.7

@hauntsaninja
Copy link
Collaborator

hauntsaninja commented Apr 22, 2020

When running under --no-implicit-optional, default values for arguments must match the type of the argument. mypy doesn't solve for type variables based on the default value; type variables vary based on what types users of the function provide. This prevents mistakes like f(x: int = "asdf") and makes it more straightforward to understand the code inside the function since you don't have to second guess the type annotation.

I recommend using @overload to type this:

@overload
def get_first(values: Sequence[E], default: None = ...) -> Optional[E]: ...
@overload
def get_first(values: Sequence[E], default: D) -> Union[E, D]: ...

@yhlam
Copy link
Author

yhlam commented Apr 22, 2020

Thank you. @overload works. But if there are 2 or more optional arguments like this, it is annoying to list out all variations. I understand there would be an error for f(x: int = "asdf"). However, in my case, D is unbound TypeVar, which should act like Any when matching the default value in --no-implicit-optional mode. That seems to be a bug of mypy in my view.

@aderbenev-intermedia-com

I don't see how I can construct an @overload sentence if the first argument is also optional.

_LOG_LEVEL_TYPE = TypeVar('_LOG_LEVEL_TYPE', int, Optional[int])

def get_configured_log_level_int(
    config_param: str = 'log_level',
    default: _LOG_LEVEL_TYPE = None,
    section: Optional[Mapping[str, str]] = None,
) -> _LOG_LEVEL_TYPE:

@RonnyPfannschmidt
Copy link

im hitting this one as well even for type vars specific types

i'd really like to b able to use the resolved type of a argument in the output expression as opposed to overloads

an example where this would really be nice is for example

https://github.com/pytest-dev/iniconfig/blob/9cae43103df70bac6fde7b9f35ad11a9f1be0cb4/src/iniconfig/__init__.py#L35-L87

instead of spelling 4 overloads, i'd like to refer to resolved types more nicely
currently in the implementation i have to write something like

def get(  
        self,
        key: str,
        default: _D | None = None,
        convert: Callable[[str], _T] | None = None,
    ) -> _D | _T | str | None:
        return self.config.get(self.name, key, convert=convert, default=default)

it would be really great if something like

def get(  
        self,
        key: str,
        default: _D | None = None,
        convert: Callable[[str], _T] | None = None,
    ) -> ResolveOr[_T, str] | ResolveOr[ _D, None]:
        return self.config.get(self.name, key, convert=convert, default=default)

completely avoiding the overload in that case

@CarliJoy
Copy link

CarliJoy commented Sep 19, 2023

I see there are two bugs considering default arguments and TypeVars and especially TypeVars allowing None.

Considering the example below in which a TypeVar can be either a Logger (Protocol) or None.

It has two issues:

  • Default Arguments seem to require an Intersection of all possible values.
  • MyPy doesn't determine the correct TypeVar when given no argument -> that is actually really bad!
from typing import TypeVar, Generic, Self, Protocol, reveal_type


class LoggerProtocol(Protocol):
    def bind(self) -> Self: ...

class Logger:
    def bind(self) -> Self:
        return self


TLoggerOrNone = TypeVar("TLoggerOrNone", None, Logger)

class Bar(Generic[TLoggerOrNone]):
    def __init__(self, val: TLoggerOrNone = None):  # ❌ error: Incompatible default for argument "val" (default has type "None", argument has type "Logger")  [assignment]
        self._val: TLoggerOrNone = val
        
    @property
    def val(self) -> TLoggerOrNone:
        if self._val is None:
            return self._val
        return self._val.bind()
        

reveal_type(Bar().val) # ❌ Revealed type is "__main__.Logger"
reveal_type(Bar(Logger()).val) # ✅ Revealed type is "__main__.Logger"
reveal_type(Bar(None).val)  # ✅ Revealed type is "None"
reveal_type(Bar("aaa").val) # ✅ error: Value of type variable "TLoggerOrNone" of "Bar" cannot be "str"  [type-var]

MyPy Play

If we modify the example to use str instead of None, the problems remains

from typing import TypeVar, Generic, Self, Protocol, reveal_type


class LoggerProtocol(Protocol):
    def bind(self) -> Self: ...

class Logger:
    def bind(self) -> Self:
        return self


TLoggerOrStr = TypeVar("TLoggerOrStr", Logger, str)

class Bar(Generic[TLoggerOrStr]):
    def __init__(self, val: TLoggerOrStr = "abc"):  # ❌ Incompatible default for argument "val" (default has type "str", argument has type "Logger")  [assignment]
        self._val: TLoggerOrStr = val
        
    @property
    def val(self) -> TLoggerOrStr:
        if isinstance(self._val, str):
            return self._val
        return self._val.bind()
        

reveal_type(Bar().val) # ❌ Revealed type is "__main__.Logger"
reveal_type(Bar(Logger()).val) # ✅ Revealed type is "__main__.Logger"
reveal_type(Bar(None).val)  # ✅ error: Value of type variable "TLoggerOrStr" of "Bar" cannot be "None"  [type-var]
reveal_type(Bar("aaa").val) # ✅ revealed type is "builtins.str"

MyPy Play

Strange enough, if the order of the TypeVars are changed, i.e. TypeVar("TLoggerOrStr", str, Logger) than MyPy detects the Bar() type "correct".

I can't use overloads to define my __init__ as well my Generic class.

So long story short: I think the issue is not special to None in TypeVars rather to MyPy's wrong handling of default values when using TypeVars...

@erictraut
Copy link

Handling default values for parameters annotated with a TypeVar is quite complicated — both from an implementation and a type soundness perspective. Refer to this issue for more details.

@CarliJoy
Copy link

Thanks for pointing this out @erictraut .
Can someone close this issue as a duplicate of #3737 ?

@JelleZijlstra JelleZijlstra closed this as not planned Won't fix, can't repro, duplicate, stale Sep 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants