Skip to content

TypeVars in some generic functions can go unsolved, resulting in an Any in the evaluated type #7928

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
18 tasks done
erictraut opened this issue May 23, 2022 · 4 comments · Fixed by #8072
Closed
18 tasks done

TypeVars in some generic functions can go unsolved, resulting in an Any in the evaluated type #7928

erictraut opened this issue May 23, 2022 · 4 comments · Fixed by #8072
Labels
priority: high This issue is more important than most

Comments

@erictraut
Copy link
Contributor

erictraut commented May 23, 2022 •
edited by AlexWaygood
Loading

Pyright users have recently reported a couple of issues where they received an Unknown (i.e. an implicit Any) type when evaluating a call expression. These problems were both related to a case where a TypeVar was used to annotate a parameter with a default argument, but the caller didn't provide an argument for that parameter and relied on the default. This left the TypeVar unsolved.

Here's an example from the pyright issue tracker.

@JelleZijlstra suggested that pyright add a check for this condition. I've enhanced the reportInvalidTypeVarUse check to detect and report this case. See this issue for details.

The new check identifies 10 cases in the stdlib typeshed stubs and 27 cases in third-party stubs. Jelle asked that I create this issue with the full list, which I have pasted below.

<redacted>/typeshed/stdlib/argparse.pyi
  <redacted>/typeshed/stdlib/argparse.pyi:215:32 - error: Type variable "_ArgumentParserT" may go unsolved if caller supplies no argument for parameter "parser_class"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
<redacted>/typeshed/stdlib/fileinput.pyi
  <redacted>/typeshed/stdlib/fileinput.pyi:32:54 - error: Type variable "AnyStr" may go unsolved if caller supplies no argument for parameter "openhook"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
<redacted>/typeshed/stdlib/gettext.pyi
  <redacted>/typeshed/stdlib/gettext.pyi:157:22 - error: Type variable "_T" may go unsolved if caller supplies no argument for parameter "class_"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
<redacted>/typeshed/stdlib/optparse.pyi
  <redacted>/typeshed/stdlib/optparse.pyi:243:41 - error: Type variable "AnyStr" may go unsolved if caller supplies no argument for parameter "args"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
<redacted>/typeshed/stdlib/tempfile.pyi
  <redacted>/typeshed/stdlib/tempfile.pyi:387:13 - error: Type variable "AnyStr" may go unsolved if caller supplies no argument for parameter "suffix"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
  <redacted>/typeshed/stdlib/tempfile.pyi:392:21 - error: Type variable "AnyStr" may go unsolved if caller supplies no argument for parameter "suffix"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
  <redacted>/typeshed/stdlib/tempfile.pyi:396:20 - error: Type variable "AnyStr" may go unsolved if caller supplies no argument for parameter "suffix"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
<redacted>/typeshed/stdlib/multiprocessing/managers.pyi
  <redacted>/typeshed/stdlib/multiprocessing/managers.pyi:127:38 - error: Type variable "_KT" may go unsolved if caller supplies no argument for parameter "sequence"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
  <redacted>/typeshed/stdlib/multiprocessing/managers.pyi:127:43 - error: Type variable "_VT" may go unsolved if caller supplies no argument for parameter "sequence"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
  <redacted>/typeshed/stdlib/multiprocessing/managers.pyi:128:39 - error: Type variable "_T" may go unsolved if caller supplies no argument for parameter "sequence"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
<redacted>/typeshed/stubs/SQLAlchemy/sqlalchemy/orm/decl_api.pyi
  <redacted>/typeshed/stubs/SQLAlchemy/sqlalchemy/orm/decl_api.pyi:117:76 - error: Type variable "_DeclT" may go unsolved if caller supplies no argument for parameter "metaclass"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
  <redacted>/typeshed/stubs/SQLAlchemy/sqlalchemy/orm/decl_api.pyi:128:37 - error: Type variable "_DeclT" may go unsolved if caller supplies no argument for parameter "metaclass"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
<redacted>/typeshed/stubs/invoke/invoke/tasks.pyi
  <redacted>/typeshed/stubs/invoke/invoke/tasks.pyi:75:17 - error: Type variable "_TaskT" may go unsolved if caller supplies no argument for parameter "klass"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
<redacted>/typeshed/stubs/mock/mock/mock.pyi
  <redacted>/typeshed/stubs/mock/mock/mock.pyi:233:14 - error: Type variable "_T" may go unsolved if caller supplies no argument for parameter "new"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
<redacted>/typeshed/stubs/mypy-extensions/mypy_extensions.pyi
  <redacted>/typeshed/stubs/mypy-extensions/mypy_extensions.pyi:24:15 - error: Type variable "_T" may go unsolved if caller supplies no argument for parameter "type"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
  <redacted>/typeshed/stubs/mypy-extensions/mypy_extensions.pyi:25:22 - error: Type variable "_T" may go unsolved if caller supplies no argument for parameter "type"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
  <redacted>/typeshed/stubs/mypy-extensions/mypy_extensions.pyi:26:20 - error: Type variable "_T" may go unsolved if caller supplies no argument for parameter "type"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
  <redacted>/typeshed/stubs/mypy-extensions/mypy_extensions.pyi:27:27 - error: Type variable "_T" may go unsolved if caller supplies no argument for parameter "type"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
  <redacted>/typeshed/stubs/mypy-extensions/mypy_extensions.pyi:28:18 - error: Type variable "_T" may go unsolved if caller supplies no argument for parameter "type"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
  <redacted>/typeshed/stubs/mypy-extensions/mypy_extensions.pyi:29:17 - error: Type variable "_T" may go unsolved if caller supplies no argument for parameter "type"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
<redacted>/typeshed/stubs/psycopg2/psycopg2/_psycopg.pyi
  <redacted>/typeshed/stubs/psycopg2/psycopg2/_psycopg.pyi:409:62 - error: Type variable "_T_cur" may go unsolved if caller supplies no argument for parameter "cursor_factory"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
<redacted>/typeshed/stubs/redis/redis/commands/core.pyi
  <redacted>/typeshed/stubs/redis/redis/commands/core.pyi:644:47 - error: Type variable "_ScoreCastFuncReturn" may go unsolved if caller supplies no argument for parameter "score_cast_func"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
  <redacted>/typeshed/stubs/redis/redis/commands/core.pyi:651:47 - error: Type variable "_ScoreCastFuncReturn" may go unsolved if caller supplies no argument for parameter "score_cast_func"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
  <redacted>/typeshed/stubs/redis/redis/commands/core.pyi:682:47 - error: Type variable "_ScoreCastFuncReturn" may go unsolved if caller supplies no argument for parameter "score_cast_func"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
  <redacted>/typeshed/stubs/redis/redis/commands/core.pyi:689:47 - error: Type variable "_ScoreCastFuncReturn" may go unsolved if caller supplies no argument for parameter "score_cast_func"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
  <redacted>/typeshed/stubs/redis/redis/commands/core.pyi:859:47 - error: Type variable "_ScoreCastFuncReturn" may go unsolved if caller supplies no argument for parameter "score_cast_func"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
  <redacted>/typeshed/stubs/redis/redis/commands/core.pyi:874:47 - error: Type variable "_ScoreCastFuncReturn" may go unsolved if caller supplies no argument for parameter "score_cast_func"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
  <redacted>/typeshed/stubs/redis/redis/commands/core.pyi:901:47 - error: Type variable "_ScoreCastFuncReturn" may go unsolved if caller supplies no argument for parameter "score_cast_func"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
  <redacted>/typeshed/stubs/redis/redis/commands/core.pyi:935:47 - error: Type variable "_ScoreCastFuncReturn" may go unsolved if caller supplies no argument for parameter "score_cast_func"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
  <redacted>/typeshed/stubs/redis/redis/commands/core.pyi:958:47 - error: Type variable "_ScoreCastFuncReturn" may go unsolved if caller supplies no argument for parameter "score_cast_func"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
  <redacted>/typeshed/stubs/redis/redis/commands/core.pyi:1023:47 - error: Type variable "_ScoreCastFuncReturn" may go unsolved if caller supplies no argument for parameter "score_cast_func"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
  <redacted>/typeshed/stubs/redis/redis/commands/core.pyi:1038:47 - error: Type variable "_ScoreCastFuncReturn" may go unsolved if caller supplies no argument for parameter "score_cast_func"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
  <redacted>/typeshed/stubs/redis/redis/commands/core.pyi:1065:47 - error: Type variable "_ScoreCastFuncReturn" may go unsolved if caller supplies no argument for parameter "score_cast_func"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
  <redacted>/typeshed/stubs/redis/redis/commands/core.pyi:1099:47 - error: Type variable "_ScoreCastFuncReturn" may go unsolved if caller supplies no argument for parameter "score_cast_func"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
  <redacted>/typeshed/stubs/redis/redis/commands/core.pyi:1122:47 - error: Type variable "_ScoreCastFuncReturn" may go unsolved if caller supplies no argument for parameter "score_cast_func"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
<redacted>/typeshed/stubs/setuptools/pkg_resources/__init__.pyi
  <redacted>/typeshed/stubs/setuptools/pkg_resources/__init__.pyi:63:104 - error: Type variable "_T" may go unsolved if caller supplies no argument for parameter "installer"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
  <redacted>/typeshed/stubs/setuptools/pkg_resources/__init__.pyi:67:83 - error: Type variable "_T" may go unsolved if caller supplies no argument for parameter "installer"
    Provide an overload that specifies the return type when the argument is not supplied (reportInvalidTypeVarUse)
37 errors, 0 warnings, 0 informations
JelleZijlstra added a commit that referenced this issue May 23, 2022
@JelleZijlstra
argparse: remove incorrect default in overload
ed3e6b0 
Part of #7928
@JelleZijlstra JelleZijlstra mentioned this issue May 23, 2022
Merged
AlexWaygood pushed a commit that referenced this issue May 23, 2022
@JelleZijlstra
argparse: remove incorrect default in overload (#7929)
b4518bf 
Part of #7928
@srittau
Copy link
Collaborator

srittau commented May 23, 2022

Thanks, that's very helpful! We should enable that check when it's available and we've fixed the issues.

@JelleZijlstra
Copy link
Member

Thanks Eric! I added a checklist to your comment so we can track the remaining work.

@JelleZijlstra JelleZijlstra mentioned this issue May 24, 2022
Merged
JelleZijlstra added a commit to JelleZijlstra/typeshed that referenced this issue May 24, 2022
@JelleZijlstra
fileinput: Fix TypeVar usage
860cd4a 
Fixes python#7922, part of python#7928.

Also discovered an apparent CPython bug in the process: python/cpython#93157
@JelleZijlstra JelleZijlstra mentioned this issue May 24, 2022
Merged
JelleZijlstra added a commit to JelleZijlstra/typeshed that referenced this issue May 24, 2022
@JelleZijlstra
optparse: fix unconstrained TypeVar
e1b841e 
Part of python#7928
@JelleZijlstra JelleZijlstra mentioned this issue May 24, 2022
Merged
hauntsaninja pushed a commit that referenced this issue May 24, 2022
@JelleZijlstra
optparse: fix unconstrained TypeVar (#7936)
1582932 
Part of #7928
AlexWaygood pushed a commit that referenced this issue May 24, 2022
@JelleZijlstra
fileinput: Fix TypeVar usage (#7934)
d950ec3 
* fileinput: Fix TypeVar usage

Fixes #7922, part of #7928.
AlexWaygood added a commit that referenced this issue May 24, 2022
@AlexWaygood
mypy_extensions: fix TypeVar usage
9cafd54 
#7928
@AlexWaygood AlexWaygood mentioned this issue May 24, 2022
Merged
JelleZijlstra pushed a commit that referenced this issue May 24, 2022
@AlexWaygood
mypy_extensions: fix TypeVar usage (#7937)
54e11a1 
#7928
AlexWaygood added a commit that referenced this issue May 24, 2022
@AlexWaygood
multiprocessing.managers: fix TypeVar usage
31962f3 
#7928

`dict()` and `list()` just return empty dictionaries and lists (respectively) if no arguments are supplied:

```python
>>> from multiprocessing.managers import SyncManager
>>> with SyncManager() as s:
...     print(s.dict())
...
{}
```
This was referenced May 24, 2022
Merged
Merged
JelleZijlstra added a commit that referenced this issue May 25, 2022
@JelleZijlstra
pkg_resources: Fix unconstrained TypeVars
f5f341c 
https://github.com/pypa/setuptools/blob/499c468a57d240e5bb450bdb6daedc3e559541dd/pkg_resources/__init__.py#L1049

Part of #7928
@JelleZijlstra JelleZijlstra mentioned this issue May 25, 2022
Merged
JelleZijlstra added a commit that referenced this issue May 25, 2022
@JelleZijlstra
redis: Fix unconstrained TypeVars in a few redis commands
e71df36 
Part of #7928
@JelleZijlstra JelleZijlstra mentioned this issue May 25, 2022
Merged
JelleZijlstra added a commit that referenced this issue May 25, 2022
@JelleZijlstra
invoke: Fix unconstrained TypeVar
dbf8cb8 
Part of #7928 

https://github.com/pyinvoke/invoke/blob/f34c6c9413146a34ede3a5b529916b4cee649887/invoke/tasks.py#L328
@JelleZijlstra JelleZijlstra mentioned this issue May 25, 2022
Merged
JelleZijlstra added a commit that referenced this issue May 25, 2022
@JelleZijlstra
builtins: Fix unconstrained overloads in set() and frozenset()
0e50e8b 
See microsoft/pyright#3501 (comment)

Related to #7928
@JelleZijlstra JelleZijlstra mentioned this issue May 25, 2022
Merged
JelleZijlstra added a commit that referenced this issue May 25, 2022
@JelleZijlstra
mock: Fix unconstrained TypeVar
d4bca29 
The first overload covers the case where `new` is not given.

Part of #7928
@JelleZijlstra JelleZijlstra mentioned this issue May 25, 2022
Merged
hauntsaninja pushed a commit that referenced this issue May 25, 2022
@JelleZijlstra
pkg_resources: Fix unconstrained TypeVars (#7941)
bb11954 
https://github.com/pypa/setuptools/blob/499c468a57d240e5bb450bdb6daedc3e559541dd/pkg_resources/__init__.py#L1049

Part of #7928
AlexWaygood pushed a commit that referenced this issue May 25, 2022
@JelleZijlstra
invoke: Fix unconstrained TypeVar (#7943)
070e6c2 
Part of #7928 

https://github.com/pyinvoke/invoke/blob/f34c6c9413146a34ede3a5b529916b4cee649887/invoke/tasks.py#L328
AlexWaygood pushed a commit that referenced this issue May 25, 2022
@JelleZijlstra
mock: Fix unconstrained TypeVar (#7945)
796a689 
The first overload covers the case where `new` is not given.

Part of #7928
AlexWaygood pushed a commit that referenced this issue May 25, 2022
@JelleZijlstra
builtins, collections: Fix unconstrained overloads for container cons…
d5bc48d 
…tructors (#7944)

See microsoft/pyright#3501 (comment)

Related to #7928
JelleZijlstra pushed a commit that referenced this issue May 26, 2022
@AlexWaygood
multiprocessing.managers: fix TypeVar usage (#7938)
d511312 
#7928

`dict()` and `list()` just return empty dictionaries and lists (respectively) if no arguments are supplied:

```python
>>> from multiprocessing.managers import SyncManager
>>> with SyncManager() as s:
...     print(s.dict())
...
{}
```
AlexWaygood pushed a commit that referenced this issue May 26, 2022
@JelleZijlstra
redis: Fix unconstrained TypeVars in a few redis commands (#7942)
466f9c2 
Part of #7928
@JelleZijlstra JelleZijlstra mentioned this issue May 27, 2022
Merged
JelleZijlstra added a commit to JelleZijlstra/typeshed that referenced this issue May 27, 2022
@JelleZijlstra
psycopg2: stub improvements
a9aebbe 
Fixes an entry from python#7928 along with a number of other improvements.

I went off the C code:
https://github.com/psycopg/psycopg2/blob/master/psycopg/connection_type.c
JelleZijlstra added a commit that referenced this issue Jun 3, 2022
@JelleZijlstra
psycopg2: stub improvements (#7964)
cb90239 
Fixes an entry from #7928 along with a number of other improvements.

I went off the C code:
https://github.com/psycopg/psycopg2/blob/master/psycopg/connection_type.c
@AlexWaygood AlexWaygood added priority: high This issue is more important than most size-large labels Jun 12, 2022
This was referenced Jun 14, 2022
Merged
Merged
Merged
Merged
Merged
@AlexWaygood
Copy link
Member

Well that was a marathon, but we got there eventually :)

@erictraut
Copy link
Contributor Author

Well done!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
priority: high This issue is more important than most
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Upgrade pyright, improve pyright config files AlexWaygood/typeshed
4 participants
@srittau @JelleZijlstra @erictraut @AlexWaygood