Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Alternate method of handling dict.get default value #13224

Closed
wants to merge 5 commits into from

Conversation

tungol
Copy link
Contributor

@tungol tungol commented Dec 8, 2024

Testing to see what this looks like compared to #13222

I also updated typing.Mapping.get() overloads to match dict. This may be disruptive. It was attempted in #10501 but backed out, so let's see if it's still a problem while we're here.

@tungol tungol marked this pull request as draft December 8, 2024 23:40

This comment has been minimized.

Copy link
Contributor

github-actions bot commented Dec 9, 2024

Diff from mypy_primer, showing the effect of this PR on open source code:

prefect (https://github.com/PrefectHQ/prefect)
+ src/prefect/locking/memory.py:194: note:     def get(self, Never, None, /) -> None
+ src/prefect/locking/memory.py:205: note:     def get(self, Never, None, /) -> None
+ src/prefect/records/memory.py:176: note:     def get(self, Never, None, /) -> None

steam.py (https://github.com/Gobot1234/steam.py)
+ steam/ext/commands/utils.py:43: note:          @overload
+ steam/ext/commands/utils.py:43: note:          def get(self, str, None, /) -> _VT | None

streamlit (https://github.com/streamlit/streamlit)
+ lib/streamlit/runtime/state/session_state.py: note: In member "get_id_from_key" of class "KeyIdMapper":
+ lib/streamlit/runtime/state/session_state.py:314:9: error: Returning Any from function declared to return "str"  [no-any-return]

operator (https://github.com/canonical/operator)
- ops/_private/harness.py:1193: error: Argument 2 to "get" of "dict" has incompatible type "None"; expected "Mapping[str, str]"  [arg-type]
+ ops/_private/harness.py:1193: error: Incompatible return value type (got "dict[str, str] | None", expected "Mapping[str, str]")  [return-value]

beartype (https://github.com/beartype/beartype)
+ beartype/_check/error/errcause.py:551: error: Incompatible types in assignment (expression has type "Callable[[ViolationCause], ViolationCause] | None", variable has type "Callable[[ViolationCause], ViolationCause]")  [assignment]
+ beartype/_check/error/errcause.py:552: error: Unused "type: ignore" comment  [unused-ignore]

mypy (https://github.com/python/mypy)
+ mypy/partially_defined.py:510: error: Never apply isinstance() to unexpanded types; use mypy.types.get_proper_type() first  [misc]
+ mypy/partially_defined.py:510: note: If you pass on the original type after the check, always use its unexpanded version

@tungol
Copy link
Contributor Author

tungol commented Dec 9, 2024

prefect and stream.py are no actual change. operator and beartype are both changing an existing error from an argument error to a return type error.

I dug into the streamlit error. It's something hinky. This produces a new error with this change:

from typing import Any

def get_id_from_key(key_id_mapping: dict[str, str], default: Any) -> str:
    return key_id_mapping.get("key", default)

But this version:

from typing import Any

def get_id_from_key(key_id_mapping: dict[str, str], default: Any) -> str:
    result = key_id_mapping.get("key", default)
    return result

Which assigns the return value to a variable before returning, causes mypy to generate Returning Any from function declared to return "str" both with and without this change.

@tungol
Copy link
Contributor Author

tungol commented Dec 9, 2024

Okay, what's going on there is that with the call in the return position, mypy uses the context of the return type being str while checking the overloads. overload 1 is discard as non-matching. Overload two is (str, _VT) -> _VT, which is (str, str) -> str. The current third overload is (str, _T) -> _T | _VT which the context infers to (str, str) -> str. That's the same for both main and this MR.

The new overload in this MR get (str, None) -> str | None. Mypy then runs a check to see if there's any ambiguity, and finds none for main, but does find ambiguity for this version.

The difference between here and assigning to result first is that in that case, mypy doesn't have the context to infer str for _T. We can give it that context with result: str = key_id_mapping.get("key", default) in which case there's no error for either main or this MR, not because the ambiguity went away but because there's no error for "Assigned Any to a typed variable" the way there is for "returning Any from function".

So that's all probably fine and sensible.

@tungol
Copy link
Contributor Author

tungol commented Dec 9, 2024

Short version of the mypy diff is that self.type_map.get(o.expr, None) evaluates to builtins.object before being passed to isinstance_proper_hook, while with this MR it evaluates to Union[mypy.types.Type, None] and the plugin considers mypy.types.Type to be improper.

I think this is similar to the streamlit error in that mypy uses context. For streamlit, it matched Any to the context of str normally, and here it's matching 'None' to the context of 'object' and skipping the union. We don't reach that with this MR applied - we match None to None first and don't reach the final branch of the overload. In a normal context, that's fine, and we'll match the union of Type and None down to object a little later, but the mypy plugin is checking before that happens in this case.

@tungol tungol closed this Dec 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant