Skip to content

[1.16 regression] descriptor not working in protocol #19054

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
JukkaL opened this issue May 8, 2025 · 12 comments
Closed

[1.16 regression] descriptor not working in protocol #19054

JukkaL opened this issue May 8, 2025 · 12 comments
Labels
bug mypy got something wrong topic-descriptors Properties, class vs. instance attributes topic-protocols

Comments

@JukkaL
Copy link
Collaborator

JukkaL commented May 8, 2025

After #18831 this example started generating an error, which looks like a regression:

import typing

A = typing.TypeVar('A')

class Attribute(typing.Generic[A]):
    @typing.overload
    def __get__(self, instance: None, owner: typing.Any) -> Attribute[A]: ...
    @typing.overload
    def __get__(self, instance: object, owner: typing.Any) -> A: ...
    def __get__(self, instance, owner): ...

class PathArg(typing.Protocol):
    path: Attribute[str]

def f(arg: PathArg) -> None:
    # No error expected
    s: str = arg.path  # error: Expression has type "Attribute[str]", variable has type "str'

If PathArg is a regular class, there is no error.

cc @ilevkivskyi as the author of the PR

@JukkaL JukkaL added bug mypy got something wrong priority-0-high labels May 8, 2025
@ilevkivskyi
Copy link
Member

This was an intentional change, to make descriptor access more consistent, see this PR comment for more motivation #18831 (comment). IMO the code in the example is a bug, such pattern should be reserved for when the actual attribute type is a descriptor, otherwise one should just use path: str in the protocol.

@JukkaL
Copy link
Collaborator Author

JukkaL commented May 8, 2025

Ok, let me play around with this a little. The impact to the Dropbox internal codebase (where I found this) is very minor, so if using path: str works, we can close this issue.

@JukkaL
Copy link
Collaborator Author

JukkaL commented May 9, 2025

I couldn't find a way to make a concrete class that uses the descriptor to be compatible with the protocol:

import typing

A = typing.TypeVar('A')

class Attribute(typing.Generic[A]):
    @typing.overload
    def __get__(self, instance: None, owner: typing.Any) -> Attribute[A]: ...
    @typing.overload
    def __get__(self, instance: object, owner: typing.Any) -> A: ...
    def __get__(self, instance, owner): ...

class PathArg(typing.Protocol):
    @property  # Attempt to use @property since the descriptor is read only
    def path(self) -> str: ...

class C:
    path = Attribute[str]()

def f(arg: PathArg) -> None:
    s: str = arg.path

def g(c: C) -> None:
    reveal_type(c.path)  # "str", as expected
    f(c)  # error: C is not compatible with PathArg <<-- this should be fine?

Is there a way to modify the old example so that it works using the new semantics?

@JukkaL
Copy link
Collaborator Author

JukkaL commented May 9, 2025

Another use case where the new behavior may cause issues is when using type[PathArg] -- here path: Attribute[str] in the protocol makes a difference:

import typing

A = typing.TypeVar('A')

class Attribute(typing.Generic[A]):
    @typing.overload
    def __get__(self, instance: None, owner: typing.Any) -> Attribute[A]: ...
    @typing.overload
    def __get__(self, instance: object, owner: typing.Any) -> A: ...
    def __get__(self, instance, owner): ...

class PathArg(typing.Protocol):
    path: Attribute[str]   # no error with Attribute[str], but error when using str

class C:
    path = Attribute[str]()

def f(arg: type[PathArg]) -> None:
    reveal_type(arg.path)  # Attribute[str]

def g() -> None:
    f(C)  # error if using "path: str" in PathArg

@ilevkivskyi
Copy link
Member

OK, I understand what happened (i.e. why someone defined such weird protocol). The problem is that protocols never worked with descriptors, see #5481, so someone found this workaround, and now this workaround doesn't work anymore.

I think the best way forward is to just wait for the proper descriptor support added in #18943. I you are really worried about this, you can revert it in the release branch. But I wouldn't bother, because IMO this is really niche workaround.

@JukkaL
Copy link
Collaborator Author

JukkaL commented May 9, 2025

This only impacts two files in the Dropbox codebase, so it's not a big deal. As you said, the code is kind of questionable anyway, so this shouldn't block the release.

@eltoder
Copy link

eltoder commented Jun 9, 2025

This is a big issue for us. We have a system where we use dataclasses with a special descriptor field to define schemas. Since there's no common base class, we use a protocol to type "generic schema".

@ilevkivskyi
Copy link
Member

@eltoder I have two questions:

  • Did you try to play with current master?
  • Can you share some code for me to understand what exactly you are doing?

@eltoder
Copy link

eltoder commented Jun 10, 2025

@ilevkivskyi

from __future__ import annotations

# Framework code
from typing import Any, Protocol, overload

class DescInfo: ...

class Desc:
    @overload
    def __get__(self, instance: None, owner: Any) -> Desc: ...
    @overload
    def __get__(self, instance: object, owner: Any) -> Callable[[], DescInfo]: ...
    def __get__(self, instance, owner):
        if instance is None: return self
        return lambda: DescInfo()  # construct from self and instance

class HasDesc(Protocol):
    desc: Desc

def write_schema(schema: HasDesc) -> None:
    print(schema.desc())  # do something with DescInfo

# User code
import dataclasses

@dataclasses.dataclass
class MySchema:
    desc = Desc()
    name: str
    value: int

write_schema(MySchema("foo", 123))

@hauntsaninja hauntsaninja changed the title Regression: descriptor not working in protocol [1.16 regression] descriptor not working in protocol Jun 10, 2025
@ilevkivskyi
Copy link
Member

@eltoder Then you should use this instead

class HasDesc(Protocol):
    desc: Callable[[], DescInfo]

to correctly specify the interface. This works on current master. This means however that you may need to wait for the next release for this to be available. Anyway, I don't think we need to keep this open, the old behavior was wrong.

@ilevkivskyi ilevkivskyi closed this as not planned Won't fix, can't repro, duplicate, stale Jun 10, 2025
@eltoder
Copy link

eltoder commented Jun 10, 2025

@ilevkivskyi That's simply wrong. It does not work when using the type. For example:

ST = TypeVar("ST", bound=HasDesc)

def read_schema(cls: type[ST]) -> ST:
    print(cls.desc)  # use desc to read

TBH I don't see how the current behavior is defensible. It makes Protocols inconsistent with regular classes and it contradicts the runtime. Please fix.

@ilevkivskyi
Copy link
Member

@eltoder It is not for you to decide. Also please watch your tone.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug mypy got something wrong topic-descriptors Properties, class vs. instance attributes topic-protocols
Projects
None yet
Development

No branches or pull requests

4 participants