Skip to content

Improve error handling for dataclass inheritance #13531

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Aug 27, 2022

Conversation

Michael0x2a
Copy link
Collaborator

@Michael0x2a Michael0x2a commented Aug 27, 2022

This pull request:

  1. Fixes Crash when overriding field as property/method on frozen derived class #8334. Overriding a dataclass attribute with a method or property now results in an error message, not a crash.

    (Overriding an attribute with a non-attribute at runtime will result in either inconsistent behavior or an exception, so I think unconditionally disallowing this is fine.)

  2. Makes mypy report an error if you try subclassing a frozen dataclass with a non-frozen one or vice versa. Attempting to do this subclassing at runtime will raise a TypeError.

This pull request:

1.  Fixes python#8334. Overriding a dataclass attribute with a method or
    property now results in an error message, not a crash.

    (Overriding an attribute with a non-attribute at runtime will
    result in either inconsistent behavior or an exception, so
    I think unconditionally disallowing this is fine.)

2.  Makes mypy report an error if you try subclassing a frozen
    dataclass with a non-frozen one or vice versa. Attempting to
    do this subclassing at runtime will raise a TypeError.
@Michael0x2a
Copy link
Collaborator Author

I'm not too enthused about the new error message for (1) -- it's a bit cryptic imo. I'm open to bikeshedding there.

@github-actions

This comment has been minimized.

@github-actions
Copy link
Contributor

According to mypy_primer, this change has no effect on the checked open source code. 🤖🎉

Copy link
Collaborator

@JukkaL JukkaL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR! Looks good. The error message is a little cryptic, but I couldn't quickly think of a clearly better one. We can update the error message later if somebody comes up with a better idea.

@JukkaL JukkaL merged commit 3f0ac21 into python:master Aug 27, 2022
Michael0x2a added a commit to Michael0x2a/mypy that referenced this pull request Aug 28, 2022
While working on python#13531, I noticed that DataclassTransformer's
`collect_attributes` method was doing basically this:

```python
all_attrs = []
known_attrs = set()
for stmt in current_class:
    attr = convert_stmt_to_dataclass_attr(stmt)
    all_attrs.append(attr)
    known_attrs.add(attr.name)

for info in current_class.mro[1:-1]:
    if info is not a dataclass:
        continue

    super_attrs = []
    for attr in info.dataclass_attributes:
        # ...snip...
        if attr.name not in known_attrs:
            super_attrs.append(attr)
            known_attrs.add(attr.name)
        elif all_attrs:
            for other_attr in all_attrs:
                if other_attr.name == attr.name:
                    all_attrs.remove(attr)
                    super_attrs.append(attr)
                    break
    all_attrs = super_attrs + all_attrs
    all_attrs.sort(key=lambda a: a.kw_only)

validate all_attrs
```

Constantly searching through and removing items from `all_attrs`,
then pre-pending the superclass attrs will result in worst-case
quadratic behavior in the edge case where subtype is overriding
every attribute defined in the supertype.

This edge case is admittedly pretty unlikely to happen, but I wanted
to clean up the code a bit by reversing the order in which we process
everything so we naturally record attrs in the correct order.

One quirk of the old implementation I found was that we do not sort
the attrs list and move kw-only attrs to the end when none of the
supertypes are dataclasses. I tried changing this logic so we
unconditionally sort the list, but this actually broke a few of our
tests for some reason. I didn't want to get too deep in the weeds,
so opted to preserve this behavior.
JukkaL pushed a commit that referenced this pull request Aug 31, 2022
While working on #13531, I noticed that DataclassTransformer's
`collect_attributes` method was doing basically this:

```python
all_attrs = []
known_attrs = set()
for stmt in current_class:
    attr = convert_stmt_to_dataclass_attr(stmt)
    all_attrs.append(attr)
    known_attrs.add(attr.name)

for info in current_class.mro[1:-1]:
    if info is not a dataclass:
        continue

    super_attrs = []
    for attr in info.dataclass_attributes:
        # ...snip...
        if attr.name not in known_attrs:
            super_attrs.append(attr)
            known_attrs.add(attr.name)
        elif all_attrs:
            for other_attr in all_attrs:
                if other_attr.name == attr.name:
                    all_attrs.remove(attr)
                    super_attrs.append(attr)
                    break
    all_attrs = super_attrs + all_attrs
    all_attrs.sort(key=lambda a: a.kw_only)

validate all_attrs
```

Constantly searching through and removing items from `all_attrs`,
then pre-pending the superclass attrs will result in worst-case
quadratic behavior in the edge case where subtype is overriding
every attribute defined in the supertype.

This edge case is admittedly pretty unlikely to happen, but I wanted
to clean up the code a bit by reversing the order in which we process
everything so we naturally record attrs in the correct order.

One quirk of the old implementation I found was that we do not sort
the attrs list and move kw-only attrs to the end when none of the
supertypes are dataclasses. I tried changing this logic so we
unconditionally sort the list, but this actually broke a few of our
tests for some reason. I didn't want to get too deep in the weeds,
so opted to preserve this behavior.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Crash when overriding field as property/method on frozen derived class
2 participants