Skip to content

Fix type checking issue with model_fields and model_computed_fields #10911

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Nov 21, 2024

Conversation

sydney-runkle
Copy link
Contributor

Fix #10907

@github-actions github-actions bot added the relnotes-fix Used for bugfixes. label Nov 20, 2024
Copy link

cloudflare-workers-and-pages bot commented Nov 20, 2024

Deploying pydantic-docs with  Cloudflare Pages  Cloudflare Pages

Latest commit: 60522af
Status: ✅  Deploy successful!
Preview URL: https://88b7606a.pydantic-docs.pages.dev
Branch Preview URL: https://fix-type-checking-issue-mode.pydantic-docs.pages.dev

View logs

@sydney-runkle sydney-runkle added the topic-type checking Related to type checking label Nov 20, 2024
Copy link

codspeed-hq bot commented Nov 20, 2024

CodSpeed Performance Report

Merging #10911 will not alter performance

Comparing fix-type-checking-issue-model-fields (60522af) with main (b985365)

Summary

✅ 46 untouched benchmarks

Copy link
Contributor

github-actions bot commented Nov 20, 2024

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  pydantic
  main.py
Project Total  

This report was generated by python-coverage-comment-action

@Viicos
Copy link
Member

Viicos commented Nov 21, 2024

We are hitting undefined behavior between pyright and mypy here, when a property is defined both on the metaclass and the class itself. I think it's a bit unfortunate to have model_fields typed as a class variable, because it won't emit deprecation warnings here. Here is an alternative approach:

  • Until V3, we use the following descriptor defined on BaseModel (not the metaclass):

Code sample in pyright playground

import warnings
from typing import Callable, Generic, TypeVar, overload

from typing_extensions import deprecated

from pydantic.warnings import PydanticDeprecatedSince211

ModelT = TypeVar('ModelT', bound='BaseModel')
R = TypeVar('R')

class classonlyproperty(Generic[ModelT, R]):
    def __init__(self, fget: Callable[[type[ModelT]], R], /) -> None:
        self.fget = fget

    @overload
    def __get__(self, instance: None, owner: type[ModelT]) -> R: ...
    @overload
    @deprecated(
        "Accessing this attribute on the instance is deprecated, and will be removed in Pydantic V3.",
        category=None,
    )
    def __get__(self, instance: ModelT, owner: type[ModelT]) -> R: ...
    def __get__(self, instance: ModelT | None, owner: type[ModelT]) -> R:
        if instance is not None:
            warnings.warn(
                "Accessing this attribute on the instance is deprecated, and will be removed in Pydantic V3.",
                category=PydanticDeprecatedSince211,
            )
        return self.fget(owner)


class BaseModel:
    @classonlyproperty
    @classmethod
    def model_fields(cls) -> dict[str, str]:
        ...


BaseModel.model_fields
BaseModel().model_fields  # type checker error

pyright will emit an error, and mypy will probably be able to do so soon implements it in 1.14.

  • In V3, we remove the classonlyproperty helper and the model_fields from BaseModel to move it back to the model metaclass, as it is currently.

WDYT?

@sydney-runkle
Copy link
Contributor Author

@Viicos,

Interesting idea. 2 reservations I have about this:

  • The reason we included the model_fields property on BaseModel in the first place was for backwards compatibility - folks sometimes access model_fields on a BaseModel instance. Thus, I think decorating with the classonlyproperty is a bit misleading.

I sort of poorly explained this in a comment:

# TODO: V3 - remove `model_fields` and `model_computed_fields` properties from the `BaseModel` class - they should only
# be accessible on the model class, not on instances. We have these purely for backwards compatibility with Pydantic <v2.10.
  • Class properties aren't supported by Python, so I think this might be doing a bit more type hacking / jumping through hoops than we really should be doing?

@Viicos
Copy link
Member

Viicos commented Nov 21, 2024

  • Thus, I think decorating with the classonlyproperty is a bit misleading.

The naming of the decorator can be improved, but with this approach, accessing the property does work at runtime. However, it will emit a runtime deprecation warning and a type checker error/warning

  • Class properties aren't supported by Python, so I think this might be doing a bit more type hacking / jumping through hoops than we really should be doing?

Well the usage of the descriptor can feel a bit too much for such a small use case, but descriptor support is implemented by all major type checkers and this seems to be the only viable approach to emit a deprecation warning. I'm a bit worried that in V3 this will come as a surprise, because we currently don't emit anything (as an example, SQLModel relies on this)

@sydney-runkle
Copy link
Contributor Author

Yeah, fair point.

What if:

  • For v2.10, we go with the approach on this PR
  • For v2.11, we go with your approach (I don't want to introduce a deprecation warning in a patch release 😓)

@Viicos
Copy link
Member

Viicos commented Nov 21, 2024

Sounds good, could you please add some tests in the typechecking folder? Probably in base_model.py.

@sydney-runkle
Copy link
Contributor Author

Oh so odd, I think I have them locally but didn't push...

sydney-runkle and others added 2 commits November 21, 2024 09:30
Co-authored-by: Victorien <65306057+Viicos@users.noreply.github.com>
Copy link
Member

@Viicos Viicos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for adding the tests

@sydney-runkle sydney-runkle merged commit 9915abf into main Nov 21, 2024
54 checks passed
@sydney-runkle sydney-runkle deleted the fix-type-checking-issue-model-fields branch November 21, 2024 15:02
sydney-runkle added a commit that referenced this pull request Nov 22, 2024
…s` (#10911)

Co-authored-by: Victorien <65306057+Viicos@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
relnotes-fix Used for bugfixes. topic-type checking Related to type checking
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2.10: Type checking error on model_fields and model_computed_fields with mypy
2 participants