Skip to content

Add and improve docblock typing for static analysis #40783

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 13 commits into from

Conversation

mdeboer
Copy link
Contributor

@mdeboer mdeboer commented Apr 12, 2021

Q A
Branch? 5.4
Bug fix? no
New feature? no
Deprecations? no
License MIT

I noticed that both PhpStorm and Psalm were both not inferring the correct return type for PassportInterface::getBadge(), instead they would type it as BadgeInterface|null. Essentially this is correct but if a not-null value is returned, it's always an instance of the type of $badgeFqcn.

In PhpStorm I at first fixed this with the following snippet in .phpstorm.meta.php but Psalm does not pick it up for some reason:

<?php

namespace PHPSTORM_META {
    use Symfony\Component\Security\Http\Authenticator\Passport\PassportInterface;

    override(PassportInterface::getBadge(), type(0));
}

So with this commit I propose to add the following docblock to PassportInterface::getBadge():

/**
 * @template T of BadgeInterface
 *
 * @param class-string<T> $badgeFqcn
 *
 * @return T|null
 */

This works with both PhpStorm and Psalm but if this causes issues for other IDEs, they can be prefixed with psalm-. Not sure what the policy is for docblocks in this case 😄

@mdeboer mdeboer requested review from chalasr and wouterj as code owners April 12, 2021 15:33
@mdeboer mdeboer changed the title Add return type for getBadge in PassportInterface [Security] Add return type for getBadge in PassportInterface Apr 12, 2021
derrabus
derrabus previously approved these changes Apr 12, 2021
@carsonbot carsonbot changed the title [Security] Add return type for getBadge in PassportInterface Add return type for getBadge in PassportInterface Apr 12, 2021
@fabpot
Copy link
Member

fabpot commented Apr 13, 2021

Sorry, but I don't think we want to merge this change. At least, not without having a discussion about using this new notation in Symfony (AFAIK, we have never used it anywhere else).

@nicolas-grekas nicolas-grekas added this to the 5.x milestone Apr 13, 2021
nicolas-grekas
nicolas-grekas previously approved these changes Apr 13, 2021
Copy link
Member

@nicolas-grekas nicolas-grekas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is worth the DX improvement this will provide thanks to autocompletion. Ppl keep adding local inline @var annotations in similar situations right now AFAIK. This should help save writing them.

@wouterj
Copy link
Member

wouterj commented Apr 13, 2021

I agree with @fabpot that this is something we'll have to discuss (and I'm 100% in favor of this btw). The Psalm team is maintaining https://github.com/psalm/psalm-plugin-symfony/tree/master/src/Stubs to add stub-files with these annotations for precisely this usage (in e.g. property accessor). I think it's much better to move all of these into the Symfony code, but I'm against having some of this in the code and some of this in the stubs.

@derrabus
Copy link
Member

Generics in PHPDoc have broad tooling support. PhpStorm, PHPStan and Psalm support this notation and it improves the DX a lot if we used it. As @wouterj has pointed out, people already build packages to deliver type information that we don't provide. I think it's better if we maintain the type information directly in our PHPDoc blocks.

@chalasr
Copy link
Member

chalasr commented Apr 13, 2021

I'm ok to do this as it is modern phpdoc. Hope we can remove it in a future PHP version though.

I think it's worth mentioning that this makes the contract more precise, which should be covered by our BC promise:
This method does not just guarantee to always return a BadgeInterface instance anymore, but an instance of the concrete implementation that is requested via method argument.

chalasr
chalasr previously approved these changes Apr 13, 2021
@fabpot
Copy link
Member

fabpot commented Apr 13, 2021

I'm ok-ish only if this is something "standard" that is understood by many tools (for some definition of many).

@mdeboer
Copy link
Contributor Author

mdeboer commented Apr 13, 2021

I'm ok-ish only if this is something "standard" that is understood by many tools (for some definition of many).

Like @derrabus mentioned, this has broad support in tooling and is not just for PhpStorm and Psalm.

I am glad to see so many reactions and to be honest, I completely forgot about the stubs repository @wouterj mentioned. I agree with his opinion in that having code in both a third-party stubs repository and in the Symfony code is not ideal.

As this piece of code is marked experimental, I suppose we could make the first steps of merging these annotations from the stubs into the Symfony code where it makes sense.

Thanks all for taking the time! 👍🏻

@nicolas-grekas
Copy link
Member

nicolas-grekas commented Apr 16, 2021

I personally would be OK to import all annotations that exist on the stubs.
Of course, we should import only the annotations that are supported by many tools (eg not psalm-specific annotations.)

@mdeboer would you mind updating this PR to add them all?

@dunglas
Copy link
Member

dunglas commented Apr 16, 2021

I was in the process of opening a similar PR for forms when @nicolas-grekas commented: 5.x...dunglas:feat/form-psalm-template.

Having such annotations will allow to not introduce "useless" code just to have autocompletion working properly such as proposed in #40799 (comment).

+1 on my side to have these annotations directly in the code base. It will increase the DX of Symfony for both PHPStorm and Psalm/PHPStan users.

@mdeboer
Copy link
Contributor Author

mdeboer commented Apr 17, 2021

I will update the PR to add all the relevant stubs that are not just for psalm (e.g. without psalm prefix or supported by others with psalm prefix).

@mdeboer mdeboer marked this pull request as draft April 17, 2021 18:49
@mdeboer
Copy link
Contributor Author

mdeboer commented May 10, 2021

So I made some fixes, hopefully this ticks all the boxes. Unfortunately fabbot complains about a comment about the return value in Symfony\Component\Form\Extension\Core\DataTransformer::transform, wanting to replace /** @var string[] */ with for example /* @var string[] */. Would it help to make it a multi-line comment?

I haven't been able to set up PhpStorm for this project properly yet I reckon but I really haven't got much time at the moment to deep dive into getting all this set up properly.

@@ -42,6 +44,7 @@ public function transform($dateTimeZone)
throw new TransformationFailedException('Expected an array of \DateTimeZone objects.');
}

/** @var string[] */
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the reason why fabbot does not like this is that this is not a valid inline phpdoc variable declaration: it does not have the variable name (which is kind of expected, as this is not applied on a variable declaration). This should be removed IMO

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a bit of a slippery slope about how far to go to help type hint things and please static analysers. This was one of those things.

I'll remove it 👍🏻

@OskarStark
Copy link
Contributor

Are you able to finish this PR? Thanks

@zmitic
Copy link

zmitic commented Aug 15, 2021

@mdeboer
Copy link
Contributor Author

mdeboer commented Aug 16, 2021

Are you able to finish this PR? Thanks

Sorry, really busy weeks. I might give this another go at the end of this week as things probably slow down.

@mdeboer There is another file for PostSubmit event: https://github.com/psalm/psalm-plugin-symfony/blob/master/src/Stubs/common/forms/PostSubmitEvent.stubphp

Okay, probably missed it. I will see if that was intentional or not and add it accordingly 👍🏻

@stof
Copy link
Member

stof commented Oct 13, 2021

I don't think that those generic annotations should be added in the form component. Most of my review comments about types being wrong relate to the fact that the Form component is very flexible, which makes it very hard to define type definitions for that (adding a data transformer changes the view data or the model data for that form type for instance)

@zmitic
Copy link

zmitic commented Oct 13, 2021

@stof

I don't think that those generic annotations should be added in the form component.

If unused, they would still work as before (mixed). But adding them prevents lots of errors and @var annotations.

The only (somewhat) drawback is the need for this:

$form = $this->createForm(UserType::class);   //   must use 'empty_data' callable to populate

// submit, handle...
if ($user = $form->getData()) {    // must be this way because Form::getData() returns nullable T

    // here psalm **knows** it is instance of User
    $this->dispatch(
        new SomeUserMessage($user);
    );
}

Message:

class SomeUserMessage
{
    private string $id;

    public function __construct(User $user)
    {
        $this->id = $user->getId();
    }
}

@nicolas-grekas
Copy link
Member

@mdeboer the code base evolved a lot 😬 Can you please rebase?

@stof
Copy link
Member

stof commented Oct 27, 2021

@zmitic the issue is that you are not adding generic types only for the Form instance, but for many parts of the components, and those don't match the behavior of the component.

A Form instance has 3 types associated to it (the model data, the normalized data and the view data).
The various data transformers are about converting between those, but they are not DataTransformerInterface<TModel, TNormalized>. They are a chain of transformers being composed, when the input of one of them is the output of the previous one.
Properly defining generic types for the form building API is probably impossible, as the FormBuilder API is about mutating an existing object, and so adding transformers is changing the data types of that instance (which is something that type systems cannot represent AFAIK), and event listeners added on it are expected to use the final data types anyway (even if they are configured before the transformers).

Having wrong generic types will be worse for static analyzers than not having them at all.

@fabpot fabpot modified the milestones: 5.4, 6.1 Oct 30, 2021
@fabpot
Copy link
Member

fabpot commented Dec 11, 2021

/cc @derrabus

@mdeboer
Copy link
Contributor Author

mdeboer commented Dec 13, 2021

Thing is I have very, very limited time and in regard of the Form stuff, I never ever touched it so I really have no clue with what is the correct way to proceed there.

This only really started as I noticed that the PassportInterface was lacking docblocks with generics. Though I'd love to be able to help, I simply cannot afford to put work aside and focus on this, specially with the holidays and the on-going crisis. Trying hard to not end up in a burn-out.

If anyone would like repo access, please let me know and I'll grant you access. Also feel free to submit pull requests, I'd be happy to merge them so they end up in this PR.

Take care all and happy holidays!

@derrabus
Copy link
Member

@mdeboer No worries. I intend to take over this PR for 6.1 because I am probably the one who caused most of the conflicts. 🙂

@fabpot
Copy link
Member

fabpot commented Mar 26, 2022

@derrabus Friendly ping here :)

@wouterj
Copy link
Member

wouterj commented Jul 21, 2022

Hi @mdeboer! I tried to reignite this pull request, but unfortunately there are just too many conflicts with version 6.2 (given we've added lots of native types and PHPdoc for iterator generics in 5.4/6.0, most of the changes in this PR conflict).

So I've decided that it is best to start all over in a new PR: #47016 I've kept your original commit (adding a template to passport) and the discussions in this PR were very useful in order to judge which changes are "quick wins" and which need a more thorough review (the new PR only contains the quick wins).

I hope you can understand and hopefully we'll see more contributions from you in the future. We've now learned the lesson (once again) that big PRs are very hard to merge... :(

@wouterj wouterj closed this Jul 21, 2022
@mdeboer
Copy link
Contributor Author

mdeboer commented Jul 21, 2022

No worries @wouterj I completely understand! I am glad it was of use 😁 I appreciate your follow up.

nicolas-grekas added a commit that referenced this pull request Aug 2, 2022
…PHPstan and Psalm (mdeboer, wouterj)

This PR was merged into the 6.2 branch.

Discussion
----------

Improve some PHPdocs based on existing Symfony stubs in PHPstan and Psalm

| Q             | A
| ------------- | ---
| Branch?       | 6.2
| Bug fix?      | no
| New feature?  | no
| Deprecations? | no (strictly spoken yes, as I guess DebugClassLoader will trigger deprecations for some now?)
| Tickets       | Replaces #40783
| License       | MIT
| Doc PR        | symfony/symfony-docs#17064

Todo
---

* [x] Review the Psalm check of this PR
* [x] Test the changes with DebugClassLoader in a real app

Description
---

This PR adds some more information to the PHPdoc of Symfony classes. By moving the information from the current stubs of static analyzers into Symfony itself: (1) we can improve the experience for all users, (2) reduce false-positives from our own Psalm check and (3) make sure they are in sync with changes done on these classes.

<s>To avoid a PR that is too big to review and maintain, the scope of this PR is deliberately kept narrow:

* Only PHPdoc from either [Psalm's Symfony stubs](https://github.com/psalm/psalm-plugin-symfony/tree/master/src/Stubs) or [PHPStan's Symfony stubs](https://github.com/phpstan/phpstan-symfony/tree/1.2.x/stubs) is added
* The PHPdoc MUST NOT break PHPstorm
* The PHPdoc MUST be supported by both PHPstan and Psalm (those are the only two static analyzers that currently ship an official Symfony plugin afaik)
* The PHPdoc SHOULD NOT duplicate anything that can already be deduced from either PHP's type declarations or already existing PHPdoc.
* The PHPdoc MUST document the contract and NOT be added to fit a 95% use-case or improve auto-completion.</s>

EDIT: Replaced the guidelines by symfony/symfony-docs#17064

On top of this, to get this PR approved quicker, I've left out all stubs from the Form (based on the discussions in #40783) and most of [PHPStan's `Envelope` stub](https://github.com/psalm/psalm-plugin-symfony/blob/master/src/Stubs/common/Component/Messenger/Envelope.stubphp) (due to complexity of the PHPdoc).

Commits
-------

f5a802a [DependencyInjection] Add nice exception when using non-scalar parameter as array key
fa7703b [ErrorHandler] Do not patch return types that are not a valid PHP type
38ab6de Improve some PHPdocs based on existing Symfony stubs in PHPstan and Psalm
PhilETaylor pushed a commit to PhilETaylor/symfony that referenced this pull request Sep 6, 2023
…ubs in PHPstan and Psalm (mdeboer, wouterj)

This PR was merged into the 6.2 branch.

Discussion
----------

Improve some PHPdocs based on existing Symfony stubs in PHPstan and Psalm

| Q             | A
| ------------- | ---
| Branch?       | 6.2
| Bug fix?      | no
| New feature?  | no
| Deprecations? | no (strictly spoken yes, as I guess DebugClassLoader will trigger deprecations for some now?)
| Tickets       | Replaces symfony#40783
| License       | MIT
| Doc PR        | symfony/symfony-docs#17064

Todo
---

* [x] Review the Psalm check of this PR
* [x] Test the changes with DebugClassLoader in a real app

Description
---

This PR adds some more information to the PHPdoc of Symfony classes. By moving the information from the current stubs of static analyzers into Symfony itself: (1) we can improve the experience for all users, (2) reduce false-positives from our own Psalm check and (3) make sure they are in sync with changes done on these classes.

<s>To avoid a PR that is too big to review and maintain, the scope of this PR is deliberately kept narrow:

* Only PHPdoc from either [Psalm's Symfony stubs](https://github.com/psalm/psalm-plugin-symfony/tree/master/src/Stubs) or [PHPStan's Symfony stubs](https://github.com/phpstan/phpstan-symfony/tree/1.2.x/stubs) is added
* The PHPdoc MUST NOT break PHPstorm
* The PHPdoc MUST be supported by both PHPstan and Psalm (those are the only two static analyzers that currently ship an official Symfony plugin afaik)
* The PHPdoc SHOULD NOT duplicate anything that can already be deduced from either PHP's type declarations or already existing PHPdoc.
* The PHPdoc MUST document the contract and NOT be added to fit a 95% use-case or improve auto-completion.</s>

EDIT: Replaced the guidelines by symfony/symfony-docs#17064

On top of this, to get this PR approved quicker, I've left out all stubs from the Form (based on the discussions in symfony#40783) and most of [PHPStan's `Envelope` stub](https://github.com/psalm/psalm-plugin-symfony/blob/master/src/Stubs/common/Component/Messenger/Envelope.stubphp) (due to complexity of the PHPdoc).

Commits
-------

f5a802a [DependencyInjection] Add nice exception when using non-scalar parameter as array key
fa7703b [ErrorHandler] Do not patch return types that are not a valid PHP type
38ab6de Improve some PHPdocs based on existing Symfony stubs in PHPstan and Psalm
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.