Skip to content

[PhpUnitBridge] testing for deprecations is not risky #21786

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 28, 2017

Conversation

xabbuh
Copy link
Member

@xabbuh xabbuh commented Feb 27, 2017

Q A
Branch? master
Bug fix? no
New feature? yes
BC breaks? no
Deprecations? no
Tests pass? yes
Fixed tickets
License MIT
Doc PR

@xabbuh xabbuh force-pushed the deprecations-not-risky branch from 864e459 to 044cc8f Compare February 27, 2017 20:04
@@ -172,6 +173,10 @@ public function addSkippedTest($test, \Exception $e, $time)
public function startTest($test)
{
if (-2 < $this->state && ($test instanceof \PHPUnit_Framework_TestCase || $test instanceof TestCase)) {
if (null !== $test->getTestResultObject()) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

getTestResultObject() can return null, for example, when the test case leads to a PHP warning. In this case, $test is an instance of PHPUnit\Framework\WarningTestCase which does not emit a test result object.

Copy link
Member

@nicolas-grekas nicolas-grekas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@nicolas-grekas nicolas-grekas added this to the 3.3 milestone Feb 28, 2017
@fabpot
Copy link
Member

fabpot commented Feb 28, 2017

Thank you @xabbuh.

@fabpot fabpot merged commit 044cc8f into symfony:master Feb 28, 2017
fabpot added a commit that referenced this pull request Feb 28, 2017
…(xabbuh)

This PR was merged into the 3.3-dev branch.

Discussion
----------

[PhpUnitBridge] testing for deprecations is not risky

| Q             | A
| ------------- | ---
| Branch?       | master
| Bug fix?      | no
| New feature?  | yes
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets |
| License       | MIT
| Doc PR        |

Commits
-------

044cc8f testing for deprecations is not risky
@stof
Copy link
Member

stof commented Feb 28, 2017

couldn't we ensure that checks for @expectedDeprecation are registered in the assertion count instead ? This would also allow getting the assertion count right.

@nicolas-grekas
Copy link
Member

Good idea! Is that the case for @expectedException? @xabbuh could you look at it again please?

@xabbuh xabbuh deleted the deprecations-not-risky branch February 28, 2017 15:58
@xabbuh
Copy link
Member Author

xabbuh commented Feb 28, 2017

It's the case for @expectedException. But that's hardcoded in the base TestCase class. Not sure how we could hook into that.

@nicolas-grekas
Copy link
Member

Trigger a dummy $test->assertTrue(true) to increase the counter?

@stof
Copy link
Member

stof commented Feb 28, 2017

better: use $test->addToAssertionCount($count)

@stof
Copy link
Member

stof commented Feb 28, 2017

and this even allows to add the proper number of assertions (one per expected deprecation)

@xabbuh
Copy link
Member Author

xabbuh commented Mar 1, 2017

see #21828

symfony-splitter pushed a commit to symfony/phpunit-bridge that referenced this pull request Mar 2, 2017
…tion counter (xabbuh)

This PR was merged into the 3.3-dev branch.

Discussion
----------

[PhpUnitBridge] include expected deprecations in assertion counter

| Q             | A
| ------------- | ---
| Branch?       | master
| Bug fix?      | no
| New feature?  | yes
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | symfony/symfony#21786 (comment)
| License       | MIT
| Doc PR        |

We still need to include the changes from #21786 as we cannot increment the number of assertions in the `startTest()` method (the PHPUnit test runner resets the counter after the listeners have been executed).

Commits
-------

cdcd5ae include expected deprecations in assertion counter
nicolas-grekas added a commit that referenced this pull request Mar 2, 2017
…tion counter (xabbuh)

This PR was merged into the 3.3-dev branch.

Discussion
----------

[PhpUnitBridge] include expected deprecations in assertion counter

| Q             | A
| ------------- | ---
| Branch?       | master
| Bug fix?      | no
| New feature?  | yes
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | #21786 (comment)
| License       | MIT
| Doc PR        |

We still need to include the changes from #21786 as we cannot increment the number of assertions in the `startTest()` method (the PHPUnit test runner resets the counter after the listeners have been executed).

Commits
-------

cdcd5ae include expected deprecations in assertion counter
@fabpot fabpot mentioned this pull request May 1, 2017
nicolas-grekas added a commit that referenced this pull request Jan 20, 2020
…nneeded code (fancyweb)

This PR was merged into the 3.4 branch.

Discussion
----------

[PhpUnitBridge][SymfonyTestsListenerTrait] Remove some unneeded code

| Q             | A
| ------------- | ---
| Branch?       | 3.4
| Bug fix?      | no
| New feature?  | no
| Deprecations? | no
| Tickets       | -
| License       | MIT
| Doc PR        | -

Risky errors when there are no assertions are added before the test end listeners are called (ie, before the code in endTest is executed) so forcing beStrictAboutTestsThatDoNotTestAnything to false when there is a expectedDeprecation annotation is enough.

If the goal is to reset the value to the original value, then I think we should not do it since we basically "lie" to the next listeners. Let's assume that when a test expect a deprecation, it can have 0 assertions. Also this flag is not used anymore by PHPUnit after we reset it.

Ref #21786 btw

Commits
-------

fb48bbc [PhpUnitBridge][SymfonyTestsListenerTrait] Remove some unneeded code
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants