Skip to content

[DependencyInjection] Fix the priority order of compiler pass trait #20995

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jan 3, 2017
Merged

[DependencyInjection] Fix the priority order of compiler pass trait #20995

merged 1 commit into from
Jan 3, 2017

Conversation

francoispluchino
Copy link
Contributor

@francoispluchino francoispluchino commented Dec 20, 2016

Q A
Branch? 3.2
Bug fix? yes
New feature? no
BC breaks? no
Deprecations? no
Tests pass? yes
Fixed tickets #20332, #20993
License MIT
Doc PR ~

A regression has appeared between the version 3.1 and 3.2 on the compilation passes using a priority option (see #20332).

Indeed, the order is no longer preserved for service definitions with the same priority. This is caused by the SplPriorityQueue class that does not have a FIFO implementation (see this comment).

The PR #20993 fixes the problem but only for Forms, not for all compiler passes using the PriorityTaggedServiceTrait trait since 3.2.

@xabbuh
Copy link
Member

xabbuh commented Dec 20, 2016

Please add a test to prevent future regressions.

Copy link
Member

@nicolas-grekas nicolas-grekas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be nice with a test case :)

}
}

return iterator_to_array($queue, false);
if (count($services) > 0) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if ($services) has my pref, the count wrapper adds nothing

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok.

@francoispluchino
Copy link
Contributor Author

@xabbuh I do it right away.

@francoispluchino francoispluchino changed the title [FrameworkBundle] Fix the priority order of compiler pass trait [DependencyInjection] Fix the priority order of compiler pass trait Dec 20, 2016
@francoispluchino
Copy link
Contributor Author

@nicolas-grekas @xabbuh Test added.

@francoispluchino
Copy link
Contributor Author

The test does not work with SplPriorityQueue.

@nicolas-grekas
Copy link
Member

👍

1 similar comment
@xabbuh
Copy link
Member

xabbuh commented Dec 20, 2016

👍

@Koc
Copy link
Contributor

Koc commented Dec 20, 2016

maybe better also add note iside trait why SplPriorityQueue not used.

@javiereguiluz
Copy link
Member

Will this be added back for 4.0? As @francoispluchino said we're using this in other places and we're promoting it with a special trait.

@stof
Copy link
Member

stof commented Dec 20, 2016

@javiereguiluz the SplPriorityQueue does not give any additional feature compared to the implementation used here (and which was used almost everywhere in Symfony before adding the trait btw, which is why switching to SplPriorityQueue was a BC break due to unstable sorting). So there is no drawback keeping this code in 4.0 (except it could use array_merge(...$services) instead of call_user_func_array to be faster once PHP 5.5 does not need to be supported anymore)

@francoispluchino
Copy link
Contributor Author

@javiereguiluz See this commit.

I do not think we need to use SplPriorityQueue, see my comment.

@francoispluchino
Copy link
Contributor Author

@Koc It's done.

@javiereguiluz
Copy link
Member

Another minor comment: if the problem is that A and B passes have 100 priority and previously they were executed in A -> B order and now they are executed in B -> A order ... then the real problem is that they should have a different priority, right?

@francoispluchino
Copy link
Contributor Author

Yes and no. Let me explain, ideally, I agree with you, but in reality, when you use different Bundles, it happens very often (if not all the time) that no priority is defined in the services. Priority is defined only when it is really necessary.

I'm not saying it's a good practice or not, but it's just an observation.

However, there is some kind of implicit priority, namely the order of adding the bundles (and others, as explained in this comment). It is therefore necessary to keep this order for services with the same priority.

In addition, we can more easily debug a priority error, because from what I understood on the tracker of PHP, is that the SplPriorityQueue class destroys the order arbitrarily, and therefore unpredictable (see the last comment on PHP Bugs).

@HeahDude
Copy link
Contributor

👍

Status: Reviewed

@@ -24,24 +24,34 @@
/**
* Finds all services with the given tag name and order them by their priority.
*
* The order of additions must to be respected for services having the same priority,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo: must to be

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops! Fixed!

@javiereguiluz
Copy link
Member

javiereguiluz commented Dec 20, 2016

Update: I change my vote to "abstain". Thanks!

My vote is -1 for now.

Reasons:

1. As the author of this PR recognized, this reverts the new behavior in some part of Symfony ... but leaves the rest of Symfony using the new behavior.
2. I still think that the reason to revert the new behavior looks weak and fragile. You say that if you change the order of your bundles in the kernel, you may break the application. But we provided features like priorities to avoid those problems.

@francoispluchino
Copy link
Contributor Author

Why some tests of Form Component are marked incomplete, while I have corrected a one line phpdoc, and that the previous tests were ok?

@francoispluchino
Copy link
Contributor Author

@javiereguiluz Why? Your arguments?

@HeahDude
Copy link
Contributor

HeahDude commented Dec 20, 2016

  1. As the author of this PR recognized, this reverts the new behavior in some part of Symfony ... but leaves the rest of Symfony using the new behavior.
  2. I still think that the reason to revert the new behavior looks weak and fragile. You say that if you change the order of your bundles in the kernel, you may break the application. But we provided features like priorities to avoid those problems.
  1. This does not revert the behavior, as I understand it, it keeps the feature, allowing to handle priorities AND keeps the original order for identical priorities.

  2. Yes but this problem was already there and is actually solved by this feature.

Am I missing something?

@HeahDude
Copy link
Contributor

Why some tests of Form Component are marked incomplete, while I have corrected a one line phpdoc, and that the previous tests were ok?

@francoispluchino This has nothing to do with your PR, some tests are skipped in the form component depending of the installed version of ICU.

@francoispluchino
Copy link
Contributor Author

Yes, the addition of the new priority option is present, and in addition, we maintain order for services with the same priority.

@stof
Copy link
Member

stof commented Dec 20, 2016

@javiereguiluz there was no priority for form extensions in Symfony 3.1, and so bundles were not defining a priority, but the order of registration was preserved. This PR is about reverting a BC break where a project would stop working when updating to 3.2 because the order of extensions defined without priority would be different (and unknown btw, making things hard to debug as it could change over time)

@nicolas-grekas
Copy link
Member

@javiereguiluz please reconsider. Stof is right.

@GuilhemN
Copy link
Contributor

👍, this should also fix FriendsOfSymfony/FOSRestBundle#1602.

To complete the other arguments, even inside a bundle, someone might rely on the order of his services definition (when using normalizers for instance):

services:
    foo:
        // ...
        tags: [ serializer.normalizer ]

    bar:
        // ...
        tags: [ serializer.normalizer ]

foo would be expected to be called before bar.

@francoispluchino
Copy link
Contributor Author

The most disturbing, is that the behavior of the SplPriorityQueue class is not really intuitive: the first item will be emitted first, and then the remaining items in reverse order of when enqueued.

2 tests to illustrate this affirmation:

class SplPriorityQueueTest extends \PHPUnit_Framework_TestCase
{
    public function testSinglePriority()
    {
        $queue = new \SplPriorityQueue();
        $values = array();

        $queue->insert('foo', 1000);// 0
        $queue->insert('bar', 1000);// 1
        $queue->insert('baz', 1000);// 2
        $queue->insert('bat', 1000);// 3
        $queue->insert('tar', 1000);// 4

        foreach ($queue as $value) {
            $values[] = $value;
        }

        // The obtained result
        $this->assertSame(array(
            'foo',// 0
            'tar',// 4
            'bat',// 3
            'baz',// 2
            'bar',// 1
        ), $values);

        // The expected result
        $this->assertSame(array(
            'foo',// 0
            'bar',// 1
            'baz',// 2
            'bat',// 3
            'tar',// 4
        ), $values);
    }

    public function testSeveralPriorities()
    {
        $queue = new \SplPriorityQueue();
        $values = array();

        $queue->insert('foo', 1000);// 0
        $queue->insert('bar', 1000);// 1
        $queue->insert('baz', 1000);// 2
        $queue->insert('bat', 1000);// 3
        $queue->insert('tar', 2000);// 4

        foreach ($queue as $value) {
            $values[] = $value;
        }

        // The obtained result
        $this->assertSame(array(
            'tar',// 4
            'bar',// 1
            'bat',// 3
            'baz',// 2
            'foo',// 0
        ), $values);

        // The expected result
        $this->assertSame(array(
            'tar',// 4
            'foo',// 0
            'bar',// 1
            'baz',// 2
            'bat',// 3
        ), $values);
    }
}

@nicolas-grekas To answer of your comment, the result is same in PHP 5.6 and PHP 7.0.

And the PHP doc is not very detailed on this, there is only the description for the insert method: Inserts an element in the queue by sifting it up.

@dmaicher
Copy link
Contributor

I actually noticed this behaviour when working on the priorities for form type extensions and had some concerns regarding BC 😢

#19790 (comment)

@francoispluchino
Copy link
Contributor Author

There is other compiler passes that uses either SplPriorityQueue or the old method:

If we later simplify the code using the PriorityTaggedServiceTrait trait, we may encounter the same problem with the TwigLoaderPass class without this fix.

@javiereguiluz
Copy link
Member

Thanks for providing more examples and comments about this. I've removed my -1 vote and now I abstain.

@ro0NL
Copy link
Contributor

ro0NL commented Dec 20, 2016

https://bugs.php.net/bug.php?id=60926

Cool.. that's mine :) I also settled with arrays back then.

Good fix 👍

@nicolas-grekas nicolas-grekas added this to the 3.2 milestone Dec 26, 2016
@nicolas-grekas
Copy link
Member

Thank you @francoispluchino.

@nicolas-grekas nicolas-grekas merged commit aac9a7e into symfony:3.2 Jan 3, 2017
nicolas-grekas added a commit that referenced this pull request Jan 3, 2017
…ass trait (francoispluchino)

This PR was merged into the 3.2 branch.

Discussion
----------

[DependencyInjection] Fix the priority order of compiler pass trait

| Q             | A
| ------------- | ---
| Branch?       | 3.2
| Bug fix?      | yes
| New feature?  | no
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | #20332, #20993
| License       | MIT
| Doc PR        | ~

A regression has appeared between the version `3.1` and `3.2` on the compilation passes using a priority option (see #20332).

Indeed, the order is no longer preserved for service definitions with the same priority. This is caused by the `SplPriorityQueue` class that does not have a FIFO implementation (see this [comment](#20332 (comment))).

The PR #20993 fixes the problem but only for Forms, not for all compiler passes using the `PriorityTaggedServiceTrait` trait since `3.2`.

Commits
-------

aac9a7e Fix the priority order of compiler pass trait
@francoispluchino francoispluchino deleted the fix-compiler-pass-priority branch January 3, 2017 17:13
@fabpot fabpot mentioned this pull request Jan 12, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.