Skip to content

[Messenger] Fix worker-only Doctrine middleware from running always #34066

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 22, 2019

Conversation

weaverryan
Copy link
Member

@weaverryan weaverryan commented Oct 22, 2019

Q A
Branch? 4.4 (or 4.3?, this is a bug fix)
Bug fix? yes
New feature? no
Deprecations? no
Tickets Fix #32436 Depends on #34069
License MIT
Doc PR not needed

Several Doctrine middleware are only meant to be run in a "worker" context: we want to "ping" the connection, "close" the connection and "clear" the entity manager ONLY when we are receiving messages. Before this PR, it was done always, which causes bad behavior for sync messages (imagine your Doctrine connection being closed in the middle of a controller or see #31334 (comment)).

This fixes that in a pragmatic way: no new system for "worker-only" middleware or anything like that: just make the middleware smart enough to only do their work when a message is being received async.

@B-Galati
Copy link
Contributor

@weaverryan Do you think the doc could be improved to make it more explicit how middlewares works with async transport/workers?

@weaverryan
Copy link
Member Author

Do you think the doc could be improved to make it more explicit how middlewares works with async transport/workers

Good idea :). See symfony/symfony-docs#12535

I'm going to update this PR a bit based on feedback from Nicolas, but the core idea will remain the same.

@weaverryan weaverryan force-pushed the fix-doctrine-middleware branch from cbeb3f9 to 290a729 Compare October 22, 2019 14:54
@weaverryan
Copy link
Member Author

weaverryan commented Oct 22, 2019

This now depends on #34069, but otherwise, is ready to go and is now much simpler.

I can't think of a downside to this - I can't think of why we would ever want to ping/close/clear except when inside a worker context. Ping @Koc

fabpot added a commit that referenced this pull request Oct 22, 2019
… with config trick (weaverryan)

This PR was merged into the 4.4 branch.

Discussion
----------

[Messenger] Removing "sync" transport and replacing it with config trick

| Q             | A
| ------------- | ---
| Branch?       | 4.4
| Bug fix?      | no (but removes some hackiness)
| New feature?  | no
| Deprecations? | yes (BC break yes)
| Tickets       | Helps #34066
| License       | MIT
| Doc PR        | not needed

I introduced `SyncTransport` in 4.3 as a way to allow people to, for example, route a message to an `async` transport and then override that to a `sync` transport in the dev environment. The implementation was always a bit of a hack, and makes this one transport behave different than others *and* makes messages routed to this transport behave a *bit* different than sync messages that are simply not routed at all.

Thanks for an idea from Nicolas, this fixes that. Gone are `SyncTransport`. Instead, the `sync://` transport string is still supported, but it's just a "config" trick: messages routed to transports that are "sync" are simply filtered out of the "senders" in the DI extension. Nice & clean.

Commits
-------

3d4e59a [Messenger] Removing "sync" transport and replacing it with much nicer config trick
@fabpot
Copy link
Member

fabpot commented Oct 22, 2019

Thank you @weaverryan.

fabpot added a commit that referenced this pull request Oct 22, 2019
…ng always (weaverryan)

This PR was merged into the 4.4 branch.

Discussion
----------

[Messenger] Fix worker-only Doctrine middleware from running always

| Q             | A
| ------------- | ---
| Branch?       | 4.4 (or 4.3?, this is a bug fix)
| Bug fix?      | yes
| New feature?  | no
| Deprecations? | no
| Tickets       | Fix #32436 Depends on #34069
| License       | MIT
| Doc PR        | not needed

Several Doctrine middleware are only meant to be run in a "worker" context: we want to "ping" the connection, "close" the connection and "clear" the entity manager ONLY when we are receiving messages. Before this PR, it was done always, which causes bad behavior for sync messages (imagine your Doctrine connection being closed in the middle of a controller or see #31334 (comment)).

This fixes that in a pragmatic way: no new system for "worker-only" middleware or anything like that: just make the middleware smart enough to only do their work when a message is being received async.

Commits
-------

290a729 Fixing issue where worker-only middleware were run in all contexts
@fabpot fabpot merged commit 290a729 into symfony:4.4 Oct 22, 2019
@weaverryan weaverryan deleted the fix-doctrine-middleware branch October 22, 2019 15:53
@nicolas-grekas nicolas-grekas modified the milestones: next, 4.4 Oct 27, 2019
Tobion added a commit that referenced this pull request Oct 31, 2019
…averryan)

This PR was squashed before being merged into the 4.4 branch (closes #34155).

Discussion
----------

Revert SyncTransport simplification and fix properly

| Q             | A
| ------------- | ---
| Branch?       | 4.4
| Bug fix?      | yes
| New feature?  | no
| Deprecations? | no
| Tickets       | Fix #34115 (and also related to #34066)
| License       | MIT
| Doc PR        | Not needed

In #34069, I made `SyncTransport` simpler by removing that transport class and making the whole things a config trick. I felt GREAT about that solution... until i realized two big problems:

1) It kills using env vars for `sync://` because we read the config values at build time - #34115 - that could probably be fixed by adding a factory, but then there is also the next problem

2) If someone routed a message to `[async, sync]` (weird, but allowed), my #34069 config solution basically maps this internally to `[async]`, which actually causes the message to *not* be handled immediately. Basically, my solution only worked if you route a message ONLY to one sync transport, but fails if you route to multiple transports.

So... this fixes things in a less-cool, but sensible way:

A) The first commit reverts #34069 exactly
B) The second commit solves the issue that we need to know if a message is being handled in a "worker" context or not, so middleware can decide if they should reset things before/after handling things. Previously we were using `ReceivedStamp` to know this. But because `SyncTransport` also "receives" the message and adds this stamp, it's not enough. To fix this, I added a new `ConsumedByWorkerStamp` that clearly means: "This message is being handled by a worker" (and so, you might want to "reset" some things before/after handling).

Thanks!

Commits
-------

01a9fef Adding ConsumedByWorkerStamp as way to mark a message in a "worker context"
38f19a9 Revert "[Messenger] Removing "sync" transport and replacing it with much nicer config trick"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Messenger] Usecase for Doctrine middlewares
5 participants