[Messenger] batch handler and delayed flush #46869
-
We have a need to make requests to external service that has harsh rate limits. So we've decided to not flush batches until some time passes. Lots of issues occurred, some related to Redis Transport, some to Messenger component itself. But the main question I have is this: is Messenger meant to handle delayed batches? Specifically the following issues made me question this:
|
Beta Was this translation helpful? Give feedback.
Replies: 2 comments 2 replies
-
We faced a similar situation - we wanted the handler to process a batch of messages OR if there are not enough messages for a batch - process what is there anyways after a short delay. Due to PHP's nature, the only way I can think of implementing it is using |
Beta Was this translation helpful? Give feedback.
-
Thanks a lot, @AdamKatzDev we actually found the |
Beta Was this translation helpful? Give feedback.
We've built an event listener that flushes our handlers with the
false
flag after everyWorkerRunningEvent
. Every batch handler in our case has a timer, when flush is called handlers process current batch if enough time has passed or the batch is full.The event listener is quite a thing in itself, I'll provide the code just in case someone needs something similar.
src/EventSubscriber/Messenger/FlushBatchesOnWorkerEventsListener.php