-
-
Notifications
You must be signed in to change notification settings - Fork 9.6k
[Messenger] Growing size of ErrorDetailsStamp breaks AMQP for RecoverableMessageHandlingException #45944
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
We had the same issue. It's quite troublesome because when it happens the message can be lost forever.. The feature @Mantra-Media is suggesting would be really nice. Meanwhile users should be aware of this issue. Otherwise, another way of getting around the bug is to create a custom This is still a "get around" solution as you will be coupled with vendor |
The workaround I'm using currently is this: services:
messenger.failure.add_error_details_stamp_listener:
class: App\Library\Symfony\Messenger\AddErrorDetailsStampListener <?php
declare(strict_types=1);
namespace App\Library\Symfony\Messenger;
use Symfony\Component\EventDispatcher\EventSubscriberInterface;
use Symfony\Component\Messenger\Event\WorkerMessageFailedEvent;
use Symfony\Component\Messenger\Exception\HandlerFailedException;
use Symfony\Component\Messenger\Stamp\ErrorDetailsStamp;
/**
* This class is meant to be used instead of \Symfony\Component\Messenger\EventListener\AddErrorDetailsStampListener.
* The difference is that the ErrorDetailsStamp will be created without FlattenException.
* This is important because FlattenException contains stack trace which can be quite large,
* potentially causing AmqpSender to throw "Library error: table too large for buffer".
*/
final class AddErrorDetailsStampListener implements EventSubscriberInterface
{
public function onMessageFailed(WorkerMessageFailedEvent $event): void
{
$throwable = $event->getThrowable();
if ($throwable instanceof HandlerFailedException) {
$throwable = $throwable->getPrevious();
}
if ($throwable === null) {
return;
}
$stamp = new ErrorDetailsStamp($throwable::class, $throwable->getCode(), $throwable->getMessage());
$previousStamp = $event->getEnvelope()->last(ErrorDetailsStamp::class);
// Do not append duplicate information
if ($previousStamp === null || ! $previousStamp->equals($stamp)) {
$event->addStamps($stamp);
}
}
public static function getSubscribedEvents(): array
{
return [
// must have higher priority than SendFailedMessageForRetryListener
WorkerMessageFailedEvent::class => ['onMessageFailed', 200],
];
}
} |
@enumag Great solution! Big thanks! |
Hey, thanks for your report! |
Yes there is a workaround as I described above but I'd prefer a better solution. |
Hey, thanks for your report! |
The status didn't change since you last asked. |
- add a new class that adds a lot less exception stamp details to prevent the "Table too large for buffer" error. original author: https://github.com/enumag code taken from: symfony/symfony#45944 (comment)
I really hope Symfony could fix this issue. |
Did someone investigate this further? |
Found this: #44943 it is the same issue only for different Transports. |
Uh oh!
There was an error while loading. Please reload this page.
Symfony version(s) affected
6.0.3
Description
After the 4th or 5th retry the
ErrorDetailsStamp
header of a message that was sent back to the queue using theRecoverableMessageHandlingException
grows to 60k and RabbitMQ rejects the message with the errorLibrary error: table too large for buffer
I did find a related and merged issue (#34082) but this does not seem to have solved it, there are recent comments from people who ran into the same issue.
I am able to solve this in using a custom serializer that removes the
ErrorDetailsStamp
but I think that this is not expected behaviour.This also has implications for the other transports like Doctrine where very large entries for simple messages are being inserted into the databae.
How to reproduce
Just create a TestMessage and watch the
ErrorDetailsStamp
header grow out of proportion:Possible Solution
Create a custom serializer and remove the
ErrorDetailsStamp
in the encode method:I guess another solution is to not even add the
ErrorDetailsStamp
to a user initiated recoverable message as it is not really an error but intended behaviour? Or make headers configurable, egtraceCount: 1
to only add the latest trace or none with 0?Does it make logical sense that this is an Exception and not a stamp? With it being an Exception although not being an error it also has to be removed from monitoring tools (Sentry etc.) to not be a false negative.
Additional Context
No response
The text was updated successfully, but these errors were encountered: