Skip to content

[Messenger] Ability to use transactions while sending messages with AMQP transport #51656

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
michalsitek opened this issue Sep 14, 2023 · 0 comments · May be fixed by #51719
Open

[Messenger] Ability to use transactions while sending messages with AMQP transport #51656

michalsitek opened this issue Sep 14, 2023 · 0 comments · May be fixed by #51719
Labels
Feature Messenger

Comments

@michalsitek
Copy link

michalsitek commented Sep 14, 2023 •
edited
Loading

Description

By default, AMQP doesn't guarantee message delivery to the queue. There are 2 ways of ensuring delivery:

  1. using Publisher Confirms (specifically using RabbitMQ: https://www.rabbitmq.com/confirms.html#publisher-confirms). It is already implemented as confirm_timeout option.
  2. using transactions

Currently there is no option to use transactions with Symfony Messenger to ensure message deliverability. Using this option wouldn't even have a big effect on performance, since all the messages are published one-by-one by this component.

Example

Introducing simple transport attribute, like transactional:

    email:
        dsn: '%env(resolve:MESSENGER_DSN)%'
        options:
            transactional: true
            exchange:
                name: some-name
            queues:
                some-name: ~

Which could be implemented with a simple change to Symfony\Component\Messenger\Bridge\Amqp\Transport\Connection method publishOnExchange.

Before:

    private function publishOnExchange(\AMQPExchange $exchange, string $body, string $routingKey = null, array $headers = [], AmqpStamp $amqpStamp = null)
    {
        $attributes = $amqpStamp ? $amqpStamp->getAttributes() : [];
        $attributes['headers'] = array_merge($attributes['headers'] ?? [], $headers);
        $attributes['delivery_mode'] ??= 2;
        $attributes['timestamp'] ??= time();

        $this->lastActivityTime = time();

        $exchange->publish(
            $body,
            $routingKey,
            $amqpStamp ? $amqpStamp->getFlags() : \AMQP_NOPARAM,
            $attributes
        );

        if ('' !== ($this->connectionOptions['confirm_timeout'] ?? '')) {
            $this->channel()->waitForConfirm((float) $this->connectionOptions['confirm_timeout']);
        }
    }

After:

    private function publishOnExchange(\AMQPExchange $exchange, string $body, string $routingKey = null, array $headers = [], AmqpStamp $amqpStamp = null)
    {
        $attributes = $amqpStamp ? $amqpStamp->getAttributes() : [];
        $attributes['headers'] = array_merge($attributes['headers'] ?? [], $headers);
        $attributes['delivery_mode'] ??= 2;
        $attributes['timestamp'] ??= time();
        $transactional = $this->connectionOptions['transactional'] ??= false;

        $this->lastActivityTime = time();
        
        if ($transactional) {
            $exchange->getChannel()->startTransaction();
        }

        $exchange->publish(
            $body,
            $routingKey,
            $amqpStamp ? $amqpStamp->getFlags() : \AMQP_NOPARAM,
            $attributes
        );
        
        if ($transactional) {
            $exchange->getChannel()->commitTransaction();
        }

        if ('' !== ($this->connectionOptions['confirm_timeout'] ?? '')) {
            $this->channel()->waitForConfirm((float) $this->connectionOptions['confirm_timeout']);
        }
    }
@xabbuh xabbuh added the Feature label Sep 15, 2023
michalsitek added a commit to michalsitek/symfony that referenced this issue Sep 22, 2023
@michalsitek
[Messenger] Add option option transactional to publish on AMQP queu…
6f92195 
…es using transactions

Resolves symfony#51656
michalsitek added a commit to michalsitek/symfony that referenced this issue Sep 22, 2023
@michalsitek
[Messenger] Add option option transactional to publish on AMQP queu…
d9ab06e 
…es using transactions

Resolves symfony#51656
michalsitek added a commit to michalsitek/symfony that referenced this issue Sep 22, 2023
@michalsitek
[Messenger] Add option option transactional to publish on AMQP queu…
6064a73 
…es using transactions

Resolves symfony#51656
@michalsitek michalsitek linked a pull request Sep 22, 2023 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Feature Messenger
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Messenger] Ability to use transactions while sending messages with AMQP michalsitek/symfony
3 participants
@xabbuh @carsonbot @michalsitek