[RFC][Mailer][FrameworkBundle] Add support for rate limited mailer transports #59985

fritzmg · 2025-03-16T12:37:53Z

Q	A
Branch?	7.3
Bug fix?	no
New feature?	yes
Deprecations?	no
Issues
License	MIT

Assume you have to use multiple SMTP servers in your application in order to be able to send emails on behalf of specific email accounts on a variety of hosting providers. Some of these hosting providers may impose limits on how many emails you can send per timeframe. For example on Hetzner the limit is 500 emails per hour, on webgo 50 emails per hour. Hetzner does not force this limit - but they may disable your account completely, if you break this rule. The same applies to services like sendgrid etc., where you will have certain limits.

Now, if you use symfony/messenger in order to send your emails asynchronously, you are able to rate limit the messenger transport:

framework:
    messenger:
        transports:
            mailer:
                dsn: 'doctrine://default?queue_name=mailer'
                rate_limiter: mailer
        routing:
            'Symfony\Component\Mailer\Messenger\SendEmailMessage': mailer
    rate_limiter:
        mailer:
            policy: fixed_window
            limit: 500
            interval: 60 minutes

However, this will only rate limit the email queue globally - and not per SMTP server. So if you have to use multiple SMTP servers, where some use different rate limits and others do not need to be rate limited at all, this is not ideal.

Thus this PR introduces support for rate limited mailer transports. The changes are fairly straight forward:

For each mailer transport you can now also optionally define a rate_limiter, just as with messenger transports.
This rate limiter will then be set by the Transport factory, if applicable.
The actual rate limiting is done in the AbstractTransport::send() method - using ensureAccepted(), so that a RateLimitExceededException will be thrown.
This means that - if you do not use a messenger queue for sending emails, your application is responsible for handling this exception, when you defined a rate limiter on a mailer transport.
If you do use a messenger queue then the email message will be put back on the queue, with available_at set to the respective time in the future according to the rate limit.

framework:
    mailer:
        transports:
            hetzner: 
                dsn: smtp://foo:bar@mail.your-server.de:587
                rate_limiter: hetzner
            webgo:
                dsn: smtp://lorem:ipsum@v123.goserver.host:587
                rate_limiter: webgo
            not_rate_limited: smtp://dolor:sit@example.com:587

    rate_limiter:
        # In actuality you will want to set a lower rate limit, depending on whether
        # other applications will also send via the same credentials.
        hetzner:
            policy: fixed_window
            limit: 500
            interval: 60 minutes
        webgo:
            policy: fixed_window
            limit: 50
            interval: 60 minutes

    messenger:
        transports:
            mailer: 
                dsn: 'doctrine://default?queue_name=mailer'
                failure_transport: mailer_failed
            mailer_failed: 'doctrine://default?queue_name=mailer_failed'
        routing:
            'Symfony\Component\Mailer\Messenger\SendEmailMessage': mailer

Tests are still missing, as I wanted to hear comments first :)

src/Symfony/Component/Mailer/Messenger/MessageHandler.php

Spomky

Hi,

Many thanks for the proposal. THis is a nice and useful idea.
I reviewed this PR and it is a good start except the missing tests and a few things I spotted.

Spomky · 2025-04-08T14:31:49Z

src/Symfony/Bundle/FrameworkBundle/DependencyInjection/FrameworkExtension.php

+
+        foreach ($transports as $name => $transport) {
+            if ($transport['rate_limiter']) {
+                if (!interface_exists(LimiterInterface::class)) {


This if could be moved outside the for loop.
The verification can be done just after if $transportRateLimiterReferences is not empty and the interface is missing.

Yes - however, the same would apply to this existing code:

symfony/src/Symfony/Bundle/FrameworkBundle/DependencyInjection/FrameworkExtension.php

Lines 2458 to 2460 in 3ec30b6

if (!interface_exists(LimiterInterface::class)) {

throw new LogicException('Rate limiter cannot be used within Messenger as the RateLimiter component is not installed. Try running "composer require symfony/rate-limiter".');

}

Do you want me to change it in the unrelated section as well?

Spomky · 2025-04-08T14:34:36Z

src/Symfony/Component/Mailer/Transport/AbstractTransport.php


 /**
 * @author Fabien Potencier <fabien@symfony.com>
 */
 abstract class AbstractTransport implements TransportInterface
 {
    private LoggerInterface $logger;
+    private ?RateLimiterFactoryInterface $rateLimiterFactory = null;


New argument should be placed after to avoid BC breaks.

Hm, how is this related to BC? This is just a private member variable - and not a constructor variable with constructor property promotion.

Spomky · 2025-04-08T14:38:42Z

src/Symfony/Bundle/FrameworkBundle/DependencyInjection/FrameworkExtension.php

+        $transportRateLimiterReferences = [];
+
+        foreach ($transports as $name => $transport) {
+            if ($transport['rate_limiter']) {


isset or array_key_exists?

Not necessary as the key will always exist due to the processed configuration. See also:

symfony/src/Symfony/Bundle/FrameworkBundle/DependencyInjection/FrameworkExtension.php

Line 2457 in 3ec30b6

if ($transport['rate_limiter']) {

stof · 2025-04-08T15:44:27Z

src/Symfony/Bundle/FrameworkBundle/DependencyInjection/FrameworkExtension.php

+
+        foreach ($transports as $name => $transport) {
+            if ($transport['rate_limiter']) {
+                if (!interface_exists(LimiterInterface::class)) {


shouldn't this rather check isConfigEnabled() for the rate_limiter config ? the fact that the component is installed does not guarantee that the config is enabled for it (smart defaults are based on the availability of the component, but projects can still configure things explicitly)

I am using the same logic as here:

symfony/src/Symfony/Bundle/FrameworkBundle/DependencyInjection/FrameworkExtension.php

Lines 2421 to 2464 in 3ec30b6

$senderAliases = [];

$transportRetryReferences = [];

$transportRateLimiterReferences = [];

foreach ($config['transports'] as $name => $transport) {

$serializerId = $transport['serializer'] ?? 'messenger.default_serializer';

$tags = [

'alias' => $name,

'is_failure_transport' => \in_array($name, $failureTransports, true),

];

if (str_starts_with($transport['dsn'], 'sync://')) {

$tags['is_consumable'] = false;

}

$transportDefinition = (new Definition(TransportInterface::class))

->setFactory([new Reference('messenger.transport_factory'), 'createTransport'])

->setArguments([$transport['dsn'], $transport['options'] + ['transport_name' => $name], new Reference($serializerId)])

->addTag('messenger.receiver', $tags)

;

$container->setDefinition($transportId = 'messenger.transport.'.$name, $transportDefinition);

$senderAliases[$name] = $transportId;

if (null !== $transport['retry_strategy']['service']) {

$transportRetryReferences[$name] = new Reference($transport['retry_strategy']['service']);

} else {

$retryServiceId = \sprintf('messenger.retry.multiplier_retry_strategy.%s', $name);

$retryDefinition = new ChildDefinition('messenger.retry.abstract_multiplier_retry_strategy');

$retryDefinition

->replaceArgument(0, $transport['retry_strategy']['max_retries'])

->replaceArgument(1, $transport['retry_strategy']['delay'])

->replaceArgument(2, $transport['retry_strategy']['multiplier'])

->replaceArgument(3, $transport['retry_strategy']['max_delay'])

->replaceArgument(4, $transport['retry_strategy']['jitter']);

$container->setDefinition($retryServiceId, $retryDefinition);

$transportRetryReferences[$name] = new Reference($retryServiceId);

}

if ($transport['rate_limiter']) {

if (!interface_exists(LimiterInterface::class)) {

throw new LogicException('Rate limiter cannot be used within Messenger as the RateLimiter component is not installed. Try running "composer require symfony/rate-limiter".');

}

$transportRateLimiterReferences[$name] = new Reference('limiter.'.$transport['rate_limiter']);

}

}

Should this not employ the same logic as the rate_limiter config for messenger for consistency?

stof · 2025-04-08T15:49:33Z

src/Symfony/Component/Mailer/Transport/AbstractTransport.php

+    {
+        $this->rateLimiterFactory = $rateLimiterFactory;
+
+        return $this;


I don't think we should return $this here. It makes the API less clear whether this is a method mutating the instance or returning a new one. Our setters generally don't return $this in Symfony.

The setters in Symfony\Component\Mailer\Transport\AbstractTransport all return $this - so I think we should also return $this here, otherwise it would be inconsistent, would it not?

fritzmg added 3 commits March 15, 2025 18:23

First draft

85ae910

Fix retryDelay

2db50eb

update the CHANGELOG.md files

f311ac4

carsonbot added Status: Needs Review Feature labels Mar 16, 2025

carsonbot added this to the 7.3 milestone Mar 16, 2025

fritzmg mentioned this pull request Mar 16, 2025

Introduce separate contao_mailer messenger transport contao/contao#7495

Closed

fritzmg changed the title ~~Add support for rate limited mailer transports~~ [Mailer][FrameworkBundle] Add support for rate limited mailer transports Mar 16, 2025

OskarStark reviewed Mar 16, 2025

View reviewed changes

src/Symfony/Component/Mailer/Messenger/MessageHandler.php Outdated Show resolved Hide resolved

do not use named arguments

ff86149

fritzmg changed the title ~~[Mailer][FrameworkBundle] Add support for rate limited mailer transports~~ [RFC][Mailer][FrameworkBundle] Add support for rate limited mailer transports Mar 18, 2025

Spomky reviewed Apr 8, 2025

View reviewed changes

stof reviewed Apr 8, 2025

View reviewed changes

fabpot modified the milestones: 7.3, 7.4 May 26, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

[RFC][Mailer][FrameworkBundle] Add support for rate limited mailer transports #59985

[RFC][Mailer][FrameworkBundle] Add support for rate limited mailer transports #59985

fritzmg commented Mar 16, 2025 •

edited

Loading

Uh oh!

Uh oh!

Spomky left a comment

Uh oh!

Spomky Apr 8, 2025

Uh oh!

fritzmg Apr 8, 2025

Uh oh!

Spomky Apr 8, 2025

Uh oh!

fritzmg Apr 8, 2025

Uh oh!

Spomky Apr 8, 2025

Uh oh!

fritzmg Apr 8, 2025 •

edited

Loading

Uh oh!

stof Apr 8, 2025

Uh oh!

fritzmg Apr 8, 2025

Uh oh!

stof Apr 8, 2025

Uh oh!

fritzmg Apr 8, 2025 •

edited

Loading

Uh oh!

Uh oh!

	if (!interface_exists(LimiterInterface::class)) {
	throw new LogicException('Rate limiter cannot be used within Messenger as the RateLimiter component is not installed. Try running "composer require symfony/rate-limiter".');
	}

	$senderAliases = [];
	$transportRetryReferences = [];
	$transportRateLimiterReferences = [];
	foreach ($config['transports'] as $name => $transport) {
	$serializerId = $transport['serializer'] ?? 'messenger.default_serializer';
	$tags = [
	'alias' => $name,
	'is_failure_transport' => \in_array($name, $failureTransports, true),
	];
	if (str_starts_with($transport['dsn'], 'sync://')) {
	$tags['is_consumable'] = false;
	}
	$transportDefinition = (new Definition(TransportInterface::class))
	->setFactory([new Reference('messenger.transport_factory'), 'createTransport'])
	->setArguments([$transport['dsn'], $transport['options'] + ['transport_name' => $name], new Reference($serializerId)])
	->addTag('messenger.receiver', $tags)
	;
	$container->setDefinition($transportId = 'messenger.transport.'.$name, $transportDefinition);
	$senderAliases[$name] = $transportId;

	if (null !== $transport['retry_strategy']['service']) {
	$transportRetryReferences[$name] = new Reference($transport['retry_strategy']['service']);
	} else {
	$retryServiceId = \sprintf('messenger.retry.multiplier_retry_strategy.%s', $name);
	$retryDefinition = new ChildDefinition('messenger.retry.abstract_multiplier_retry_strategy');
	$retryDefinition
	->replaceArgument(0, $transport['retry_strategy']['max_retries'])
	->replaceArgument(1, $transport['retry_strategy']['delay'])
	->replaceArgument(2, $transport['retry_strategy']['multiplier'])
	->replaceArgument(3, $transport['retry_strategy']['max_delay'])
	->replaceArgument(4, $transport['retry_strategy']['jitter']);
	$container->setDefinition($retryServiceId, $retryDefinition);

	$transportRetryReferences[$name] = new Reference($retryServiceId);
	}

	if ($transport['rate_limiter']) {
	if (!interface_exists(LimiterInterface::class)) {
	throw new LogicException('Rate limiter cannot be used within Messenger as the RateLimiter component is not installed. Try running "composer require symfony/rate-limiter".');
	}

	$transportRateLimiterReferences[$name] = new Reference('limiter.'.$transport['rate_limiter']);
	}
	}

Uh oh!

[RFC][Mailer][FrameworkBundle] Add support for rate limited mailer transports #59985

Are you sure you want to change the base?

[RFC][Mailer][FrameworkBundle] Add support for rate limited mailer transports #59985

Conversation

fritzmg commented Mar 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Spomky left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fritzmg Apr 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fritzmg Apr 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

fritzmg commented Mar 16, 2025 •

edited

Loading

fritzmg Apr 8, 2025 •

edited

Loading

fritzmg Apr 8, 2025 •

edited

Loading