Skip to content

Commit 7cd5bbf

Browse files
fabpotfabpot
committed
feature #37546 [RFC] Introduce a RateLimiter component (wouterj)
This PR was squashed before being merged into the 5.2-dev branch. Discussion ---------- [RFC] Introduce a RateLimiter component | Q | A | ------------- | --- | Branch? | master | Bug fix? | no | New feature? | yes | Deprecations? | no | Tickets | Refs #37444 | License | MIT | Doc PR | tbd Based on the discussions in #37444, I decided to write a general purpose RateLimiter component. This implementation uses the token bucket algorithm, inspired by the [Go's time/rate](https://github.com/golang/time/blob/3af7569d3a1e776fc2a3c1cec133b43105ea9c2e/rate/rate.go) library and the [PHP `bandwidth-throttle/token-bucket` package](https://github.com/bandwidth-throttle/token-bucket) (which is [unmaintained for years](bandwidth-throttle/token-bucket#19)). ### Usage The component has two main methods: * `Limiter::reserve(int $tokens, int $maxTime)`, allocates `$tokens` and returns a `Reservation` containing the wait time. Use this method if your process wants to wait before consuming the token. * `Limiter::consume(int $tokens)`, checks if `$tokens` are available now and discards the reservation if that's not the case. Use this method if you want to skip when there are not enough tokens at this moment. The component uses the Lock component to make sure it can be used in parallel processes. Example: ```php <?php namespace App\EventListener; use Symfony\Component\EventDispatcher\EventSubscriberInterface; use Symfony\Component\HttpFoundation\Response; use Symfony\Component\HttpKernel\Event\RequestEvent; use Symfony\Component\RateLimiter\LimiterFactory; class LimitListener { private $limiterFactory; public function __construct(LimiterFactory $apiLimiterFactory) { $this->limiterFactory = $apiLimiterFactory; } public function __invoke(RequestEvent $event) { $ip = $event->getRequest()->getClientIp(); $limiter = $this->limiterFactory->createLimiter(preg_replace('/[^a-zA-Z0-9]/', '-', $ip)); if (!$limiter->consume()) { $event->setResponse(new Response('Too many requests', 429)); } } } ``` ### Usefullness of the component I think a generic rate limiter is usefull in quite some places: * Add a login throttling feature in Symfony * <s>Rate limiting outgoing API calls (e.g. HttpClient), to prevent hitting upstream API limits.</s> See #37471 (and https://blog.heroku.com/rate-throttle-api-client ) * Allowing users to easily implement API rate limits in their own Symfony-based APIs. ### State of the art There are some rate limiting packages in PHP, but I think there is no precendent for this component: * [`graham-campbell/throttle`](https://github.com/GrahamCampbell/Laravel-Throttle) is heavily relying on Laravel. It is however very popular, proofing there is a need for such feature * [`nikolaposa/rate-limit`](https://github.com/nikolaposa/rate-limit) does not implement reservation of tokens and as such less feature complete. Also its architecture combines the rate limiter and storage, making it harder to implement different storages. ### Todo If it is agreed that this component can bring something to Symfony, it needs some more love: * [x] Add more tests * [x] Integrate with the FrameworkBundle * [x] Add sliding window implementation * [x] Add integration with the Security component * <s>Maybe add more storage implementations? I didn't want to duplicate storage functionalities already existing in the Lock and Cache component, thus I for now focused mostly on integrating the Cache adapters. But maybe a special Doctrine adapter makes sense?</s> Commits ------- 67417a6 [RFC] Introduce a RateLimiter component
2 parents 128bbd1 + 67417a6 commit 7cd5bbf

36 files changed

+1652
-4
lines changed

‎src/Symfony/Bundle/FrameworkBundle/DependencyInjection/Configuration.php

+48
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@
3030
use Symfony\Component\Messenger\MessageBusInterface;
3131
use Symfony\Component\Notifier\Notifier;
3232
use Symfony\Component\PropertyInfo\PropertyInfoExtractorInterface;
33+
use Symfony\Component\RateLimiter\TokenBucketLimiter;
3334
use Symfony\Component\Serializer\Serializer;
3435
use Symfony\Component\Translation\Translator;
3536
use Symfony\Component\Validator\Validation;
@@ -134,6 +135,7 @@ public function getConfigTreeBuilder()
134135
$this->addMailerSection($rootNode);
135136
$this->addSecretsSection($rootNode);
136137
$this->addNotifierSection($rootNode);
138+
$this->addRateLimiterSection($rootNode);
137139

138140
return $treeBuilder;
139141
}
@@ -1707,4 +1709,50 @@ private function addNotifierSection(ArrayNodeDefinition $rootNode)
17071709
->end()
17081710
;
17091711
}
1712+
1713+
private function addRateLimiterSection(ArrayNodeDefinition $rootNode)
1714+
{
1715+
$rootNode
1716+
->children()
1717+
->arrayNode('rate_limiter')
1718+
->info('Rate limiter configuration')
1719+
->{!class_exists(FullStack::class) && class_exists(TokenBucketLimiter::class) ? 'canBeDisabled' : 'canBeEnabled'}()
1720+
->fixXmlConfig('limiter')
1721+
->beforeNormalization()
1722+
->ifTrue(function ($v) { return \is_array($v) && !isset($v['limiters']) && !isset($v['limiter']); })
1723+
->then(function (array $v) {
1724+
$newV = [
1725+
'enabled' => $v['enabled'],
1726+
];
1727+
unset($v['enabled']);
1728+
1729+
$newV['limiters'] = $v;
1730+
1731+
return $newV;
1732+
})
1733+
->end()
1734+
->children()
1735+
->arrayNode('limiters')
1736+
->useAttributeAsKey('name')
1737+
->arrayPrototype()
1738+
->children()
1739+
->scalarNode('lock')->defaultValue('lock.factory')->end()
1740+
->scalarNode('storage')->defaultValue('cache.app')->end()
1741+
->scalarNode('strategy')->isRequired()->end()
1742+
->integerNode('limit')->isRequired()->end()
1743+
->scalarNode('interval')->end()
1744+
->arrayNode('rate')
1745+
->children()
1746+
->scalarNode('interval')->isRequired()->end()
1747+
->integerNode('amount')->defaultValue(1)->end()
1748+
->end()
1749+
->end()
1750+
->end()
1751+
->end()
1752+
->end()
1753+
->end()
1754+
->end()
1755+
->end()
1756+
;
1757+
}
17101758
}

‎src/Symfony/Bundle/FrameworkBundle/DependencyInjection/FrameworkExtension.php

+55-1
Original file line numberDiff line numberDiff line change
@@ -123,6 +123,9 @@
123123
use Symfony\Component\PropertyInfo\PropertyReadInfoExtractorInterface;
124124
use Symfony\Component\PropertyInfo\PropertyTypeExtractorInterface;
125125
use Symfony\Component\PropertyInfo\PropertyWriteInfoExtractorInterface;
126+
use Symfony\Component\RateLimiter\Limiter;
127+
use Symfony\Component\RateLimiter\LimiterInterface;
128+
use Symfony\Component\RateLimiter\Storage\CacheStorage;
126129
use Symfony\Component\Routing\Loader\AnnotationDirectoryLoader;
127130
use Symfony\Component\Routing\Loader\AnnotationFileLoader;
128131
use Symfony\Component\Security\Core\Security;
@@ -173,6 +176,7 @@ class FrameworkExtension extends Extension
173176
private $mailerConfigEnabled = false;
174177
private $httpClientConfigEnabled = false;
175178
private $notifierConfigEnabled = false;
179+
private $lockConfigEnabled = false;
176180

177181
/**
178182
* Responds to the app.config configuration parameter.
@@ -405,10 +409,18 @@ public function load(array $configs, ContainerBuilder $container)
405409
$this->registerPropertyInfoConfiguration($container, $loader);
406410
}
407411

408-
if ($this->isConfigEnabled($container, $config['lock'])) {
412+
if ($this->lockConfigEnabled = $this->isConfigEnabled($container, $config['lock'])) {
409413
$this->registerLockConfiguration($config['lock'], $container, $loader);
410414
}
411415

416+
if ($this->isConfigEnabled($container, $config['rate_limiter'])) {
417+
if (!interface_exists(LimiterInterface::class)) {
418+
throw new LogicException('Rate limiter support cannot be enabled as the RateLimiter component is not installed. Try running "composer require symfony/rate-limiter".');
419+
}
420+
421+
$this->registerRateLimiterConfiguration($config['rate_limiter'], $container, $loader);
422+
}
423+
412424
if ($this->isConfigEnabled($container, $config['web_link'])) {
413425
if (!class_exists(HttpHeaderSerializer::class)) {
414426
throw new LogicException('WebLink support cannot be enabled as the WebLink component is not installed. Try running "composer require symfony/weblink".');
@@ -2170,6 +2182,48 @@ private function registerNotifierConfiguration(array $config, ContainerBuilder $
21702182
}
21712183
}
21722184

2185+
private function registerRateLimiterConfiguration(array $config, ContainerBuilder $container, PhpFileLoader $loader)
2186+
{
2187+
if (!$this->lockConfigEnabled) {
2188+
throw new LogicException('Rate limiter support cannot be enabled without enabling the Lock component.');
2189+
}
2190+
2191+
$loader->load('rate_limiter.php');
2192+
2193+
$locks = [];
2194+
$storages = [];
2195+
foreach ($config['limiters'] as $name => $limiterConfig) {
2196+
$limiter = $container->setDefinition($limiterId = 'limiter.'.$name, new ChildDefinition('limiter'));
2197+
2198+
if (!isset($locks[$limiterConfig['lock']])) {
2199+
$locks[$limiterConfig['lock']] = new Reference($limiterConfig['lock']);
2200+
}
2201+
$limiter->addArgument($locks[$limiterConfig['lock']]);
2202+
unset($limiterConfig['lock']);
2203+
2204+
if (!isset($storages[$limiterConfig['storage']])) {
2205+
$storageId = $limiterConfig['storage'];
2206+
// cache pools are configured by the FrameworkBundle, so they
2207+
// exists in the scoped ContainerBuilder provided to this method
2208+
if ($container->has($storageId)) {
2209+
if ($container->findDefinition($storageId)->hasTag('cache.pool')) {
2210+
$container->register('limiter.storage.'.$storageId, CacheStorage::class)->addArgument(new Reference($storageId));
2211+
$storageId = 'limiter.storage.'.$storageId;
2212+
}
2213+
}
2214+
2215+
$storages[$limiterConfig['storage']] = new Reference($storageId);
2216+
}
2217+
$limiter->replaceArgument(1, $storages[$limiterConfig['storage']]);
2218+
unset($limiterConfig['storage']);
2219+
2220+
$limiterConfig['id'] = $name;
2221+
$limiter->replaceArgument(0, $limiterConfig);
2222+
2223+
$container->registerAliasForArgument($limiterId, Limiter::class, $name.'.limiter');
2224+
}
2225+
}
2226+
21732227
private function resolveTrustedHeaders(array $headers): int
21742228
{
21752229
$trustedHeaders = 0;

‎src/Symfony/Bundle/FrameworkBundle/Resources/config/rate_limiter.php

+25
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
<?php
2+
3+
/*
4+
* This file is part of the Symfony package.
5+
*
6+
* (c) Fabien Potencier <fabien@symfony.com>
7+
*
8+
* For the full copyright and license information, please view the LICENSE
9+
* file that was distributed with this source code.
10+
*/
11+
12+
namespace Symfony\Component\DependencyInjection\Loader\Configurator;
13+
14+
use Symfony\Component\RateLimiter\Limiter;
15+
16+
return static function (ContainerConfigurator $container) {
17+
$container->services()
18+
->set('limiter', Limiter::class)
19+
->abstract()
20+
->args([
21+
abstract_arg('config'),
22+
abstract_arg('storage'),
23+
])
24+
;
25+
};

‎src/Symfony/Bundle/FrameworkBundle/Resources/config/schema/symfony-1.0.xsd

+27
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@
3434
<xsd:element name="http-client" type="http_client" minOccurs="0" maxOccurs="1" />
3535
<xsd:element name="mailer" type="mailer" minOccurs="0" maxOccurs="1" />
3636
<xsd:element name="http-cache" type="http_cache" minOccurs="0" maxOccurs="1" />
37+
<xsd:element name="rate-limiter" type="rate_limiter" minOccurs="0" maxOccurs="1" />
3738
</xsd:choice>
3839

3940
<xsd:attribute name="http-method-override" type="xsd:boolean" />
@@ -634,4 +635,30 @@
634635
<xsd:enumeration value="full" />
635636
</xsd:restriction>
636637
</xsd:simpleType>
638+
639+
<xsd:complexType name="rate_limiter">
640+
<xsd:sequence>
641+
<xsd:element name="limiter" type="rate_limiter_limiter" minOccurs="0" maxOccurs="unbounded" />
642+
</xsd:sequence>
643+
<xsd:attribute name="enabled" type="xsd:boolean" />
644+
<xsd:attribute name="max-host-connections" type="xsd:integer" />
645+
<xsd:attribute name="mock-response-factory" type="xsd:string" />
646+
</xsd:complexType>
647+
648+
<xsd:complexType name="rate_limiter_limiter">
649+
<xsd:sequence>
650+
<xsd:element name="rate" type="rate_limiter_rate" minOccurs="0" />
651+
</xsd:sequence>
652+
<xsd:attribute name="name" type="xsd:string" />
653+
<xsd:attribute name="lock" type="xsd:string" />
654+
<xsd:attribute name="storage" type="xsd:string" />
655+
<xsd:attribute name="strategy" type="xsd:string" />
656+
<xsd:attribute name="limit" type="xsd:int" />
657+
<xsd:attribute name="interval" type="xsd:string" />
658+
</xsd:complexType>
659+
660+
<xsd:complexType name="rate_limiter_rate">
661+
<xsd:attribute name="interval" type="xsd:string" />
662+
<xsd:attribute name="amount" type="xsd:int" />
663+
</xsd:complexType>
637664
</xsd:schema>

‎src/Symfony/Bundle/FrameworkBundle/Tests/DependencyInjection/ConfigurationTest.php

+4
Original file line numberDiff line numberDiff line change
@@ -531,6 +531,10 @@ class_exists(SemaphoreStore::class) && SemaphoreStore::isSupported() ? 'semaphor
531531
'debug' => '%kernel.debug%',
532532
'private_headers' => [],
533533
],
534+
'rate_limiter' => [
535+
'enabled' => false,
536+
'limiters' => [],
537+
],
534538
];
535539
}
536540
}

‎src/Symfony/Component/Lock/CHANGELOG.md

+4-3
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ CHANGELOG
66

77
* `MongoDbStore` does not implement `BlockingStoreInterface` anymore, typehint against `PersistingStoreInterface` instead.
88
* added support for shared locks
9+
* added `NoLock`
910

1011
5.1.0
1112
-----
@@ -25,10 +26,10 @@ CHANGELOG
2526
* added InvalidTtlException
2627
* deprecated `StoreInterface` in favor of `BlockingStoreInterface` and `PersistingStoreInterface`
2728
* `Factory` is deprecated, use `LockFactory` instead
28-
* `StoreFactory::createStore` allows PDO and Zookeeper DSN.
29-
* deprecated services `lock.store.flock`, `lock.store.semaphore`, `lock.store.memcached.abstract` and `lock.store.redis.abstract`,
29+
* `StoreFactory::createStore` allows PDO and Zookeeper DSN.
30+
* deprecated services `lock.store.flock`, `lock.store.semaphore`, `lock.store.memcached.abstract` and `lock.store.redis.abstract`,
3031
use `StoreFactory::createStore` instead.
31-
32+
3233
4.2.0
3334
-----
3435

‎src/Symfony/Component/Lock/NoLock.php

+51
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
<?php
2+
3+
/*
4+
* This file is part of the Symfony package.
5+
*
6+
* (c) Fabien Potencier <fabien@symfony.com>
7+
*
8+
* For the full copyright and license information, please view the LICENSE
9+
* file that was distributed with this source code.
10+
*/
11+
12+
namespace Symfony\Component\Lock;
13+
14+
/**
15+
* A non locking lock.
16+
*
17+
* This can be used to disable locking in classes
18+
* requiring a lock.
19+
*
20+
* @author Wouter de Jong <wouter@wouterj.nl>
21+
*/
22+
final class NoLock implements LockInterface
23+
{
24+
public function acquire(bool $blocking = false): bool
25+
{
26+
return true;
27+
}
28+
29+
public function refresh(float $ttl = null)
30+
{
31+
}
32+
33+
public function isAcquired(): bool
34+
{
35+
return true;
36+
}
37+
38+
public function release()
39+
{
40+
}
41+
42+
public function isExpired(): bool
43+
{
44+
return false;
45+
}
46+
47+
public function getRemainingLifetime(): ?float
48+
{
49+
return null;
50+
}
51+
}

‎src/Symfony/Component/RateLimiter/.gitattributes

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
/Tests export-ignore
2+
/phpunit.xml.dist export-ignore
3+
/.gitattributes export-ignore
4+
/.gitignore export-ignore

‎src/Symfony/Component/RateLimiter/.gitignore

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
composer.lock
2+
phpunit.xml
3+
vendor/

‎src/Symfony/Component/RateLimiter/CHANGELOG.md

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
CHANGELOG
2+
=========
3+
4+
5.2.0
5+
-----
6+
7+
* added the component

‎src/Symfony/Component/RateLimiter/CompoundLimiter.php

+47
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
<?php
2+
3+
/*
4+
* This file is part of the Symfony package.
5+
*
6+
* (c) Fabien Potencier <fabien@symfony.com>
7+
*
8+
* For the full copyright and license information, please view the LICENSE
9+
* file that was distributed with this source code.
10+
*/
11+
12+
namespace Symfony\Component\RateLimiter;
13+
14+
/**
15+
* @author Wouter de Jong <wouter@wouterj.nl>
16+
*
17+
* @experimental in 5.2
18+
*/
19+
final class CompoundLimiter implements LimiterInterface
20+
{
21+
private $limiters;
22+
23+
/**
24+
* @param LimiterInterface[] $limiters
25+
*/
26+
public function __construct(array $limiters)
27+
{
28+
$this->limiters = $limiters;
29+
}
30+
31+
public function consume(int $tokens = 1): bool
32+
{
33+
$allow = true;
34+
foreach ($this->limiters as $limiter) {
35+
$allow = $limiter->consume($tokens) && $allow;
36+
}
37+
38+
return $allow;
39+
}
40+
41+
public function reset(): void
42+
{
43+
foreach ($this->limiters as $limiter) {
44+
$limiter->reset();
45+
}
46+
}
47+
}

‎src/Symfony/Component/RateLimiter/Exception/MaxWaitDurationExceededException.php

+21
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
<?php
2+
3+
/*
4+
* This file is part of the Symfony package.
5+
*
6+
* (c) Fabien Potencier <fabien@symfony.com>
7+
*
8+
* For the full copyright and license information, please view the LICENSE
9+
* file that was distributed with this source code.
10+
*/
11+
12+
namespace Symfony\Component\RateLimiter\Exception;
13+
14+
/**
15+
* @author Wouter de Jong <wouter@wouterj.nl>
16+
*
17+
* @experimental in 5.2
18+
*/
19+
class MaxWaitDurationExceededException extends \RuntimeException
20+
{
21+
}

0 commit comments

Comments
 (0)