Skip to content

Use RateLimiter to build a GuzzleRateLimiter middleware #40504

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
kdefives opened this issue Mar 17, 2021 · 2 comments
Closed

Use RateLimiter to build a GuzzleRateLimiter middleware #40504

kdefives opened this issue Mar 17, 2021 · 2 comments

Comments

@kdefives
Copy link

kdefives commented Mar 17, 2021

Description
Seems the new feature RateLimiter has been designed to limit the number of request that is coming to an application (inbound). What do you think to use this new feature to control the RateLimit for http request outbound? In order to never reach the rate limit when we are calling an external API which limit the number of request per second, per minute...

Example
I had the idea to use that because existing rate limiter middleware for Guzzle (i.e. https://github.com/spatie/guzzle-rate-limiter-middleware) does not manage "shared rate limiter" across several processes running in same time in parallel. Seems symfony/rate-limiter manages that natively. I tested and seems it works but I would like feedback from experts :-).

The code of the RateLimiterMiddleware for guzzle:

<?php
namespace App\GuzzleMiddleware\RateLimiterMiddleware;
use Closure;
use Psr\Http\Message\RequestInterface;
use Symfony\Component\RateLimiter\RateLimiterFactory;
class RateLimiter
{
    /** @var RateLimiterFactory */
    protected $guzzleRateLimiter;
    /**
     * RateLimiter constructor.
     * @param RateLimiterFactory $guzzleRateLimiter
     */
    public function __construct(RateLimiterFactory $guzzleRateLimiter)
    {
        $this->guzzleRateLimiter = $guzzleRateLimiter;
    }
    /**
     * @param callable $handler
     * @return Closure
     */
    public function __invoke(callable $handler): Closure
    {
        return function (RequestInterface $request, array $options) use ($handler) {
            $limiter = $this->guzzleRateLimiter->create('custom-guzzle-rate-limiter');
            while(!$limiter->consume()->isAccepted()){
                sleep(1);
            }
            return $handler($request, $options);
        };
    }
}

How to add the middleware to guzzle (inside services.yaml:

GuzzleMiddleware\GuzzleLimiterMiddleware\RateLimiter:
        class: App\GuzzleMiddleware\RateLimiterMiddleware\RateLimiter

    GuzzleHttp\HandlerStack:
        class: GuzzleHttp\HandlerStack
        factory: [ 'GuzzleHttp\HandlerStack', create ]
        calls:
            - [ push, [ '@log_middleware', 'log' ] ]
            - [ push, [ '@GuzzleMiddleware\GuzzleLimiterMiddleware\RateLimiter' ] ]
@xabbuh
Copy link
Member

xabbuh commented Mar 18, 2021

IMO such a class should rather be part of a third-party package.

@wouterj
Copy link
Member

wouterj commented Mar 18, 2021

I agree with @xabbuh here. I don't see anything Symfony can do here, other than to provide the low level APIs like the RateLimiter component does. Feel free to transform this class into an open source package that you own :)

As for the Symfony HttpClient component, this usecase is already solved in 5.2 by #37471. See also #37471 (comment) for a bit of explanation why this feature doesn't use the RateLimiter component (and I'm unsure if it's the perfect fit for this type of backoff algorithms).

@wouterj wouterj closed this as completed Mar 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants