Skip to content

[HttpClient] Make the client aware of throttling / rate limits #37471

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
NicoHaase opened this issue Jul 1, 2020 · 12 comments
Closed

[HttpClient] Make the client aware of throttling / rate limits #37471

NicoHaase opened this issue Jul 1, 2020 · 12 comments

Comments

@NicoHaase
Copy link
Contributor

Description
When making API requests, some providers implement rate limits: for example, an API might define that you can only perform 50 requests in five seconds. All exceeding requests will be answered with error code 429 ("Too many requests").

Currently, this error is not caught or handled anywhere. A future solution could catch this directly somewhere in the core HttpClient and react to it: simply wait for some time (maybe the response's headers already tell you how long to wait?) and retry the request.

@nicolas-grekas already provided the first steps in #36779, and I'd like to see how this could help to resolve 429 responses

@stof
Copy link
Member

stof commented Jul 1, 2020

Well, that's a perfect job for a decorating client rather than having it in the core HttpClient class. Then, whether this decorator is implemented in core or in third-party remains to be decided (to have it in core would require that the way to determine the action to do is standard, to avoid needing to have multiple such decorators in core).

@NicoHaase
Copy link
Contributor Author

That's my plan :) With "the core HttpClient", I wanted to refer to the component, not to the class itself.

As you are speaking of the "standard", would you have one in mind? I've asked @nicolas-grekas about his thoughts, and he recommended to evalute whatever the Retry-After header provides.

@mynameisbogdan
Copy link
Contributor

More likely HttpClient needs middleware support. I'm already using a throttling middleware in Guzzle which doesn't mess with the "core".

@nicolas-grekas
Copy link
Member

5.2 provides all the required infrastructure, see #36779

@NicoHaase
Copy link
Contributor Author

@nicolas-grekas I don't want to sound stupid, but could you explain that further? All that I came up with so far is https://github.com/NicoHaase/symfony/blob/feature/37471-http-throttling/src/Symfony/Component/HttpClient/ThrottleAwareHttpClient.php (while still lacking the evaluation of all 429 related headers to delay the next request attempt), and it does not look related to #36779 after all

@nicolas-grekas
Copy link
Member

nicolas-grekas commented Jul 6, 2020

The linked code defeats async. Making it both async and non blocking (aka no usleep() to pause) requires leveraging AsyncResponse.

@bohanyang
Copy link
Contributor

bohanyang commented Jul 6, 2020

@NicoHaase You may take a look at this and this to have a basic idea of how to use it.
But honestly I think it should be explained more. Because I still don't have idea how to build to a practical thing with it.

@nicolas-grekas
Copy link
Member

nicolas-grekas commented Jul 7, 2020

Here is another PR where AsyncResponse is used: #36692
Help welcome to improve the doc!

NicoHaase added a commit to NicoHaase/symfony that referenced this issue Jul 8, 2020
@NicoHaase
Copy link
Contributor Author

@nicolas-grekas thanks for linking that PR - I think I'm starting to understand. Is https://github.com/NicoHaase/symfony/blob/feature/37471-http-throttling/src/Symfony/Component/HttpClient/ThrottleAwareHttpClient.php going in the right direction?

@nicolas-grekas
Copy link
Member

Yes, that's a good start!
You'll need to yield the chunk in the else case.

@wouterj
Copy link
Member

wouterj commented Jul 10, 2020

In my RateLimiter component PR (which we decided does not relate to this one, as it's not API throttling), I was made aware of this blog posts showing a more advanced backoff algorithm to handle the wait time: https://blog.heroku.com/rate-throttle-api-client Might be of interest for this implementation as well :) (or maybe some sort of customizable backoff algorithm?)

fabpot added a commit that referenced this issue Sep 16, 2020
This PR was squashed before being merged into the 5.2-dev branch.

Discussion
----------

[RFC] Introduce a RateLimiter component

| Q             | A
| ------------- | ---
| Branch?       | master
| Bug fix?      | no
| New feature?  | yes
| Deprecations? | no
| Tickets       | Refs #37444
| License       | MIT
| Doc PR        | tbd

Based on the discussions in #37444, I decided to write a general purpose RateLimiter component. This implementation uses the token bucket algorithm, inspired by the [Go's time/rate](https://github.com/golang/time/blob/3af7569d3a1e776fc2a3c1cec133b43105ea9c2e/rate/rate.go) library and the [PHP `bandwidth-throttle/token-bucket` package](https://github.com/bandwidth-throttle/token-bucket) (which is [unmaintained for years](bandwidth-throttle/token-bucket#19)).

### Usage

The component has two main methods:

* `Limiter::reserve(int $tokens, int $maxTime)`, allocates `$tokens` and returns a `Reservation` containing the wait time. Use this method if your process wants to wait before consuming the token.
* `Limiter::consume(int $tokens)`, checks if `$tokens` are available now and discards the reservation if that's not the case. Use this method if you want to skip when there are not enough tokens at this moment.

The component uses the Lock component to make sure it can be used in parallel processes.

Example:

```php
<?php

namespace App\EventListener;

use Symfony\Component\EventDispatcher\EventSubscriberInterface;
use Symfony\Component\HttpFoundation\Response;
use Symfony\Component\HttpKernel\Event\RequestEvent;
use Symfony\Component\RateLimiter\LimiterFactory;

class LimitListener
{
    private $limiterFactory;

    public function __construct(LimiterFactory $apiLimiterFactory)
    {
        $this->limiterFactory = $apiLimiterFactory;
    }

    public function __invoke(RequestEvent $event)
    {
        $ip = $event->getRequest()->getClientIp();

        $limiter = $this->limiterFactory->createLimiter(preg_replace('/[^a-zA-Z0-9]/', '-', $ip));
        if (!$limiter->consume()) {
            $event->setResponse(new Response('Too many requests', 429));
        }
    }
}
```

### Usefullness of the component

I think a generic rate limiter is usefull in quite some places:

* Add a login throttling feature in Symfony
* <s>Rate limiting outgoing API calls (e.g. HttpClient), to prevent hitting upstream API limits.</s> See #37471 (and https://blog.heroku.com/rate-throttle-api-client )
* Allowing users to easily implement API rate limits in their own Symfony-based APIs.

### State of the art

There are some rate limiting packages in PHP, but I think there is no precendent for this component:

* [`graham-campbell/throttle`](https://github.com/GrahamCampbell/Laravel-Throttle) is heavily relying on Laravel. It is however very popular, proofing there is a need for such feature
* [`nikolaposa/rate-limit`](https://github.com/nikolaposa/rate-limit) does not implement reservation of tokens and as such less feature complete. Also its architecture combines the rate limiter and storage, making it harder to implement different storages.

### Todo

If it is agreed that this component can bring something to Symfony, it needs some more love:

* [x] Add more tests
* [x] Integrate with the FrameworkBundle
* [x] Add sliding window implementation
* [x] Add integration with the Security component
* <s>Maybe add more storage implementations? I didn't want to duplicate storage functionalities already existing in the Lock and Cache component, thus I for now focused mostly on integrating the Cache adapters. But maybe a special Doctrine adapter makes sense?</s>

Commits
-------

67417a6 [RFC] Introduce a RateLimiter component
@NicoHaase
Copy link
Contributor Author

As far as I see, this looks solved through #38182

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants