Thanks for this contribution! Looks pretty good. Any way to easily test this? (e.g. against a public GitLab instance without Retry-After header?)

@max-wittig Hm, this is a bit difficult to test. But here's how I validated the code path - let me know what you think.

1. Created a virtual environment and built the library with the following code:

while True:
    # result = self.session.send(prepped, timeout=timeout, **settings)
    result = requests.get('https://httpstat.us/429')

    self._check_redirects(result)

    if 200 <= result.status_code < 300:
        return result

    if 429 == result.status_code and obey_rate_limit:
        if max_retries == -1 or cur_retries < max_retries:
            wait_time = 2 ** cur_retries * 0.1
            if "Retry-AfterXYZ" in result.headers:
                wait_time = int(result.headers["Retry-After"])
            cur_retries += 1
            print('DEBUG: backing off', str(wait_time))
            time.sleep(wait_time)
            continue

Note that https://httpstat.us returns a response with the Retry-After header set to 3600. I first validated that we backed off 3600 seconds; then I changed the header we were checking for (Retry-AfterXYZ as shown above) and validated that we were backing off exponentially (see below).

2. Wrote this small script:

from getpass import getpass
import gitlab

private_token = getpass('GitLab API Token: ')
gl = gitlab.Gitlab('https://gitlab.com/', private_token=private_token)
project = gl.projects.get(<some-project-id-we-own>)

project.members.list(all=True)

3. Validated the exponential backoff:

(env) ➜  python-gitlab python throttle.py 
GitLab API Token: 
DEBUG: backing off 0.1
DEBUG: backing off 0.2
DEBUG: backing off 0.4
DEBUG: backing off 0.8
DEBUG: backing off 1.6
DEBUG: backing off 3.2
DEBUG: backing off 6.4
DEBUG: backing off 12.8
DEBUG: backing off 25.6
DEBUG: backing off 51.2
Traceback (most recent call last):
  File "/Users/srikanth.chelluri/repo/python-gitlab/gitlab/exceptions.py", line 251, in wrapped_f
    return f(*args, **kwargs)
  File "/Users/srikanth.chelluri/repo/python-gitlab/gitlab/mixins.py", line 49, in get
    server_data = self.gitlab.http_get(path, **kwargs)
  File "/Users/srikanth.chelluri/repo/python-gitlab/gitlab/__init__.py", line 542, in http_get
    streamed=streamed, **kwargs)
  File "/Users/srikanth.chelluri/repo/python-gitlab/gitlab/__init__.py", line 520, in http_request
    response_body=result.content)
gitlab.exceptions.GitlabHttpError: 429: 429 Too Many Requests

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "throttle.py", line 10, in <module>
    project = gl.projects.get(8244461)
  File "/Users/srikanth.chelluri/repo/python-gitlab/gitlab/exceptions.py", line 253, in wrapped_f
    raise error(e.error_message, e.response_code, e.response_body)
gitlab.exceptions.GitlabGetError: 429: 429 Too Many Requests

Thoughts/feedback on this approach? Didn't try spinning up and configuring a GitLab instance without the header since we don't really need it to test the code path.

@srikanthchelluri Works as expected. Many thanks for the contribution 👍

Just a hint for future contributions: It's best practice to use a separate branch in your fork. Makes it easier for us to checkout the changes and easier for you to maintain your next contribution. Otherwise your master in your fork is polluted by another commit etc.

@max-wittig This was a separate branch, no? I forked the repo into my org (appian), then checked out a new branch called backoff-requests. Let me know if there's a better practice.

Thanks for the review and merge 👍

ah really. Okay weird. Nevermind. Maybe I miss-typed something and didn't see it. Not sure 😄