-
Notifications
You must be signed in to change notification settings - Fork 1.8k
Add API request throttling to find-inactive-members script to limit requests to 5000/hour #789
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
Co-authored-by: thedave42 <50186003+thedave42@users.noreply.github.com>
@copilot update the readme file to ensure that the user know to install an additional gem for the throttling. |
…ware Co-authored-by: thedave42 <50186003+thedave42@users.noreply.github.com>
Co-authored-by: thedave42 <50186003+thedave42@users.noreply.github.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR adds API request throttling to the find-inactive-members.rb
script to prevent GitHub API rate limit violations when processing large organizations. The implementation uses a custom Faraday middleware to automatically limit requests to 5000 per hour.
- Implements
ThrottleMiddleware
class with precise rate limiting (0.72s minimum delay between requests) - Enhances rate limit reporting with throttling status and reset time information
- Updates installation documentation to include required Faraday dependency
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
File | Description |
---|---|
find_inactive_members.rb | Adds throttling middleware class and integrates it into the Faraday stack |
README.md | Updates gem installation instructions to include faraday-retry dependency |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
elapsed_hour = Time.now - @hour_start_time | ||
rate_per_hour = elapsed_hour > 0 ? (@request_count / elapsed_hour * 3600).round(1) : 0 | ||
$stderr.print "Throttle debug: #{@request_count} requests in last #{elapsed_hour.round(1)}s (#{rate_per_hour}/hour rate)\n" if ENV['THROTTLE_DEBUG'] | ||
end |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The log_throttle_status
method is called on every request but only outputs when THROTTLE_DEBUG
is set. This creates unnecessary computation overhead. Consider removing the method call from line 23 or implementing a more efficient debug check.
Copilot uses AI. Check for mistakes.
@copilot implement a more efficient debug check |
Co-authored-by: thedave42 <50186003+thedave42@users.noreply.github.com>
Implemented a more efficient debug check by caching the |
This PR implements API request throttling for the
find-inactive-members.rb
script to ensure it doesn't exceed GitHub's recommended rate limits while processing large organizations.Problem
The existing script makes numerous API calls when analyzing organization members and repositories:
For large organizations with many repositories, this can easily exceed GitHub's rate limits, causing the script to fail or get blocked.
Solution
Added a custom Faraday middleware (
ThrottleMiddleware
) that automatically throttles all API requests to stay under 5000 requests per hour (approximately 1.39 requests per second):Key Features
Implementation Details
The middleware integrates seamlessly into the existing Faraday stack:
Enhanced rate limit checking now shows:
Installation
Updated installation instructions include the required Faraday gem:
Backward Compatibility
Testing
Comprehensive testing verified:
The script will now run reliably for large organizations without hitting rate limits, while maintaining the exact same user experience and output format.
💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.