The ImprovedProxyScanner class is designed to scan, verify, and manage a list of proxies. It uses GeoIP databases to gather information about the proxies and stores the results in an SQLite database. The scanner can be run manually or scheduled to run at regular intervals.
- Classes and Enums
- Methods
- initialize_geoip
- initialize_db
- fetch_proxies
- quick_protocol_check
- check_proxy
- check_anonymity_level
- scan_proxies
- geoip_lookup
- update_proxy_database
- get_filtered_proxies
- run_scan
- view_results
- filter_proxies
- export_proxies
- display_proxy_count
- display_last_scan_time
- refresh_data
- blacklist_proxy
- schedule_scan
- main
Enum representing the anonymity level of a proxy.
- TRANSPARENT: The proxy reveals the client's IP address.
- ANONYMOUS: The proxy does not reveal the client's IP address but does not hide its presence as a proxy.
- ELITE: The proxy hides both the client's IP address and its presence as a proxy.
A class to scan, verify, and manage a list of proxies.
Initializes the GeoIP database readers.
Returns:
tuple: A tuple containing the city and ISP GeoIP readers.
Initializes the SQLite database, creating necessary tables and indexes.
Fetches proxies from the provided sources.
Returns:
Set[str]: A set of proxies fetched from the sources.
Quickly checks which protocols a proxy supports.
Parameters:
proxy (str): The proxy address to check.
Returns:
List[str]: A list of supported protocols.
Checks the functionality of a proxy.
Parameters:
proxy (str): The proxy address to check.
Returns:
Dict[str, any]: A dictionary with the results of the proxy check.
Checks the anonymity level of a proxy.
Parameters:
proxy (str): The proxy address to check.
Returns:
AnonymityLevel: The anonymity level of the proxy.
Scans a set of proxies to determine their functionality.
Parameters:
proxies (Set[str]): A set of proxies to scan.progress_bar: A progress bar to update during the scan (optional).
Returns:
List[Dict[str, any]]: A list of dictionaries with the results of the proxy checks.
Looks up GeoIP information for a given IP address.
Parameters:
ip (str): The IP address to look up.
Returns:
Dict[str, str]: A dictionary containing the country, city, and ISP of the IP address.
Updates the proxy database with scan results.
Parameters:
proxy_data (Dict[str, any]): A dictionary containing the proxy scan results.
Filters proxies based on the given criteria.
Parameters:
country (str, optional): The country to filter by.max_latency (float, optional): The maximum latency to filter by.min_anonymity (AnonymityLevel, optional): The minimum anonymity level to filter by.protocols (List[str], optional): The protocols to filter by.
Returns:
List[Dict[str, any]]: A list of dictionaries containing the filtered proxies.
Runs the proxy scan process.
Views the best proxies from the database.
Filters proxies based on user-defined criteria.
Exports filtered proxies to a JSON file.
Displays the total number of proxies in the database.
Displays the last time a scan was run.
Refreshes the displayed data.
Adds a proxy to the blacklist.
Parameters:
proxy (str): The proxy address to blacklist.
Schedules regular proxy scans. Not setup - Easy refactor"
Parameters:
interval (str, optional): The interval at which to run the scans. Defaults to "daily".
The main method to run the Streamlit app.
- Scan Proxies: Run the proxy scanner to fetch and verify proxies.
- View Results: View the best proxies from the database.
- Filter Proxies: Filter proxies based on custom criteria.
- Export Proxies: Export filtered proxies to a JSON file.
- Refresh Data: Refresh the displayed data.
- Blacklist Proxy: Add a proxy to the blacklist.
- Schedule Scan: Schedule proxy scans to run at regular intervals.