-
Notifications
You must be signed in to change notification settings - Fork 1.4k
Comparing changes
Open a pull request
base repository: dpkp/kafka-python
base: master
head repository: hackaugusto/kafka-python
compare: master
- 5 commits
- 7 files changed
- 3 contributors
Commits on Jan 11, 2021
-
Avoid 100% CPU usage while socket is closed (sleep)
After stop/start kafka service, kafka-python may use 100% CPU caused by busy-retry while the socket was closed. This fix the issue by sleep 0.1 second if the fd is negative.
Configuration menu - View commit details
-
Copy full SHA for b061f3d - Browse repository at this point
Copy the full SHA b061f3dView commit details -
Support connecting through SOCKS5 proxies (#2169)
Implement support for SOCKS5 proxies. Implement a new proxy wrapper that handles SOCKS5 connection, authentication and requesting connections to the actual Kafka broker endpoint. The proxy can be configured via a new keyword argument `socks5_proxy` to consumers, producers or admin client. The value is URL with optional username and password. E.g. `socks5://user:secret@proxy.example.com:10800` The implementation is done in state machine that emulates repeated calls to connect_ex. The rationale with this design is minimal changes to the actual BrokerConnection object.
Configuration menu - View commit details
-
Copy full SHA for 346c01e - Browse repository at this point
Copy the full SHA 346c01eView commit details
Commits on Jan 14, 2021
-
bugfix: fix infinite loop on KafkaAdminClient
An infinite loop may happen with the following pattern: self._send_request_to_node(self._client.least_loaded_node(), request) The problem happens when `self._client`'s cluster metadata is out-of-date, and the result of `least_loaded_node()` is a node that has been removed from the cluster but the client is unware of it. When this happens `_send_request_to_node` will enter an infinite loop waiting for the chosen node to become available, which won't happen, resulting in an infinite loop. This commit introduces a new method named `_send_request_to_least_loaded_node` which handles the case above. This is done by regularly checking if the target node is available in the cluster metadata, and if not, a new node is chosen. Notes: - This does not yet cover every call site to `_send_request_to_node`, there are some other places were similar race conditions may happen. - The code above does not guarantee that the request itself will be sucessful, since it is still possible for the target node to exit, however, it does remove the infinite loop which can render client code unusable.
Configuration menu - View commit details
-
Copy full SHA for 1e2ac76 - Browse repository at this point
Copy the full SHA 1e2ac76View commit details -
bugfix: race among _connecting and cluster metadata
A call to `maybe_connect` can be performed while the cluster metadata is being updated. If that happens, the assumption that every entry in `_connecting` has metadata won't hold. The existing assert will then raise on every subsequent call to `poll` driving the client instance unusable. This fixes the issue by ignoring connetion request to nodes that do not have the metadata available anymore.
Configuration menu - View commit details
-
Copy full SHA for 8c6ab20 - Browse repository at this point
Copy the full SHA 8c6ab20View commit details -
bugfix: infinite loop when send msgs to controller
If the value `_controller_id` is out-of-date and the node was removed from the cluster, `_send_request_to_node` would enter an infinite loop.
Configuration menu - View commit details
-
Copy full SHA for 2315638 - Browse repository at this point
Copy the full SHA 2315638View commit details
This comparison is taking too long to generate.
Unfortunately it looks like we can’t render this comparison for you right now. It might be too big, or there might be something weird with your repository.
You can try running this command locally to see the comparison on your machine:
git diff master...master