Bump KafkaConsumer's request_timeout to 305000ms #1002
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The Java Consumer [changed the value to 305000 in `0.10.1.0`](https://kafka.apache.org/documentation/#upgrade_1010_notable): > The new Java Consumer now supports heartbeating from a background thread. There is a new configuration max.poll.interval.ms which controls the maximum time between poll invocations before the consumer will proactively leave the group (5 minutes by default). The value of the configuration request.timeout.ms must always be larger than max.poll.interval.ms because this is the maximum time that a JoinGroup request can block on the server while the consumer is rebalancing, so we have changed its default value to just above 5 minutes. Finally, the default value of session.timeout.ms has been adjusted down to 10 seconds, and the default value of max.poll.records has been changed to 500. See the current upstream value here: https://kafka.apache.org/documentation/#configuration
|
It does seem strange to hit 40sec timeouts in an environment like that. |
|
Would like to wait on changing this default until the background heartbeat thread is implemented |
|
Hi @dpkp, |
|
Closing in favor of #1266 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The Java Consumer changed the value to 305000 in
0.10.1.0:See the current upstream value here: https://kafka.apache.org/documentation/#configuration
This value was only changed in
KafkaConsumer. So we don't also need to updateKafkaProducer/KafkaClientas the upstream Java implementations still default to30000. The broker default is also30000.Note:
Merging this may be slightly premature since #948 hasn't been done to add
max.poll.interval.msyet, but I don't see any harm in adding this now.Sessions will still timeout, this just gives the brokers/cluster a bit longer to handle things like consumer group coordination etc.
In one of our dev environments, we were seeing a lot of these timeouts at the current default of
40000using the newKafkaConsumer. Weirdly, the timeouts were being hit even though the consumer had no messages to consume and they were starting with default offset oflatest.This doesn't quite feel right. It's a dev cluster, so everything on VMs and the network can be flaky/slow. But I still wouldn't have expected a consumer that was simply sitting there would be hitting this request timeout at 40 seconds.
Still, when I bumped their request timeouts up to this new
305000value, the timeout errors went away, and the consumers remained stable as best I could tell.