Kafka 的消费组时不时就会断开连接

loongshawn

于 2021-09-08 12:53:35 发布

阅读量8.3k

点赞数

CC 4.0 BY-SA版权

分类专栏： Kafka 职场 Message Queue 文章标签： kafka java 数据结构

本文链接：https://blog.csdn.net/loongshawn/article/details/120177636

本文分析了Kafka消费组断开连接的原因，包括一个topic多个consumer导致的rebalancing和consumer poll超时。解决方案是调整Kafka参数如max.poll.records和max.poll.interval.ms，以及优化消费逻辑，确保消费速度与poll间隔匹配。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

问题现象

Kafka消费组时不时就会断开连接¹，提示如下异常

2019-19-28 12:36:44.711 |-INFO  [kafka-coordinator-heartbeat-thread | loongshawn] org.apache.kafka.clients.consumer.internals.AbstractCoordinator [1234] -| [Consumer clientId=loongshawn-xxxxxxxxxxxxxxxxxxxxxxxxxxxxx-127.0.0.1, groupId=loongshawn] Member loongshawn-xxxxxxxxxxxxxxxxxxxxxxxxxxxxx-127.0.0.1 sending LeaveGroup request to coordinator xx.xx.xx.xx:9092 (id: 23456223346 rack: null) due to consumer poll timeout has expired. This means the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms, which typically implies that the poll loop is spending too much time processing messages. You can address this either by increasing max.poll.interval.ms or by reducing the maximum size o

2019-19-28 12:36:44.711 |-INFO [kafka-coordinator-heartbeat-thread | loongshawn] org.apache.kafka.clients.consumer.internals.AbstractCoordinator [1234] -| [Consumer clientId=loongshawn-xxxxxxxxxxxxxxxxxxxxxxxxxxxxx-127.0.0.1, groupId=loongshawn] Member loongshawn-xxxxxxxxxxxxxxxxxxxxxxxxxxxxx-127.0.0.1 sending LeaveGroup request to coordinator xx.xx.xx.xx:9092 (id: 23456223346 rack: null) due to consumer poll tim