docker 部署 kafka 2.8 集群

### 使用 Docker 部署 Kafka 2.8 版本集群 #### 准备工作 为了成功部署基于 Docker 的 Kafka 2.8 版本集群，需先确保主机已安装并配置好 Docker 和 Docker Compose。 #### 创建 Zookeeper 节点 Zookeeper 是 Kafka 运作所必需的服务之一。创建多个 Zookeeper 实例来提高系统的可靠性： ```yaml version: '3' services: zoo1: image: wurstmeister/zookeeper container_name: zoo1 ports: - "2181:2181" environment: ZOO_MY_ID: 1 ZOO_SERVERS: server.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888 zoo2: image: wurstmeister/zookeeper container_name: zoo2 ports: - "2182:2181" environment: ZOO_MY_ID: 2 ZOO_SERVERS: server.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888 zoo3: image: wurstmeister/zookeeper container_name: zoo3 ports: - "2183:2181" environment: ZOO_MY_ID: 3 ZOO_SERVERS: server.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888 ``` 保存上述 YAML 文件为 `docker-compose-zk.yml` 并执行命令启动 Zookeeper 集群： ```bash docker-compose -f docker-compose-zk.yml up -d ``` #### 构建 Kafka Broker 节点 接下来定义三个 Kafka broker 容器，在同一网络下连接到之前建立的 Zookeeper 集群: ```yaml version: '3' services: kafka1: image: wurstmeister/kafka:2.8.0-kafka-2.8.0 hostname: kafka1 container_name: kafka1 depends_on: - zoo1 - zoo2 - zoo3 ports: - "9092:9092" environment: KAFKA_BROKER_ID: 1 KAFKA_ZOOKEEPER_CONNECT: zoo1:2181,zoo2:2181,zoo3:2181 KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092 KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 3 kafka2: image: wurstmeister/kafka:2.8.0-kafka-2.8.0 hostname: kafka2 container_name: kafka2 depends_on: - zoo1 - zoo2 - zoo3 ports: - "9093:9093" environment: KAFKA_BROKER_ID: 2 KAFKA_ZOOKEEPER_CONNECT: zoo1:2181,zoo2:2181,zoo3:2181 KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9093 KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 3 kafka3: image: wurstmeister/kafka:2.8.0-kafka-2.8.0 hostname: kafka3 container_name: kafka3 depends_on: - zoo1 - zoo2 - zoo3 ports: - "9094:9094" environment: KAFKA_BROKER_ID: 3 KAFKA_ZOOKEEPER_CONNECT: zoo1:2181,zoo2:2181,zoo3:2181 KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9094 KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 3 networks: default: external: name: bridge ``` 同样地，将此文件命名为 `docker-compose-kafka.yml` 后运行如下指令以启动 Kafka brokers[^4]: ```bash docker-compose -f docker-compose-kafka.yml up -d ``` #### 测试与验证 完成以上步骤之后可以尝试发送消息测试集群是否正常运作。也可以通过 CMAK (Cluster Manager And Monitor) 来监控管理 Kafka 集群状态[^2]: ```bash docker run -d --name cmak \ --restart=always \ -p 9000:9000 \ -e ZK_HOSTS="zoo1:2181,zoo2:2181,zoo3:2181" \ vimagick/cmak:latest ``` 访问 http://localhost:9000 可进入 CMAK Web UI 页面查看集群详情。