发送消息失败

报错：Invalid Replication factor, 1 larger than available brokers: 0

解决1：参考：无法启动

无法启动

查看日志 logs/server.log，发现启动时警告：Found a corrupted index file corresponding to log file /data/kafka/FirstTopic-6/00000000000000000494.log due to Corrupt time index found, time index file (/apps/kafka/data/FirstTopic-6/00000000000000000494.timeindex) has non-zero size but the last timestamp is 0 which is less than the first timestamp 1553720469480}, recovering segment and rebuilding index files… (kafka.log.Log) }}

解决1

可能是因为之前强制kill生成了错误的index文件，找到kafka data目录，把 /data/kafka/*.log   *.timeindex 删掉就好了

find . -name “*.log” | xargs  rm

find . -name “*.timeindex” | xargs  rm

延伸

• https://blog.experteer.engineering/kafka-corrupted-index-file-warnings-after-broker-restart.html – 如何防止强制kill，导致生成错误index文件

TODO

• kafka data文件中，log、index、timeindex 分别是什么文件？

常用命令

查看group

./bin/kafka-consumer-groups.sh --bootstrap-server 127.0.0.1:9092 localhost:2181 --list

查看指定消费组的消费位置offset（消费情况）

./bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group md-actionlog-consumer

查看topic

./bin/kafka-topics.sh --zookeeper localhost:2181 --list

创建topic

./bin/kafka-topics.sh -zookeeper localhost:2181 --create --partitions 5 --replication-factor 1 --topic ws-publicform

查看topic分布情况

./bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic actionlog

查看topic配置

./bin/kafka-configs.sh --zookeeper localhost:2181 --describe --entity-name __consumer_offsets --entity-type topics

修改topic的清理策略，防止log过多占用磁盘

./bin/kafka-topics.sh --zookeeper localhost:2181 --topic __consumer_offsets --alter --config cleanup.policy=delete

等几分钟，比较老的log就会被自动清理掉

发消息

./bin/kafka-console-producer.sh --broker-list localhost:9092 --topic ws-publicform