kafka动态修改配置

如何修改一个已存在的topic的配置

动态修改配置

修改topic的数据删除周期,默认是1天,配置为delete.retention.ms,通过kafka的shell脚本可以修改配置,改成30天
kafka-topics.sh --zookeeper ip:2181 --alter --topic glance_source_mr_job --config delete.retention.ms=2592000000
修改后可以查看kafka的日志,看到该topic的配置已经被覆盖
[2019-01-22 16:41:25,351] INFO Processing override for entityPath: topics/glance_source_mr_job with config: {delete.retention.ms=2592000000} (kafka.server.DynamicConfigManager)

原理分析

shell脚本通过java连接到zookeeper,修改/config/topics/glance_source_mr_job这个topic的配置,如下所示

1
2
3
4
5
6
7
8
9
10
11
12
13
get /config/topics/glance_source_mr_job
{"version":1,"config":{"delete.retention.ms":"2592000000"}}
cZxid = 0x500b258dd
ctime = Tue Jan 15 19:36:47 CST 2019
mZxid = 0x500f8d172
mtime = Tue Jan 22 16:41:25 CST 2019
pZxid = 0x500b258dd
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 59
numChildren = 0

kafka会一直监听这些zookeeper节点,当发生修改时,kafka获取到该事件,并做完成对配置的更新,可以参考类DynamicConfigManager

ulysses wechat
订阅+