kafka叢集搭建

安裝vim

安裝ssh sudo apt-get install openssh-server

service ssh start

192.168.1.102

192.168.1.103

192.168.1.104

sudo mkdir /data/zookeeper/data

sudo mkdir /data/zookeeper/log

建立myid

三台機器分别執行如下操作

搭建zookeeper叢集

進入kafka根目錄 /home/feng/kafka

修改zookeeper配置檔案

dataDir=/tmp/zookeeper

# the port at which the clients will connect

clientPort=2181

# disable the per-ip limit on the number of connections since this is a non-production config

maxClientCnxns=200

initLimit=5

syncLimit=2

server.1=192.168.1.102:2888:3888

server.2=192.168.1.103:2888:3888

server.3=192.168.1.104:2888:3888

啟動zookeeper

nohup ./bin/zookeeper-server-start.sh config/zookeeper.properties >logs/zookeeper.log 2>1 &

檢視zookeeper日志是否報錯，如果報錯，更改三台機器的啟動順序，重新啟動，用來确定leader和follower

最後檢視2181端口是否啟動以确定是否成功

netstat -anput | grep 2181

搭建kafka叢集

進入kafka根目錄 /home/feng/kafka

修改kafka配置檔案server.properties

<a href="https://s4.51cto.com/wyfs02/M00/96/F0/wKioL1km5A6DlPWJAAEROo-4w8E401.jpg-wh_500x0-wm_3-wmp_4-s_3176663316.jpg" target="_blank"></a>

<a href="https://s4.51cto.com/wyfs02/M01/96/EF/wKiom1km5BGyY4MsAAEbimkkyTU443.jpg-wh_500x0-wm_3-wmp_4-s_2336311125.jpg" target="_blank"></a>

分别修改

broker.id=1    //本機myid

host.name=192.168.1.102   //本機ip

advertised.host.name=192.168.1.102   //本機ip

zookeeper.connect=192.168.1.102:2181,192.168.1.103:2181,192.168.1.104:2181   //zookeeper叢集

啟動kafka叢集

nohup ./bin/kafka-server-start.sh config/server.properties &gt;logs/kafka.log 2&gt;1 &amp;

測試kafka叢集

1-進入kafka根目錄，建立topic--test

bin/kafka-topics.sh --create --zookeeper 192.168.1.xx:2181,192.168.1.xx:2181,192.168.1.xx:2181 --replication-factor 1 --partitions 1 --topic test

2-列出已建立的topic清單

bin/kafka-topics.sh --list --zookeeper localhost:2181

3-模拟用戶端去發送消息

bin/kafka-console-producer.sh --broker-list 192.168.1.xx:9092,192.168.1.xx:9092,192.168.1.xx:9092 --topic test

4-模拟用戶端去接受消息

bin/kafka-console-consumer.sh --zookeeper 192.168.1.xx:2181,192.168.1.xx:2181,192.168.1.xx:2181 --from-beginning --topic test

7、自己測試結果

bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 3 --partitions 1 --topic test

本文轉自 無心低語 51CTO部落格，原文連結：http://blog.51cto.com/fengzhankui/1927753，如需轉載請自行聯系原作者