天天看點
傳回

搭建zookeeper+kafka叢集

 　　　　　　　　　　　　　　　　　　　搭建zookeeper+kafka叢集

一、環境及準備

叢集環境：

搭建zookeeper+kafka叢集

軟體版本：

搭建zookeeper+kafka叢集

部署前操作：

關閉防火牆，關閉selinux(生産環境按需關閉或打開)
同步伺服器時間，選擇公網ntpd伺服器或者自建ntpd伺服器
[root@es1 ~]# crontab -l #為了友善直接使用公網伺服器
#update time
*/5 * * * *  /usr/bin/rdate -s time-b.nist.gov &>/dev/null

二、zookeeper叢集安裝配置

1.安裝jvm依賴環境(三台)

安裝JDK

[root@node01 ~]# rpm -ivh jdk1.8.0_162-x64.rpm  #為了以後更新麻煩直接安裝1.8
Preparing...                ########################################### [100%]
   1:jdk1.8.0_162           ########################################### [100%]

設定Java環境

[root@node01 ~]# cat /etc/profile.d/java.sh  #編輯Java環境配置檔案
export JAVA_HOME=/usr/java/latest
export CLASSPATH=$JAVA_HOME/lib/tools.jar
export PATH=$JAVA_HOME/bin:$PATH
[root@node01 ~]# . /etc/profile.d/java.sh  
[root@node01 ~]# java -version   #确認配置
java version "1.8.0_162"
Java(TM) SE Runtime Environment (build 1.8.0_162-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.162-b12, mixed mode)
2.安裝配置zookeeper 
[root@node01 ~]#wget http://mirror.bit.edu.cn/apache/zookeeper/zookeeper-3.4.13/zookeeper-3.4.13.tar.gz
[root@node01 ~]#tar xf zookeeper-3.4.13.tar.gz -C /usr/local
[root@node01 ~]#cd /usr/local
[root@node01 local]#ln -sv zookeeper-3.4.13 zookeeper
[root@node01 local]#cd zookeeper/conf
[root@node01 conf]# cp  zoo_sample.cfg  zoo.cfg
[root@node01 conf]# vim zoo.cfg
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/Data/zookeeper
clientPort=2181
server.1=172.16.150.154:2888:3888
server.2=172.16.150.155:2888:3888
server.3=172.16.150.156:2888:3888

＃配置參數說明：
tickTime:用戶端與伺服器或者伺服器與伺服器之間每個tickTime時間就會發送一次心跳。通過心跳不僅能夠用來監聽機器的工作狀态，還可以通過心跳來控制Flower跟Leader的通信時間，預設2秒
initLimit：叢集中的follower伺服器(F)與leader伺服器(L)之間初始連接配接時能容忍的最多心跳數（tickTime的數量）。
syncLimit：叢集中flower伺服器（F）跟leader（L）伺服器之間的請求和答應最多能容忍的心跳數。   
dataDir：該屬性對應的目錄是用來存放myid資訊跟一些版本，日志，跟伺服器唯一的ID資訊等。
clientPort：用戶端連接配接的接口，用戶端連接配接zookeeper伺服器的端口，zookeeper會監聽這個端口，接收用戶端的請求通路！這個端口預設是2181。
service.N=YYY:A:B
N：代表伺服器編号（也就是myid裡面的值）
YYY：伺服器位址
A：表示 Flower 跟 Leader的通信端口，簡稱服務端内部通信的端口（預設2888）
B：表示 是選舉端口（預設是3888）

建立zookeeper所需要的目錄和myid檔案

[root@node01 conf]# mkdir -pv /Data/zookeeper 
mkdir: 已建立目錄 "/Data"
mkdir: 已建立目錄 "/Data/zookeeper"
[root@node01 conf]# echo "1" > /Data/zookeeper/myid #myid檔案，裡面的内容為數字，用于辨別主機，如果這個檔案沒有的話，zookeeper無法啟動

其他節點配置相同，除以下配置：

echo "x" > /Data/zookeeper/myid #唯一
3.啟動zookeeper（三台） 
[root@node01 zookeeper]# cd /usr/local/zookeeper/bin
[root@node01 bin]# ./zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... taSTARTED
[root@node01 bin]# tailf zookeeper.out 
2019-02-13 14:05:28,088 [myid:] - INFO  [main:QuorumPeerConfig@136] - Reading configuration from: /usr/local/zookeeper/bin/../conf/zoo.cfg
2019-02-13 14:05:28,102 [myid:] - INFO  [main:QuorumPeer$QuorumServer@184] - Resolved hostname: 172.16.150.154 to address: /172.16.150.154
2019-02-13 14:05:28,102 [myid:] - INFO  [main:QuorumPeer$QuorumServer@184] - Resolved hostname: 172.16.150.156 to address: /172.16.150.156
2019-02-13 14:05:28,103 [myid:] - INFO  [main:QuorumPeer$QuorumServer@184] - Resolved hostname: 172.16.150.155 to address: /172.16.150.155
2019-02-13 14:05:28,103 [myid:] - INFO  [main:QuorumPeerConfig@398] - Defaulting to majority quorums
2019-02-13 14:05:28,108 [myid:1] - INFO  [main:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3
2019-02-13 14:05:28,108 [myid:1] - INFO  [main:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0
2019-02-13 14:05:28,108 [myid:1] - INFO  [main:DatadirCleanupManager@101] - Purge task is not scheduled.
2019-02-13 14:05:28,119 [myid:1] - INFO  [main:QuorumPeerMain@130] - Starting quorum peer
2019-02-13 14:05:28,128 [myid:1] - INFO  [main:ServerCnxnFactory@117] - Using org.apache.zookeeper.server.NIOServerCnxnFactory as server connection factory
2019-02-13 14:05:28,134 [myid:1] - INFO  [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:2181
2019-02-13 14:05:28,144 [myid:1] - INFO  [main:QuorumPeer@1158] - tickTime set to 2000
2019-02-13 14:05:28,144 [myid:1] - INFO  [main:QuorumPeer@1204] - initLimit set to 10
2019-02-13 14:05:28,144 [myid:1] - INFO  [main:QuorumPeer@1178] - minSessionTimeout set to -1
2019-02-13 14:05:28,144 [myid:1] - INFO  [main:QuorumPeer@1189] - maxSessionTimeout set to -1
2019-02-13 14:05:28,151 [myid:1] - INFO  [main:QuorumPeer@1467] - QuorumPeer communication is not secured!
2019-02-13 14:05:28,153 [myid:1] - INFO  [main:QuorumPeer@1496] - quorum.cnxn.threads.size set to 20
2019-02-13 14:05:28,196 [myid:1] - INFO  [ListenerThread:QuorumCnxManager$Listener@736] - My election bind port: /172.16.150.154:3888
........

zookeeper服務檢查

[root@node01 bin]#  netstat -nlpt | grep -E "2181|2888|3888"
tcp        0      0 0.0.0.0:2181            0.0.0.0:*               LISTEN      6242/java           
tcp        0      0 172.16.150.154:3888     0.0.0.0:*               LISTEN      6242/java 
[root@node02 ~]#  netstat -nlpt | grep -E "2181|2888|3888"
tcp        0      0 0.0.0.0:2181            0.0.0.0:*               LISTEN      5197/java           
tcp        0      0 172.16.150.155:3888     0.0.0.0:*               LISTEN      5197/java  
[root@node03 ~]#  netstat -nlpt | grep -E "2181|2888|3888"
tcp        0      0 0.0.0.0:2181            0.0.0.0:*               LISTEN      5304/java           
tcp        0      0 172.16.150.156:2888     0.0.0.0:*               LISTEN      5304/java   #哪台是leader，那麼他就擁有2888端口，可以看到目前node3節點為leader
tcp 0 0 172.16.150.156:3888 0.0.0.0:* LISTEN 5304/java

測試伺服器是否正常

[root@node01 bin]# yum install telnet nc -y
[root@node01 bin]# telnet 172.16.150.154 2181
Trying 172.16.150.154...
Connected to 172.16.150.154.
Escape character is '^]'.
exit
Connection closed by foreign host.
[root@node01 bin]# echo "stat"|nc 172.16.150.154 2181  #conf 可以顯示配置資訊，cons可以顯示所有用戶端連接配接的詳細資訊，mntr指令比stat指令更詳細
Zookeeper version: 3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03, built on 06/29/2018 04:05 GMT
Clients:
 /172.16.150.154:54989[0](queued=0,recved=1,sent=0)

Latency min/avg/max: 0/0/0
Received: 1
Sent: 0
Connections: 1
Outstanding: 0
Zxid: 0x1000000d4
Mode: follower
Node count: 138

連接配接zookeeper

[root@node01 bin]# ./zkCli.sh  -server 172.16.150.154:2181
Connecting to 172.16.150.154:2181
2019-02-13 14:25:24,060 [myid:] - INFO  [main:Environment@100] - Client environment:zookeeper.version=3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03, built on 06/29/2018 04:05 GMT
....
[zk: 172.16.150.154:2181(CONNECTED) 0] h  #檢視指令幫助
ZooKeeper -server host:port cmd args
    stat path [watch]
    set path data [version]
    ls path [watch]
    delquota [-n|-b] path
    ls2 path [watch]
    setAcl path acl
    setquota -n|-b val path
    history 
    redo cmdno
    printwatches on|off
    delete path [version]
    sync path
    listquota path
    rmr path
    get path [watch]
    create [-s] [-e] path data acl
    addauth scheme auth
    quit 
    getAcl path
    close 
    connect host:port
[zk: 172.16.150.154:2181(CONNECTED) 1] quit #退出

設定jconsole連接配接zookeeper 

[root@node01 bin]# vim zkServer.sh #修改54行，172.16.150.154是本機的ip位址，8899是jconsole的連接配接位址，關閉ssl和認證
 ZOOMAIN="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.local.only=$JMXLOCALONLY -Djava.rmi.server.hostname=172.16.150.154     -Dcom.sun.management.jmxremote.port=8899 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false org.apache    .zookeeper.server.quorum.QuorumPeerMain"
搭建zookeeper+kafka叢集
./zkServer.sh stop && ./zkServer.sh start #重新開機服務，使用jconsole連接配接zookeeper伺服器，選擇遠端連接配接，輸入172.16.150.154:8899 即可

#登入jconsole

搭建zookeeper+kafka叢集
搭建zookeeper+kafka叢集
搭建zookeeper+kafka叢集
zookeeper開啟超級使用者   #關于zookeeper ACL權限請參考官方文檔

當設定了znode權限，但是密碼忘記了怎麼辦？如果忘記了該子節點的授權使用者名還有密碼。這裡是比較蛋疼的事情。由于我們基本上找不到because在base64反編碼後再sha1反編碼後的樣子，是以基本上這個節點的控制權可以說是失去了。還好Zookeeper提供了超級管理者機制。

[root@node01 bin]# cd /usr/local/zookeeper/lib/
[root@node01 lib]# java -cp ../zookeeper-3.4.13.jar:./log4j-1.2.17.jar:./slf4j-api-1.7.25.jar:./slf4j-log4j12-1.7.25.jar org.apache.zookeeper.server.auth.DigestAuthenticationProvider super:super
super:super->super:gG7s8t3oDEtIqF6DM9LlI/R+9Ss=   #生成密文
[root@node01 lib]# vim ../bin/zkServer.sh SUPER_ACL="-Dzookeeper.DigestAuthenticationProvider.superDigest=super:gG7s8t3oDEtIqF6DM9LlI/R+9Ss="
搭建zookeeper+kafka叢集

#添加以上标記的内容

驗證使用者是否有效

[root@node01 lib]# cd ../bin/
[root@node01 bin]# ./zkServer.sh stop #修改配置檔案後重新開機服務
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED
[root@node01 bin]# ./zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@node01 bin]# ./zkCli.sh -server 172.16.150.154
......
[zk: 172.16.150.154(CONNECTED) 0] addauth digest super:super  #對之前添加的使用者進行認證
[zk: 172.16.150.154(CONNECTED) 1] quit

三、kafka叢集安裝

kafka同樣依賴Java環境，由于和zookeeper在相同的機器上，之前已經安裝過了，所有可以直接跳過Java環境安裝

1.安裝kafka 
[root@node01 ~]#wget http://mirrors.tuna.tsinghua.edu.cn/apache/kafka/2.0.1/kafka_2.11-2.0.1.tgz
[root@node01 ~]#tar xf kafka_2.11-2.0.1.tgz -C /usr/local
[root@node01 ~]#cd /usr/local
[root@node01 local]# ln -sv kafka_2.11-2.0.1 kafka
[root@node01 local]# cd kafka/config/
[root@node01 config]#cp server.properties server.properties-bak
[root@node01 config]# grep "^[a-Z]" server.properties 
broker.id=1  #唯一
listeners=PLAINTEXT://172.16.150.154:9092  #修改為本機位址
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
log.dirs=/Data/kafka-logs #資料目錄，kafka-logs會自動采集
num.partitions=3
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=172.16.150.154:2181,172.16.150.155:2181,172.16.150.156:2181  #zokeeper叢集位址，以","為分割
zookeeper.connection.timeout.ms=6000
group.initial.rebalance.delay.ms=0

其他節點配置相同，除以下内容：

broker.id=1  #唯一
listeners=PLAINTEXT://172.16.150.154:9092  #修改為本機位址

啟動服務

[root@node01 config]# cd ../bin
[root@node01 bin]# 
./kafka-server-start.sh -daemon ../config/server.properties #背景運作

驗證服務是否正常

登入zookeeper驗證：

[zk: 172.16.150.154(CONNECTED) 5] get  /brokers/ids/1 #檢視節點broker id為1的資訊
{"listener_security_protocol_map":{"PLAINTEXT":"PLAINTEXT"},"endpoints":["PLAINTEXT://172.16.150.154:9092"],"jmx_port":-1,"host":"172.16.150.154","timestamp":"1549953312989","port":9092,"version":4}
cZxid = 0x10000002e
ctime = Tue Feb 12 14:35:13 CST 2019
mZxid = 0x10000002e
mtime = Tue Feb 12 14:35:13 CST 2019
pZxid = 0x10000002e
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x10077feb7bc0001
dataLength = 198
numChildren = 0

建立topic驗證

#154上建立一個生産者
[root@node01 ~]# cd /usr/local/kafka/bin/
[root@node01 bin]# ./kafka-topics.sh --create --zookeeper 172.16.150.154:2181 --replication-factor 1 --partitions 1 --topic Test
Created topic "Test".
[root@node01 bin]# ./kafka-console-producer.sh --broker-list 172.16.150.154:9092 --topic Test
#其他伺服器上建立一個消費者
[root@node02 ~]# cd /usr/local/kafka/bin/
[root@node02 bin]# ./kafka-console-consumer.sh --bootstrap-server 172.16.150.155:9092 --topic Test --from-beginning
#啟動成功後，在154上輸入任意内容，在另一台機器上檢視是否會同步顯示

四、zookeeper及kafka監控工具

1.zookeeper監控工具(沒有安裝過，有需要請參考官方文檔) 
zookeeper監控工具位址：https://github.com/soabase/exhibitor
2.kafka監控工具

1)KafkaOffsetMonitor

[root@node01 ~]#mkdir KafkaMonitor
[root@node01 ~]#cd KafkaMonitor/
[root@node01 ~]#wget https://github.com/quantifind/KafkaOffsetMonitor/releases/download/v0.2.1/KafkaOffsetMonitor-assembly-0.2.1.jar
[root@node01 ~]#nohup java -cp KafkaOffsetMonitor-assembly-0.2.0.jar  com.quantifind.kafka.offsetapp.OffsetGetterWeb --zk 172.16.150.154:2181,172.16.150.155:2181,172.16.150.156:2181 -port 8088 --refresh 5.seconds --retain 1.days &

通路(由于測試環境沒有資料，所有我直接登入生産環境來示範)：

檢視曾經消費者的情況

搭建zookeeper+kafka叢集

檢視其中任意一個消費者資訊

搭建zookeeper+kafka叢集

主意lag字段，表示是否有延遲

搭建zookeeper+kafka叢集

檢視topic

搭建zookeeper+kafka叢集
搭建zookeeper+kafka叢集
搭建zookeeper+kafka叢集
搭建zookeeper+kafka叢集

2)kafka-manager

[root@node01 ~]# unzip kafka-manager-1.3.3.7.zip  #直接使用已經編譯完成的軟體包(連結: https://pan.baidu.com/s/12sswyPo7-e9R3mZQ3ba-dA 提取碼: jz6s)
[root@node01 ~]# cd kafka-manager-1.3.3.7
[root@node01 ~]# cd conf/
[root@node01 ~]# vim application.conf
kafka-manager.zkhosts="172.16.150.154:2181,172.16.150.155:2181,172.16.150.156:2181"  #填寫zookeeper伺服器位址和端口
[root@node01 ~]#cd ../bin/
[root@node01 ~]# ./kafka-manager -Dconfig.file=../conf/application.conf -Dhttp.port=8888  #8888表示監聽端口，啟動後直接通路

#kafka-manager安裝需要編譯，并且過程複雜、成功率低建議使用其他人已經編譯過得直接使用

3)kafka eagle

未實驗，聽說不錯有興趣的可以學習一下

作為一個初學者，有很多地方都沒有了解，寫的比較簡單，望海涵！

參考文檔：

https://zookeeper.apache.org/doc/r3.4.13/zookeeperAdmin.html

https://blog.csdn.net/pdw2009/article/details/73794525

https://blog.csdn.net/lizhitao/article/details/25667831

https://www.cnblogs.com/dadonggg/p/8242682.html

https://www.cnblogs.com/dadonggg/p/8205302.html

"一勞永逸" 的話，有是有的，而 "一勞永逸" 的事卻極少

上一篇: oracle資料庫從入門到精通
下一篇: Flink叢集搭建