Ubuntu快速安裝kafka叢集

搭建叢集做少需要三台伺服器 一個centos和兩個ubuntu

每台都按照下面做：

安裝JDK

sudo apt-get install default-jre
    java -version
    apt-get install default-jdk
    java -version
           

1. 安裝：

sudo apt-get install zookeeper
/etc/init.d/zookeeper start  //自啟動
           

預設資訊：

#安裝路徑

/usr/share/zookeeper

#配置檔案

/etc/zookeeper/conf/zoo.cfg

2. 啟動zookeeper

cd /usr/share/zookeeper/bin
sudo sh zkServer.sh start
           

ZooKeeper JMX enabled by default
Using config: /etc/zookeeper/conf/zoo.cfg
Starting zookeeper ... STARTED
           

啟動zookeeper時遇到錯誤以下錯誤

zkServer.sh: 157: zkServer.sh: Syntax error: "(" unexpected (expecting ";;")
原因是：zookeeper使用的shell版本和系統使用的shell版本不相容，目前ubuntu系統的shell預設使用的是dash,而zookeeper使用的是bash
解決辦法：
dpkg-reconfigure dash
Tab 移動到NO(選擇否) 回車即可
驗證是否啟動成功
           

/usr/share/zookeeper/bin$ sudo zkCli.sh -server localhost:2181

Connecting to localhost:2181
Welcome to ZooKeeper!
JLine support is enabled
           

WATCHER::

WatchedEvent state:SyncConnected type:None path:null

zookeeper基本操作

啟動ZK服務: sh bin/zkServer.sh start

檢視ZK服務狀态: sh bin/zkServer.sh status

停止ZK服務: sh bin/zkServer.sh stop

重新開機ZK服務: sh bin/zkServer.sh restart 
           

3.1 配置檔案

需要修改server連接配接位址 dataDir可以不修改 新增logDir=(可選)

# http://hadoop.apache.org/zookeeper/docs/current/zookeeperAdmin.html
  
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial 
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between 
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
dataDir=/var/lib/zookeeper
# Place the dataLogDir to a separate physical disc for better performance
# dataLogDir=/disk2/zookeeper

# the port at which the clients will connect
clientPort=2181

# specify all zookeeper servers
# The fist port is used by followers to connect to the leader
# The second one is used for leader election
server.1=IP1:2888:3888
server.2=IP2:2888:3888
server.3=IP3:2888:3888


# To avoid seeks ZooKeeper allocates space in the transaction log file in
# blocks of preAllocSize kilobytes. The default block size is 64M. One reason
# for changing the size of the blocks is to reduce the block size if snapshots
# are taken more often. (Also, see snapCount).
#preAllocSize=65536

# Clients can submit requests faster than ZooKeeper can process them,
# especially if there are a lot of clients. To prevent ZooKeeper from running
# out of memory due to queued requests, ZooKeeper will throttle clients so that
# there is no more than globalOutstandingLimit outstanding requests in the
# system. The default limit is 1,000.ZooKeeper logs transactions to a
# transaction log. After snapCount transactions are written to a log file a
# snapshot is started and a new transaction log file is started. The default
# snapCount is 10,000.
#snapCount=1000

# If this option is defined, requests will be will logged to a trace file named
# traceFile.year.month.day. 
#traceFile=

# Leader accepts client connections. Default value is "yes". The leader machine
# coordinates updates. For higher update throughput at thes slight expense of
# read throughput the leader can be configured to not accept clients and focus
# on coordination.
#leaderServes=yes
           

安裝kafka

wget http://mirror.bit.edu.cn/apache/kafka/2.5.0/kafka_2.12-2.5.0.tgz
tar -zxvf kafka_2.12-2.5.0.tgz
mv  kafka_2.12-2.5.0.tgz kafka
cd kafka
//x修改配置檔案 server.properties  其他可不變
broker.id=0   每台伺服器設定不同的值  
zookeeper.connect=ip1.3:2181,ip2:2181,ip3:2181

bin/kafka-server-start.sh -daemon  config/server.properties//啟動服務

bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic test //建立topic
bin/kafka-topics.sh --list --bootstrap-server localhost:9092  //檢視topic
bin/kafka-console-producer.sh --bootstrap-server localhost:9092 --topic test //生産消息
> hello
> good moring
bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning //消費者擷取消息