安装storm需要安装下面一些依赖组件:
1. 推荐版本
ZeroMQ2.1.7 Java6 Python2.6.6
2. python和jdk安装
jdk安装完成后需要设置下JAVA_HOME。
vi /etc/profile
export JAVA_HOME=/opt/soft/jdk
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib/tools.jar:$JAVA_HOME/lib/rt.jar
source /etc/profile
3. zookeeper安装
在 http://zookeeper.apache.org/releases.html 上选择合适版本下载,直接解压zookeeper包到指定目录即可。启动zookeeper前需要修改下zoo.cfg的配置:
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/data/zookeeper
dataLogDir=/data/zookeeper
# the port at which the clients will connect
clientPort=2181
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1
server.1=zookeeper-node1:2888:3888
server.2=zookeeper-node2:2888:3888
server.3=zookeeper-node3:2888:3888
zookeeper-node1的映射在/etc/hosts中添加,完成zoo.cfg的修改后,还需要在dataDir下生成一加个myid文件,并在文件中输入中对应的编号。例如:
在server.1的机器上输入myid文件的内容为:
echo 1 >> /data/zookeeper/myid
现在可以一次启动各个服务其上的zookeeper节点了,启动命令:
bin/zkServer.sh start
zookeeper的日志输出在zookeeper.out文件中,节点的运行状态可以通过下面命令查询:
bin/zkServer.sh status
4.zeromq 安装
通过下面命令完成zeromq的安装
wget http://download.zeromq.org/zeromq-2.1.7.tar.gz
tar -xzf zeromq-2.1.7.tar.gz
cd zeromq-2.1.7
./configure
make
sudo make install
安装过程中可能会出现一些依赖包找不到的情况:
如果报错
configure: error: in `/usr/local/download/zeromq-2.1.7':
configure: error: no acceptable C compiler found in $PATH
See `config.log' for more details
原因为没有安装c compiler
解决方法
# yum install gcc*
如果遇到Error:cannot link with -luuid, install uuid-dev
原因为缺少uuid相关package
解决方法
# yum install uuid*
# yum install e2fsprogs*
# yum install libuuid*
5.jzmq 安装
通过下面命令完成jzmq安装
git clone https://github.com/nathanmarz/jzmq.git
cd jzmq./autogen.sh
./configure
make
sudo make install
这些依赖组件安装完成后就可以开始storm之旅了。
从https://github.com/nathanmarz/storm/downloads下载合适的storm版本,解压到指定目录即可。
启动前需要配置下conf/storm.yml文件。
nimbus节点主要需要修改的地方:
storm.zookeeper.servers:
- "server-hd01.bj"
- "server-hd02.bj"
- "server-hd04.bj"
storm.local.dir: "/home/data/storm"
worker节点主要需要修改的地方
storm.zookeeper.servers:
- "server-hd01.bj"
- "server-hd02.bj"
- "server-hd04.bj"
storm.local.dir: "/home/data/storm"
nimbus.host: "server-dn02.bj"
更加详细的配置可以参照https://github.com/nathanmarz/storm/wiki/Setting-up-a-Storm-cluster,修改完配置后就可以依次启动相应的节点了。
通过下面命令启动nimbus节点
bin/storm nimbus
在nimbus节点还可以启动ui界面
bin/storm ui
启动完成后可以通过http://{nimbus host}:8080进行访问
bin/storm supervisor