【Hadoop】 分布式Hadoop集群安装配置

第一次安装hadoop 集群，记录其安装过程！

1 前期准备，配置所有的机器的/etc/hosts 文件

# cat /etc/hosts

# do not remove the following line, or various programs

# that require network functionality will fail.

127.0.0.1       rac localhost

::1             rac3 localhost

10.250.7.225    rac1

10.250.7.249    rac2

10.250.7.241    rac3

10.250.7.220    rac4 

1 获取hadoop安装文件

可以从一下网址获取hadoop 安装文件 hadoop-0.20.2.tar.gz：

http://apache.etoak.com//hadoop/common/ 

http://mirror.bjtu.edu.cn/apache//hadoop/common/ 

http://labs.renren.com/apache-mirror//hadoop/common/ 

#tar zxvf hadoop-0.20.2.tar.gz

#mv hadoop-0.20.2.tar.gz hadoop

#cd hadoop/conf

配置conf文件夹里面的core-site.xml,hdfs-site.xml,mapread-site.xml,mapred-site.xml，hadoop-env.sh。具体各个配置的含义请参考hadoop帮助文档。

core-site.xml文件

    fs.default.name

    hdfs://rac2:9000

<b>2.其次编辑所有节点的hdfs-site.xml，命令如下：</b>

#vi /root/hadoop/conf/hdfs-site.xml

 dfs.name.dir

 /opt/hadoop/namedata

 dfs.permissions

 false

 dfs.replication

 1

<b>3 编辑所有节点mapred-site.xml文件：</b>

#vi /home/hadoop/conf/mapred-site.xml

mapred.job.tracker

192.168.0.10:9001

<b>4 在所有节点编辑hadoop-env.sh </b>

export hadoop_home=/root/hadoop

export hadoop_conf_dir=$hadoop_home/conf 

export path=$path:$hadoop_home/bin 

export java_home=/usr/java/jdk1.6.0_29 

export classhome=/usr/java/jdk1.6.0_29/lib/tools.jar:/usr/java/jdk1.6.0_29/lib/dt.jar

<b>5 在所有节点修改masters ，slaves 文件，其中masters 文件写入master 节点的ip，slaves 文件写入slaves 节点的ip</b>

[root@rac1 conf]#  cat masters 

10.250.7.225

[root@rac1 conf]# cat slaves 

10.250.7.220

10.250.7.249

10.250.7.241

<b>6 启动hadoop</b>

[root@rac1 bin]# sh start-all.sh 

starting namenode, logging to /root/hadoop/logs/hadoop-root-namenode-rac1.out

10.250.7.220: starting datanode, logging to /root/hadoop/logs/hadoop-root-datanode-rac4.out

10.250.7.241: starting datanode, logging to /root/hadoop/logs/hadoop-root-datanode-rac3.out

10.250.7.249: starting datanode, logging to /root/hadoop/logs/hadoop-root-datanode-rac2.out

10.250.7.225: starting secondarynamenode, logging to /root/hadoop/logs/hadoop-root-secondarynamenode-rac1.out

jobtracker running as process 20175. stop it first.

10.250.7.220: starting tasktracker, logging to /root/hadoop/logs/hadoop-root-tasktracker-rac4.out

10.250.7.241: starting tasktracker, logging to /root/hadoop/logs/hadoop-root-tasktracker-rac3.out

10.250.7.249: starting tasktracker, logging to /root/hadoop/logs/hadoop-root-tasktracker-rac2.out