輕松搭建hadoop-1.2.1僞分布
以CentOS為例：
CentOS虛拟機安裝：http://blog.csdn.net/baolibin528/article/details/32918565
網絡設定：http://blog.csdn.net/baolibin528/article/details/43797107
PieTTY用法：http://blog.csdn.net/baolibin528/article/details/43822509
WinSCP用法：http://blog.csdn.net/baolibin528/article/details/43819289
隻要把虛拟系統裝好之後，一切就可以用軟體遠端操作它。
1、設定IP
配置僞分布隻在一台機器上運作，是以用 localhost 代替 IP 位址也可以。
如果要用遠端工具（PieTTY、WinSCP等）連接配接 Linux，那就必須要設定 IP 位址。
在裝系統的時候就可以順便把IP位址配好。
如果現配執行指令： service network restart //重新開機網絡
ifconfig //檢視IP 資訊
[[email protected] ~]# ifconfig
eth0 Linkencap:Ethernet HWaddr 00:0C:29:44:A3:A5
inetaddr:192.168.1.100 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fe44:a3a5/64 Scope:Link
UP BROADCAST RUNNINGMULTICAST MTU:1500 Metric:1
RX packets:97040errors:0 dropped:0 overruns:0 frame:0
TX packets:10935errors:0 dropped:0 overruns:0 carrier:0
collisions:0txqueuelen:1000
RX bytes:141600363(135.0 MiB) TX bytes:1033124 (1008.9KiB)
lo Link encap:LocalLoopback
inetaddr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128Scope:Host
UP LOOPBACKRUNNING MTU:16436 Metric:1
RX packets:16 errors:0 dropped:0 overruns:0frame:0
TX packets:16errors:0 dropped:0 overruns:0 carrier:0
collisions:0txqueuelen:0
RX bytes:1184 (1.1KiB) TX bytes:1184 (1.1 KiB) 2、關閉防火牆
執行指令： service iptables stop
驗證： service iptables status
上面那個指令執行完之後，再次開機防火牆還會打開。
永久關閉防火牆： chkconfig iptables off
驗證： chkconfig --list | grep iptables
[[email protected]~]# chkconfig --list | grep iptables
iptables 0:關閉 1:關閉 2:關閉 3:關閉 4:關閉 5:關閉 6:關閉 3、設定主機名：
指令1： hostname 主機名
這個指令是暫時修改主機名，重新開機後無效。下面這個指令是永久修改主機名。
指令2： vim /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=baolibin
GATEWAY=192.168.1.1 4、IP與hostname 綁定：
指令： vim /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4localhost4.localdomain4
::1 localhost localhost.localdomainlocalhost6 localhost6.localdomain6
192.168.1.100 baolibin 5、設定SSH免密碼登陸：
執行指令： ssh-keygen -t rsa -P ''
cat ~/.ssh/id_rsa.pub >>~/.ssh/authorized_keys
chmod 600~/.ssh/authorized_keys
[[email protected]~]# ssh 192.168.1.100
Last login: SunFeb 15 21:01:30 2015 from baolibin
[[email protected]~]# 6、安裝JDK：
給JDK安裝賦予可執行權限： chmodu+x jdk-6u45-linux-x64.bin
解壓JDK ： ./jdk-6u45-linux-x64.bin
給解壓的檔案夾改名： mv jdk1.6.0_45jdk
修改權限： chown -R hadoop:hadoop jdk
在/etc/profile 檔案中添加：
#set java environment
export JAVA_HOME=/usr/local/jdk
export JRE_HOME=/usr/local/jdk/jre
export CLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/jre/lib
export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH [[email protected]]# vim /etc/profile
[[email protected]]# source /etc/profile
[[email protected]]# java -version
java version"1.6.0_45"
Java(TM) SERuntime Environment (build 1.6.0_45-b06)
JavaHotSpot(TM) 64-Bit Server VM (build 20.45-b01, mixed mode)
[[email protected]]# 7、安裝Hadoop：
解壓hadoop： tar -zxvf hadoop-1.2.1.tar.gz
修改權限： chown -R hadoop:hadoop hadoop-1.2.1
重命名： mv hadoop-1.2.1hadoop
編輯 ： vim /etc/profile
#sethadoop environment
export HADOOP_HOME=/usr/hadoop
export PATH=$PATH:$HADOOP_HOME/bin source /etc/profile
修改conf目錄下的配置檔案hadoop-env.sh、core-site.xml、hdfs-site.xml、mapred-site.xml
hadoop-env.sh：
export JAVA_HOME=/usr/local/jdk
core-site.xml：
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://192.168.1.100:9000</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/home/hadoop/dfs</value>
</property>
</configuration> 注意：之前寫的是 <value>/usr/hadoop/tmp</value> 每次使用需要重新格式化，要不然JobTracker過一會會自動消失，改成非tmp目錄就正常了。
hdfs-site.xml：
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.permissions</name>
<value>false</value>
</property>
</configuration> mapred-site.xml：
<configuration>
<property>
<name>mapred.job.tracker</name>
<value>192.168.1.100:9001</value>
</property>
</configuration> 在conf下再修改 vim masters 和 vim slaves
内容均為192.168.1.100
8、格式化：
執行指令： hadoop namenode -format
[[email protected] ~]$ jps
29363 Jps
[[email protected] ~]$ cd /usr/hadoop/bin
[[email protected] bin]$ hadoop namenode -format
Warning: $HADOOP_HOME is deprecated.
15/02/15 21:04:06 INFO namenode.NameNode:STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = baolibin/192.168.1.100
STARTUP_MSG: args = [-format]
STARTUP_MSG: version = 1.2.1
STARTUP_MSG: build =https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.2 -r 1503152;compiled by 'mattf' on Mon Jul 22 15:23:09 PDT 2013
STARTUP_MSG: java = 1.6.0_45
************************************************************/
15/02/15 21:04:06 INFO util.GSet: Computingcapacity for map BlocksMap
15/02/15 21:04:06 INFO util.GSet: VMtype = 64-bit
15/02/15 21:04:06 INFO util.GSet: 2.0% maxmemory = 1013645312
15/02/15 21:04:06 INFO util.GSet:capacity = 2^21 = 2097152 entries
15/02/15 21:04:06 INFO util.GSet:recommended=2097152, actual=2097152
15/02/15 21:04:07 INFOnamenode.FSNamesystem: fsOwner=hadoop
15/02/15 21:04:07 INFOnamenode.FSNamesystem: supergroup=supergroup
15/02/15 21:04:07 INFOnamenode.FSNamesystem: isPermissionEnabled=false
15/02/15 21:04:07 INFOnamenode.FSNamesystem: dfs.block.invalidate.limit=100
15/02/15 21:04:07 INFOnamenode.FSNamesystem: isAccessTokenEnabled=false accessKeyUpdateInterval=0min(s), accessTokenLifetime=0 min(s)
15/02/15 21:04:07 INFO namenode.FSEditLog:dfs.namenode.edits.toleration.length = 0
15/02/15 21:04:07 INFO namenode.NameNode:Caching file names occuring more than 10 times
15/02/15 21:04:08 INFO common.Storage:Image file /usr/hadoop/tmp/dfs/name/current/fsimage of size 112 bytes saved in0 seconds.
15/02/15 21:04:08 INFO namenode.FSEditLog:closing edit log: position=4, editlog=/usr/hadoop/tmp/dfs/name/current/edits
15/02/15 21:04:08 INFO namenode.FSEditLog:close success: truncate to 4, editlog=/usr/hadoop/tmp/dfs/name/current/edits
15/02/15 21:04:08 INFO common.Storage:Storage directory /usr/hadoop/tmp/dfs/name has been successfully formatted.
15/02/15 21:04:08 INFO namenode.NameNode:SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode atbaolibin/192.168.1.100
************************************************************/
[[email protected] bin]$ 9、啟動hadoop ：
執行指令： start-all.sh
[[email protected]]$ start-all.sh
Warning:$HADOOP_HOME is deprecated.
startingnamenode, logging to/usr/hadoop/libexec/../logs/hadoop-hadoop-namenode-baolibin.out
192.168.1.100:starting datanode, logging to/usr/hadoop/libexec/../logs/hadoop-hadoop-datanode-baolibin.out
192.168.1.100:starting secondarynamenode, logging to/usr/hadoop/libexec/../logs/hadoop-hadoop-secondarynamenode-baolibin.out
startingjobtracker, logging to/usr/hadoop/libexec/../logs/hadoop-hadoop-jobtracker-baolibin.out
192.168.1.100:starting tasktracker, logging to /usr/hadoop/libexec/../logs/hadoop-hadoop-tasktracker-baolibin.out
[[email protected]]$ 檢視程序：
[[email protected]]$ jps
29707SecondaryNameNode
29804JobTracker
29928TaskTracker
29585DataNode
30049 Jps
29470NameNode
[[email protected]]$ 10、在浏覽器檢視： http://192.168.1.100:50070 http://192.168.1.100:50030
用遠端軟體登入，直接在Windows 網頁上打開即可：
50070：
50030：
11、關閉hadoop
執行指令： stop-all.sh
[[email protected]]$ stop-all.sh
Warning:$HADOOP_HOME is deprecated.
stoppingjobtracker
192.168.1.100:stopping tasktracker
stoppingnamenode
192.168.1.100:stopping datanode
192.168.1.100:stopping secondarynamenode
[[email protected]]$ 12、啟動時沒有NameNode的可能原因:
(1)沒有格式化
(2)環境變量設定錯誤
(3)ip與hostname綁定失敗