关于64位Linux编译hadoop2

Apache官方提供hadoop2的安装包是在32位机器下编译的，生产环境一般使用64的Linux，那么需要在64位机器下重新编译

可以查看hadoop-2.2.0-src下的BUILDING.txt

Build instructions for Hadoop

----------------------------------------------------------------------------------

Requirements:

* Unix System

* JDK 1.6+

* Maven 3.0 or later

* Findbugs 1.3.9 (if running findbugs)

* ProtocolBuffer 2.5.0

* CMake 2.6 or newer (if compiling native code)

* Internet connection for first build (to fetch all Maven and Hadoop dependencies)

...

步骤：

1.安装JDK 1.6+ (验证:java -version) 

2.安装Maven 3.0 or later (验证:mvn -version)

3.ProtocolBuffer 2.5.0 (验证:protoc --version)下载地址：https://protobuf.googlecode.com/files/protobuf-2.5.0.tar.gz

#为了编译安装protobuf，Linux需要上网，使用YUM在线安装依赖。如不能上网，就比较麻烦，需要一一下载每个依赖包再安装。

sudo yum install gcc 

sudo yum install gcc-c++ 

sudo yum install make

#解压protobuf

sudo tar -zxvf protobuf-2.5.0.tar.gz

#进入到protobuf-2.5.0

cd protobuf-2.5.0

#编译安装

sudo ./configure

sudo make

sudo make install

4.安装CMake 2.6 or newer

sudo yum install cmake

sudo yum install openssl-devel

sudo yum install ncurses-devel

5.编译hadoop-2.2.0

#解压hadoop-2.2.0-src.tar.gz

tar -zxvf hadoop-2.2.0-src.tar.gz

#进入到hadoop-2.2.0-src

cd hadoop-2.2.0-src

#hadoop-2.2.0-src/hadoop-common-project/hadoop-auth/pom.xml有个bug，这里需要修改一下

vim hadoop-common-project/hadoop-auth/pom.xml

#在<dependencies>标签内添加如下内容

<dependency>

<groupId>org.mortbay.jetty</groupId>

<artifactId>jetty-util</artifactId>

<scope>test</scope>

</dependency>

#编译

mvn package -DskipTests -Pdist,native

#编译好的hadoop-2.2.0再hadoop-2.2.0-src/hadoop-dist/target目录下

[INFO] ------------------------------------------------------------------------

[INFO] Reactor Summary:

[INFO] 

[INFO] Apache Hadoop Main ................................ SUCCESS [ 1.228 s]

[INFO] Apache Hadoop Project POM ......................... SUCCESS [ 0.894 s]

[INFO] Apache Hadoop Annotations ......................... SUCCESS [ 1.809 s]

[INFO] Apache Hadoop Assemblies .......................... SUCCESS [ 0.222 s]

[INFO] Apache Hadoop Project Dist POM .................... SUCCESS [ 1.198 s]

[INFO] Apache Hadoop Maven Plugins ....................... SUCCESS [ 2.205 s]

[INFO] Apache Hadoop Auth ................................ SUCCESS [ 2.169 s]

[INFO] Apache Hadoop Auth Examples ....................... SUCCESS [ 1.583 s]

[INFO] Apache Hadoop Common .............................. SUCCESS [01:02 min]

[INFO] Apache Hadoop NFS ................................. SUCCESS [ 5.132 s]

[INFO] Apache Hadoop Common Project ...................... SUCCESS [ 0.038 s]

[INFO] Apache Hadoop HDFS ................................ SUCCESS [01:02 min]

[INFO] Apache Hadoop HttpFS .............................. SUCCESS [ 9.002 s]

[INFO] Apache Hadoop HDFS BookKeeper Journal ............. SUCCESS [ 4.995 s]

[INFO] Apache Hadoop HDFS-NFS ............................ SUCCESS [ 2.647 s]

[INFO] Apache Hadoop HDFS Project ........................ SUCCESS [ 0.058 s]

[INFO] hadoop-yarn ....................................... SUCCESS [ 0.138 s]

[INFO] hadoop-yarn-api ................................... SUCCESS [ 31.854 s]

[INFO] hadoop-yarn-common ................................ SUCCESS [ 20.121 s]

[INFO] hadoop-yarn-server ................................ SUCCESS [ 0.105 s]

[INFO] hadoop-yarn-server-common ......................... SUCCESS [ 5.776 s]

[INFO] hadoop-yarn-server-nodemanager .................... SUCCESS [ 10.490 s]

[INFO] hadoop-yarn-server-web-proxy ...................... SUCCESS [ 3.321 s]

[INFO] hadoop-yarn-server-resourcemanager ................ SUCCESS [ 8.311 s]

[INFO] hadoop-yarn-server-tests .......................... SUCCESS [ 0.510 s]

[INFO] hadoop-yarn-client ................................ SUCCESS [ 3.929 s]

[INFO] hadoop-yarn-applications .......................... SUCCESS [ 0.060 s]

[INFO] hadoop-yarn-applications-distributedshell ......... SUCCESS [ 1.720 s]

[INFO] hadoop-mapreduce-client ........................... SUCCESS [ 0.062 s]

[INFO] hadoop-mapreduce-client-core ...................... SUCCESS [ 16.204 s]

[INFO] hadoop-yarn-applications-unmanaged-am-launcher .... SUCCESS [ 1.779 s]

[INFO] hadoop-yarn-site .................................. SUCCESS [ 0.111 s]

[INFO] hadoop-yarn-project ............................... SUCCESS [ 2.287 s]

[INFO] hadoop-mapreduce-client-common .................... SUCCESS [ 11.855 s]

[INFO] hadoop-mapreduce-client-shuffle ................... SUCCESS [ 2.560 s]

[INFO] hadoop-mapreduce-client-app ....................... SUCCESS [ 6.985 s]

[INFO] hadoop-mapreduce-client-hs ........................ SUCCESS [ 3.319 s]

[INFO] hadoop-mapreduce-client-jobclient ................. SUCCESS [ 4.021 s]

[INFO] hadoop-mapreduce-client-hs-plugins ................ SUCCESS [ 1.508 s]

[INFO] Apache Hadoop MapReduce Examples .................. SUCCESS [ 4.176 s]

[INFO] hadoop-mapreduce .................................. SUCCESS [ 2.367 s]

[INFO] Apache Hadoop MapReduce Streaming ................. SUCCESS [ 2.902 s]

[INFO] Apache Hadoop Distributed Copy .................... SUCCESS [ 5.365 s]

[INFO] Apache Hadoop Archives ............................ SUCCESS [ 1.673 s]

[INFO] Apache Hadoop Rumen ............................... SUCCESS [ 4.095 s]

[INFO] Apache Hadoop Gridmix ............................. SUCCESS [ 2.962 s]

[INFO] Apache Hadoop Data Join ........................... SUCCESS [ 2.089 s]

[INFO] Apache Hadoop Extras .............................. SUCCESS [ 2.190 s]

[INFO] Apache Hadoop Pipes ............................... SUCCESS [ 5.887 s]

[INFO] Apache Hadoop Tools Dist .......................... SUCCESS [ 1.149 s]

[INFO] Apache Hadoop Tools ............................... SUCCESS [ 0.028 s]

[INFO] Apache Hadoop Distribution ........................ SUCCESS [ 6.514 s]

[INFO] Apache Hadoop Client .............................. SUCCESS [ 2.199 s]

[INFO] Apache Hadoop Mini-Cluster ........................ SUCCESS [ 0.121 s]

[INFO] BUILD SUCCESS (####看到BUILD SUCCESS说明编译成功####)

[INFO] Total time: 05:39 min

[INFO] Finished at: 2014-05-23T13:37:06+08:00

[INFO] Final Memory: 135M/300M

本文转自SummerChill博客园博客，原文链接：http://www.cnblogs.com/DreamDrive/p/4546792.html，如需转载请自行联系原作者