使用cloudera manager安裝cdh5.8.0

節點規劃：

所有節點data-01 - data-06 6台阿裡雲伺服器8核32G記憶體

管理節點data-01

基礎環境準備工作：

1：hosts映射及系統設定

—所有節點執行—

echo ”>> /etc/hosts

echo ‘10.0.0.1 data-01’ >> /etc/hosts

echo ‘10.0.0.2 data-02’ >> /etc/hosts

echo ‘10.0.0.3 data-03’ >> /etc/hosts

echo ‘10.0.0.4 data-04’ >> /etc/hosts

echo ‘10.0.0.5 data-05’ >> /etc/hosts

echo ‘10.0.0.6 data-06’ >> /etc/hosts

echo ‘01 * * * * root ntpdate -u data-06’ >> /etc/crontab

sed -i ‘s/SELINUX=enforcing/SELINUX=disabled/’ /etc/selinux/config

setenforce 0

service iptables stop

chkconfig iptables off

echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag

echo ‘echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag’ >> /etc/rc.local

echo ‘0’ > /proc/sys/vm/swappiness

echo ‘echo 0 > /proc/sys/vm/swappiness’ >> /etc/sysctl.conf

sysctl -p

2：ssh免密碼登入

—所有節點執行—

cd ~

ssh-keygen

之後一路回車

cat ~/.ssh/id_rsa.pub

将所有機器的id_rsa.pub内容放到一個文本内命名為authorized_keys，将authorized_keys上傳到各機器/root/.ssh/ 下

驗證是否實作各機器之間免密登入

3：配置cdh源（用于各節點下載下傳安裝包）

—data-06執行（源節點）—

cdh.5.8.0.20161009.tar.gz此包包含了所有安裝cdh要用到的東西

mkdir -p /var/www/html/

tar -xvf cdh.5.8.0.20161009.tar.gz -C /var/www/html/

yum install expat-devel-* cyrus-sasl-devel-* openldap-devel-* apr-* httpd-* deltarpm-* python-deltarpm-* createrepo-*

service httpd restart

chkconfig httpd on

4：配置ntp時間服務

—data-06執行（時間服務節點）—

cp /etc/ntp.conf /etc/ntp.conf.bak

sed -i ‘s/restrict default kod nomodify notrap nopeer noquery/restrict default kod nomodify /’ /etc/ntp.conf

echo ” >> /etc/ntp.conf

echo ‘server 127.127.1.0’ >> /etc/ntp.conf

echo ‘fudge 127.127.1.0 stratum 10’ >> /etc/ntp.conf

service ntpd restart

chkconfig ntpd on

—所有節點執行—

yum install -y ntp

service ntpd start

chkconfig ntpd on

5：Java環境

—所有節點執行—

mkdir -p /usr/java/

wget http://data-06/cdh/jdk-8u51-linux-x64.tar.gz -P ~

tar -xvf jdk-8u51-linux-x64.tar.gz -C /usr/java

vim /etc/profile

export JAVA_HOME=/usr/java/jdk1.8.0_51

export CLASSPATH=.: JAVAHOME/jre/lib/rt.jar: JAVA_HOME/lib/dt.jar: JAVAHOME/lib/tools.jarexportPATH= PATH:$JAVA_HOME/bin

source /etc/profile

驗證Java安裝正确

java -version

6：mysql安裝

—data-01執行（mysql資料庫位置）—

yum install -y mysql-server

/usr/bin/mysql_install_db

service mysqld start

chkconfig mysqld on

建立資料

mysql

create database hive DEFAULT CHARSET utf8 COLLATE utf8_general_ci;

create database amon DEFAULT CHARSET utf8 COLLATE utf8_general_ci;

create database hue DEFAULT CHARSET utf8 COLLATE utf8_general_ci;

create database reports_manager DEFAULT CHARSET utf8 COLLATE utf8_general_ci;

create database oozie DEFAULT CHARSET utf8 COLLATE utf8_general_ci;

grant all on . to root@”%” Identified by “bigdata”;

use mysql;

UPDATE user SET password=PASSWORD(‘bigdata’) WHERE user=’root’;

flush privileges;

exit;

7：安裝cloudera manager

—所有節點—（以下指令中安裝包版本需按照自己安裝版本更改）

wget http://data-06/cdh/cloudera-manager-el6-cm5.8.2_x86_64.tar.gz -P ~

tar -xvf cloudera-manager-el6-cm5.8.2_x86_64.tar.gz -C /opt/

rm -f cloudera-manager-el6-cm5.8.2_x86_64.tar.gz

wget http://data-06/cdh/CDH-5.8.0-1.cdh5.8.0.p0.42-el6.parcel -P /opt/cloudera/parcel-repo/

wget http://data-06/cdh/CDH-5.8.0-1.cdh5.8.0.p0.42-el6.parcel.sha -P /opt/cloudera/parcel-repo/

wget http://data-06/cdh/manifest.json -P /opt/cloudera/parcel-repo/

wget http://data-06/cdh/mysql-connector-java.jar -P /opt/cm-5.8.2/share/cmf/lib/

wget http://data-06/cdh/mysql-connector-java.jar -P /usr/share/java/

sed -i ‘s/server_host=localhost/server_host=data-01/’ /opt/cm-5.8.2/etc/cloudera-scm-agent/config.ini

useradd –system –home=/opt/cm-5.8.2/run/cloudera-scm-server –no-create-home –shell=/bin/false –comment “Cloudera SCM User” cloudera-scm

—管理節點—

連接配接資料庫

/opt/cm-5.8.2/share/cmf/schema/scm_prepare_database.sh mysql cm -hdata-01 -uroot -pbigdata –scm-host data-01 scm scm scm

啟動服務

/opt/cm-5.8.2/etc/init.d/cloudera-scm-server start &

—所有節點—

/opt/cm-5.8.2/etc/init.d/cloudera-scm-agent start &

8：安裝cdh

http://10.27.229.69:7180 (admin:admin)

根據頁面提示選擇安裝