Ambari介紹
Apache Ambari是一個基于Web的支援Apache Hadoop叢集的供應、管理和監控的開源工具， Ambari已支援大多數Hadoop元件，包括HDFS、MapReduce、Hive、Pig、 Hbase、Zookeeper、Sqoop和Hcatalog等。提供Web UI進行可視化的叢集管理，簡化了大資料平台的安裝、使用難度。
Ambari體系結構
Ambari 自身也是一個分布式架構的軟體，主要由兩部分組成：Ambari Server 和 Ambari Agent。
簡單來說，使用者通過Ambari Server通知 Ambari Agent 安裝對應的軟體；Agent 會定時地發送各個機器每個軟體子產品的狀态給 Ambari Server，最終這些狀态資訊會呈現在 Ambari 的 GUI，友善使用者了解到叢集的各種狀态，并進行相應的維護。Ambari Server 從整個叢集上收集資訊。每個主機上都有 Ambari Agent, Ambari Server 通過 Ambari Agent 控制每部主機。
下面介紹CentOS7系統下利用Ambari搭建HDP大資料平台
參考官方文檔
https://docs.cloudera.com/HDPDocuments/Ambari-2.6.2.2/bk_ambari-installation/content/install-ambari-server-rhel7.html https://docs.cloudera.com/HDPDocuments/Ambari-2.6.2.2/bk_ambari-installation/content/hdp_26_repositories.html https://docs.cloudera.com/HDPDocuments/Ambari-2.6.2.2/bk_ambari-installation/content/ambari_repositories.html
一、準備條件
1、jdk安裝包 jdk-8u261-linux-x64.tar.gz
2、提前下載下傳好ambari,HDP,HDP-UTILS 本地倉庫Tarball檔案
http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.2.2/ambari-2.6.2.2-centos7.tar.gz
http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.5.0/HDP-2.6.5.0-centos7-rpm.tar.gz
http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7/HDP-UTILS-1.1.0.22-centos7.tar.gz
複制
二、作業系統基礎環境準備
先在ambari機器上配置JDK，準備基礎環境
1)、關閉防火牆
systemctl disable firewalld
systemctl stop firewalld
複制
2)、關閉SELINUX
sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
setenforce 0
複制
3)、設定ulimit 檔案打開數為10000
在/etc/profile最後一行添加ulimit -n 10000
4)、安裝JDK，配置環境變量
cd /opt
tar -zxvf jdk-8u261-linux-x64.tar.gz
mv jdk1.8.0_261 /usr/local/
cd /usr/local/
mv jdk1.8.0_261 jdk
複制
#在/etc/profile中添加如下行
vi /etc/profile
export JAVA_HOME=/usr/local/jdk
export CLASSPATH=.:$JAVA_HOME/lib
PATH=$PATH:$JAVA_HOME/bin 複制
5)、source /etc/profile生效
最後檢查java，ulimit值
以上配置做好之後，可以poweroff關機做一個虛拟機快照，按這個虛拟機模闆克隆4台機器 分别為bigdata1,bigdata2,bigdata3,bigdata4
三、主機名修改與免密鑰登入配置
1)5台主機分别修改主機名
hostnamectl set-hostname ambari
hostnamectl set-hostname bigdata1
hostnamectl set-hostname bigdata2
hostnamectl set-hostname bigdata3
hostnamectl set-hostname bigdata4 複制
2)配置ambari-server免密鑰登入其他4台節點
ssh-keygen
複制
3)配置主機域名解析
vi /etc/hosts
添加如下行
10.20.90.59 ambari
10.20.90.60 bigdata1
10.20.90.61 bigdata2
10.20.90.62 bigdata3
10.20.90.63 bigdata4 複制
4)拷貝公鑰
ssh-copy-id root@bigdata1
ssh-copy-id root@bigdata2
ssh-copy-id root@bigdata3
ssh-copy-id root@bigdata4
ssh-copy-id root@ambari
複制
5)主機域名解析檔案拷貝到4台節點上
scp /etc/hosts root@bigdata1:/etc
scp /etc/hosts root@bigdata2:/etc
scp /etc/hosts root@bigdata3:/etc
scp /etc/hosts root@bigdata4:/etc
複制
并驗證可以正常免密登入
四、配置Ambari HDP以及HDP-UTILS 本地yum源
在ambari伺服器上配置Ambari HDP以及HDP-UTILS 本地yum源
yum install -y httpd
cd /opt
mkdir /var/www/html/ambari
mkdir /var/www/html/hdp
tar -zxf ambari-2.6.2.2-centos7.tar.gz -C /var/www/html/ambari/
tar -zxf HDP-2.6.5.0-centos7-rpm.tar.gz -C /var/www/html/hdp/
tar -zxf HDP-UTILS-1.1.0.22-centos7.tar.gz -C /var/www/html/hdp/
systemctl start httpd
systemctl enable httpd
複制
SFTP上傳到/opt目錄下
驗證Ambari HDP以及HDP-UTILS 本地yum源均可以正常通路
http://10.20.90.59/ambari/ambari/centos7/2.6.2.2-1/
http://10.20.90.59/hdp/HDP/centos7/2.6.5.0-292/
http://10.20.90.59/hdp/HDP-UTILS/centos7/1.1.0.22/
複制
針對下面這個錯誤的解決辦法
删除或者移走html檔案
配置ambari YUM源
cat >/etc/yum.repos.d/ambari.repo<<EOF
#VERSION_NUMBER=2.6.2.2-1
[Ambari-2.6.2.2-1]
name=2.6.2.2-1 - Updates
baseurl=http://10.20.90.59/ambari/ambari/centos7/2.6.2.2-1/
gpgcheck=1
gpgkey=http://10.20.90.59/ambari/ambari/centos7/2.6.2.2-1/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
EOF
複制
#配置hdp YUM源
cat >/etc/yum.repos.d/hdp.repo<<EOF
#VERSION_NUMBER=2.6.5.0-292
[HDP-2.6.5.0]
name=HDP Version - HDP-2.6.5.0-292
baseurl=http://10.20.90.59/hdp/HDP/centos7/2.6.5.0-292/
gpgcheck=1
gpgkey=http://10.20.90.59/hdp/HDP/centos7/2.6.5.0-292/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
[HDP-UTILS-1.1.0.22]
name=HDP-UTILS Version - HDP-UTILS-1.1.0.22
baseurl=http://10.20.90.59/hdp/HDP-UTILS/centos7/1.1.0.22/
gpgcheck=1
gpgkey=http://10.20.90.59/hdp/HDP-UTILS/centos7/1.1.0.22/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
EOF
複制
cat /etc/yum.repos.d/ambari.repo
cat /etc/yum.repos.d/hdp.repo 複制
五、安裝ambari-server
yum -y install ambari-server
複制
六、安裝MariaDB資料庫并配置資料庫
1、安裝MariaDB資料庫
yum -y install mariadb-server
systemctl enable mariadb.service
systemctl start mariadb.service
netstat -anp | grep 3306
複制
2、設定資料庫初始密碼
mysqladmin -uroot password "ambari" 複制
3、設定可以遠端登入
mysql -uroot -pambari
grant all privileges on *.* to 'root'@'ambari' identified by 'ambari' with grant option;
grant all privileges on *.* to 'root'@'10.20.90.59' identified by 'ambari' with grant option;
flush privileges;
systemctl restart mariadb.service
複制
4、建立ambari資料庫
mysql -uroot -pambari -hambari
create database ambari DEFAULT CHARSET utf8;
use ambari
source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
複制
七、ambari-server初始化配置
[root@ambari home]# ambari-server setup
Using python /usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is 'disabled'
Customize user account for ambari-server daemon [y/n] (n)? y
Enter user account for ambari-server daemon (root):root
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7
[3] Custom JDK
==============================================================================
Enter choice (1): 3
WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
Path to JAVA_HOME: /usr/local/jdk
Validating JDK on Ambari Server...done.
Checking GPL software agreement...
GPL License for LZO: https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html
Enable Ambari Server to download and install GPL Licensed LZO packages [y/n] (n)? n
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)? y
Configuring database...
==============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL / MariaDB
[4] - PostgreSQL
[5] - Microsoft SQL Server (Tech Preview)
[6] - SQL Anywhere
[7] - BDB
==============================================================================
Enter choice (1): 3
Hostname (localhost):
Port (3306):
Database name (ambari):
Username (ambari): root
Enter Database Password (bigdata): ambari
Re-enter password:
Configuring ambari database...
WARNING: Before starting Ambari Server, you must copy the MySQL JDBC driver JAR file to /usr/share/java and set property "server.jdbc.driver.path=[path/to/custom_jdbc_driver]" in ambari.properties.
Press <enter> to continue.
Configuring remote database connection properties...
WARNING: Before starting Ambari Server, you must run the following DDL against the database to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
Proceed with configuring remote database connection properties [y/n] (y)? y
Extracting system views...
....ambari-admin-2.6.2.2.1.jar
.......
Adjusting ambari-server permissions and ownership...
Ambari Server 'setup' completed successfully. 複制
拷貝jdbc驅動包，并在ambari.properties配置檔案中添加jdbc驅動的配置
mkdir /usr/share/java
cp /home/mysql-connector-java-8.0.20.jar /usr/share/java/mysql-connector-java.jar
vi /etc/ambari-server/conf/ambari.properties
#追加一行
server.jdbc.driver.path=/usr/share/java/mysql-connector-java.jar
複制
ambari-server start
複制
八、ambari web界面向導搭建叢集
注意：這裡的URL最後面的/斜杠需要去掉，（圖誤）
九、排錯
1、Confirm Hosts時出現安裝ambari-agent的報錯的解決方法
修改 /etc/ambari-agent/conf/ambari-agent.ini 檔案，在 [security] 配置部分添加
[security]
...
force_https_protocol=PROTOCOL_TLSv1_2 複制
2、yum.conf配置檔案修改
vi /etc/yum.conf
installonly_limit=5改為
installonly_limit=600 複制