天天看點
傳回

centos7下mysql雙主+keepalived

 一、keepalived簡介

    keepalived是vrrp協定的實作，原生設計目的是為了高可用ipvs服務，keepalived能夠配置檔案中的定義生成ipvs規則，并能夠對各RS的健康狀态進行檢測；通過共用的虛拟IP位址對外提供服務；每個熱備組内同一時刻隻有一台主伺服器提供服務，其他伺服器處于備援狀态，若目前線上的伺服器當機，其虛拟IP位址将會被其他伺服器接替（優先級決定接替順序），實作高可用為後端主機提供服務。

   二、keepalived元件

   Keepalived元件介紹

core:keepalived核心元件，主程序的啟動和維護，全局配置等。

vrrp stack:keepalived是基于vrrp協定實作高可用vps服務，vrrp則為相關子程序為其提供服務

check:檢測keepalived的健康狀态相關程序 

system call:系統調用

watch dog:監控check和vrrp程序的看管者，check負責檢測器子程序的健康狀态，當其檢測到master上的服務不可用時則通告vrrp将其轉移至backup伺服器上。

   三 環境準備

作業系統：centos7.1.1511（core）

資料庫： mysql5.7.21 社群版

master1 ：10.0.0.11　　      安裝mysql 和keeplived

master2 ：10.0.0.12             安裝mysql 和keeplived

VIP：10.0.0.20

centos7下mysql雙主+keepalived

要實作互為主從，就必須 mster1-->master2設定主從同步 同時 master2--->master1 也設定主從同步

四、Mysql主主同步環境部署

---------------master1伺服器操作記錄---------------
在my.cnf檔案的[mysqld]配置區域添加下面内容：
[root@master1 ~]# vim /usr/local/mysql/my.cnf
server-id = 1         
log-bin = mysql-bin     
sync_binlog = 1
binlog_checksum = none
binlog_format = mixed
auto-increment-increment = 2     
auto-increment-offset = 1    
slave-skip-errors = all      
  
[root@master1 ~]# /etc/init.d/mysql restart
Shutting down MySQL. SUCCESS!
Starting MySQL.. SUCCESS!

建立一個複制使用者

出了小問題，由于之前root使用者的密碼設定過于簡單在建立複制使用者時報如下錯誤

mysql> grant replication slave,replication client on *.* to repl@'10.0.0.%' identified by '1qaz@WSX';
ERROR 1820 (HY000): You must reset your password using ALTER USER statement before executing this statement.

按照提示将密碼設定的複雜一點 在授權建立就沒有問題了 

mysql> alter user 'root'@'localhost' identified by '1qaz@WSX';
Query OK, 0 rows affected (0.00 sec)

mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)
      
mysql> grant replication slave,replication client on *.* to repl@'10.0.0.%' identified by '1qaz@WSX';  
Query OK, 0 rows affected, 1 warning (0.00 sec)

mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)

鎖表，待同步配置完成在解鎖

mysql> flush tables with read lock;
Query OK, 0 rows affected (0.00 sec)

檢視目前的binlog以及資料所在位置

mysql> show master status;
+------------------+----------+--------------+------------------+-------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-------------------+
| mysql-bin.000006 |      996 |              |                  |                   |
+------------------+----------+--------------+------------------+-------------------+
1 row in set (0.00 sec)      
---------------master2伺服器操作記錄---------------
在my.cnf檔案的[mysqld]配置區域添加下面内容：
[root@master2 ~]# vim /usr/local/mysql/my.cnf
server-id = 2        
log-bin = mysql-bin    
sync_binlog = 1
binlog_checksum = none
binlog_format = mixed
auto-increment-increment = 2     
auto-increment-offset = 2    
slave-skip-errors = all
  
[root@master2 ~]# /etc/init.d/mysql restart
Shutting down MySQL.. SUCCESS!
Starting MySQL.. SUCCESS!      
mysql> grant replication slave,replication client on *.* to repl@'10.0.0.%' identified by '1qaz@WSX';  
Query OK, 0 rows affected, 1 warning (0.01 sec)

mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)

  mysql> flush tables with read lock;

  Query OK, 0 rows affected (0.00 sec)

檢視 master情況

mysql> show master status;
+------------------+----------+--------------+------------------+-------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-------------------+
| mysql-bin.000001 |      150 |              |                  |                   |
+------------------+----------+--------------+------------------+-------------------+
1 row in set (0.00 sec)

分别開啟同步對方

---------------master1伺服器做同步操作---------------
mysql> unlock tables;     //先解鎖，将對方資料同步到自己的資料庫中
mysql> slave stop；
mysql> change  master to master_host='10.0.0.12',master_user='repl',master_password='1qaz@WSX',master_log_file='mysql-bin.000001',master_log_pos=150;         
Query OK, 0 rows affected, 2 warnings (0.01 sec)      
mysql> start slave;
Query OK, 0 rows affected (0.01 sec)

檢視兩個線程狀态是否為YES 

mysql> show slave status \G;

Slave_IO_Running: Yes

Slave_SQL_Running: Yes

-------------master2伺服器做同步操作---------------
mysql> unlock tables;     //先解鎖，将對方資料同步到自己的資料庫中
mysql> slave stop；
mysql> change  master to master_host='10.0.0.11',master_user='repl',master_password='1qaz@WSX',master_log_file='mysql-bin.000006',master_log_pos=996;  
Query OK, 0 rows affected, 2 warnings (0.06 sec)
  
mysql> start slave;
Query OK, 0 rows affected (0.01 sec)
  
mysql> show slave status \G;

Master_Log_File: mysql-bin.000006

Read_Master_Log_Pos: 996

Relay_Log_File: master2-relay-bin.000002

Relay_Log_Pos: 312

Relay_Master_Log_File: mysql-bin.000006

以上表明雙方已經實作了mysql主主同步。

當運作一段時間後，要是發現同步有問題，比如隻能單向同步，雙向同步失效。可以重新執行下上面的change master同步操作，隻不過這樣同步後，隻能同步在此之後的更新資料。下面開始進行資料驗證：

 
-----------------主主同步效果驗證---------------------
1）在master1資料庫上寫入新資料
mysql> unlock tables;
Query OK, 0 rows affected (0.00 sec)
  
mysql> create database huanqiu;
Query OK, 1 row affected (0.01 sec)
  
mysql> use huanqiu;
Database changed

mysql> create table if not exists haha ( id int(10) PRIMARY KEY AUTO_INCREMENT, name varchar(50) NOT NULL);

Query OK, 0 rows affected (0.04 sec)

mysql> insert into haha values(2,'guojing');

Query OK, 1 row affected (0.00 sec)

mysql> insert into haha values(1,"huangrong");

mysql> select * from haha;

+----+-----------+

| id | name |

| 1 | huangrong |

| 2 | guojing |

2 rows in set (0.00 sec)

然後在master2資料庫上檢視，發現資料已經同步過來了!

mysql> select * from huanqiu.haha;

2）在master2資料庫上寫入新資料
mysql> create database hehe;
Query OK, 1 row affected (0.00 sec)

mysql> insert into huanqiu.haha values(3,"haha"),(4,"haha");

Query OK, 2 rows affected (0.00 sec)

Records: 2 Duplicates: 0 Warnings: 0

然後在master1資料庫上檢視，發現資料也已經同步過來了!
mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| hehe               |
| huanqiu            |
| mysql              |
| performance_schema |
| test               |
+--------------------+
6 rows in set (0.00 sec)

| 3 | haha |

| 4 | haha |

4 rows in set (0.00 sec)

至此，Mysql主主同步環境已經實作。

五 配置mysql+keepalived 高可用環境 

1）安裝keepalived并将其配置成系統服務。master1和master2兩台機器上同樣進行如下操作：
[root@master1 ~]# yum install -y openssl-devel
[root@master1 ~]# cd /usr/local/src/
[root@master1 src]# wget http://www.keepalived.org/software/keepalived-1.3.5.tar.gz
[root@master1 src]# tar -zvxf keepalived-1.3.5.tar.gz
[root@master1 src]# cd keepalived-1.3.5
[root@master1 keepalived-1.3.5]# ./configure --prefix=/usr/local/keepalived
[root@master1 keepalived-1.3.5]# make && make install
     
[root@master1 keepalived-1.3.5]# cp /usr/local/src/keepalived-1.3.5/keepalived/etc/init.d/keepalived /etc/rc.d/init.d/
[root@master1 keepalived-1.3.5]# cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
[root@master1 keepalived-1.3.5]# mkdir /etc/keepalived/
[root@master1 keepalived-1.3.5]# cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
[root@master1 keepalived-1.3.5]# cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
[root@master1 keepalived-1.3.5]# echo "/etc/init.d/keepalived start" >> /etc/rc.local

2）master1機器上的keepalived.conf配置。（下面配置中沒有使用lvs的負載均衡功能，是以不需要配置虛拟伺服器virtual server）

[root@master1 ~]# cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.bak
[root@master1 ~]# vim /etc/keepalived/keepalived.conf       ＃清空預設内容，直接采用下面配置：
! Configuration File for keepalived
       
global_defs {
notification_email {
[email protected]
[email protected]
}
       
notification_email_from [email protected]
smtp_server 127.0.0.1 
smtp_connect_timeout 30
router_id MASTER-HA
}
       
vrrp_script chk_mysql_port {     #檢測mysql服務是否在運作。有很多方式，比如程序，用腳本檢測等等
    script "/opt/chk_mysql.sh"   #這裡通過腳本監測
    interval 2                   #腳本執行間隔，每2s檢測一次
    weight -5                    #腳本結果導緻的優先級變更，檢測失敗（腳本傳回非0）則優先級 -5
    fall 2                    #檢測連續2次失敗才算确定是真失敗。會用weight減少優先級（1-255之間）
    rise 1                    #檢測1次成功就算成功。但不修改優先級
}
       
vrrp_instance VI_1 {
    state MASTER    
    interface eth0      #指定虛拟ip的網卡接口
    mcast_src_ip 10.0.0.11
    virtual_router_id 51    #路由器辨別，MASTER和BACKUP必須是一緻的
    priority 101            #定義優先級，數字越大，優先級越高，在同一個vrrp_instance下，MASTER的優先級必須大于BACKUP的優先級。這樣MASTER故障恢複後，就可以将VIP資源再次搶回來 
    advert_int 1         
    authentication {   
        auth_type PASS 
        auth_pass 1111     
    }
    virtual_ipaddress {    
        10.0.0.20
    }
      
track_script {               
   chk_mysql_port             
}
}

編寫切換腳本。KeepAlived做心跳檢測，如果Master的MySQL服務挂了(3306端口挂了),那麼它就會選擇自殺。Slave的KeepAlived通過心跳檢測發現這個情況，就會将VIP的請求接管

[root@master1 ~]# vim /opt/chk_mysql.sh
#!/bin/bash
counter=$(netstat -na|grep "LISTEN"|grep "3306"|wc -l)
if [ "${counter}" -eq 0 ]; then
    /etc/init.d/keepalived stop
fi      
[root@master1 ~]# chmod 755 /opt/chk_mysql.sh
     
啟動keepalived服務
[root@master1 ~]# /etc/init.d/keepalived start
正在啟動 keepalived：                                      [确定]

4）master2機器上的keepalived配置。master2機器上的keepalived.conf檔案隻修改priority為90、nopreempt不設定、real_server設定本地IP。

[root@master2 ~]# cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.bak
[root@master2 ~]# >/etc/keepalived/keepalived.conf
[root@master2 ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
       
global_defs {
notification_email {
[email protected]
[email protected]
}
       
notification_email_from [email protected]
smtp_server 127.0.0.1 
smtp_connect_timeout 30
router_id MASTER-HA
}
       
vrrp_script chk_mysql_port {
    script "/opt/chk_mysql.sh"
    interval 2            
    weight -5                 
    fall 2                 
    rise 1               
}
       
vrrp_instance VI_1 {
    state BACKUP
    interface eth0    
    mcast_src_ip 10.0.0.12
    virtual_router_id 51    
    priority 99          
    advert_int 1         
    authentication {   
        auth_type PASS 
        auth_pass 1111     
    }
    virtual_ipaddress {    
        10.0.0.20
    }
      
track_script {               
   chk_mysql_port             
}
}
     
     
[root@master2 ~]# cat /opt/chk_mysql.sh
#!/bin/bash
counter=$(netstat -na|grep "LISTEN"|grep "3306"|wc -l)
if [ "${counter}" -eq 0 ]; then
    /etc/init.d/keepalived stop
fi
 
[root@master2 ~]# chmod 755 /opt/chk_mysql.sh
     
[root@master2 ~]# /etc/init.d/keepalived start
正在啟動 keepalived：                                      [确定]

我這裡啟動時出現了問題，分析日志

tail -f /var/log/message
Mar 31 14:28:14 master1 systemd: Configuration file /usr/lib/systemd/system/ebtables.service is marked executable. Please remove executable permission bits. Proceeding anyway.
centos7下mysql雙主+keepalived

檢視keepalived.service

# vi /lib/systemd/system/keepalived.service

[Unit]

Description=LVS and VRRP High Availability Monitor

After=syslog.target network-online.target

[Service]

Type=forking

# PIDFile=/usr/local/keepalived/var/run/keepalived.pid 

# 上面這個注釋掉 改成下面 因為這個預設路徑不存在是以就無法寫入程序ID檔案

PIDFile=/var/run/keepalived.pid 

KillMode=process

EnvironmentFile=-/usr/local/keepalived/etc/sysconfig/keepalived

ExecStart=/usr/local/keepalived/sbin/keepalived $KEEPALIVED_OPTIONS

ExecReload=/bin/kill -HUP $MAINPID

[Install]

WantedBy=multi-user.target

centos7下mysql雙主+keepalived

5）master1和master2兩台伺服器都要授權允許root使用者遠端登入，用于在用戶端登陸測試！

 
mysql> grant all on *.* to root@'10.0.0.%' identified by "1qaz@WSX";
Query OK, 0 rows affected (0.00 sec)
     
mysql> flush privileges;
Query OK, 0 rows affected (0.01 sec)

6）在master1和master2兩台機器上設定iptables防火牆規則，如下：

 
[root@master1 ~]# cat /etc/sysconfig/iptables
........
-A INPUT -s 10.0.0.0/24 -d 224.0.0.18 -j ACCEPT       #允許多點傳播位址通信
-A INPUT -s 10.0.0.0/24 -p vrrp -j ACCEPT             #允許VRRP（虛拟路由器備援協）通信
-A INPUT -m state --state NEW -m tcp -p tcp --dport 3306 -j ACCEPT    #開放mysql的3306端口
   
[root@master1 ~]# /etc/init.d/iptables restart

 六 Mysql＋keepalived故障轉移的高可用測試

1）通過Mysql用戶端通過VIP連接配接，看是否連接配接成功。

比如，在遠端一台測試機上連接配接，通過vip位址可以正常連接配接（下面的連接配接權限要是在服務端提前授權的）

 
[root@master1 ~]# mysql -uroot -p1qaz@WSX -h10.0.0.20
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 11
Server version: 5.7.21-log MySQL Community Server (GPL)

Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

4 rows in set (0.02 sec)

2）預設情況下，vip是在master1上的。使用

"ip addr"

指令檢視vip切換情況 

 
[root@master1 ~]# ip addr |grep 10.0                 
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    inet 10.0.0.11/8 brd 10.255.255.255 scope global eth0
    inet 10.0.0.20/32 scope global eth0

[root@master2 ~]# ip addr |grep 10.0

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000

inet 10.0.0.12/8 brd 10.255.255.255 scope global eth0

停止master1機器上的mysql服務，根據配置中的腳本，mysql服務停了，keepalived也會停，進而vip資源将會切換到master2機器上。（mysql服務沒有起來的時候，keepalived服務也無法順利啟動！）

[root@master1 ~]# systemctl stop mysqld      
[root@master1 ~]# ps -ef|grep mysql
root       4431   2423  0 15:08 pts/0    00:00:00 grep --color=auto mysql
[root@master1 ~]# ps -ef|grep keepalived
root       4433   2423  0 15:08 pts/0    00:00:00 grep --color=auto keepalived      
[root@master1 ~]# ip addr |grep 10.0                 
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    inet 10.0.0.11/8 brd 10.255.255.255 scope global eth0

檢視master2主機

[root@master2 ~]# ip addr |grep 10.0   
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    inet 10.0.0.12/8 brd 10.255.255.255 scope global eth0
    inet 10.0.0.20/32 scope global eth0

3）再次啟動master1的mysql和keepalived服務。（注意：如果restart重新開機mysql，那麼還要啟動下keepalived，因為mysql重新開機，根據腳本會造成keepalived關閉）

注意：一定要先啟動mysql服務，然後再啟動keepalived服務。如果先啟動keepalived服務，按照上面的配置，mysql沒有起來，就會自動關閉keepalived。

 
[root@master1 ~]# systemctl start mysqld
[root@master1 ~]# /etc/init.d/keepalived start
Starting keepalived (via systemctl):                       [  OK  ]      
[root@master1 ~]# ip addr |grep 10.0          
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    inet 10.0.0.11/8 brd 10.255.255.255 scope global eth0
    inet 10.0.0.20/32 scope global eth0

此時虛拟ip又回到了 master1 主機上了

 以上在vip資源切換過程中，對于用戶端連接配接mysql（使用vip連接配接）來說幾乎是沒有任何影響的。

---------------------------------溫馨提示（Keepalived的搶占和非搶占模式）---------------------------------------

keepalive是基于vrrp協定在linux主機上以守護程序方式，根據配置檔案實作健康檢查。

VRRP是一種選擇協定，它可以把一個虛拟路由器的責任動态配置設定到區域網路上的VRRP路由器中的一台。

控制虛拟路由器IP位址的VRRP路由器稱為主路由器，它負責轉發資料包到這些虛拟IP位址。

一旦主路由器不可用，這種選擇過程就提供了動态的故障轉移機制，這就允許虛拟路由器的IP位址可以作為終端主機的預設第一跳路由器。

keepalive通過多點傳播，單點傳播等方式（自定義），實作keepalive主備推選。工作模式分為搶占和非搶占（通過參數nopreempt來控制）。

1）搶占模式：

主服務正常工作時，虛拟IP會在主上，備不提供服務，當主服務優先級低于備的時候，備會自動搶占虛拟IP，這時，主不提供服務，備提供服務。

也就是說，工作在搶占模式下，不分主備，隻管優先級。

如上配置，不管keepalived.conf裡的state配置成master還是backup，隻看誰的priority優先級高（一般而言，state為MASTER的優先級要高于BACKUP）。

priority優先級高的那一個在故障恢複後，會自動将VIP資源再次搶占回來！！

2）非搶占模式：

這種方式通過參數nopreempt（一般設定在advert_int的那一行下面）來控制。不管priority優先級，隻要MASTER機器發生故障，VIP資源就會被切換到BACKUP上。

并且當MASTER機器恢複後，也不會去将VIP資源搶占回來，直至BACKUP機器發生故障時，才能自動切換回來。

千萬注意：

nopreempt這個參數隻能用于state為backup的情況，是以在配置的時候要把master和backup的state都設定成backup，這樣才會實作keepalived的非搶占模式！

也就是說：

a）當state狀态一個為master，一個為backup的時候，加不加nopreempt這個參數都是一樣的效果。即都是根據priority優先級來決定誰搶占vip資源的，是搶占模式！

b）當state狀态都設定成backup，如果不配置nopreempt參數，那麼也是看priority優先級決定誰搶占vip資源，即也是搶占模式。

c）當state狀态都設定成backup，如果配置nopreempt參數，那麼就不會去考慮priority優先級了，是非搶占模式！即隻有vip目前所在機器發生故障，另一台機器才能接管vip。即使優先級高的那一台機器恢複  後也不會主動搶回vip，隻能等到對方發生故障，才會将vip切回來。

---------------------------------mysql狀态檢測腳本優化---------------------------------

上面的mysql監測腳本有點過于簡單且粗暴，即腳本一旦監測到Master的mysql服務關閉，就立刻把keepalived服務關閉，進而實作vip轉移！

下面對該腳本進行優化，優化後，當監測到Master的mysql服務關閉後，就會将vip切換到Backup上（但此時Master的keepalived服務不會被暴力

kill

）

當Master的mysql服務恢複後，就會再次将VIP資源切回來！

 
[root@master ~]# cat /opt/chk_mysql.sh
#!/bin/bash
MYSQL=/usr/bin/mysql
MYSQL_HOST=localhost
MYSQL_USER=root
MYSQL_PASSWORD=1qaz@WSX
CHECK_TIME=3
  
#mysql  is working MYSQL_OK is 1 , mysql down MYSQL_OK is 0
  
MYSQL_OK=1
  
function check_mysql_helth (){
    $MYSQL -h $MYSQL_HOST -u $MYSQL_USER -p${MYSQL_PASSWORD} -e "show status;" >/dev/null 2>&1
    if [ $? = 0 ] ;then
    MYSQL_OK=1
    else
    MYSQL_OK=0
    fi
    return $MYSQL_OK
}
while [ $CHECK_TIME -ne 0 ]
do
    let "CHECK_TIME -= 1"
    check_mysql_helth
if [ $MYSQL_OK = 1 ] ; then
    CHECK_TIME=0
    exit 0
fi
if [ $MYSQL_OK -eq 0 ] &&  [ $CHECK_TIME -eq 0 ]
then
    pkill keepalived
    exit 1
fi
sleep 1
done

來源：https://www.cnblogs.com/benjamin77/p/8682360.html#auto_id_0

mysql
上一篇: 使用Kibana 分析Nginx 日志并在 Dashboard上展示
下一篇: 值傳遞還是位址傳遞

繼續閱讀