首先來解釋下上面的圖，當keepalived主節點正常時，資料封包會和原來LVS叢集的資料封包流向一樣，如圖中紅色或綠色實線是請求封包的走向，紅色或綠色虛線是響應封包的走向；當keepalived主節點當機後，備份節點會通過心跳資訊去判斷主節點是否線上，如果在規定的時間探測到主節點沒線上後，備份節點會立刻把VIP搶過來，然後提供服務。這時新來的請求就會通過備份節點去處理新的請求，進而實作，服務高可用，避免單點失敗的問題；圖上藍色虛線表示主節點當機後，備份節點處理請求和響應封包的過程。
1、簡述lvs四種叢集特點及使用場景
LVS叢集有4種類型，分别是NAT、DR、TUN、FULLNAT
從工作方式來講，NAT和FULLNAT都要修改請求封包的目标IP和目标端口（NAT）或源IP目标IP或源端口目标端口（FULLNAT），通常情況下，不建議修改源端口。這兩種叢集的特點是，請求封包和響應封包都要經由DIRECTOR（排程器），在NAT類型的叢集中，後端real server一般都是和director在同一網段，且為私網位址。director應該是後端各real server 的網關。而FULLNAT各real server 的ip未必都在同一IP網絡，但後端主機必須能與director通信即可。通常這兩種類型的叢集應用比較多的是NAT，FULLNAT應用比較少，且FULLNAT是非标準應用，是以我們要在使用FULLNAT時還需要額外的給Linux核心打更新檔才能使用；NAT通常應用在一些請求流量沒有太大的叢集環境中，且director和各後端real server在同一IP網網絡，一般用于隐藏後端主機的真實位址；FULLNAT常用于後端主機和director不再同一IP網絡，但他們又可以正常通行的跨網段的内網叢集環境中使用；
DR和TUN這兩種類型的叢集在使用者請求封包上都沒有修改操作，隻是在原來的請求封包上各自封裝了一個新的mac首部（DR）或ip首部（TUN）；這兩種類型的叢集都有這樣的特點，請求封包經由director，響應封包則由各real server各自響應給用戶端，這樣一來，各real server上就必須配置VIP位址；DR類型的叢集的director必須同後端各real server在同一實體網絡，簡單說就是他們之間不能有路由器，原因是DR是基于在原來的請求封包前封裝一個MAC首部，源MAC為DIP，目标MAC為後端real server 中的其中一個RS的MAC；通常這種叢集應用在流量特别大，用來做進站流量接收器來用，通常情況LVS把前端流量接進來交給後端的排程器（這種排程器是基于更好的比對算法來排程，比如基于請求封包的URL做排程），這種常用于多級排程的叢集環境中；而TUN類型的叢集，它和DR類似，都是請求封包經由director 響應封包由各real server 自己響應給用戶端，響應封包不經由director，這種類型的叢集DIP和各後端real server都不在同一機房或同一區域網路，通常都是各real server在一個公網環境中（對real server來講 出口位址都不是同一個）；這種叢集的實作是通過在原請求封包的外邊封裝IP首部，其中源IP是DIP目标IP是RIP，且這種叢集環境中各real server必須能夠支援并識别tun類型的封包（簡單說就是雙IP首部的封包）；通常這種叢集應用在多個real server 不同一公網ip位址下，且各real server相距很遠時（跨機房、跨地市州等）；
2、描述LVS-DR工作原理，并配置實作。
如上圖所示，LVS-DR類型的叢集它的工作邏輯上這樣的，用戶端請求vip，當封包到達LVS伺服器時，LVS伺服器會對收到的封包進行檢視，它看到目标IP是VIP，且自己有VIP，然後它就把封包送到INPUT鍊上去，在INPUT鍊上進行規則比對，這時我們在LVS上寫的規則，定義的叢集，當比對到該封包是叢集服務時，它會把原來的請求封包原封不動的，然後在該封包外邊封裝一個MAC首部，這個MAC首部的源MAC就是DIP所在的接口MAC位址，目标MAC是經過排程算法得到一個RIP，然後通過ARP廣播拿到對應RIP對應接口上的MAC，然後把這個封裝好的封包直接轉發出去。當封包從LVS伺服器的DIP所在接口發出後，交換機會通過目标MAC把這個封包送到對應的RIP接口上去，RIP收到封裝後的封包後，一看目标MAC是自己，然後它就把MAC首部給拆了，然後拿到用戶端的請求封包，一看目标IP也是自己，然後它就拆IP首部，然後拿到用戶端的請求資訊，然後它會根據用戶端的請求資訊給出對應的響應；在RS封裝響應封包時，它會把VIP封裝成源IP，把用戶端IP封裝成目标IP，然後通過VIP所在接口發送出去（這是因為它收到封包時，是從VIP所在接口拿到的請求封包，響應封包會從請求封包進來的接口發送出去）；這時的響應封包會根據目标IP，層層路由最後送到用戶端；用戶端收到封包後，看目标IP是自己的IP，然後它就拆除IP首部拿到服務端給出的響應；
以上就是LVS-DR類型處理封包的過程，接下來我們搭建一個以上面top圖為例的實驗環境
環境說明：
用戶端：192.168.0.99 LVS伺服器：VIP是192.168.0.222 DIP是：192.168.0.10 後端RS1的IP是：192.168.0.20 RS2的IP是192.168.0.30 VIP是192.168.0.222
1）準備2台RS并安裝配置好web服務（2台RS目前IP是192.168.0.20和30先配置好web服務，并提供一個測試頁面，這兩個測試頁面有意不同，友善我們看出排程到那台RS上去了）
[root@dr ~]# curl http://192.168.0.20/test.html
<h1>RS1,192.168.0.20</h1>
[root@dr ~]# curl http://192.168.0.30/test.html
<h1>RS2,192.168.0.30</h1>
[root@dr ~]#
提示：配置好web服務後，dr是能夠通路得到
2）修改核心參數，讓其兩個RS配置的VIP不對區域網路做ARP通報和ARP請求響應，同時配置路由，目标IP為VIP的封包轉發至VIP所在接口，并在各RS上配置好VIP
為了配置友善我們可以寫腳本，然後執行腳本即可
[root@rs1 ~]# cat setparam.sh
#/bin/bash
vip='192.168.0.222'
mask='255.255.255.255'
interface='lo:0'
case $1 in
start)
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
ifconfig $interface $vip netmask $mask broadcast $vip up
route add -host $vip dev $interface
;;
stop)
ifconfig $interface down
echo 0 >/proc/sys/net/ipv4/conf/all/arp_announce
echo 0 >/proc/sys/net/ipv4/conf/lo/arp_announce
echo 0 >/proc/sys/net/ipv4/conf/all/arp_ignore
echo 0 >/proc/sys/net/ipv4/conf/lo/arp_ignore
;;
*)
echo "Usage:bash $0 start|stop"
exit 1
;;
esac
[root@rs1 ~]#
提示：以上腳本的意思就是設定核心參數，然後把VIP綁定到lo:0，添加主機路由
[root@rs1 ~]# ifconfig
ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.20 netmask 255.255.255.0 broadcast 192.168.0.255
ether 00:0c:29:96:23:23 txqueuelen 1000 (Ethernet)
RX packets 31990 bytes 42260814 (40.3 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 23112 bytes 1983590 (1.8 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1 (Local Loopback)
RX packets 259 bytes 21752 (21.2 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 259 bytes 21752 (21.2 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@rs1 ~]# bash -x setparam.sh start
+ vip=192.168.0.222
+ mask=255.255.255.255
+ interface=lo:0
+ case $1 in
+ echo 2
+ echo 2
+ echo 1
+ echo 1
+ ifconfig lo:0 192.168.0.222 netmask 255.255.255.255 broadcast 192.168.0.222 up
+ route add -host 192.168.0.222 dev lo:0
[root@rs1 ~]# cat /proc/sys/net/ipv4/conf/all/arp_announce
2
[root@rs1 ~]# cat /proc/sys/net/ipv4/conf/all/arp_ignore
1
[root@rs1 ~]# cat /proc/sys/net/ipv4/conf/lo/arp_announce
2
[root@rs1 ~]# cat /proc/sys/net/ipv4/conf/lo/arp_ignore
1
[root@rs1 ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.0.1 0.0.0.0 UG 100 0 0 ens33
192.168.0.0 0.0.0.0 255.255.255.0 U 100 0 0 ens33
192.168.0.222 0.0.0.0 255.255.255.255 UH 0 0 0 lo
[root@rs1 ~]# ifconfig
ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.20 netmask 255.255.255.0 broadcast 192.168.0.255
ether 00:0c:29:96:23:23 txqueuelen 1000 (Ethernet)
RX packets 32198 bytes 42279504 (40.3 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 23266 bytes 2001218 (1.9 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1 (Local Loopback)
RX packets 259 bytes 21752 (21.2 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 259 bytes 21752 (21.2 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo:0: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 192.168.0.222 netmask 255.255.255.255
loop txqueuelen 1 (Local Loopback)
[root@rs1 ~]#
提示：可以看到我們執行腳本後，對應的核心參數都已經設定好，VIP和相關路由也都添加成功；同樣RS2也隻需要把上面的腳本拿過去運作一遍即可
3）在LVS伺服器上配置VIP，定義叢集服務
3.1）首先給director綁定VIP
[root@dr ~]# ifconfig
ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.10 netmask 255.255.255.0 broadcast 192.168.0.255
inet6 fe80::20c:29ff:fef2:820c prefixlen 64 scopeid 0x20<link>
ether 00:0c:29:f2:82:0c txqueuelen 1000 (Ethernet)
RX packets 11135 bytes 9240712 (8.8 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 7705 bytes 754318 (736.6 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1 (Local Loopback)
RX packets 70 bytes 5804 (5.6 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 70 bytes 5804 (5.6 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@dr ~]# ifconfig ens33:0 192.168.0.222 netmask 255.255.255.255 broadcast 192.168.0.222 up
[root@dr ~]# ifconfig
ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.10 netmask 255.255.255.0 broadcast 192.168.0.255
inet6 fe80::20c:29ff:fef2:820c prefixlen 64 scopeid 0x20<link>
ether 00:0c:29:f2:82:0c txqueuelen 1000 (Ethernet)
RX packets 11277 bytes 9253418 (8.8 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 7800 bytes 765238 (747.3 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ens33:0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.222 netmask 255.255.255.255 broadcast 192.168.0.222
ether 00:0c:29:f2:82:0c txqueuelen 1000 (Ethernet)
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1 (Local Loopback)
RX packets 70 bytes 5804 (5.6 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 70 bytes 5804 (5.6 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@dr ~]#
3.2）添加叢集服務
[root@dr ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
[root@dr ~]# ipvsadm -A -t 192.168.0.222:80 -s rr
[root@dr ~]# ipvsadm -a -t 192.168.0.222:80 -r 192.168.0.20 -g
[root@dr ~]# ipvsadm -a -t 192.168.0.222:80 -r 192.168.0.30 -g
[root@dr ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.0.222:80 rr
-> 192.168.0.20:80 Route 1 0 0
-> 192.168.0.30:80 Route 1 0 0
[root@dr ~]#
提示：以上規則表示添加一個叢集服務192.168.0.222：80，排程算法是rr(輪詢)，在叢集服務下添加了2個real server 分别是192.168.0.20和192.168.0.30,并且添加為DR類型
4）測試
用用戶端192.168.0.99 去通路VIP
提示：可以看到用戶端是能夠通路的，并且是輪詢的方式通路後端伺服器。我們更換一個排程算法，再來試試
[root@dr ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.0.222:80 rr
-> 192.168.0.20:80 Route 1 0 5
-> 192.168.0.30:80 Route 1 0 5
[root@dr ~]# ipvsadm -E -t 192.168.0.222:80 -s sh
[root@dr ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.0.222:80 sh
-> 192.168.0.20:80 Route 1 0 0
-> 192.168.0.30:80 Route 1 0 0
[root@dr ~]#
提示：以上規則表示更改叢集服務192.168.0.222：80的排程算法為sh
提示：可以看到我們更換了算法後，也就立刻生效了。以上就是LVS-DR類型的叢集實作，這裡需要提醒一點的是如果VIP和DIP不再同一網段，需要考慮後端real server 怎麼将響應封包送出去；
3、實作LVS+Keepalived高可用。
首先來解釋下上面的圖，當keepalived主節點正常時，資料封包會和原來LVS叢集的資料封包流向一樣，如圖中紅色或綠色實線是請求封包的走向，紅色或綠色虛線是響應封包的走向；當keepalived主節點當機後，備份節點會通過心跳資訊去判斷主節點是否線上，如果在規定的時間探測到主節點沒線上後，備份節點會立刻把VIP搶過來，然後提供服務。這時新來的請求就會通過備份節點去處理新的請求，進而實作，服務高可用，避免單點失敗的問題；圖上藍色虛線表示主節點當機後，備份節點處理請求和響應封包的過程。
按照上圖，我們需要在LVS叢集上加一台伺服器，并且需要在原有的LVS叢集的排程器上安裝配置keepalived服務，如上圖
1）在兩個排程器上安裝keepalived
[root@dr1 ~]# yum install -y keepalived
Loaded plugins: fastestmirror
epel | 5.4 kB 00:00:00
my_base | 3.6 kB 00:00:00
(1/2): epel/x86_64/updateinfo | 1.0 MB 00:00:00
(2/2): epel/x86_64/primary_db | 6.7 MB 00:00:01
Loading mirror speeds from cached hostfile
Resolving Dependencies
--> Running transaction check
---> Package keepalived.x86_64 0:1.3.5-1.el7 will be installed
--> Processing Dependency: libnetsnmpmibs.so.31()(64bit) for package: keepalived-1.3.5-1.el7.x86_64
--> Processing Dependency: libnetsnmpagent.so.31()(64bit) for package: keepalived-1.3.5-1.el7.x86_64
--> Processing Dependency: libnetsnmp.so.31()(64bit) for package: keepalived-1.3.5-1.el7.x86_64
--> Running transaction check
---> Package net-snmp-agent-libs.x86_64 1:5.7.2-28.el7 will be installed
--> Processing Dependency: libsensors.so.4()(64bit) for package: 1:net-snmp-agent-libs-5.7.2-28.el7.x86_64
---> Package net-snmp-libs.x86_64 1:5.7.2-28.el7 will be installed
--> Running transaction check
---> Package lm_sensors-libs.x86_64 0:3.4.0-4.20160601gitf9185e5.el7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
==================================================================================================
Package Arch Version Repository Size
==================================================================================================
Installing:
keepalived x86_64 1.3.5-1.el7 my_base 327 k
Installing for dependencies:
lm_sensors-libs x86_64 3.4.0-4.20160601gitf9185e5.el7 my_base 41 k
net-snmp-agent-libs x86_64 1:5.7.2-28.el7 my_base 704 k
net-snmp-libs x86_64 1:5.7.2-28.el7 my_base 748 k
Transaction Summary
==================================================================================================
Install 1 Package (+3 Dependent packages)
Total download size: 1.8 M
Installed size: 6.0 M
Downloading packages:
(1/4): lm_sensors-libs-3.4.0-4.20160601gitf9185e5.el7.x86_64.rpm | 41 kB 00:00:00
(2/4): keepalived-1.3.5-1.el7.x86_64.rpm | 327 kB 00:00:00
(3/4): net-snmp-agent-libs-5.7.2-28.el7.x86_64.rpm | 704 kB 00:00:00
(4/4): net-snmp-libs-5.7.2-28.el7.x86_64.rpm | 748 kB 00:00:00
--------------------------------------------------------------------------------------------------
Total 1.9 MB/s | 1.8 MB 00:00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : 1:net-snmp-libs-5.7.2-28.el7.x86_64 1/4
Installing : lm_sensors-libs-3.4.0-4.20160601gitf9185e5.el7.x86_64 2/4
Installing : 1:net-snmp-agent-libs-5.7.2-28.el7.x86_64 3/4
Installing : keepalived-1.3.5-1.el7.x86_64 4/4
Verifying : 1:net-snmp-libs-5.7.2-28.el7.x86_64 1/4
Verifying : 1:net-snmp-agent-libs-5.7.2-28.el7.x86_64 2/4
Verifying : lm_sensors-libs-3.4.0-4.20160601gitf9185e5.el7.x86_64 3/4
Verifying : keepalived-1.3.5-1.el7.x86_64 4/4
Installed:
keepalived.x86_64 0:1.3.5-1.el7
Dependency Installed:
lm_sensors-libs.x86_64 0:3.4.0-4.20160601gitf9185e5.el7
net-snmp-agent-libs.x86_64 1:5.7.2-28.el7
net-snmp-libs.x86_64 1:5.7.2-28.el7
Complete!
[root@dr1 ~]#
提示：keepalived包來自base包，不需要額外配置epel源，在DR2上也是同樣的操作安裝好keepalived包
2）編寫郵件通知腳本
[root@dr1 ~]# cat /etc/keepalived/notify.sh
#!/bin/bash
#
contact='root@localhost'
notify() {
local mailsubject="$(hostname) to be $1, vip floating"
local mailbody="$(date +'%F %T'): vrrp transition, $(hostname) changed to be $1"
echo "$mailbody" | mail -s "$mailsubject" $contact
}
case $1 in
master)
notify master
;;
backup)
notify backup
;;
fault)
notify fault
;;
*)
echo "Usage: $(basename $0) {master|backup|fault}"
exit 1
;;
esac
[root@dr1 ~]# chmod +x /etc/keepalived/notify.sh
[root@dr1 ~]# ll /etc/keepalived/notify.sh
-rwxr-xr-x 1 root root 405 Feb 21 19:52 /etc/keepalived/notify.sh
[root@dr1 ~]# 提示：以上腳本主要思想是通過傳遞不同狀态的參數，然後相應的發送不同狀态的郵件
3）在DR上面安裝sorry_sever
[root@dr1 ~]# yum install -y nginx
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
Resolving Dependencies
--> Running transaction check
---> Package nginx.x86_64 1:1.16.1-1.el7 will be installed
--> Processing Dependency: nginx-all-modules = 1:1.16.1-1.el7 for package: 1:nginx-1.16.1-1.el7.x86_64
--> Processing Dependency: nginx-filesystem = 1:1.16.1-1.el7 for package: 1:nginx-1.16.1-1.el7.x86_64
--> Processing Dependency: nginx-filesystem for package: 1:nginx-1.16.1-1.el7.x86_64
……省略部分資訊
Installed:
nginx.x86_64 1:1.16.1-1.el7
Dependency Installed:
centos-indexhtml.noarch 0:7-9.el7.centos fontconfig.x86_64 0:2.10.95-11.el7
fontpackages-filesystem.noarch 0:1.44-8.el7 gd.x86_64 0:2.0.35-26.el7
gperftools-libs.x86_64 0:2.4-8.el7 libX11.x86_64 0:1.6.5-1.el7
libX11-common.noarch 0:1.6.5-1.el7 libXau.x86_64 0:1.0.8-2.1.el7
libXpm.x86_64 0:3.5.12-1.el7 libjpeg-turbo.x86_64 0:1.2.90-5.el7
libpng.x86_64 2:1.5.13-7.el7_2 libunwind.x86_64 2:1.2-2.el7
libxcb.x86_64 0:1.12-1.el7 libxslt.x86_64 0:1.1.28-5.el7
lyx-fonts.noarch 0:2.2.3-1.el7 nginx-all-modules.noarch 1:1.16.1-1.el7
nginx-filesystem.noarch 1:1.16.1-1.el7 nginx-mod-http-image-filter.x86_64 1:1.16.1-1.el7
nginx-mod-http-perl.x86_64 1:1.16.1-1.el7 nginx-mod-http-xslt-filter.x86_64 1:1.16.1-1.el7
nginx-mod-mail.x86_64 1:1.16.1-1.el7 nginx-mod-stream.x86_64 1:1.16.1-1.el7
Complete!
[root@dr1 ~]#
提示：在RS2上也是同樣的操作
給sorry server 一個測試首頁
[root@dr1 ~]# cat /usr/share/nginx/html/index.html
<h1>sorry server 192.168.0.10</h1>
[root@dr1 ~]#
[root@dr2 ~]# cat /usr/share/nginx/html/index.html
<h1>sorry server 192.168.0.11<h1>
[root@dr2 ~]#
提示：這兩個頁面可以是一樣的内容，我們為了區分，故意給出不一樣的頁面。
啟動服務
[root@dr1 ~]# systemctl start nginx
[root@dr1 ~]# curl http://127.0.0.1
<h1>sorry server 192.168.0.10</h1>
[root@dr1 ~]#
[root@dr2 ~]# systemctl start nginx
[root@dr2 ~]# curl http://127.0.0.1
<h1>sorry server 192.168.0.11<h1>
[root@dr2 ~]#
提示：可以看到兩個DR上面各自的sorry server都啟動，并能夠通路
4）配置主節點keepalived
1）在配置前需要确認伺服器的各時間是否同步，通常情況下，我們會把一個叢集的所有主機都指向一台時間伺服器，用來同步時間，有關時間伺服器搭建可參考本人部落格https://www.cnblogs.com/qiuhom-1874/p/12079927.html
[root@dr1 ~]# grep "^server" /etc/chrony.conf
server 192.168.0.99 iburst
[root@dr1 ~]#
[root@dr2 ~]# grep "^server" /etc/chrony.conf
server 192.168.0.99 iburst
[root@dr2 ~]#
[root@rs1 ~]# grep "^server" /etc/chrony.conf
server 192.168.0.99 iburst
[root@rs1 ~]#
[root@rs2 ~]# grep "^server" /etc/chrony.conf
server 192.168.0.99 iburst
[root@rs2 ~]#
提示：把時間伺服器位址執行同一個時間伺服器，然後重新開機服務即可同步時間
2）確定iptables及selinux不會成為阻礙；
[root@dr1 ~]# getenforce
Disabled
[root@dr1 ~]# iptables -nvL
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
[root@dr1 ~]#
提示：我們可以選擇關閉selinux和iptables，在centos7上可能會有很多規則，我們可以選擇添加IPTABLES或者直接将規則情況，把預設的處理動作改為ACCEPT也行；
3）各節點之間可通過主機名互相通信（對KA并非必須），建議使用/etc/hosts檔案實作；
[root@dr1 ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.0.10 dr1.ilinux.io dr1
192.168.0.11 dr2.ilinux.io dr2
192.168.0.20 rs1.ilinux.io rs1
192.168.0.30 rs2.ilinux.io rs2
[root@dr1 ~]# scp /etc/hosts 192.168.0.11:/etc/
The authenticity of host '192.168.0.11 (192.168.0.11)' can't be established.
ECDSA key fingerprint is SHA256:EG9nua4JJuUeofheXlgQeL9hX5H53JynOqf2vf53mII.
ECDSA key fingerprint is MD5:57:83:e6:46:2c:4b:bb:33:13:56:17:f7:fd:76:71:cc.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.0.11' (ECDSA) to the list of known hosts.
[email protected]'s password:
hosts 100% 282 74.2KB/s 00:00
[root@dr1 ~]# scp /etc/hosts 192.168.0.20:/etc/
[email protected]'s password:
hosts 100% 282 144.9KB/s 00:00
[root@dr1 ~]# scp /etc/hosts 192.168.0.30:/etc/
[email protected]'s password:
hosts 100% 282 85.8KB/s 00:00
[root@dr1 ~]# ping dr1
PING dr1.ilinux.io (192.168.0.10) 56(84) bytes of data.
64 bytes from dr1.ilinux.io (192.168.0.10): icmp_seq=1 ttl=64 time=0.031 ms
64 bytes from dr1.ilinux.io (192.168.0.10): icmp_seq=2 ttl=64 time=0.046 ms
^C
--- dr1.ilinux.io ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1000ms
rtt min/avg/max/mdev = 0.031/0.038/0.046/0.009 ms
[root@dr1 ~]# ping dr2
PING dr2.ilinux.io (192.168.0.11) 56(84) bytes of data.
64 bytes from dr2.ilinux.io (192.168.0.11): icmp_seq=1 ttl=64 time=1.36 ms
64 bytes from dr2.ilinux.io (192.168.0.11): icmp_seq=2 ttl=64 time=0.599 ms
64 bytes from dr2.ilinux.io (192.168.0.11): icmp_seq=3 ttl=64 time=0.631 ms
^C
--- dr2.ilinux.io ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 0.599/0.865/1.366/0.355 ms
[root@dr1 ~]# ping rs1
PING rs1.ilinux.io (192.168.0.20) 56(84) bytes of data.
64 bytes from rs1.ilinux.io (192.168.0.20): icmp_seq=1 ttl=64 time=0.614 ms
64 bytes from rs1.ilinux.io (192.168.0.20): icmp_seq=2 ttl=64 time=0.628 ms
^C
--- rs1.ilinux.io ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 0.614/0.621/0.628/0.007 ms
[root@dr1 ~]# ping rs2
PING rs2.ilinux.io (192.168.0.30) 56(84) bytes of data.
64 bytes from rs2.ilinux.io (192.168.0.30): icmp_seq=1 ttl=64 time=0.561 ms
64 bytes from rs2.ilinux.io (192.168.0.30): icmp_seq=2 ttl=64 time=0.611 ms
64 bytes from rs2.ilinux.io (192.168.0.30): icmp_seq=3 ttl=64 time=0.653 ms
^C
--- rs2.ilinux.io ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 0.561/0.608/0.653/0.042 ms
[root@dr1 ~]#
提示：配置好的hosts檔案可以通過scp拷貝到各節點即可
4）確定各節點的用于叢集服務的接口支援MULTICAST通信；
以上4點沒有問題的情況下，我們可以來配置keepalived
[root@dr1 ~]# cat /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from keepalived@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DR1
vrrp_mcast_group4 224.10.10.222
}
vrrp_instance VI_1 {
state MASTER
interface ens33
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass Yc15tnWa
}
virtual_ipaddress {
192.168.0.222/24 dev ens33 label ens33:0
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
virtual_server 192.168.0.222 80 {
delay_loop 2
lb_algo rr
lb_kind DR
protocol TCP
sorry_server 127.0.0.1 80
real_server 192.168.0.20 80 {
weight 1
TCP_CHECK {
connect_timeout 3
}
}
real_server 192.168.0.30 80 {
weight 1
HTTP_GET {
url {
path /test.html
status_code 200
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
[root@dr1 ~]#
提示：多點傳播位址是用來傳遞心跳資訊的，我們在配置時需要配置成一個D類位址。
5）複制郵件發送腳本，并配置備份節點keepalived
[root@dr1 ~]# scp /etc/keepalived/notify.sh 192.168.0.11:/etc/keepalived/
[email protected]'s password:
notify.sh 100% 405 116.6KB/s 00:00
[root@dr1 ~]# scp /etc/keepalived/keepalived.conf 192.168.0.11:/etc/keepalived/keepalived.conf.bak
[email protected]'s password:
keepalived.conf 100% 1162 506.4KB/s 00:00
[root@dr1 ~]#
提示：我們可以把主節點配置檔案發送到備份節點上，然後改下就可以了
[root@dr2 ~]# ls /etc/keepalived/
epalived.conf keepalived.conf.bak notify.sh
[root@dr2 ~]# cp /etc/keepalived/keepalived.conf{,.backup}
[root@dr2 ~]# ls /etc/keepalived/
keepalived.conf keepalived.conf.backup keepalived.conf.bak notify.sh
[root@dr2 ~]# mv /etc/keepalived/keepalived.conf.bak /etc/keepalived/keepalived.conf
mv: overwrite ‘/etc/keepalived/keepalived.conf’? y
[root@dr2 ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from keepalived@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DR2
vrrp_mcast_group4 224.10.10.222
}
vrrp_instance VI_1 {
state BACKUP
interface ens33
virtual_router_id 51
priority 99
advert_int 1
authentication {
auth_type PASS
auth_pass Yc15tnWa
}
virtual_ipaddress {
192.168.0.222/24 dev ens33 label ens33:0
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
virtual_server 192.168.0.222 80 {
delay_loop 2
lb_algo rr
lb_kind DR
protocol TCP
sorry_server 127.0.0.1 80
real_server 192.168.0.20 80 {
weight 1
TCP_CHECK {
connect_timeout 3
}
}
real_server 192.168.0.30 80 {
weight 1
HTTP_GET {
url {
path /test.html
status_code 200
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
"/etc/keepalived/keepalived.conf" 60L, 1161C written
[root@dr2 ~]#
提示：如果我們是從主節點複制配置檔案到備份節點上去，我們隻需要更改global_defs裡面的route_id；vrrp_instances裡更改state 為BACKUP,priority 為99,這個值表示優先級，數字越小優先級越低
6）啟動主備節點，看啟VIP是否都已配，以及LVS規則是否生成
[root@dr1 ~]# ifconfig
ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.10 netmask 255.255.255.0 broadcast 192.168.0.255
inet6 fe80::20c:29ff:fef2:820c prefixlen 64 scopeid 0x20<link>
ether 00:0c:29:f2:82:0c txqueuelen 1000 (Ethernet)
RX packets 16914 bytes 14760959 (14.0 MiB)
RX errors 0 dropped 1 overruns 0 frame 0
TX packets 12058 bytes 1375703 (1.3 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1 (Local Loopback)
RX packets 15 bytes 1304 (1.2 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 15 bytes 1304 (1.2 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@dr1 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
[root@dr1 ~]# systemctl start keepalived
[root@dr1 ~]# ifconfig
ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.10 netmask 255.255.255.0 broadcast 192.168.0.255
inet6 fe80::20c:29ff:fef2:820c prefixlen 64 scopeid 0x20<link>
ether 00:0c:29:f2:82:0c txqueuelen 1000 (Ethernet)
RX packets 17003 bytes 14768581 (14.0 MiB)
RX errors 0 dropped 1 overruns 0 frame 0
TX packets 12150 bytes 1388509 (1.3 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ens33:0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.222 netmask 255.255.255.0 broadcast 0.0.0.0
ether 00:0c:29:f2:82:0c txqueuelen 1000 (Ethernet)
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1 (Local Loopback)
RX packets 15 bytes 1304 (1.2 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 15 bytes 1304 (1.2 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@dr1 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.0.222:80 rr
-> 192.168.0.20:80 Route 1 0 0
-> 192.168.0.30:80 Route 1 0 0
[root@dr1 ~]#
提示：可看到keepalived啟動後，VIP和LVS規則就自動生成了，接下我們在備份幾點抓包看看主節點是否在向多點傳播位址發送心跳資訊
提示：可看到主節點在向多點傳播位址通告自己的心跳資訊
啟動備份節點
[root@dr2 ~]# systemctl start keepalived
[root@dr2 ~]# ifconfig
ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.11 netmask 255.255.255.0 broadcast 192.168.0.255
inet6 fe80::20c:29ff:fe50:13f1 prefixlen 64 scopeid 0x20<link>
ether 00:0c:29:50:13:f1 txqueuelen 1000 (Ethernet)
RX packets 12542 bytes 14907658 (14.2 MiB)
RX errors 0 dropped 1 overruns 0 frame 0
TX packets 7843 bytes 701839 (685.3 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1 (Local Loopback)
RX packets 10 bytes 879 (879.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 10 bytes 879 (879.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@dr2 ~]# tcpdump -i ens33 -nn host 224.10.10.222
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens33, link-type EN10MB (Ethernet), capture size 262144 bytes
20:59:33.620661 IP 192.168.0.10 > 224.10.10.222: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20
20:59:34.622645 IP 192.168.0.10 > 224.10.10.222: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20
20:59:35.624590 IP 192.168.0.10 > 224.10.10.222: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20
20:59:36.626588 IP 192.168.0.10 > 224.10.10.222: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20
20:59:37.628675 IP 192.168.0.10 > 224.10.10.222: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20
20:59:38.630562 IP 192.168.0.10 > 224.10.10.222: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20
20:59:39.632673 IP 192.168.0.10 > 224.10.10.222: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20
20:59:40.634658 IP 192.168.0.10 > 224.10.10.222: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20
20:59:41.636699 IP 192.168.0.10 > 224.10.10.222: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20
^C
9 packets captured
9 packets received by filter
0 packets dropped by kernel
[root@dr2 ~]#
提示：可以看到備份節點啟動後，它不會去拿VIP，這是因為主節點的優先級要比備份節點高，同時主節點在向多點傳播位址通告自己的心跳資訊
用用戶端192.168.0.99 去通路叢集服務
[qiuhom@test ~]$ ip a s enp2s0
2: enp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:30:18:51:af:3c brd ff:ff:ff:ff:ff:ff
inet 192.168.0.99/24 brd 192.168.0.255 scope global noprefixroute enp2s0
valid_lft forever preferred_lft forever
inet 172.16.1.2/16 brd 172.16.255.255 scope global noprefixroute enp2s0:0
valid_lft forever preferred_lft forever
inet6 fe80::230:18ff:fe51:af3c/64 scope link
valid_lft forever preferred_lft forever
[qiuhom@test ~]$ curl http://192.168.0.222/test.html
<h1>RS2,192.168.0.30</h1>
[qiuhom@test ~]$ curl http://192.168.0.222/test.html
<h1>RS1,192.168.0.20</h1>
[qiuhom@test ~]$ curl http://192.168.0.222/test.html
<h1>RS2,192.168.0.30</h1>
[qiuhom@test ~]$
提示：現在主節點正常的情況，叢集服務是可以正常通路的
把主節點停掉，看看叢集服務是否能夠正常通路
[root@dr1 ~]# systemctl stop keepalived
[root@dr1 ~]# ifconfig
ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.10 netmask 255.255.255.0 broadcast 192.168.0.255
inet6 fe80::20c:29ff:fef2:820c prefixlen 64 scopeid 0x20<link>
ether 00:0c:29:f2:82:0c txqueuelen 1000 (Ethernet)
RX packets 18001 bytes 15406859 (14.6 MiB)
RX errors 0 dropped 1 overruns 0 frame 0
TX packets 14407 bytes 1548635 (1.4 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1 (Local Loopback)
RX packets 15 bytes 1304 (1.2 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 15 bytes 1304 (1.2 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@dr1 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
[root@dr1 ~]#
提示：可以看到當我們停掉了主節點後，vip和lvs規則也就自動删除了，接下來，我們再用用戶端來通路下叢集服務，看看是否可通路？
[qiuhom@test ~]$ curl http://192.168.0.222/test.html
<h1>RS2,192.168.0.30</h1>
[qiuhom@test ~]$ curl http://192.168.0.222/test.html
<h1>RS1,192.168.0.20</h1>
[qiuhom@test ~]$ curl http://192.168.0.222/test.html
<h1>RS2,192.168.0.30</h1>
[qiuhom@test ~]$ curl http://192.168.0.222/test.html
<h1>RS1,192.168.0.20</h1>
[qiuhom@test ~]$ curl http://192.168.0.222/test.html
<h1>RS2,192.168.0.30</h1>
[qiuhom@test ~]$
提示：可看到在主節點當機的情況，叢集服務是不受影響的，這是因為備份節點接管了主節點的工作，把VIP和LVS規則在自己的節點上應用了
我們再看看備份節點上的IP資訊和LVS規則
[root@dr2 ~]# ifconfig
ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.11 netmask 255.255.255.0 broadcast 192.168.0.255
inet6 fe80::20c:29ff:fe50:13f1 prefixlen 64 scopeid 0x20<link>
ether 00:0c:29:50:13:f1 txqueuelen 1000 (Ethernet)
RX packets 13545 bytes 15227354 (14.5 MiB)
RX errors 0 dropped 1 overruns 0 frame 0
TX packets 9644 bytes 828542 (809.1 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ens33:0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.222 netmask 255.255.255.0 broadcast 0.0.0.0
ether 00:0c:29:50:13:f1 txqueuelen 1000 (Ethernet)
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1 (Local Loopback)
RX packets 10 bytes 879 (879.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 10 bytes 879 (879.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@dr2 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.0.222:80 rr
-> 192.168.0.20:80 Route 1 0 0
-> 192.168.0.30:80 Route 1 0 0
[root@dr2 ~]#
我們把主節點恢複，再看看備份節點是否把VIP和LVS規則删除？
[root@dr1 ~]# systemctl start keepalived
[root@dr1 ~]# ifconfig
ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.10 netmask 255.255.255.0 broadcast 192.168.0.255
inet6 fe80::20c:29ff:fef2:820c prefixlen 64 scopeid 0x20<link>
ether 00:0c:29:f2:82:0c txqueuelen 1000 (Ethernet)
RX packets 18533 bytes 15699933 (14.9 MiB)
RX errors 0 dropped 1 overruns 0 frame 0
TX packets 14808 bytes 1589148 (1.5 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ens33:0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.222 netmask 255.255.255.0 broadcast 0.0.0.0
ether 00:0c:29:f2:82:0c txqueuelen 1000 (Ethernet)
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1 (Local Loopback)
RX packets 17 bytes 1402 (1.3 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 17 bytes 1402 (1.3 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@dr1 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.0.222:80 rr
-> 192.168.0.20:80 Route 1 0 0
-> 192.168.0.30:80 Route 1 0 0
[root@dr1 ~]#
提示：可以看到主節點啟動keepalived後，VIP和LVS規則都自動生成，我們再來看看備份節點上的VIP和LVS是否存在？
[root@dr2 ~]# ifconfig
ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.11 netmask 255.255.255.0 broadcast 192.168.0.255
inet6 fe80::20c:29ff:fe50:13f1 prefixlen 64 scopeid 0x20<link>
ether 00:0c:29:50:13:f1 txqueuelen 1000 (Ethernet)
RX packets 13773 bytes 15243276 (14.5 MiB)
RX errors 0 dropped 1 overruns 0 frame 0
TX packets 10049 bytes 857748 (837.6 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1 (Local Loopback)
RX packets 12 bytes 977 (977.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 12 bytes 977 (977.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
You have mail in /var/spool/mail/root
[root@dr2 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.0.222:80 rr
-> 192.168.0.20:80 Route 1 0 0
-> 192.168.0.30:80 Route 1 0 0
[root@dr2 ~]#
提示：可以看到主節點啟動後，備份節點上的VIP沒有了，但是LVS規則還存在；我們看看備份節點上是否有郵件呢？
[root@dr2 ~]# mail
Heirloom Mail version 12.5 7/5/10. Type ? for help.
"/var/spool/mail/root": 1 message 1 new
>N 1 root Fri Feb 21 08:13 18/673 "dr2.ilinux.io to be backup, vip floatin"
& 1
Message 1:
From [email protected] Fri Feb 21 08:13:00 2020
Return-Path: <[email protected]>
X-Original-To: root@localhost
Delivered-To: [email protected]
Date: Fri, 21 Feb 2020 08:13:00 -0500
To: [email protected]
Subject: dr2.ilinux.io to be backup, vip floating
User-Agent: Heirloom mailx 12.5 7/5/10
Content-Type: text/plain; charset=us-ascii
From: [email protected] (root)
Status: R
2020-02-21 08:13:00: vrrp transition, dr2.ilinux.io changed to be backup
&
提示：可看到有一封郵件，告訴我們DR2切換至backup狀态了，按理說，主節點上也有郵件，不妨我們也去看看吧
[root@dr1 ~]# mail
Heirloom Mail version 12.5 7/5/10. Type ? for help.
"/var/spool/mail/root": 1 message 1 new
>N 1 root Fri Feb 21 08:13 18/673 "dr1.ilinux.io to be master, vip floatin"
& 1
Message 1:
From [email protected] Fri Feb 21 08:13:01 2020
Return-Path: <[email protected]>
X-Original-To: root@localhost
Delivered-To: [email protected]
Date: Fri, 21 Feb 2020 08:13:01 -0500
To: [email protected]
Subject: dr1.ilinux.io to be master, vip floating
User-Agent: Heirloom mailx 12.5 7/5/10
Content-Type: text/plain; charset=us-ascii
From: [email protected] (root)
Status: R
2020-02-21 08:13:01: vrrp transition, dr1.ilinux.io changed to be master
&
提示：在主節點上也收到了一封郵件，說dr1切換成master狀态了
到此LVS+keepalived高可用LVS測試沒有問題，接下來我們在測試，當一個real server當機後，DR上的LVS是否能夠及時的把對應的RS下線？
[root@rs1 ~]# systemctl stop nginx
[root@rs1 ~]# ss -ntl
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 *:22 *:*
LISTEN 0 100 127.0.0.1:25 *:*
LISTEN 0 128 :::22 :::*
LISTEN 0 100 ::1:25 :::*
[root@rs1 ~]#
[root@dr1 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.0.222:80 rr
-> 192.168.0.30:80 Route 1 0 0
[root@dr1 ~]#
提示：可以看到當一個RS1故障時，DR會馬上把rs1從叢集服務下線
再把RS2都停掉看看 對應的sorry server是否能夠正常加到叢集服務
[root@rs2 ~]# systemctl stop nginx
[root@rs2 ~]# ss -ntl
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 *:22 *:*
LISTEN 0 100 127.0.0.1:25 *:*
LISTEN 0 128 :::22 :::*
LISTEN 0 100 ::1:25 :::*
[root@rs2 ~]#
[root@dr1 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.0.222:80 rr
-> 127.0.0.1:80 Route 1 0 0
[root@dr1 ~]#
提示：可看到當後端兩個RS都當機後，sorry server 會立刻加到叢集來，如果這時用戶端再通路叢集服務，就會把sorry server 的頁面響應給使用者
[qiuhom@test ~]$ curl http://192.168.0.222/
<h1>sorry server 192.168.0.10</h1>
[qiuhom@test ~]$
提示：這個頁面主要是告訴使用者，網站正在維護等資訊，專用于給使用者說sorry的，是以叫sorry server ;當然我們也可以把它配置成和叢集服務頁面一模一樣的也是可以的，一般不建議這樣做
我們把RS1 啟動起來，再看看叢集是否把sorry server 下線把RS1加到叢集？
[root@rs1 ~]# systemctl start nginx
[root@rs1 ~]# ss -ntl
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 *:80 *:*
LISTEN 0 128 *:22 *:*
LISTEN 0 100 127.0.0.1:25 *:*
LISTEN 0 128 :::80 :::*
LISTEN 0 128 :::22 :::*
LISTEN 0 100 ::1:25 :::*
[root@rs1 ~]#
[root@dr1 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.0.222:80 rr
-> 192.168.0.20:80 Route 1 0 0
You have new mail in /var/spool/mail/root
[root@dr1 ~]#
提示：可看到當後端real server 正常後，sorry server會從叢集服務裡下線，然後real server繼續提供服務
到此LVS叢集+keepalived高可用搭建和測試就完畢了！！！
作者：Linux-1874
出處：https://www.cnblogs.com/qiuhom-1874/
本文版權歸作者和部落格園共有，歡迎轉載，但未經作者同意必須保留此段聲明，且在文章頁面明顯位置給出原文連接配接，否則保留追究法律責任的權利.