天天看点
返回

LVS+KeepAlived+Nginx+Tomcat高可用解决方案

1、环境：

1、操作系统：Centos7.6

2、服务器配置如下：

服务器 软件
192.168.0.13 LVS+KeepAlived
192.168.0.14 LVS+KeepAlived
192.168.0.15 Nginx
192.168.0.16 Nginx
192.168.0.17 Tomcat
192.168.0.18 Tomcat
VIP 192.168.1.200

2、安装lvs+keepalived

本文安装采用源代码编译方式进行安装。

2.1 Lvs

从2.4版本开始，linux内核默认支持LVS。要使用LVS的能力，只需安装一个LVS的管理工具：ipvsadm。

yum -y install ipvsadm

2.2 keepalived

同时在

192.168.0.13

和

192.168.0.14

两台服务器上操作：

技巧：

在Xshell的

撰写窗格

里同时对两台服务器进行操作。

2.2.1 下载 

`进入到/usr/local/src目录下`
[[email protected] ~]# cd /usr/local/src

`下载keepalived`
[[email protected] src]# wget https://www.keepalived.org/software/keepalived-2.0.20.tar.gz

`解压缩`
[[email protected] src]# tar -zxvf keepalived-2.0.20.tar.gz

2.2.2 安装 

`在/usr/local目录下创建keepalived文件夹`
[[email protected] keepalived-2.0.20]# mkdir /usr/local/keepalived

`将keepalived安装到/usr/local/keepalived下，conf配置文件指定到目录/etc下`
[[email protected] keepalived-2.0.20]# ./configure --prefix=/usr/local/keepalived --sysconf=/etc

`编译安装`
[[email protected] keepalived-2.0.20]# make && make install

编译过程中可能会出现如下常见问题：

1、缺少

OpenSSL

 
`-------错误信息---------------`
hecking openssl/ssl.h usability... no
checking openssl/ssl.h presence... no
checking for openssl/ssl.h... no
configure: error: 
  !!! OpenSSL is not properly installed on your system. !!!
  !!! Can not include OpenSSL headers files.            !!!
  
`----- ---解决方案--------------------`
  yum -y install openssl-devel

2、缺少

libnl/libnl-3

 
`--------错误信息---------------`
*** WARNING - this build will not support IPVS with IPv6. Please install libnl/libnl-3 dev libraries to support IPv6 with IPVS.

`-------解决方案--------------------`
yum -y install libnl libnl-devel

2.2.3 配置 

`进入安装后的路径 cd /data/program/keepalived, 创建软连接`
[[email protected] sbin]# ln -s /usr/local/keepalived/sbin/keepalived  /sbin/

`把 keepalived的启动文件复制到init.d下，加入开机启动项`
[[email protected] keepalived-2.0.20]# cp /usr/local/src/keepalived-2.0.20/keepalived/etc/init.d/keepalived /etc/init.d

`添加keepalived到系统服务`
[[email protected] sbin]# chkconfig –add keepalived
chkconfig version 1.7.4 - Copyright (C) 1997-2000 Red Hat, Inc.
This may be freely redistributed under the terms of the GNU Public License.

usage:   chkconfig [--list] [--type <type>] [name]
         chkconfig --add <name>
         chkconfig --del <name>
         chkconfig --override <name>
         chkconfig [--level <levels>] [--type <type>] <name> <on|off|reset|resetpriorities>
`检测是否添加成功`
[[email protected] sbin]# chkconfig keepalived on
Note: Forwarding request to 'systemctl enable keepalived.service'.
Created symlink from /etc/systemd/system/multi-user.target.wants/keepalived.service to /usr/lib/systemd/system/keepalived.service.

2.2.4 启动keepalived 

`启动keepalived服务`
[[email protected] sbin]# systemctl start keepalived.service
`查看keepalived状态`
[[email protected] sbin]# systemctl status keepalived.service
● keepalived.service - LVS and VRRP High Availability Monitor
   Loaded: loaded (/usr/lib/systemd/system/keepalived.service; enabled; vendor preset: disabled)
   Active: active (running) since Sat 2020-03-07 22:13:54 CST; 3s ago
  Process: 25684 ExecStart=/usr/local/keepalived/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS)
 Main PID: 25685 (keepalived)
   CGroup: /system.slice/keepalived.service
           ├─25685 /usr/local/keepalived/sbin/keepalived -D
           ├─25686 /usr/local/keepalived/sbin/keepalived -D
           └─25687 /usr/local/keepalived/sbin/keepalived -D

-------------------------------------------------

操作keepalived的命令有如下：

`----启动-----`
systemctl start keepalived.service
`----重启-----`
systemctl restart keepalived.service
`----停止-----`
systemctl stop keepalived.service
`----查看状态-----`
systemctl status keepalived.service

2.3 防火墙

为方便测试，我直接关闭了防火墙，在实际应用中可以根据需要开启防火墙的端口，此外还要设置服务器的安全策略，我的是阿里云的服务器，就在阿里云服务器控制台设置了安全策略，开放了需要的端口。

关闭防火墙：

`关闭防火墙`
[[email protected] sysconfig]# systemctl stop firewalld

`查看防火墙状态`
[[email protected] sysconfig]# systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
   Active: `inactive (dead)`
     Docs: man:firewalld(1)

Mar 07 22:39:08 henry001 systemd[1]: Starting firewalld - dynamic firewall daemon...
Mar 07 22:39:08 henry001 systemd[1]: Started firewalld - dynamic firewall daemon.
Mar 07 22:39:23 henry001 systemd[1]: Stopping firewalld - dynamic firewall daemon...
Mar 07 22:39:24 henry001 systemd[1]: Stopped firewalld - dynamic firewall daemon.

3、安装Nginx

同时在192.168.0.15和192.168.0.16两台服务器上操作，对

Nginx+Tomcat

的安装请参考文章：

实现Nginx+Tomcat负载均衡 https://blog.csdn.net/qq_33996921/article/details/104999852

4、配置keepalived

4.1master服务器

先来配置

192.168.0.13

的主机，指定其为

master

服务器；

`进入配置文件目录`
[[email protected] ~]#  cd /etc/keepalived
[[email protected] keepalived]# ls
keepalived.conf  samples
`编辑配置文件信息`
[[email protected] keepalived]# vim keepalived.conf

编辑

keepalived.conf

文件

global_defs {
   #notification_email {
   #      [email protected]
   #}
  # notification_email_from [email protected]
  # smtp_server 192.168.80.1
   #smtp_connection_timeout 30
   router_id LVS_DEVEL  # 设置lvs的id，在一个网络内应该是唯一的
}
vrrp_instance VI_1 {
    state MASTER   #指定Keepalived的角色，MASTER为主，BACKUP为备 记得大写
    interface eth0  #网卡id 不同的电脑网卡id会有区别 可以使用:ip a查看
    virtual_router_id 51  #虚拟路由编号，主备要一致
    priority 100  #定义优先级，数字越大，优先级越高，主DR必须大于备用DR
    advert_int 1  #检查间隔，默认为1s
    authentication {   #这里配置的密码最多为8位，主备要一致，否则无法正常通讯
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.0.200  #定义虚拟IP(VIP)为192.168.1.200，可多设，每行一个
    }
}
# 定义对外提供服务的LVS的VIP以及port
virtual_server 192.168.0.200 80 {
    delay_loop 6 # 设置健康检查时间，单位是秒
    lb_algo rr # 设置负载调度的算法为wlc
    lb_kind DR # 设置LVS实现负载的机制，有NAT、TUN、DR三个模式
    #nat_mask 255.255.255.0
    persistence_timeout 0
    protocol TCP
    real_server 192.168.0.15 80 {  # 指定real server1的IP地址
        weight 3   # 配置节点权值，数字越大权重越高
        TCP_CHECK {
        connect_timeout 10
        nb_get_retry 3
        delay_before_retry 3
        connect_port 80
        }
    }
    real_server 192.168.0.16 80 {  # 指定real server2的IP地址
        weight 3  # 配置节点权值，数字越大权重越高
        TCP_CHECK {
        connect_timeout 10
        nb_get_retry 3
        delay_before_retry 3
        connect_port 80
        }
     }
}

4.1 backup服务器

配置

192.168.0.14

的主机，指定其为

backup

服务器；

`进入配置文件目录`
[[email protected] ~]#  cd /etc/keepalived
[[email protected] keepalived]# ls
keepalived.conf  samples
`编辑配置文件信息`
[[email protected] keepalived]# vim keepalived.conf

编辑

keepalived.conf

文件

global_defs {
   #notification_email {
   #      [email protected]
   #}
  # notification_email_from [email protected]
  #  smtp_server 192.168.80.1
  #  smtp_connection_timeout 30
   router_id LVS_DEVEL  # 设置lvs的id，在一个网络内应该是唯一的
}
vrrp_instance VI_1 {
    state BACKUP #指定Keepalived的角色，MASTER为主，BACKUP为备 记得大写
    interface eth0  #网卡id 不同的电脑网卡id会有区别 可以使用:ip a查看
    virtual_router_id 51  #虚拟路由编号，主备要一致
    priority 50  #定义优先级，数字越大，优先级越高，主DR必须大于备用DR
    advert_int 1  #检查间隔，默认为1s
    authentication {   #这里配置的密码最多为8位，主备要一致，否则无法正常通讯
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.0.200  #定义虚拟IP(VIP)为192.168.1.200，可多设，每行一个
    }
}
# 定义对外提供服务的LVS的VIP以及port
virtual_server 192.168.0.200 80 {
    delay_loop 6 # 设置健康检查时间，单位是秒
    lb_algo rr # 设置负载调度的算法为wlc
    lb_kind DR # 设置LVS实现负载的机制，有NAT、TUN、DR三个模式
    #nat_mask 255.255.255.0
    persistence_timeout 0
    protocol TCP
    real_server 192.168.0.16 80 {  # 指定real server1的IP地址
        weight 3   # 配置节点权值，数字越大权重越高
        TCP_CHECK {
        connect_timeout 10
        nb_get_retry 3
        delay_before_retry 3
        connect_port 80
        }
    }
    real_server 192.168.0.15 80 {  # 指定real server2的IP地址
        weight 3  # 配置节点权值，数字越大权重越高
        TCP_CHECK {
        connect_timeout 10
        nb_get_retry 3
        delay_before_retry 3
        connect_port 80
        }
     }
}

5、查看虚拟IP

查看

master

服务器：

[[email protected] keepalived]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:16:3e:30:cc:a2 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.13/24 brd 192.168.0.255 scope global dynamic eth0
       valid_lft 315332386sec preferred_lft 315332386sec
    `inet 192.168.1.200/32 scope global eth0`
       valid_lft forever preferred_lft forever

查看backup服务器

[[email protected] keepalived]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:16:3e:30:9f:0f brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.14/24 brd 192.168.0.255 scope global dynamic eth0
       valid_lft 315348979sec preferred_lft 315348979sec

下面我们停止掉

master

服务器上的keepalived，虚拟ip将会漂移到

backup

服务器上

[[email protected] keepalived]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:16:3e:30:9f:0f brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.14/24 brd 192.168.0.255 scope global dynamic eth0
       valid_lft 315348845sec preferred_lft 315348845sec
      `inet 192.168.0.200/32 scope global eth0`
       valid_lft forever preferred_lft forever

至此,一个

LVS+KeepAlived+Nginx+Tomcat

的高可用方案就搭建完成了。

tomcat nginx lvs keepalived 集群
上一篇: glusterfs双副本原理解析和脑裂解决方案2．Glusterfs双副本在虚拟化中的应用3．Glusterfs双副本实现过程4.  Glusterfs双副本一致性和可用性分析5．  Glusterfs针对虚拟化应用优化策略双副本spp状态机详解spp协议流程图
下一篇: 2. Keepalived-高可用列脑

继续阅读