LVS + keepalived + tomcat负载均衡及高可用实现(初级)

时间:2022-01-07 22:05:55

1、首先检测Linux服务器是否支持ipvs

  执行如下命令:modprobe -l|grep ipvs

  输出:

kernel/net/netfilter/ipvs/ip_vs.ko
kernel/net/netfilter/ipvs/ip_vs_rr.ko
kernel/net/netfilter/ipvs/ip_vs_wrr.ko
kernel/net/netfilter/ipvs/ip_vs_lc.ko
kernel/net/netfilter/ipvs/ip_vs_wlc.ko
kernel/net/netfilter/ipvs/ip_vs_lblc.ko
kernel/net/netfilter/ipvs/ip_vs_lblcr.ko
kernel/net/netfilter/ipvs/ip_vs_dh.ko
kernel/net/netfilter/ipvs/ip_vs_sh.ko
kernel/net/netfilter/ipvs/ip_vs_sed.ko
kernel/net/netfilter/ipvs/ip_vs_nq.ko
kernel/net/netfilter/ipvs/ip_vs_ftp.ko
kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko

表示支持!

否则需要手动下载安装ipvs

 

2、检查是否已经安装所必须的包

  执行如下命令:

  rpm -q kernel-devel   
  rpm -q gcc   
  rpm -q openssl   
  rpm -q openssl-devel
  rpm -q popt

  rpm -q popt-static

   rpm -q kernel-headers

  输出:package ** is not installed  需要手动安装;

  yum命令是安装,更新,删除依赖包的命令;相当于一个软件包管理器;

  安装方法:yum install kernel-devel -y 

      yum install gcc -y

      ……

3、安装ipvsadm

  首先确定安装的版本,安装的版本应该与Linux内核版本一致;

  通过:rpm -q kernel-devel  查看内核版本,根据内核版本,下载相对应的ipvsadm;

  我的内核版本:kernel-devel-2.6.32-642.3.1.el6.x86_64

  对应的ipvsadm版本:ipvsadm-1.26.tar.gz

4、解压

  tar -zxvf ipvsadm-1.26.tar.gz

5、建立软连接

  ln -s /usr/src/kernels/2.6.32-642.3.1.el6.x86_64 /usr/src/linux

6、编译安装

  进入到ipvsadm-1.26下

  执行 make && make install

  报错可能是依赖的包缺失,需要安装:yum install -y libnl* popt*

  至此,安装ipvsadm成功  

7、keepalive安装

  下载地址:http://www.keepalived.org/software/

  解压;

  进入到keepalive目录:cd keepalived-1.2.19

  执行  ./configure

  等待…………

  执行  make

  等待…………

  执行  make install

  安装完成。

 8、配置主从服务器

  进入如下目录: /usr/local/etc/keepalived/   打开keepalived.conf文件

 

LVS + keepalived + tomcat负载均衡及高可用实现(初级)LVS + keepalived + tomcat负载均衡及高可用实现(初级)
 1 ! Configuration File for keepalived
 2 
 3 global_defs {
 4    notification_email {
 5      acassen@firewall.loc
 6      failover@firewall.loc
 7      sysadmin@firewall.loc
 8    }
 9    notification_email_from Alexandre.Cassen@firewall.loc
10    smtp_server 127.0.0.1
11    smtp_connect_timeout 30
12    router_id LVS_DEVEL
13 }
14 
15 vrrp_instance VI_1 {
16     state MASTER
17     interface eth0
18     virtual_router_id 230 
19     priority 100 
20     advert_int 1
21     authentication {
22         auth_type PASS
23         auth_pass 1111
24     }
25     virtual_ipaddress {
26         192.168.91.230
27     }
28 }
29 
30 virtual_server 192.168.91.230 8080 {
31     delay_loop 6
32     lb_algo rr
33     lb_kind DR
34     nat_mask 255.255.255.0
35     persistence_timeout 0 
36     protocol TCP
37 
38     real_server 192.168.91.231 8080 {
39         weight 1
40         TCP_CHECK {
41             connect_timeout 3
42             nb_get_retry 3
43             delay_before_retry 3
44             connect_port 8080
45         }
46     }
47     real_server 192.168.91.232 8080 {
48         weight 1
49         TCP_CHECK {
50             connect_timeout 3
51             nb_get_retry 3
52             delay_before_retry 3
53             connect_port 8080
54         }
55     }
56 }
主服务器
LVS + keepalived + tomcat负载均衡及高可用实现(初级)LVS + keepalived + tomcat负载均衡及高可用实现(初级)
! Configuration File for keepalived

global_defs {
   notification_email {
     acassen@firewall.loc
     failover@firewall.loc
     sysadmin@firewall.loc
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 230 
    priority 99
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.91.230
    }
}

virtual_server 192.168.91.230 8080 {
    delay_loop 6
    lb_algo rr
    lb_kind DR
    nat_mask 255.255.255.0
    persistence_timeout 0 
    protocol TCP

    real_server 192.168.91.231 8080 {
        weight 1
        TCP_CHECK {
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
            connect_port 8080
        }
    }
    real_server 192.168.91.232 8080 {
        weight 1
        TCP_CHECK {
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
            connect_port 8080
        }
    }
}
备服务器

 

9、配置LVS server客户端

  执行如下脚本即可:

#!/bin/bash  
# description: Config realserver  
LVS_VIP=192.168.91.230  
/etc/rc.d/init.d/functions  
case "$1" in  
start)  
       /sbin/ifconfig lo:0 $LVS_VIP netmask 255.255.255.255 broadcast $LVS_VIP  
       /sbin/route add -host $LVS_VIP dev lo:0  
       echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore  
       echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce  
       echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore  
       echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce  
       sysctl -p >/dev/null 2>&1  
       echo "RealServer Start OK"  
       ;;  
stop)  
       /sbin/ifconfig lo:0 down  
       /sbin/route del $LVS_VIP >/dev/null 2>&1  
       echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore  
       echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce  
       echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore  
       echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce  
       echo "RealServer Stoped"  
       ;;  
*)  
       echo "Usage: $0 {start|stop}"  
       exit 1  
esac  
exit 0  

 

10、keepalived的启动

  service keepalived start  或者 keepalived -D -f /usr/local/etc/keepalived/keepalived.conf

  查看日志:tail -f /var/log/messages

  在默认情况下,Keepalived在启动时会查找/etc/Keepalived/Keepalived.conf配置文件

 

小结:到此为止,当主服务器或者从服务器上的tomcat挂掉之后,不会对客户端产生影响;

上述keepalived健康检测是协议检测,也就是说跟业务无关;

遗留问题:

1、  本机realserver宕掉后,不会切换到从服务器;(已解决,未配置LVS server客户端)

2、  Kill掉keepalived后,另一个不会自动接管VIP(已解决  是防火墙的原因,关闭防火墙即可)

3、  通过ip a命令,查看,两个机器同时占有VIP(已解决  是防火墙的原因,关闭防火墙即可)

  同时keepalived  kill掉之后,另一个会主动接管虚IP  killall keepalived   (杀掉keepalive进程 ,kill 与 all之间没有空格)

 

一些常用命令:

  ip a 查看此时VIP是否启用

  正常情况,输出:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet 192.168.91.230/32 brd 192.168.91.230 scope global lo:0
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:23:ae:9e:0e:20 brd ff:ff:ff:ff:ff:ff
    inet 192.168.91.231/24 brd 192.168.91.255 scope global eth0
    inet 192.168.91.230/32 scope global eth0
    inet6 fe80::223:aeff:fe9e:e20/64 scope link 
       valid_lft forever preferred_lft forever

  VIP未被此机抢占,输出:

  

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet 192.168.91.230/32 brd 192.168.91.230 scope global lo:0
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:23:ae:ae:e4:e8 brd ff:ff:ff:ff:ff:ff
    inet 192.168.91.232/24 brd 192.168.91.255 scope global eth0
    inet6 fe80::223:aeff:feae:e4e8/64 scope link 
       valid_lft forever preferred_lft forever

 

  查看转发路由、转发规则命令:ipvsadm -ln

IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.91.230:8080 rr
  -> 192.168.91.231:8080          Local   1      0          0         
  -> 192.168.91.232:8080          Route   1      0          0