利用LVS+Keepalived DR 实现高性能高可用负载均衡

最近公司用做负责均衡，研究了一下lvs，做为我的第一篇博客吧。

一． LVS+Keepalived 介绍

1. LVS

LVS是Linux Virtual Server的简写，意即Linux虚拟服务器，是一个虚拟的服务器集群系统。本项目在1998年5月由章文嵩博士成立，是中国国内最早出现的*软件项目之一。目前有三种IP负载均衡技术（VS/NAT、VS/TUN和VS/DR）；

八种调度算法（rr,wrr,lc,wlc,lblc,lblcr,dh,sh）。

2. Keepalvied

Keepalived在这里主要用作RealServer的健康状态检查以及LoadBalance主机和BackUP主机之间failover的实现

二. 网站负载均衡拓朴图

三. 安装LVS和Keepalvied软件包

#lsmod | rep ip_vs
#uname -r
显示2.6.18-53.el5PAE
#ln -s /usr/src/kernels/2.6.18-53.el5PAE-i686/  /usr/src/linux

#tar zxvf ipvsadm-1.24.tar.gz
#cd ipvsadm-1.24
#make all && make install(安装相关包：gcc*)
#find / -name ipvsadm  # 查看ipvsadm的位置

#tar zxvf keepalived-1.1.15.tar.gz
#cd keepalived-1.1.15
#./configure (提示安装openssl*)
# make && make install
#find / -name keepalived  # 查看keepalived位置

#cp /usr/local/etc/rc.d/init.d/keepalived /etc/rc.d/init.d/
#cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/
#mkdir /etc/keepalived
#cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/
#cp /usr/local/sbin/keepalived /usr/sbin/
#chkconfig —add keepalived
#chkconfig keepalived on

四. 配置LVS

在LVS机上配置（两台都要配置）

＃vim /usr/local/sbin/lvsdr.sh
#!/bin/bash
# description: start LVS of DirectorServer
#Written by :NetSeek 
GW=192.168.5.2

# website director vip.
WEB_VIP=192.168.6.200
WEB_RIP1=192.168.6.152
WEB_RIP2=192.168.6.159
. /etc/rc.d/init.d/functions

logger $0 called with $1

case "$1" in
start)
        # Clear all iptables rules.
        #/sbin/iptables -F
        # Reset iptables counters.
        #/sbin/iptables -Z
        # Clear all ipvsadm rules/services.
        /sbin/ipvsadm -C
#set lvs vip for dr
        /sbin/ipvsadm --set 30 5 60
        #/sbin/ifconfig eth0:0 $WEB_VIP broadcast $WEB_VIP netmask 255.255.255.255 up
        /sbin/route add -host $WEB_VIP dev eth0
        /sbin/ipvsadm -A -t $WEB_VIP:1758 -s wrr -p 3
        /sbin/ipvsadm -a -t $WEB_VIP:1758 -r $WEB_RIP1:1758 -g -w 1
        /sbin/ipvsadm -a -t $WEB_VIP:1758 -r $WEB_RIP2:1758 -g -w 1
        touch /var/lock/subsys/ipvsadm >/dev/null 2>&1
        # set Arp
        /sbin/arping -I eth0 -c 5 -s $WEB_VIP $GW >/dev/null 2>&1 #备份节点上把这行注释掉
       ;;
stop)
        /sbin/ipvsadm -C
        /sbin/ipvsadm -Z
        #ifconfig eth0:0 down
        route del $WEB_VIP  >/dev/null 2>&1
        rm -rf /var/lock/subsys/ipvsadm >/dev/null 2>&1
        /sbin/arping -I eth0 -c 5 -s $WEB_VIP $GW #备份节点上把这行注释掉
        echo "ipvsadm stoped"
       ;;
status)
        if [ ! -e /var/lock/subsys/ipvsadm ];then
                echo "ipvsadm is stoped"
                exit 1
        else
                ipvsadm -ln
                echo "..........ipvsadm is OK."
        fi
      ;;
*)
        echo "Usage: $0 {start|stop|status}"
        exit 1
esac
exit 0

五. 利用Keepalvied实现负载均衡和和高可用性

1．主LVS上：修改/etc/keepalived/keepalived.conf

global_defs {
   router_id LVS_DEVEL
}
vrrp_sync_group lvs_1 {
           group {
                 VI_1
                   }
}
vrrp_instance VI_1 {
    state MASTER 
    interface eth0
    virtual_router_id 50
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.6.200         
    }
}

virtual_server 192.168.6.200 1758 {
     delay_loop 6
     lb_algo rr
     lb_kind DR
    persistence_timeout 50
    protocol TCP
       real_server 192.168.6.152 1758 {
              weight 1
                TCP_CHECK {
                   connect_timeout 3
                   nb_get_retry 3
                   delay_before_retry 3
                   connect_port 1758
              }
         }
      real_server 192.168.6.159 1758 {
              weight 1
                TCP_CHECK {
                   connect_timeout 3
                   nb_get_retry 3
                   delay_before_retry 3
                   connect_port 1758
              }
     }
}

2．从LVS：

global_defs {
   router_id LVS_DEVEL
}
vrrp_sync_group lvs_1 {
           group {
                 VI_1
                   }
}
vrrp_instance VI_1 {
    state BACKUP 
    interface eth0
    virtual_router_id 50
    priority 80
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.6.200  
}

virtual_server 192.168.6.200 1758 {
     delay_loop 6
     lb_algo rr
     lb_kind DR
    persistence_timeout 50
    protocol TCP
       real_server 192.168.6.152 1758 {
              weight 1
                TCP_CHECK {
                   connect_timeout 3
                   nb_get_retry 3
                   delay_before_retry 3
                   connect_port 1758
              }
         }
      real_server 192.168.6.159 1758 {
              weight 1
                TCP_CHECK {
                   connect_timeout 3
                   nb_get_retry 3
                   delay_before_retry 3
                   connect_port 1758
              }
     }
}

3.开启IP转发，可以不用,如果是nat方式就要配置了，用sysctl命令配置

net.ipv4.ip_forward=1

六、配置xcloud（我们项目的名字）端（192.168.6.152和192.168.6.159）

1. 配置IP转发功能

＃vim /etc/rc.d/init.d/realserver.sh

#!/bin/bash
# description: Config realserver lo and apply noarp

SNS_VIP=192.168.6.200
/etc/rc.d/init.d/functions

case "$1" in
start)
       ifconfig lo:0 $SNS_VIP netmask 255.255.255.255 broadcast $SNS_VIP
       /sbin/route add -host $SNS_VIP dev lo:0
       echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
       echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
       echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
       echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
       sysctl -p >/dev/null 2>&1
       echo "RealServer Start OK"
       ;;
stop)
       ifconfig lo:0 down
       route del $SNS_VIP >/dev/null 2>&1
       echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
       echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
       echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
       echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
       echo "RealServer Stoped"
       ;;
*)
       echo "Usage: $0 {start|stop}"
       exit 1
esac
exit 0

七、启动服务

＃/usr/local/sbin/lvsdr.sh start （把这条语句写到/etc/rc.local中，开机启动）

＃/etc/init.d/keepalivedstart 启动keepalived 服务，keepalived就能利用keepalived.conf 配置文件，实现负载均衡和高可用.

真是服务器上：运行 /etc/rc.d/init.d/realserver.shstart 也可以加入/etc/rc.local里面

八、测试

在主lvs上：注意红色的ip

1. ＃ ip addr show

1: lo: <LOOPBACK,UP,LOWER_UP> mtu16436 qdisc noqueue state UNKNOWN

link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

inet 127.0.0.1/8 scope host lo

inet6 ::1/128 scope host

valid_lft forever preferred_lft forever

2: eth0:<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000

link/ether e4:1f:13:ec:d2:aa brd ff:ff:ff:ff:ff:ff

inet 192.168.6.175/16 brd 192.168.255.255 scope global eth0

inet 192.168.6.200/32 scope global eth0

inet6 fe80::e61f:13ff:feec:d2aa/64 scope link

valid_lft forever preferred_lft forever

3: eth1:<NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen1000

link/ethere4:1f:13:ec:d2:ab brd ff:ff:ff:ff:ff:ff

………

最后把主机宕机(通过在主机上用/etc/init.d/keepalived stop也可以)，到从机上使用ip addr show 查看，同上则OK。

2.用客户端连接一下xcloudd（192.168.6.200），如果连接成功，通过下面命令可以看到，真实连接到了哪台服务器。

#watch ipvsadm –ln

IP Virtual Server version 1.2.1 (size=4096)

ProtLocalAddress:Port Scheduler Flags

-> RemoteAddress:Port Forward Weight ActiveConn InActConn

TCP 192.168.6.200:1758 wrr persistent 3

-> 192.168.6.152:1758 Route 1 0 0

-> 192.168.6.159:1758 Route 1 1 0

#tail –f /var/log/message 监听日志，查看状态。

3．将一台xcloudd服务器关闭，然后在LVS 上用ipvsadm 命令查看，关闭的服务器应该从

lvs集群中剔除了，再将关闭的服务器启动起来，用ipvsadm查看，又回来了。

秒客网

利用LVS+Keepalived DR 实现高性能高可用负载均衡

相关文章