Lvs+keepalived 实现高可用负载均衡文档

时间:2021-11-24 03:04:06

一、LVS简介

LVS是Linux Virtual Server的简写,意即Linux虚拟服务器,是一个虚拟的服务器集群系统。本项目在1998年5月由章文嵩博士成立,是中国国内最早出现的*软件项目之一。目前有三种IP负载均衡技术(VS/NAT、VS/TUN和VS/DR);

十种调度算法(rrr|wrr|lc|wlc|lblc|lblcr|dh|sh|sed|nq)。

二、Keepalvied简介

Keepalived在这里主要用作RealServer的健康状态检查以及LoadBalance主机和BackUP主机之间failover的实现。

三、负载均衡网络拓扑图

四、安装ipvsadm和keepalived

1.安装ipvsadm

下载相关软件包

#wget http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.24.tar.gz

       注意:安装两个软件包前,请首先将gcc*和openssl*相关包安装,不然装的过程中会报错。

 安装ipvsadm:

#ln -s /usr/src/kernels/2.6.18-53.el5PAE-i686(根据自己系统实际情况)/  /usr/src/linux(建立一个软链接)

#tar zxvf ipvsadm-1.24.tar.gz

#cd ipvsadm-1.24

#make all && make install

注意:如果中间过程中无报错,则证明安装成功。

#find / -name ipvsadm  # 查看ipvsadm的位置

2.安装keepalived:

#wget http://www.keepalived.org/software/keepalived-1.1.15.tar.gz

#tar zxvf keepalived-1.1.15.tar.gz

#cd keepalived-1.1.15

#./configure

# make && make install

#find / -name keepalived  # 查看keepalived位置            

#cp /usr/local/etc/rc.d/init.d/keepalived/etc/rc.d/init.d/

#cp /usr/local/etc/sysconfig/keepalived/etc/sysconfig/

#mkdir /etc/keepalived

#cp/usr/local/etc/keepalived/keepalived.conf /etc/keepalived/

#cp /usr/local/sbin/keepalived /usr/sbin/

#chkconfig —add keepalived

#chkconfig keepalived on 设置为开机自启动

五、keepalived相关配置及说明

1.负载均衡服务器配置说明:

在两台负载服务器上修改/etc/sysctl.conf  修改完毕后用 sysctl -p 生效

net.ipv4.ip_forward = 0

net.ipv4.conf.all.send_redirects = 1

net.ipv4.conf.default.send_redirects = 1

net.ipv4.conf.eth0.send_redirects = 1

 

2.配置keepalived配置文件:

Vi /e       tc/keepalived/keepalived.conf

 

global_defs {

  router_id LVS_DEVEL

}

vrrp_sync_group lvs_1 {

          group {

                 VI_1

                   }

}

vrrp_instance VI_1 {

state BACKUP(设置lvs主机角色,分为两种masterback为争抢模式,简单说就是只要master主机keepalived存在就会争抢过来,反之backupbackup不存在争抢,只有当一台keepalived死掉或者restart时才会漂到另一台)

   interface eth2 (配置检测网卡)

   virtual_router_id 51

   priority 90(权重配置,如果state同时设定为backup时,则权重值需一样,如果一台为master时,则master主机权重值要比backup高)

   advert_int 1

   nopreempt

   authentication {

       auth_type PASS

       auth_pass 1111

     }

    virtual_ipaddress {(配置vip,可配置多个)

172.29.154.3  bond0

             172.29.154.14  bond0 配置vip,可配置多个

     }

}

virtual_server 172.29.154.3 80 {在此请注意,负载的端口必须和tomcatnginx等发布的端口一致,不能vip80端口,到realserver配置8080或者其他端口。

    delay_loop 6

    lb_algo rr(调度算法)

     lb_kind DR(负载均衡模式)

       protocol TCP

       real_server 172.29.154.4 80 {(配置realserver ip和端口)

              weight 3

                TCP_CHECK {

                   connect_timeout 5

                   nb_get_retry 3

                   delay_before_retry 3

                   connect_port 80

              }

         }为一个完整的realserver配置,可添加多个realserver

        real_server 172.29.154.5 80 {

             weight 3

                TCP_CHECK {

                   connect_timeout 5

                   nb_get_retry 3

                   delay_before_retry 3

                   connect_port 80

             }

     }

}

virtual_server 172.29.154.14 80 {

    delay_loop 6

    lb_algo rr

    lb_kind DR

   persistence_timeout 10

   protocol TCP

      real_server 172.29.154.6 80 {

             weight 3

                TCP_CHECK {

                   connect_timeout 3

                   nb_get_retry 3

                   delay_before_retry 3

             }

        }

        real_server 172.29.154.7 80 {

             weight 3

                TCP_CHECK {

                   connect_timeout 3

                   nb_get_retry 3

                   delay_before_retry 3

             }

     }

}

virtual_server 172.29.154.14 8080 {

    delay_loop 6

    lb_algo rr

    lb_kind DR

   persistence_timeout 10

   protocol TCP

      real_server 172.29.154.6 8080 {

             weight 3

                TCP_CHECK {

                   connect_timeout 3

                   nb_get_retry 3

                   delay_before_retry 3

              }

        }

        real_server 172.29.154.7 8080 {

             weight 3

                TCP_CHECK {

                   connect_timeout 3

                   nb_get_retry 3

                   delay_before_retry 3

             }

     }

}

virtual_server 172.29.154.3 8443 {

    delay_loop 6

    lb_algo rr

    lb_kind DR

#    nat_mask 255.255.255.192

   persistence_timeout 10

   protocol TCP

      real_server 172.29.154.4 8443 {

             weight 3

                TCP_CHECK {

                  connect_timeout 3

                   nb_get_retry 3

                   delay_before_retry 3

             }

        }

        real_server 172.29.154.5 8443 {

             weight 3

                TCP_CHECK {

                   connect_timeout 3

                   nb_get_retry 3

                   delay_before_retry 3

             }

     }

}

如果多个vip配置相同,注意{}的配对一定要正确。另一台负载均衡机子配置同第一台相同。

3.vip挂载和漂移测试

两台操作完毕后,启动keepalived:

Service keepalived start(启动) restart(重启)stop(停止)

启动后使用ip add名称看vip是否存在:

 

使用ipvsadm –ln命令可查看负载情况:

 

4.realserver配置:

 

在每台realserver的/etc/sysctl.conf 添加:

net.ipv4.ip_forward=0

net.ipv4.conf.lo.arp_ignore = 1

net.ipv4.conf.lo.arp_announce = 2

net.ipv4.conf.all.arp_ignore = 1

net.ipv4.conf.all.arp_announce = 2

使用 sysctl -p 生效

 

在/etc/inid.d/下新建一个文件,例如名称为lvs,编辑该文件,在文件中添加:

#!/bin/bash

# Written by NetSeek

# description: Config realserver lo andapply noarp

WEB_VIP=172.29.154.3(修改成自己的vip

 

. /etc/init.d/functions

 

case "$1" in

start)

      ifconfig lo:0 $WEB_VIP netmask 255.255.255.255 broadcast $WEB_VIP

      /sbin/route add -host $WEB_VIP dev lo:0

      echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore

      echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce

      echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore

      echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce

      sysctl -p >/dev/null 2>&1

      echo "RealServer Start OK"

 

      ;;

stop)

      ifconfig lo:0 down

      route del $WEB_VIP >/dev/null 2>&1

      echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore

      echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce

      echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore

      echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce

      echo "RealServer Stoped"

      ;;

status)

       # Status of LVS-DR real server.

       islothere=`/sbin/ifconfig lo:0 | grep $WEB_VIP`

       isrothere=`netstat -rn | grep "lo:0" | grep $web_VIP`

       if [ ! "$islothere" -o ! "isrothere" ];then

           # Either the route or the lo:0 device

           # not found.

           echo "LVS-DR real server Stopped."

       else

           echo "LVS-DR Running."

       fi

;;

*)

       # Invalid entry.

       echo "$0: Usage: $0 {start|status|stop}"

       exit 1

;;

esac

exit 0

 

脚本添加完毕后,使用chmod+x lvs 赋予执行权限。

#chkconfig —add lvs

#chkconfig lvs on 设置为开机自启动

 

使用/etc/init.d/lvsstart 启动脚本。

使用ifconfig查看vip是否存在:

通过ping命令查看是否可以ping同vip。

六、整体测试

Readlserver配置完毕后,可通过搭建测试页面进行测试,通过vip访问要测试的url,可用ipvsadm -ln在负载均衡服务器上看到每台服务器的分发情况。