最近公司用做负责均衡,研究了一下lvs,做为我的第一篇博客吧。
一. LVS+Keepalived 介绍
1. LVS
LVS是Linux Virtual Server的简写,意即Linux虚拟服务器,是一个虚拟的服务器集群系统。本项目在1998年5月由章文嵩博士成立,是中国国内最早出现的*软件项目之一。目前有三种IP负载均衡技术(VS/NAT、VS/TUN和VS/DR);
八种调度算法(rr,wrr,lc,wlc,lblc,lblcr,dh,sh)。
2. Keepalvied
Keepalived在这里主要用作RealServer的健康状态检查以及LoadBalance主机和BackUP主机之间failover的实现
二. 网站负载均衡拓朴图
三. 安装LVS和Keepalvied软件包
#lsmod | rep ip_vs
#uname -r
显示2.6.18-53.el5PAE
#ln -s /usr/src/kernels/2.6.18-53.el5PAE-i686/ /usr/src/linux
#tar zxvf ipvsadm-1.24.tar.gz
#cd ipvsadm-1.24
#make all && make install(安装相关包:gcc*)
#find / -name ipvsadm # 查看ipvsadm的位置
#tar zxvf keepalived-1.1.15.tar.gz
#cd keepalived-1.1.15
#./configure (提示安装openssl*)
# make && make install
#find / -name keepalived # 查看keepalived位置
#cp /usr/local/etc/rc.d/init.d/keepalived /etc/rc.d/init.d/
#cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/
#mkdir /etc/keepalived
#cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/
#cp /usr/local/sbin/keepalived /usr/sbin/
#chkconfig —add keepalived
#chkconfig keepalived on
四. 配置LVS
在LVS机上配置(两台都要配置)
#vim /usr/local/sbin/lvsdr.sh
#!/bin/bash
# description: start LVS of DirectorServer
#Written by :NetSeek
GW=192.168.5.2
# website director vip.
WEB_VIP=192.168.6.200
WEB_RIP1=192.168.6.152
WEB_RIP2=192.168.6.159
. /etc/rc.d/init.d/functions
logger $0 called with $1
case "$1" in
start)
# Clear all iptables rules.
#/sbin/iptables -F
# Reset iptables counters.
#/sbin/iptables -Z
# Clear all ipvsadm rules/services.
/sbin/ipvsadm -C
#set lvs vip for dr
/sbin/ipvsadm --set 30 5 60
#/sbin/ifconfig eth0:0 $WEB_VIP broadcast $WEB_VIP netmask 255.255.255.255 up
/sbin/route add -host $WEB_VIP dev eth0
/sbin/ipvsadm -A -t $WEB_VIP:1758 -s wrr -p 3
/sbin/ipvsadm -a -t $WEB_VIP:1758 -r $WEB_RIP1:1758 -g -w 1
/sbin/ipvsadm -a -t $WEB_VIP:1758 -r $WEB_RIP2:1758 -g -w 1
touch /var/lock/subsys/ipvsadm >/dev/null 2>&1
# set Arp
/sbin/arping -I eth0 -c 5 -s $WEB_VIP $GW >/dev/null 2>&1 #备份节点上把这行注释掉
;;
stop)
/sbin/ipvsadm -C
/sbin/ipvsadm -Z
#ifconfig eth0:0 down
route del $WEB_VIP >/dev/null 2>&1
rm -rf /var/lock/subsys/ipvsadm >/dev/null 2>&1
/sbin/arping -I eth0 -c 5 -s $WEB_VIP $GW #备份节点上把这行注释掉
echo "ipvsadm stoped"
;;
status)
if [ ! -e /var/lock/subsys/ipvsadm ];then
echo "ipvsadm is stoped"
exit 1
else
ipvsadm -ln
echo "..........ipvsadm is OK."
fi
;;
*)
echo "Usage: $0 {start|stop|status}"
exit 1
esac
exit 0
五. 利用Keepalvied实现负载均衡和和高可用性
1.主LVS上:修改/etc/keepalived/keepalived.conf
global_defs {
router_id LVS_DEVEL
}
vrrp_sync_group lvs_1 {
group {
VI_1
}
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 50
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.6.200
}
}
virtual_server 192.168.6.200 1758 {
delay_loop 6
lb_algo rr
lb_kind DR
persistence_timeout 50
protocol TCP
real_server 192.168.6.152 1758 {
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 1758
}
}
real_server 192.168.6.159 1758 {
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 1758
}
}
}
2.从LVS:
global_defs {
router_id LVS_DEVEL
}
vrrp_sync_group lvs_1 {
group {
VI_1
}
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 50
priority 80
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.6.200
}
virtual_server 192.168.6.200 1758 {
delay_loop 6
lb_algo rr
lb_kind DR
persistence_timeout 50
protocol TCP
real_server 192.168.6.152 1758 {
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 1758
}
}
real_server 192.168.6.159 1758 {
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 1758
}
}
}
3.开启IP转发,可以不用,如果是nat方式就要配置了,用sysctl命令配置
net.ipv4.ip_forward=1
六、配置xcloud(我们项目的名字)端(192.168.6.152和192.168.6.159)
1. 配置IP转发功能
#vim /etc/rc.d/init.d/realserver.sh
#!/bin/bash
# description: Config realserver lo and apply noarp
SNS_VIP=192.168.6.200
/etc/rc.d/init.d/functions
case "$1" in
start)
ifconfig lo:0 $SNS_VIP netmask 255.255.255.255 broadcast $SNS_VIP
/sbin/route add -host $SNS_VIP dev lo:0
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
sysctl -p >/dev/null 2>&1
echo "RealServer Start OK"
;;
stop)
ifconfig lo:0 down
route del $SNS_VIP >/dev/null 2>&1
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
echo "RealServer Stoped"
;;
*)
echo "Usage: $0 {start|stop}"
exit 1
esac
exit 0
七、启动服务
#/usr/local/sbin/lvsdr.sh start (把这条语句写到/etc/rc.local中,开机启动)
#/etc/init.d/keepalivedstart 启动keepalived 服务,keepalived就能利用keepalived.conf 配置文件,实现负载均衡和高可用.
真是服务器上:运行 /etc/rc.d/init.d/realserver.shstart 也可以加入/etc/rc.local里面
八、测试
在主lvs上:注意红色的ip
1. # ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0:<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether e4:1f:13:ec:d2:aa brd ff:ff:ff:ff:ff:ff
inet 192.168.6.175/16 brd 192.168.255.255 scope global eth0
inet 192.168.6.200/32 scope global eth0
inet6 fe80::e61f:13ff:feec:d2aa/64 scope link
valid_lft forever preferred_lft forever
3: eth1:<NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen1000
link/ethere4:1f:13:ec:d2:ab brd ff:ff:ff:ff:ff:ff
………
最后把主机宕机(通过在主机上用/etc/init.d/keepalived stop也可以),到从机上使用ip addr show 查看,同上则OK。
2.用客户端连接一下xcloudd(192.168.6.200),如果连接成功,通过下面命令可以看到,真实连接到了哪台服务器。
#watch ipvsadm –ln
IP Virtual Server version 1.2.1 (size=4096)
ProtLocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.6.200:1758 wrr persistent 3
-> 192.168.6.152:1758 Route 1 0 0
-> 192.168.6.159:1758 Route 1 1 0
#tail –f /var/log/message 监听日志,查看状态。
3.将一台xcloudd服务器关闭,然后在LVS 上用ipvsadm 命令查看,关闭的服务器应该从
lvs集群中剔除了,再将关闭的服务器启动起来,用ipvsadm查看,又回来了。