MySQL复制能够保证数据的冗余的同时可以做读写分离来分担系统压力，如果是主主复制还可以很好的避免主节点的单点故障。然而MySQL主主复制存在一些问题无法满足我们的实际需要：未提供统一访问入口来实现负载均衡，如果其中master宕掉的话需要手动切换到另外一个master，而不能自动进行切换。前面介绍了Mysql Keepalived双主热备高可用方案记录，那篇文档里没有使用到LVS（实现负载均衡），而下面要介绍的就是如何通过Keepalived LVS方式来是实现MySQL的高可用性，利用LVS实现MySQL的读写负载均衡，Keepalived避免节点出现单点故障，同时解决以上问题。
Keepalived是一个基于VRRP（虚拟路由冗余协议）可用来实现服务高可用性的软件方案，避免出现单点故障。Keepalived一般用来实现轻量级高可用性，且不需要共享存储，一般用于两个节点之间，常见有LVS Keepalived、Nginx Keepalived组合。
LVS(Linux Virtual Server)是一个高可用性虚拟的服务器集群系统。本项目在1998年5月由章文嵩博士成立，是中国国内最早出现的*软件项目之一。LVS主要用于多服务器的负载均衡，作用于网络层。LVS构建的服务器集群系统中，前端的负载均衡层被称为Director Server；后端提供服务的服务器组层被称为Real Server。通过下图可以大致了解LVS的基础架构。
LVS有三种工作模式，分别是：
1）DR（Direct Routing 直接路由）模式。 DR可以支持相当多的Real Server，但需要保证Director Server（分发器）虚拟网卡与物理网卡在同一网段，并且后端Real Server的vip要建立在本地回环口lo设备上，这样做是为了防止ip冲突；DR模式的好处是进站流量经过Director Server，出站流量不经过Director Server，减轻了Director Server的负载压力。
2）NAT（Network Address Translation 网络地址转换）模式。NAT扩展性有限，无法支持更多的Real Server，因为所有的请求包和应答包都需要Director Server进行解析再生，这样就很影响效率。
3）TUN(Tunneling IP隧道）模式。TUN模式能够支持更多的Real Server，但需要所有服务器支持IP隧道协议；
LVS负载均衡有10中调度算法，分别是：rr（轮询）、wrr（加权轮询）、lc、wlc、lblc、lblcr、dh、sh、sed、nq
以下详细记录了Mysql在主主同步环境下，利用LVS实现Mysql的读写负责均衡以及使用Keepalived心跳测试避免节点出现单点故障，实现故障转移的高可用。
1）高可用方案的环境准备
| 1 2 3 4 5 6 7 8 9 10 11 | LVS_Master： 182.148.15.237 LVS_Backup： 182.148.15.236 MySQL1 Real Server： 182.148.15.233 MySQL2 Real Server： 182.148.15.238 VIP： 182.148.15.239 OS: CentOS 6.8 温馨提示：LVS_Master和LVS_Backup充当的是Director Server分发器的角色。 这里的LVS采用DR模式，即 "进站流量经过Director Server分发器，出站流量不经过分发器" ，这就要求： 1）LVS_Master和LVS_Backup需要将VIP绑定在其正常提供服务的网卡上（这里指182.48.115.0网段所在的网卡），netmask和对于网卡设备一致。 2）后端的Real Server要在本地回环口lo上绑定vip（防止ip冲突） |
2）环境部署记录如下
a）Mysql主主热备环境部署
MySQL1 Real Server1和MySQL2 Real Server的主主热备可以参考Mysql Keepalived双主热备高可用操作记录中对应部分。
b）Keepalived安装
LVS_Master和LVS_Backup的keepalived安装，也可以参考Mysql Keepalived双主热备高可用操作记录中对应部分。
c）LVS安装
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | LVS_Master和LVS_Backup两台机器安装步骤一样 首先打开两台机器的ip_forward转发功能 [[email protected]_Master ~] # echo "1" > /proc/sys/net/ipv4/ip_forward 先下载ipvsadm [[email protected]_Master ~] # cd /usr/local/src/ [[email protected]_Master src] # wget http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.26.tar.gz 需要安装以下软件包 [[email protected]_Master src] # yum install -y libnl* popt* 查看是否加载lvs模块 [[email protected]_Master src] # modprobe -l |grep ipvs 解压安装 [[email protected]_Master src] # ln -s /usr/src/kernels/2.6.32-431.5.1.el6.x86_64/ /usr/src/linux [[email protected]_Master src] # tar -zxvf ipvsadm-1.26.tar.gz [[email protected]_Master src] # cd ipvsadm-1.26 [[email protected]_Master ipvsadm-1.26] # make && make install LVS安装完成，查看当前LVS集群 [[email protected]_Master ipvsadm-1.26] # ipvsadm -L -n IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn |
d）编写LVS启动脚本/etc/init.d/realserver
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 | 1）在MySQL1 Real Server服务器上的操作 [[email protected] ~] # vim /etc/init.d/realserver #!/bin/sh VIP=182.148.15.239 . /etc/rc .d /init .d /functions case "$1" in # 禁用本地的ARP请求、绑定本地回环地址 start) /sbin/ifconfig lo down /sbin/ifconfig lo up echo "1" > /proc/sys/net/ipv4/conf/lo/arp_ignore echo "2" > /proc/sys/net/ipv4/conf/lo/arp_announce echo "1" > /proc/sys/net/ipv4/conf/all/arp_ignore echo "2" > /proc/sys/net/ipv4/conf/all/arp_announce /sbin/sysctl -p > /dev/null 2>&1 /sbin/ifconfig lo:0 $VIP netmask 255.255.255.255 up #在回环地址上绑定VIP，设定掩码，与Direct Server（自身）上的IP保持通信 /sbin/route add -host $VIP dev lo:0 echo "LVS-DR real server starts successfully.n" ;; stop) /sbin/ifconfig lo:0 down /sbin/route del $VIP > /dev/null 2>&1 echo "1" > /proc/sys/net/ipv4/conf/lo/arp_ignore echo "2" > /proc/sys/net/ipv4/conf/lo/arp_announce echo "1" > /proc/sys/net/ipv4/conf/all/arp_ignore echo "2" > /proc/sys/net/ipv4/conf/all/arp_announce echo "LVS-DR real server stopped.n" ;; status) isLoOn=` /sbin/ifconfig lo:0 | grep "$VIP" ` isRoOn=` /bin/netstat -rn | grep "$VIP" ` if [ "$isLoON" == "" -a "$isRoOn" == "" ]; then echo "LVS-DR real server has run yet." else echo "LVS-DR real server is running." fi exit 3 ;; *) echo "Usage: $0 {start|stop|status}" exit 1 esac exit 0 将lvs脚本加入开机自启动 [[email protected] ~] # chmod x /etc/init.d/realserver [[email protected] ~] # echo "/etc/init.d/realserver" >> /etc/rc.d/rc.local 启动LVS脚本 [[email protected] ~] # service realserver start LVS-DR real server starts successfully.n 查看MySQL1 Real Server服务器，发现VIP已经成功绑定到本地回环口lo上了 [[email protected] ~] # ifconfig eth0 Link encap:Ethernet HWaddr 52:54:00:D1:27:75 inet addr:182.148.15.233 Bcast:182.148.15.255 Mask:255.255.255.224 inet6 addr: fe80::5054:ff:fed1:2775 /64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:598406 errors:0 dropped:0 overruns:0 frame:0 TX packets:12050 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:78790653 (75.1 MiB) TX bytes:33151764 (31.6 MiB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1 /128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:483 errors:0 dropped:0 overruns:0 frame:0 TX packets:483 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:55807 (54.4 KiB) TX bytes:55807 (54.4 KiB) lo:0 Link encap:Local Loopback inet addr:182.148.15.239 Mask:255.255.255.255 UP LOOPBACK RUNNING MTU:65536 Metric:1 2）在MySQL2 Real Server服务器上的操作 [[email protected] ~] # vim /etc/init.d/realserver //这个脚本在后端Real Server上都是一样的内容 #!/bin/sh VIP=182.148.15.239 . /etc/rc .d /init .d /functions case "$1" in start) /sbin/ifconfig lo down /sbin/ifconfig lo up echo "1" > /proc/sys/net/ipv4/conf/lo/arp_ignore echo "2" > /proc/sys/net/ipv4/conf/lo/arp_announce echo "1" > /proc/sys/net/ipv4/conf/all/arp_ignore echo "2" > /proc/sys/net/ipv4/conf/all/arp_announce /sbin/sysctl -p > /dev/null 2>&1 /sbin/ifconfig lo:0 $VIP netmask 255.255.255.255 up /sbin/route add -host $VIP dev lo:0 echo "LVS-DR real server starts successfully.n" ;; stop) /sbin/ifconfig lo:0 down /sbin/route del $VIP > /dev/null 2>&1 echo "1" > /proc/sys/net/ipv4/conf/lo/arp_ignore echo "2" > /proc/sys/net/ipv4/conf/lo/arp_announce echo "1" > /proc/sys/net/ipv4/conf/all/arp_ignore echo "2" > /proc/sys/net/ipv4/conf/all/arp_announce echo "LVS-DR real server stopped.n" ;; status) isLoOn=` /sbin/ifconfig lo:0 | grep "$VIP" ` isRoOn=` /bin/netstat -rn | grep "$VIP" ` if [ "$isLoON" == "" -a "$isRoOn" == "" ]; then echo "LVS-DR real server has run yet." else echo "LVS-DR real server is running." fi exit 3 ;; *) echo "Usage: $0 {start|stop|status}" exit 1 esac exit 0 将lvs脚本加入开机自启动并启动lvs脚本 [[email protected] ~] # chmod x /etc/init.d/realserver [[email protected] ~] # echo "/etc/init.d/realserver" >> /etc/rc.d/rc.local [[email protected] ~] # service realserver start LVS-DR real server starts successfully.n [[email protected] ~] # ifconfig eth0 Link encap:Ethernet HWaddr 52:54:00:3B:33:8F inet addr:182.148.15.238 Bcast:182.148.15.255 Mask:255.255.255.224 inet6 addr: fe80::5054:ff:fe3b:338f /64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:135305 errors:0 dropped:0 overruns:0 frame:0 TX packets:11256 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:17338566 (16.5 MiB) TX bytes:892363 (871.4 KiB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1 /128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) lo:0 Link encap:Local Loopback inet addr:182.148.15.239 Mask:255.255.255.255 UP LOOPBACK RUNNING MTU:65536 Metric:1 |
e）配置iptables防火墙
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | 1）后端两台机器MySQL1 Real Server和MySQL2 Real Server要在iptables里开放3306端口 [[email protected] ~] # vim /etc/sysconfig/iptables ...... -A INPUT -m state --state NEW -m tcp -p tcp --dport 3306 -j ACCEPT [[email protected] ~] # /etc/init.d/iptables restart 2）LVS_Master和LVS_Backup两台机器要在iptables开放VRRP组播地址的相关规则。 注意：这个一定要设置！！！否则会出现故障时的VIP资源漂移错乱问题！ [[email protected]_Master ~] # vim /etc/sysconfig/iptables //两台LVS机器都要设置 ....... -A INPUT -s 182.148.15.0 /24 -d 224.0.0.18 -j ACCEPT // 允许组播地址通信 -A INPUT -s 182.148.15.0 /24 -p vrrp -j ACCEPT // 允许VRRP（虚拟路由器冗余协）通信 -A INPUT -m state --state NEW -m tcp -p tcp --dport 3306 -j ACCEPT [[email protected]_Master ~] # /etc/init.d/iptables restart |
3）接着配置LVS Keepalived配置
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 | 1）LVS_Master上的操作 [[email protected]_Master ~] # vim /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { router_id LVS_Master } vrrp_instance VI_1 { state MASTER #指定instance初始状态，实际根据优先级决定.backup节点不一样 interface eth0 #虚拟IP所在网 virtual_router_id 51 #VRID，相同VRID为一个组，决定多播MAC地址 priority 100 #优先级，另一台改为90.backup节点不一样 advert_int 1 #检查间隔 authentication { auth_type PASS #认证方式，可以是pass或ha auth_pass 1111 #认证密码 } virtual_ipaddress { 182.148.15.239 #VIP } } virtual_server 182.148.15.239 3306 { delay_loop 6 #服务轮询的时间间隔 lb_algo wrr #加权轮询调度，LVS调度算法 rr|wrr|lc|wlc|lblc|sh|sh lb_kind DR #LVS集群模式 NAT|DR|TUN，其中DR模式要求负载均衡器网卡必须有一块与物理网卡在同一个网段 #nat_mask 255.255.255.0 persistence_timeout 50 #会话保持时间 protocol TCP #健康检查协议 ## Real Server设置，3306就是MySQL连接端口 real_server 182.148.15.233 3306 { weight 3 ##权重 TCP_CHECK { connect_timeout 3 nb_get_retry 3 delay_before_retry 3 connect_port 3306 } } real_server 182.148.15.238 3306 { weight 3 TCP_CHECK { connect_timeout 3 nb_get_retry 3 delay_before_retry 3 connect_port 3306 } } } 启动keepalived [[email protected]_Master ~] # /etc/init.d/keepalived start Starting keepalived: [ OK ] 注意此时网卡的变化，可以看到虚拟网卡已经分配到了realserver上。 此时查看LVS集群状态，可以看到集群下有两个Real Server，调度算法，权重等信息。ActiveConn代表当前Real Server的活跃连接数。 [[email protected]_Master ~] # ipvsadm -ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 182.148.15.239:3306 wrr persistent 50 -> 182.148.15.233:3306 Route 3 1 0 -> 182.148.15.238:3306 Route 3 0 0 2）LVS_Backup上的操作 [[email protected]_Backup ~] # vim /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { router_id LVS_Backup } vrrp_instance VI_1 { state BACKUP interface eth0 virtual_router_id 51 priority 90 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 182.148.15.239 } } virtual_server 182.148.15.239 3306 { delay_loop 6 lb_algo wrr lb_kind DR persistence_timeout 50 protocol TCP real_server 182.148.15.233 3306 { weight 3 TCP_CHECK { connect_timeout 3 nb_get_retry 3 delay_before_retry 3 connect_port 3306 } } real_server 182.148.15.238 3306 { weight 3 TCP_CHECK { connect_timeout 3 nb_get_retry 3 delay_before_retry 3 connect_port 3306 } } } 启动keepalived [[email protected]_Backup ~] # /etc/init.d/keepalived start Starting keepalived: [ OK ] [[email protected]_Backup ~] # ipvsadm -ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 182.148.15.239:3306 wrr persistent 50 -> 182.148.15.233:3306 Route 3 0 0 -> 182.148.15.238:3306 Route 3 0 0 |
此时LVS Keepalived MySQL主主复制已经搭建完成。
4）最后进行测试验证
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 | 1）先进行功能性验证 a）关闭MySQL2 Real Server服务器上的mysql [[email protected] ~] # /etc/init.d/mysql stop Shutting down MySQL.. SUCCESS! 在LVS_Master查看 /var/log/messages 中关于keepalived日志，LVS_Master检测到了MySQL2 Real Server服务器宕机，同时LVS集群自动剔除了故障节点 [[email protected]_Master ~] # tail -f /var/log/messages ....... Apr 26 15:22:19 test3-237 Keepalived_healthcheckers[4606]: TCP connection to [182.148.15.238]:3306 failed. Apr 26 15:22:19 test3-237 Keepalived_vrrp[4608]: Sending gratuitous ARP on eth0 for 182.148.15.239 ....... [[email protected]_Master ~] # ipvsadm -ln //LVS已经将MySQL2 Real Server剔除 IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 182.148.15.239:3306 wrr persistent 50 -> 182.148.15.233:3306 Route 3 1 0 从新启动MySQL2 Real Server后自动将故障节点自动加入LVS集群 [[email protected]_Master ~] # tail -f /var/log/messages ....... Apr 26 15:23:49 test3-237 Keepalived_healthcheckers[4606]: TCP connection to [182.148.15.238]:3306 success. Apr 26 15:23:49 test3-237 Keepalived_healthcheckers[4606]: Adding service [182.148.15.238]:3306 to VS [182.148.15.239]:3306 ....... [[email protected]_Master ~] # ipvsadm -ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 182.148.15.239:3306 wrr persistent 50 -> 182.148.15.233:3306 Route 3 1 0 -> 182.148.15.238:3306 Route 3 1 0 b）关闭LVS_Master上的Keepalived（模拟宕机操作），查看LVS_Master上的日志，可以看到Keepalived移出了LVS1上的VIP [[email protected]_Master ~] # /etc/init.d/keepalived stop Stopping keepalived: [ OK ] [[email protected]_Master ~] # tail -f /var/log/messages ........ Apr 26 15:29:38 test3-237 Keepalived[4976]: Stopping Apr 26 15:29:38 test3-237 Keepalived_vrrp[4979]: VRRP_Instance(VI_1) sent 0 priority Apr 26 15:29:38 test3-237 Keepalived_vrrp[4979]: VRRP_Instance(VI_1) removing protocol VIPs. Apr 26 15:29:38 test3-237 Keepalived_healthcheckers[4977]: Removing service [182.148.15.233]:3306 from VS [182.148.15.239]:3306 Apr 26 15:29:38 test3-237 Keepalived_healthcheckers[4977]: Removing service [182.148.15.238]:3306 from VS [182.148.15.239]:3306 Apr 26 15:29:38 test3-237 Keepalived_healthcheckers[4977]: Stopped Apr 26 15:29:38 test3-237 kernel: IPVS: __ip_vs_del_service: enter Apr 26 15:29:39 test3-237 Keepalived_vrrp[4979]: Stopped Apr 26 15:29:39 test3-237 Keepalived[4976]: Stopped Keepalived v1.3.5 (03 /19 ,2017), git commit v1.3.5-6-g6fa32f2 [[email protected]_Master ~] # ip addr //发现VIP资源已经不在本机了 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link /loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1 /8 scope host lo inet6 ::1 /128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link /ether 52:54:00:68: dc :b6 brd ff:ff:ff:ff:ff:ff inet 182.148.15.237 /27 brd 182.148.15.255 scope global eth0 inet 182.148.15.239 /27 brd 182.148.15.255 scope global secondary eth0:0 inet6 fe80::5054:ff:fe68:dcb6 /64 scope link valid_lft forever preferred_lft forever 同时查看LVS_Backup上日志，可以看到LVS_Backup成为了Master，并接管了VIP [[email protected]_Backup ~] # tail -f /var/log/messages ..... Apr 26 15:26:41 test4-236 Keepalived_vrrp[4711]: VRRP_Instance(VI_1) Transition to MASTER STATE Apr 26 15:26:42 test4-236 Keepalived_vrrp[4711]: VRRP_Instance(VI_1) Entering MASTER STATE Apr 26 15:26:42 test4-236 Keepalived_vrrp[4711]: VRRP_Instance(VI_1) setting protocol VIPs. Apr 26 15:26:42 test4-236 Keepalived_vrrp[4711]: Sending gratuitous ARP on eth0 for 182.148.15.239 Apr 26 15:26:42 test4-236 Keepalived_vrrp[4711]: VRRP_Instance(VI_1) Sending /queueing gratuitous ARPs on eth0 for 182.148.15.239 [[email protected]_Backup ~] # ip addr //发现VIP资源已经转移到LVS_Backup机器上了 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link /loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1 /8 scope host lo inet6 ::1 /128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link /ether 52:54:00:90:ac:0f brd ff:ff:ff:ff:ff:ff inet 182.148.15.236 /27 brd 182.148.15.255 scope global eth0 inet 182.148.15.239 /32 scope global eth0 inet6 fe80::5054:ff:fe90:ac0f /64 scope link valid_lft forever preferred_lft forever 在LVS_Backup上查看LVS集群状态，一切正常。 [[email protected]_Backup ~] # ipvsadm -ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 182.148.15.239:3306 wrr persistent 50 -> 182.148.15.233:3306 Route 3 0 0 -> 182.148.15.238:3306 Route 3 1 1 接着恢复LVS_Master上的Keepalived，发现VIP资源又会重新从LVS_Backup上转移回来。即LVS_Master重新接管服务。 [[email protected]_Master ~] # tail -f /var/log/messages ....... Apr 26 15:37:14 test3-237 Keepalived_vrrp[5263]: VRRP_Instance(VI_1) Transition to MASTER STATE Apr 26 15:37:15 test3-237 Keepalived_vrrp[5263]: VRRP_Instance(VI_1) Entering MASTER STATE Apr 26 15:37:15 test3-237 Keepalived_vrrp[5263]: VRRP_Instance(VI_1) setting protocol VIPs. Apr 26 15:37:15 test3-237 Keepalived_vrrp[5263]: Sending gratuitous ARP on eth0 for 182.148.15.239 Apr 26 15:37:15 test3-237 Keepalived_vrrp[5263]: VRRP_Instance(VI_1) Sending /queueing gratuitous ARPs on eth0 for 182.148.15.239 [[email protected]_Master ~] # ip addr 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link /loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1 /8 scope host lo inet6 ::1 /128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link /ether 52:54:00:68: dc :b6 brd ff:ff:ff:ff:ff:ff inet 182.148.15.237 /27 brd 182.148.15.255 scope global eth0 inet 182.148.15.239 /32 scope global eth0 inet 182.148.15.239 /27 brd 182.148.15.255 scope global secondary eth0:0 inet6 fe80::5054:ff:fe68:dcb6 /64 scope link valid_lft forever preferred_lft forever ----------------------------------------------------------------------------------------- 2）接着进行mysql主主热备的高可用 在MySQL1 Real Server和MySQL2 Real Server两台机器的mysql里授权，使远程客户机能正常连接。 mysql> grant all on *.* to test @ ‘%‘ identified by "123456" ; Query OK, 0 rows affected (0.03 sec) mysql> flush privileges; 经测试发现，对于上面的功能验证，不管是LVS_Master的keepalived关闭（宕机）还是后端MySQL2 Real Serve的mysql关闭，在远程客户机上都能正常连接mysql（通过VIP进行连接）（LVS_Master的keepalived关闭时，如若远程客户机在连接mysql中，可以断开重新连接一次即可） [[email protected] ~] # mysql -h182.148.15.239 -utest -p123456 Welcome to the MySQL monitor. Commands end with ; or g. Your MySQL connection id is 856 Server version: 5.6.34-log Source distribution Copyright (c) 2000, 2013, Oracle and /or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and /or its affiliates. Other names may be trademarks of their respective owners. Type ‘help;‘ or ‘h‘ for help. Type ‘c‘ to clear the current input statement. mysql> select * from huanqiu.haha; ---- ----------- | id | name | ---- ----------- | 1 | wangshibo | | 2 | guohuihui | | 22 | huihui | | 23 | bobo | ---- ----------- 4 rows in set (0.00 sec) |
总结几点：
1）MySQL主主复制是集群的基础，组成Server Array，其中每个节点作为Real Server。
2）LVS服务器提供了负载均衡的作用，将用户请求分发到Real Server，一台Real Server故障并不会影响整个集群的。
3）Keepalived搭建主备LVS服务器，避免了LVS服务器的单点故障，出现故障时可以自动切换到正常的节点。
4）VRRP虚拟IP地址和接口实际IP必须在同一个网段内，VRRP两个实际接口不在同一网段，是不能形成主备关系的。因为：
| 1 2 | VRRP的形式是组播（多播），这个和广播基本一个原理，只不过缩小了范围，让几个服务器能收到，众所周知广播必须在2层里面实现，出3层就出了广播域了。 VRRP中的ip切换是靠arp欺骗实现的，否则vip从主切到备得断多少机器，arp欺骗也是没法在出了3层以外的2个机器直接做的。 |