一、背景
通过keepalived实现nginx高可用,由于在家不想弄多台主机来搞,所以将运行环境用docker封装来模拟跨主机
docker基础镜像:centos
说之前,简单介绍一下:
Keepalived是基于vrrp协议的一款高可用软件。Keepailived有一台主服务器和多台备份服务器,在主服务器和备份服务器上面部署相同的服务配置,使用一个虚拟IP地址对外提供服务,当主服务器出现故障时,虚拟IP地址会自动漂移到备份服务器。
双机高可用方法目前分为两种:
- 双机主从(也叫双机热备)
- 双机主主 (也叫双机互备)
下述介绍,高可用中的双机主从模式,双机主主模式,主要是keepalived.conf配置会有所不同,方法都是一样。
二、具体操作
1、安装centos 镜像
docker pull centos
说明:通过用centos镜像来安装高可用所需要的所有环境,再启两个容器,再真实模拟跨主机的场景
2、在centos上安装所需环境(nginx和其它工具)
运行centos容器
docker run -it centos /bash/bin
安装依赖和所需要的包
#使用yum安装nginx需要包括Nginx的库,安装Nginx的库
rpm -Uvh http://nginx.org/packages/centos/7/noarch/RPMS/nginx-release-centos-7-0.el7.ngx.noarch.rpm
# 使用下面命令安装nginx
#yum install nginx
#安装网络包(需要使用ifconfig和ping命令)
yum install net-tools
#安装vim
yum install vim
3、在centos安装keepalvied
#安装keepalived环境依赖
yum install -y gcc openssl-devel popt-devel
#安装keepalived
通过yum install keepalived
#或者通过源码安装
wget http://124.205.69.132/files/90630000053A2BB4/www.keepalived.org/software/keepalived-1.3.4.tar.gz
tar zxvf keepalived-1.3.4.tar.gz
cd keepalived-1.3.4
./configure --prefix=/usr/local/keepalived
make && make install
拷贝几个文件到CentOS7环境中:
cp keepalived-1.3.4/keepalived/etc/init.d/keepalived /etc/init.d/
mkdir /etc/keepalived
cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
cp keepalived-1.3.4/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
4、修改/etc/keepalived/keepalived.conf文件
! Configuration File for keepalived global_defs { notification_email { 762357658@qq.com } notification_email_from itsection@example.com smtp_server mail.example.com smtp_connect_timeout 30 router_id LVS_DEVEL } vrrp_script chk_nginx { script "/etc/keepalived/nginx_check.sh" interval 2 weight -5 fall 3 rise 2 } vrrp_instance VI_1 { state MASTER interface eth0 virtual_router_id 2 priority 101 advert_int 2 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 172.17.0.210 } track_script { chk_nginx } }
/etc/keepalived/check_nginx.sh文件
A=`ps -ef | grep nginx | grep -v grep | wc -l` if [ $A -eq 0 ];then nginx sleep 2 if [ `ps -ef | grep nginx | grep -v grep | wc -l` -eq 0 ];then #killall keepalived ps -ef|grep keepalived|grep -v grep|awk '{print $2}'|xargs kill -9 fi fi
再对check_nginx.sh赋于执行权限:
chmod +x check_nginx.sh
注:keepalived是通过检测keepalived进程是否存在判断服务器是否宕机,如果keepalived进程在但是nginx进程不在了那么keepalived是不会做主备切换,所以我们需要写个脚本来监控nginx进程是否存在,如果nginx不存在就将keepalived进程杀掉。
在主nginx上需要编写nginx进程检测脚本(check_nginx.sh),判断nginx进程是否存在,如果nginx不存在就将keepalived进程杀掉,并将vip漂移到备份机器上
5、设置开机启动
chkconfig keepalived on
或者
systemctl enable keepalived.service 设置开机自动启动
启动keepalived服务:
systemctl start keepalived.service 启动
6、安装所有需要的依赖和环境后,将容器新增的内容重新提交
docker commit 5d112 centos_keepalived_nginx:v1
注:5d112为,上述安装软件所对应的容器id
6、启动含有(keepalived+nginx)的容器
docker run --privileged -tid --name keepalived_master centos_keepalived_nginx:v1 /usr/sbin/init
进入keepalived_master容器:
docker exec -it keepalived_master bash
进入/usr/share/nginx/html,修改index.html文件
修改标题为:
Welcome to nginx Master!
7、启动keepalived_salve容器
#启动一个容器
docker run --privileged -tid --name keepalived_slave centos_keepalived_nginx:v1 /usr/sbin/init
#进入容器
docker exec -it keepalived_slave bash
8、修改keepalived_salve容器中nginx index.html文件
vim /usr/share/nginx/html/index.html
修改标题为:
Welcome to nginx Slave!
9、修改keepalived_salve容器中keepalived.conf文件 (master容器中,保持和镜像中设置一样即可,不需要更改)
! Configuration File for keepalived global_defs { notification_email { 762357658@qq.com } notification_email_from itsection@example.com smtp_server mail.example.com smtp_connect_timeout 30 router_id LVS_DEVEL } vrrp_script chk_nginx { script "/etc/keepalived/nginx_check.sh" interval 2 weight -5 fall 3 rise 2 } vrrp_instance VI_1 { state BACKUP interface eth0 virtual_router_id 2 priority 100 advert_int 2 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 172.17.0.210 } track_script { chk_nginx } }
其实,由配置中可以看出,主要是state和priority两个参数的调整,其中master节点的priority值一定要比backup大才行!
原理说明:
1、 通过vrrp协议广播,每个keepalived vrrp都去争取master
2、 以virtual_router_id为组队标识。 同为一个vip服务的keepalived的virtual_router_id要保持相同
3、 以priority 为权值,同一个virtual_router_id下那个priority大那个就是master,其它为backup
改完之后,重新加载
systemctl daemon-reload
systemctl restart keepalived.service
10、验证
查看两个容器中keepalived服务状态
systemctl status keepalived.service
keepalived_master服务状态效果:
keepalived_slave服务状态效果图:
可以看到,keepalived服务运行正常
启动nginx: nginx
在master容器中 curl 172.17.0.210
在slave容器中 curl 172.17.0.210:
可以看现,此时master和slave容器两边通过虚拟vip : 172.17.0.210 访问nginx数据,请求返回的数据都是master容器中nginx配置的数据: welcome to nginx master
继续验证,关掉master容器的keepalived服务:
验证得到的结果是当master容器中的keepalived服务关掉后,curl 172.17.0.210请求返回的数据来自slave,welcome to nginx slave
再继续验证,把关掉master容器的keepalived服务再开启:
可以看到,当master容器中的keepalived服务开启后,请求返回的数据会再次转到master中。
到此,所有的验证和预期的一致,也达到我们借助docker为基础来实现了整套基于Nginx+Keepalived高可用的方案了。
三、Keepalived服务命令
- systemctl daemon-reload 重新加载
- systemctl enable keepalived.service 设置开机自动启动
- systemctl disable keepalived.service 取消开机自动启动
- systemctl start keepalived.service 启动
- systemctl stop keepalived.service停止
- systemctl status keepalived.service 查看服务状态