nginx + keepalived 双机热备

时间:2022-12-19 09:42:46

双机热备是指两台机器都在运行,但并非两台机器同时在提供服务。
当提供服务的一台出现故障的时候,另外一台会马上自动接管并且提供服务,且切换的时间非常短。

keepalived的工作原理是VRRP——虚拟路由冗余协议。

测试环境如下:

nginx + keepalived 双机热备

 

ip

vip

master

192.168.174.135

192.168.174.140

backup

192.168.174.137

192.168.174.140

nginx

安装

sudo apt-get install nginx 

查找配置文件位置

sudo find / -name nginx.conf
/etc/nginx/nginx.conf

修改配置文件(nginx.conf)

user www-data;
worker_processes 4;
pid /run/nginx.pid;

events {
    worker_connections 1024;
}

http {
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;
    types_hash_max_size 2048;

    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;

    server {
        listen 80 default_server;
        server_name test;
        charset utf-8;

        location / {
        root html;
        index index.html index.htm;
        proxy_set_header X-Real_IP $remote_addr;
        client_max_body_size 100m;
        }
    }
}

文件/usr/share/nginx/html/index.html

在192.168.174.135上加上  <h1>Welcome to nginx!  135  </h1>

在192.168.174.137上加上  <h1>Welcome to nginx!   ***137***   </h1>

启动

sudo service nginx start  

关闭

sudo service nginx stop

keepalived

安装

下载keepalived-1.2.19.tar.gz

tar –zxvf keepalived-1.2.19.tar.gz
cd keepalived-1.2.19
./configure --prefix=/usr/local/keepalived
make
sudo make install

期间可能出现问题:

!!! OpenSSL is not properly installed on your system. !!!
!!! Can not include OpenSSL headers files. !!!

解决

sudo apt-get install libssl.dev

建立软链接

sudo ln -s /usr/local/keepalived/sbin/keepalived /sbin/
sudo ln -s /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
sudo ln -s /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/

启动

sudo keepalived -D -f /usr/local/keepalived/etc/keepalived/keepalived.conf

关闭

sudo killall keepalived

配置(keepalived.conf):

global_defs {
    router_id NODEA
}

vrrp_instance VI_1 {
    state MASTER 
    interface eth0        #监测网络接口 
    virtual_router_id 50  #主、备必须一样  
    priority 100          #优先级:主>备
    advert_int 1
    authentication {
        auth_type PASS #VRRP认证,主备一致
        auth_pass 1111  #密码
}

virtual_ipaddress {
        192.168.174.140/24 #VRRP HA虚拟地址
    }
}

备用节点的配置

global_defs {
   router_id NODEB
}

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 50
    priority 90 
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }

 virtual_ipaddress {
    192.168.174.140/24
    }
}

测试

双击热备

两台机子均启动nginx和keepalived,浏览器各自访问

nginx + keepalived 双机热备

nginx + keepalived 双机热备

浏览器访问:http://192.168.174.140/,显示的是MASTER的页面。

nginx + keepalived 双机热备

同样用ip appr可以验证:

135机器:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

    inet 127.0.0.1/8 scope host lo

       valid_lft forever preferred_lft forever

    inet6 ::1/128 scope host

       valid_lft forever preferred_lft forever

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000

    link/ether 00:0c:29:39:d4:88 brd ff:ff:ff:ff:ff:ff

    inet 192.168.174.135/24 brd 192.168.174.255 scope global eth0

       valid_lft forever preferred_lft forever

    inet 192.168.174.140/24 scope global secondary eth0

       valid_lft forever preferred_lft forever

    inet6 fe80::20c:29ff:fe39:d488/64 scope link

       valid_lft forever preferred_lft forever

137机器:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

    inet 127.0.0.1/8 scope host lo

       valid_lft forever preferred_lft forever

    inet6 ::1/128 scope host

       valid_lft forever preferred_lft forever

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 1000

    link/ether 00:0c:29:cf:23:62 brd ff:ff:ff:ff:ff:ff

    inet 192.168.174.137/24 brd 192.168.174.255 scope global eth0

       valid_lft forever preferred_lft forever

    inet6 fe80::20c:29ff:fecf:2362/64 scope link

       valid_lft forever preferred_lft forever

 

现在关闭135机器的keepalived。

nginx + keepalived 双机热备

但当nginx宕掉或整个机子宕机后,这种情况不行了——通过浏览器访问192.168.174.140访问不到资源。

nginx宕掉/机器宕掉热备

为了解决上一问题,可以利用脚本,当检测到nginx进程宕掉后,自动关闭keepalived进程,从而实现热备份。

主节点的配置

global_defs {
    router_id NODEA
}

vrrp_script chk_http_port {
    script "/home/jimite/keepalived/chk_nginx_pid.sh"
    interval 2
    weight 2
}

vrrp_instance VI_1 {
    state MASTER 
    interface eth0
    virtual_router_id 50
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    track_script {
        chk_http_port
    }
    virtual_ipaddress {
        192.168.174.140/24
    }
}

备用节点的配置

global_defs {
   router_id NODEB
}

vrrp_script chk_http_port {
    script "/home/jihite/keepalived/chk_nginx_pid.sh"
    interval 2
    weight 2
}

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 50
    priority 90 
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
   track_script {
       chk_http_port
   }

    virtual_ipaddress {
    192.168.174.140/24
    }
}

其中/home/jimite/keepalived/chk_nginx_pid.sh为

#!/bin/bash
A=`ps -C nginx --no-header |wc -l`
if [ $A -eq 0 ]
then
    echo 'nginx server is died'
    sudo killall keepalived
fi

问题:杀死keepalived进程后,可以实现vip的偏移,但是原机器的vip无法自动删除

原因:VRRP协议原理是:只有MASTER对外发送消息。各BACKUP接受消息,当接受不到消息时会在剩下的BACKUP机器中选出新的MASTER。
之前用kill -9 pid 或killall pid杀死keepalived进程,导致安装keepalived不能发送信息,BACKUP收不到信息升级为MASTER,但是由于进程被杀死【非正常关闭】,导致keepalived 没有能力自己删除 vip。
解决方案:关闭keepalived时用命令
       service keepalived stop  或   kill -15 pid(注:只删除第一个进程号)
存在问题:
       非正常关闭keepalived。 禁止使用kill -9  或killall杀死keepalived。