Docker Swarm集群部署

时间:2024-10-04 06:59:36

Docker Swarm集群部署

  • 1 方案介绍
    • 1.1 概述
    • 1.2 软件包
  • 2 Swarm集群搭建
    • 2.1 IP规划
    • 2.2 基础配置
      • 2.2.1 关闭SELinux
      • 2.2.2 关闭防火墙或开放需要的业务端口
      • 2.2.3 调整max_open_files
      • 2.2.4 重启检查SELinux、firewalld和open files
    • 2.3 配置主机名解析
    • 2.4 安装Docker
    • 2.5 启动并设置开机自启
    • 2.6 创建集群
      • 2.6.1 初始化主管理节点
      • 2.6.2 增加从管理节点
      • 2.6.3 查看管理节点信息
      • 2.6.4 增加工作节点
      • 2.6.5 查看集群信息
    • 2.7 重建集群
      • 2.7.1 重新初始化管理节点
      • 2.7.2 重新增加工作节点
  • 3 添加、更新、删除标签信息

1 方案介绍

1.1 概述

要利用Swarm模式的容错功能,建议创建奇数的管理节点。

当有多个管理节点时,如果一个leader节点宕机,其余的manager节点将选择一个新的leader,并恢复集群状态的编排和维护。默认情况下,管理节点也运行任务。

三个管理节点的Swarm最大允许一个管理节点宕机。

1.2 软件包

下载地址:
/download/Zhuge_Dan/85064880

下载地址:
/download/Zhuge_Dan/85067951

本文使用作为示例。

2 Swarm集群搭建

2.1 IP规划

本示例中使用6台机器为例。IP规划如下:

角色 主机名 IP地址 防火墙规则
主管理节点(manager node) docker01 172.16.86.11 开放端口:2375-2377 tcp/udp、7946 tcp/udp、4789 udp
从管理节点(manager node) docker02 172.16.86.12 开放端口:2375-2377 tcp/udp、7946 tcp/udp、4789 udp
从管理节点(manager node) docker03 172.16.86.13 开放端口:2375-2377 tcp/udp、7946 tcp/udp、4789 udp
工作节点(worker node) docker04 172.16.86.14 开放端口:2375-2377 tcp/udp、7946 tcp/udp、4789 udp
工作节点(worker node) docker05 172.16.86.15 开放端口:2375-2377 tcp/udp、7946 tcp/udp、4789 udp
工作节点(worker node) docker06 172.16.86.16 开放端口:2375-2377 tcp/udp、7946 tcp/udp、4789 udp

默认的管理服务端口为2377,需要能被工作节点访问到。
为了支持集群的成员发现和外部服务映射,还需要在所有节点上开启7946 TCP/UDP4789 UDP端口。

2.2 基础配置

操作设备:所有节点

2.2.1 关闭SELinux

[root@docker01 ~]# setenforce 0
[root@docker01 ~]# vi /etc/selinux/config
SELINUX=disabled								#将SELINUX设置为disabled
[root@docker01 ~]# getenforce 
Permissive
  • 1
  • 2
  • 3
  • 4
  • 5

2.2.2 关闭防火墙或开放需要的业务端口

[root@docker01 ~]# systemctl stop firewalld								#关闭防火墙
[root@docker01 ~]# systemctl disable firewalld								#禁止防火墙开机自启
  • 1
  • 2

或者:

[root@docker01 ~]# firewall-cmd --zone=public --add-port=2375-2377/tcp --add-port=2375-2377/udp --add-port=7946/tcp --add-port=7946/udp --add-port=4789/udp --permanent
success
[root@docker01 ~]# firewall-cmd --reload
success
[root@docker01 ~]# firewall-cmd --list-all
public (active)
  target: default
  icmp-block-inversion: no
  interfaces: ens33
  sources:
  services: dhcpv6-client ssh
  ports: 2375-2377/tcp 2375-2377/udp 7946/tcp 7946/udp 4789/udp
  protocols:
  masquerade: no
  forward-ports:
  source-ports:
  icmp-blocks:
  rich rules:
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18

2.2.3 调整max_open_files

[root@docker01 ~]# echo '* soft nofile 65535' >> /etc/security/

[root@docker01 ~]# echo '* hard nofile 65535' >> /etc/security/

[root@docker01 ~]# cat /etc/security/ | tail -2
* soft nofile 65535
* hard nofile 65535

[root@docker01 ~]# echo 'ulimit -SHn 65535' >> /etc//

[root@docker01 ~]# cat /etc// | tail -1
ulimit -SHn 65535

[root@docker01 ~]# chmod +x /etc//								#为增加可执行权限
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14

2.2.4 重启检查SELinux、firewalld和open files

[root@docker01 ~]# reboot

[root@docker01 ~]# getenforce								#查看SELinux是否关闭成功
Disabled								#关闭成功

[root@docker01 ~]# systemctl status firewalld								#查看防火墙状态
●  - firewalld - dynamic firewall daemon
    Loaded: loaded (/usr/lib/systemd/system/; disabled; vendor preset: enabled)
    Active: inactive (dead)								#防火墙已关闭
      Docs: man:firewalld(1)

[root@docker01 ~]# systemctl is-enabled firewalld								#查看防火墙是否开机自启
disabled								#防火墙已禁止开机自启

[root@docker01 ~]# ulimit -a | grep open
open files                      (-n) 65535								#open files已调整为65535
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16

2.3 配置主机名解析

操作设备:所有节点

将所有主机的IP和主机名对应关系配置到系统中,在所有服务器上执行:

[root@docker01 ~]# echo '172.16.86.11    docker01' >> /etc/hosts
[root@docker01 ~]# echo '172.16.86.12    docker02' >> /etc/hosts
[root@docker01 ~]# echo '172.16.86.13    docker03' >> /etc/hosts
[root@docker01 ~]# echo '172.16.86.14    docker04' >> /etc/hosts
[root@docker01 ~]# echo '172.16.86.15    docker05' >> /etc/hosts
[root@docker01 ~]# echo '172.16.86.16    docker06' >> /etc/hosts
[root@docker01 ~]# cat /etc/hosts
127.0.0.1   localhost  localhost4 localhost4.localdomain4
::1         localhost  localhost6 localhost6.localdomain6
172.16.86.11    docker01
172.16.86.12    docker02
172.16.86.13    docker03
172.16.86.14    docker04
172.16.86.15    docker05
172.16.86.16    docker06
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15

然后通过ping其它主机名检查是否生效。

2.4 安装Docker

操作设备:所有节点

将离线安装包上传到服务器。在所有服务器上执行:

A.安装

[root@docker01 ~]# cd /usr/local/

[root@docker01 local]# cp -r /tmp/ /usr/local/

[root@docker01 local]# mkdir /usr/local/docker-20108

[root@docker01 local]# tar -xPf /usr/local/ -C /usr/local/docker-20108

[root@docker01 local]# cd /usr/local/docker-20108

[root@docker01 docker-20108]# ls -lh
total 106M
-rw-r--r--. 1 root root 256K Aug 23  2021 audit-2.8.5-4.el7.x86_64.rpm
-rw-r--r--. 1 root root 102K Aug 23  2021 audit-libs-2.8.5-4.el7.x86_64.rpm
-rw-r--r--. 1 root root  77K Aug 23  2021 audit-libs-python-2.8.5-4.el7.x86_64.rpm
-rw-r--r--. 1 root root 295K Aug 23  2021 checkpolicy-2.5-8.el7.x86_64.rpm
-rw-r--r--. 1 root root  30M Aug 23  2021 -1.4.9-3.1.el7.x86_64.rpm
-rw-r--r--. 1 root root  23M Aug 23  2021 docker-ce-20.10.8-3.el7.x86_64.rpm
-rw-r--r--. 1 root root  30M Aug 23  2021 docker-ce-cli-20.10.8-3.el7.x86_64.rpm
-rw-r--r--. 1 root root 8.1M Aug 23  2021 docker-ce-rootless-extras-20.10.8-3.el7.x86_64.rpm
-rw-r--r--. 1 root root 4.2M Aug 23  2021 docker-scan-plugin-0.8.0-3.el7.x86_64.rpm
-rw-r--r--. 1 root root  67K Aug 23  2021 libcgroup-0.41-21.el7.x86_64.rpm
-rw-r--r--. 1 root root  57K Aug 23  2021 libseccomp-2.3.1-4.el7.x86_64.rpm
-rw-r--r--. 1 root root 163K Aug 23  2021 libselinux-2.5-15.el7.x86_64.rpm
-rw-r--r--. 1 root root 236K Aug 23  2021 libselinux-python-2.5-15.el7.x86_64.rpm
-rw-r--r--. 1 root root 152K Aug 23  2021 libselinux-utils-2.5-15.el7.x86_64.rpm
-rw-r--r--. 1 root root 151K Aug 23  2021 libsemanage-2.5-14.el7.x86_64.rpm
-rw-r--r--. 1 root root 113K Aug 23  2021 libsemanage-python-2.5-14.el7.x86_64.rpm
-rw-r--r--. 1 root root 298K Aug 23  2021 libsepol-2.5-10.el7.x86_64.rpm
-rw-r--r--. 1 root root 917K Aug 23  2021 policycoreutils-2.5-34.el7.x86_64.rpm
-rw-r--r--. 1 root root 458K Aug 23  2021 policycoreutils-python-2.5-34.el7.x86_64.rpm
-rw-r--r--. 1 root root  33K Aug 23  2021 python-IPy-0.75-6.
-rw-r--r--. 1 root root 498K Aug 23  2021 selinux-policy-3.13.1-268.el7_9.
-rw-r--r--. 1 root root 7.0M Aug 23  2021 selinux-policy-targeted-3.13.1-268.el7_9.
-rw-r--r--. 1 root root 621K Aug 23  2021 setools-libs-3.3.8-4.el7.x86_64.rpm

[root@docker01 docker-20108]# rpm -ivh /usr/local/docker-20108/*.rpm --nodeps --force
warning: /usr/local/docker-20108/audit-2.8.5-4.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID f4a80eb5: NOKEY
warning: /usr/local/docker-20108/-1.4.9-3.1.el7.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID 621e9f35: NOKEY
Preparing...                          ################################# [100%]
Updating / installing...
   1:libsepol-2.5-10.el7              ################################# [  4%]
   2:libselinux-2.5-15.el7            ################################# [  9%]
   3:audit-libs-2.8.5-4.el7           ################################# [ 13%]
   4:libsemanage-2.5-14.el7           ################################# [ 17%]
   5:docker-scan-plugin-0:0.8.0-3.el7 ################################# [ 22%]
   6:docker-ce-cli-1:20.10.8-3.el7    ################################# [ 26%]
   7:libseccomp-2.3.1-4.el7           ################################# [ 30%]
   8:libcgroup-0.41-21.el7            ################################# [ 35%]
   9:-1.4.9-3.1.el7      ################################# [ 39%]
  10:docker-ce-rootless-extras-0:20.10################################# [ 43%]
  11:docker-ce-3:20.10.8-3.el7        ################################# [ 48%]
  12:libsemanage-python-2.5-14.el7    ################################# [ 52%]
  13:audit-libs-python-2.8.5-4.el7    ################################# [ 57%]
  14:libselinux-python-2.5-15.el7     ################################# [ 61%]
  15:libselinux-utils-2.5-15.el7      ################################# [ 65%]
  16:policycoreutils-2.5-34.el7       ################################# [ 70%]
  17:selinux-policy-3.13.1-268.el7_9.2################################# [ 74%]
  18:setools-libs-3.3.8-4.el7         ################################# [ 78%]
  19:python-IPy-0.75-6.el7            ################################# [ 83%]
  20:checkpolicy-2.5-8.el7            ################################# [ 87%]
  21:policycoreutils-python-2.5-34.el7################################# [ 91%]
  22:selinux-policy-targeted-3.13.1-26################################# [ 96%]
  23:audit-2.8.5-4.el7                ################################# [100%]
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24
  • 25
  • 26
  • 27
  • 28
  • 29
  • 30
  • 31
  • 32
  • 33
  • 34
  • 35
  • 36
  • 37
  • 38
  • 39
  • 40
  • 41
  • 42
  • 43
  • 44
  • 45
  • 46
  • 47
  • 48
  • 49
  • 50
  • 51
  • 52
  • 53
  • 54
  • 55
  • 56
  • 57
  • 58
  • 59
  • 60
  • 61
  • 62
  • 63
  • 64

B.安装

[root@docker01 ~]# cd /usr/local/

[root@docker01 local]# cp -r /tmp/ /usr/local/

[root@docker01 local]# mkdir /usr/local/docker-201012

[root@docker01 local]# unzip  -d docker-201012
Archive:  
  inflating: docker-201012/audit-libs-python-2.8.5-4.el7.x86_64.rpm
  inflating: docker-201012/checkpolicy-2.5-8.el7.x86_64.rpm
  inflating: docker-201012/container-selinux-2.119.2-1.911c772.el7_8.
 extracting: docker-201012/-1.5.11-3.1.el7.x86_64.rpm
 extracting: docker-201012/docker-ce-20.10.12-3.el7.x86_64.rpm
  inflating: docker-201012/docker-ce-cli-20.10.14-3.el7.x86_64.rpm
 extracting: docker-201012/docker-ce-rootless-extras-20.10.14-3.el7.x86_64.rpm
 extracting: docker-201012/docker-scan-plugin-0.17.0-3.el7.x86_64.rpm
  inflating: docker-201012/fuse-overlayfs-0.7.2-6.el7_8.x86_64.rpm
  inflating: docker-201012/fuse3-libs-3.6.1-4.el7.x86_64.rpm
  inflating: docker-201012/libcgroup-0.41-21.el7.x86_64.rpm
  inflating: docker-201012/libseccomp-2.3.1-4.el7.x86_64.rpm
  inflating: docker-201012/libsemanage-python-2.5-14.el7.x86_64.rpm
  inflating: docker-201012/policycoreutils-python-2.5-34.el7.x86_64.rpm
  inflating: docker-201012/python-IPy-0.75-6.
  inflating: docker-201012/setools-libs-3.3.8-4.el7.x86_64.rpm
  inflating: docker-201012/slirp4netns-0.4.3-4.el7_8.x86_64.rpm

[root@docker01 local]# cd /usr/local/docker-201012

[root@docker01 docker-201012]# ls -lh
total 96M
-rw-r--r-- 1 root root  77K Apr  2 09:18 audit-libs-python-2.8.5-4.el7.x86_64.rpm
-rw-r--r-- 1 root root 295K Apr  2 09:18 checkpolicy-2.5-8.el7.x86_64.rpm
-rw-r--r-- 1 root root  29M Apr  2 09:18 -1.5.11-3.1.el7.x86_64.rpm
-rw-r--r-- 1 root root  40K Apr  2 09:18 container-selinux-2.119.2-1.911c772.el7_8.
-rw-r--r-- 1 root root  23M Apr  2 09:18 docker-ce-20.10.12-3.el7.x86_64.rpm
-rw-r--r-- 1 root root  31M Apr  2 09:18 docker-ce-cli-20.10.14-3.el7.x86_64.rpm
-rw-r--r-- 1 root root 8.1M Apr  2 09:19 docker-ce-rootless-extras-20.10.14-3.el7.x86_64.rpm
-rw-r--r-- 1 root root 3.8M Apr  2 09:19 docker-scan-plugin-0.17.0-3.el7.x86_64.rpm
-rw-r--r-- 1 root root  82K Apr  2 09:18 fuse3-libs-3.6.1-4.el7.x86_64.rpm
-rw-r--r-- 1 root root  55K Apr  2 09:18 fuse-overlayfs-0.7.2-6.el7_8.x86_64.rpm
-rw-r--r-- 1 root root  67K Apr  2 09:18 libcgroup-0.41-21.el7.x86_64.rpm
-rw-r--r-- 1 root root  57K Apr  2 09:18 libseccomp-2.3.1-4.el7.x86_64.rpm
-rw-r--r-- 1 root root 113K Apr  2 09:18 libsemanage-python-2.5-14.el7.x86_64.rpm
-rw-r--r-- 1 root root 458K Apr  2 09:18 policycoreutils-python-2.5-34.el7.x86_64.rpm
-rw-r--r-- 1 root root  33K Apr  2 09:18 python-IPy-0.75-6.
-rw-r--r-- 1 root root 621K Apr  2 09:18 setools-libs-3.3.8-4.el7.x86_64.rpm
-rw-r--r-- 1 root root  82K Apr  2 09:18 slirp4netns-0.4.3-4.el7_8.x86_64.rpm

[root@docker01 docker-201012]# rpm -ivh /usr/local/docker-201012/*.rpm --nodeps --force
warning: /usr/local/docker-201012/-1.5.11-3.1.el7.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID 621e9f35: NOKEY
Preparing...                          ################################# [100%]
Updating / installing...
   1:libseccomp-2.3.1-4.el7           ################################# [  6%]
   2:docker-scan-plugin-0:0.17.0-3.el7################################# [ 12%]
   3:docker-ce-cli-1:20.10.14-3.el7   ################################# [ 18%]
   4:libcgroup-0.41-21.el7            ################################# [ 24%]
   5:slirp4netns-0.4.3-4.el7_8        ################################# [ 29%]
   6:setools-libs-3.3.8-4.el7         ################################# [ 35%]
   7:python-IPy-0.75-6.el7            ################################# [ 41%]
   8:libsemanage-python-2.5-14.el7    ################################# [ 47%]
   9:fuse3-libs-3.6.1-4.el7           ################################# [ 53%]
  10:fuse-overlayfs-0.7.2-6.el7_8     ################################# [ 59%]
  11:checkpolicy-2.5-8.el7            ################################# [ 65%]
  12:audit-libs-python-2.8.5-4.el7    ################################# [ 71%]
  13:policycoreutils-python-2.5-34.el7################################# [ 76%]
  14:container-selinux-2:2.119.2-1.911################################# [ 82%]
setsebool:  SELinux is disabled.
  15:-1.5.11-3.1.el7     ################################# [ 88%]
  16:docker-ce-rootless-extras-0:20.10################################# [ 94%]
  17:docker-ce-3:20.10.12-3.el7       ################################# [100%]
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24
  • 25
  • 26
  • 27
  • 28
  • 29
  • 30
  • 31
  • 32
  • 33
  • 34
  • 35
  • 36
  • 37
  • 38
  • 39
  • 40
  • 41
  • 42
  • 43
  • 44
  • 45
  • 46
  • 47
  • 48
  • 49
  • 50
  • 51
  • 52
  • 53
  • 54
  • 55
  • 56
  • 57
  • 58
  • 59
  • 60
  • 61
  • 62
  • 63
  • 64
  • 65
  • 66
  • 67
  • 68
  • 69
  • 70

2.5 启动并设置开机自启

操作设备:所有节点

[root@docker01 ~]# systemctl start docker

[root@docker01 ~]# systemctl enable docker
Created symlink from /etc/systemd/system// to /usr/lib/systemd/system/.

[root@docker01 ~]# systemctl is-enabled docker
enabled
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7

2.6 创建集群

2.6.1 初始化主管理节点

操作设备:主管理节点(docker01)

[root@docker01 ~]# docker swarm init --advertise-addr 172.16.86.11:2377
Swarm initialized: current node (hod82m1r4tykhkg7dugvkr626) is now a manager.

To add a worker to this swarm, run the following command:

    docker swarm join --token SWMTKN-1-22921lkf4gy74u1ovey2cwmaf8s6yh774oqqzcmiswq3z9fpjg-2z2okt46vuensumj8voy4lsya 172.16.86.11:2377

To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8

2.6.2 增加从管理节点

1、在主管理节点,使用docker swarm join-token manager命令,生成从管理节点加入集群需要执行的命令。
操作设备:主管理节点(docker01)

[root@docker01 ~]# docker swarm join-token manager
To add a manager to this swarm, run the following command:

    docker swarm join --token SWMTKN-1-22921lkf4gy74u1ovey2cwmaf8s6yh774oqqzcmiswq3z9fpjg-6e4mc8y4jfslmzdubg99q0999 172.16.86.11:2377
  • 1
  • 2
  • 3
  • 4

2、根据上一步的命令在从管理节点服务器执行。
操作设备:从管理节点(docker02、docker03)

[root@docker02 ~]# docker swarm join --token SWMTKN-1-22921lkf4gy74u1ovey2cwmaf8s6yh774oqqzcmiswq3z9fpjg-6e4mc8y4jfslmzdubg99q0999 172.16.86.11:2377
This node joined a swarm as a manager.

[root@docker03 ~]# docker swarm join --token SWMTKN-1-22921lkf4gy74u1ovey2cwmaf8s6yh774oqqzcmiswq3z9fpjg-6e4mc8y4jfslmzdubg99q0999 172.16.86.11:2377
This node joined a swarm as a manager.
  • 1
  • 2
  • 3
  • 4
  • 5

2.6.3 查看管理节点信息

操作设备:主管理节点(docker01)

[root@docker01 ~]# docker node ls
ID                            HOSTNAME   STATUS    AVAILABILITY   MANAGER STATUS   ENGINE VERSION
hod82m1r4tykhkg7dugvkr626 *   docker01   Ready     Active         Leader           20.10.8
vwsm18thom4kkh9qrtbdy8rpu     docker02   Ready     Active         Reachable        20.10.8
pvqfoz6ybo9sf7oue5dapaysj     docker03   Ready     Active         Reachable        20.10.8
  • 1
  • 2
  • 3
  • 4
  • 5

2.6.4 增加工作节点

执行命令为“2.5.1 初始化主管理节点”输出命令。
操作设备:工作节点(docker04、docker05、docker06)

[root@docker04 ~]# docker swarm join --token SWMTKN-1-22921lkf4gy74u1ovey2cwmaf8s6yh774oqqzcmiswq3z9fpjg-2z2okt46vuensumj8voy4lsya 172.16.86.11:2377
This node joined a swarm as a worker.

[root@docker05 ~]# docker swarm join --token SWMTKN-1-22921lkf4gy74u1ovey2cwmaf8s6yh774oqqzcmiswq3z9fpjg-2z2okt46vuensumj8voy4lsya 172.16.86.11:2377
This node joined a swarm as a worker.

[root@docker06 ~]# docker swarm join --token SWMTKN-1-22921lkf4gy74u1ovey2cwmaf8s6yh774oqqzcmiswq3z9fpjg-2z2okt46vuensumj8voy4lsya 172.16.86.11:2377
This node joined a swarm as a worker.
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8

2.6.5 查看集群信息

1、查看节点情况
操作设备:主管理节点(docker01)

[root@docker01 ~]# docker node ls
ID                            HOSTNAME   STATUS    AVAILABILITY   MANAGER STATUS   ENGINE VERSION
hod82m1r4tykhkg7dugvkr626 *   docker01   Ready     Active         Leader           20.10.8
vwsm18thom4kkh9qrtbdy8rpu     docker02   Ready     Active         Reachable        20.10.8
pvqfoz6ybo9sf7oue5dapaysj     docker03   Ready     Active         Reachable        20.10.8
kuje4tn1zq0onf4z1s0790v9p     docker04   Ready     Active                          20.10.8
y16cgewarsec50wq5gnthosgo     docker05   Ready     Active                          20.10.8
xtjhgk33jit6ceykjhgtftltg     docker06   Ready     Active                          20.10.8
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8

AVAILABILITY:表示调度程序是否可以将任务分配给节点。
Active:可以将任务分配给节点。
Pause:调度程序不向节点分配新任务,但现有的任务仍然运行。
Drain:调度程序不向节点分配新任务,已经存在的任务也将被调用到Active节点上。

MANAGER STATUS:没有值表示不参与群集管理的工作节点。
Leader:为集群做出所有的集群管理和编排决策。
Reachable:表示节点参与Raft仲裁的manager节点。如果leader节点不可用,则该节点有资格成为新的leader。
Unavailable:表示节点是一个无法与其他manager通信的节点。如果manager节点变为此状态应该加入一个新的manager节点到集群中,或者将一个工作节点提升为一个manager。

2、查看节点详细信息
操作设备:主管理节点(docker01)
(1)主管理节点。

[root@docker01 ~]# docker node inspect docker01 --pretty
ID:                     hod82m1r4tykhkg7dugvkr626
Hostname:               docker01
Joined at:              2022-03-04 13:47:09.655581079 +0000 utc
Status:
 State:                 Ready
 Availability:          Active
 Address:               172.16.86.11
Manager Status:
 Address:               172.16.86.11:2377
 Raft Status:           Reachable
 Leader:                Yes
Platform:
 Operating System:      linux
 Architecture:          x86_64
Resources:
 CPUs:                  2
 Memory:                1.777GiB
Plugins:
 Log:           awslogs, fluentd, gcplogs, gelf, journald, json-file, local, logentries, splunk, syslog
 Network:               bridge, host, ipvlan, macvlan, null, overlay
 Volume:                local
Engine Version:         20.10.8
TLS Info:
 TrustRoot:
-----BEGIN CERTIFICATE-----
MIIBazCCARCgAwIBAgIUKTcfZTB4XFgTpgIAi454UQ5KSgIwCgYIKoZIzj0EAwIw
EzERMA8GA1UEAxMIc3dhcm0tY2EwHhcNMjIwMzA0MTM0MjAwWhcNNDIwMjI3MTM0
MjAwWjATMREwDwYDVQQDEwhzd2FybS1jYTBZMBMGByqGSM49AgEGCCqGSM49AwEH
A0IABP0g1c91Gm9C2kY8d9AmRQ6sNgKagRKgMi/6G04nRxsi2/3lQbqXz9OiJivC
nyd76d2BTrDaCS2Hh7m0YcfWQVCjQjBAMA4GA1UdDwEB/wQEAwIBBjAPBgNVHRMB
Af8EBTADAQH/MB0GA1UdDgQWBBRN+qybDBzzAbq8aErv9cHSFFP9NzAKBggqhkjO
PQQDAgNJADBGAiEA0sZNhoz3ylDLSDkxGNW+kc4b2qOuhUeySjNc3uIHh0MCIQCj
HJWoJrh8HtZWGs/tNrdHNMWl3Y3rSBOANdQqVDh7cA==
-----END CERTIFICATE-----

 Issuer Subject:        MBMxETAPBgNVBAMTCHN3YXJtLWNh
 Issuer Public Key:     MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE/SDVz3Uab0LaRjx30CZFDqw2ApqBEqAyL/obTidHGyLb/eVBupfP06ImK8KfJ3vp3YFOsNoJLYeHubRhx9ZBUA==
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24
  • 25
  • 26
  • 27
  • 28
  • 29
  • 30
  • 31
  • 32
  • 33
  • 34
  • 35
  • 36
  • 37
  • 38

(2)从管理节点。

[root@docker01 ~]# docker node inspect docker02 --pretty
ID:                     vwsm18thom4kkh9qrtbdy8rpu
Hostname:               docker02
Joined at:              2022-03-04 13:49:57.555041807 +0000 utc
Status:
 State:                 Ready
 Availability:          Active
 Address:               172.16.86.12
Manager Status:
 Address:               172.16.86.12:2377
 Raft Status:           Reachable
 Leader:                No
Platform:
 Operating System:      linux
 Architecture:          x86_64
Resources:
 CPUs:                  2
 Memory:                1.777GiB
Plugins:
 Log:           awslogs, fluentd, gcplogs, gelf, journald, json-file, local, logentries, splunk, syslog
 Network:               bridge, host, ipvlan, macvlan, null, overlay
 Volume:                local
Engine Version:         20.10.8
TLS Info:
 TrustRoot:
-----BEGIN CERTIFICATE-----
MIIBazCCARCgAwIBAgIUKTcfZTB4XFgTpgIAi454UQ5KSgIwCgYIKoZIzj0EAwIw
EzERMA8GA1UEAxMIc3dhcm0tY2EwHhcNMjIwMzA0MTM0MjAwWhcNNDIwMjI3MTM0
MjAwWjATMREwDwYDVQQDEwhzd2FybS1jYTBZMBMGByqGSM49AgEGCCqGSM49AwEH
A0IABP0g1c91Gm9C2kY8d9AmRQ6sNgKagRKgMi/6G04nRxsi2/3lQbqXz9OiJivC
nyd76d2BTrDaCS2Hh7m0YcfWQVCjQjBAMA4GA1UdDwEB/wQEAwIBBjAPBgNVHRMB
Af8EBTADAQH/MB0GA1UdDgQWBBRN+qybDBzzAbq8aErv9cHSFFP9NzAKBggqhkjO
PQQDAgNJADBGAiEA0sZNhoz3ylDLSDkxGNW+kc4b2qOuhUeySjNc3uIHh0MCIQCj
HJWoJrh8HtZWGs/tNrdHNMWl3Y3rSBOANdQqVDh7cA==
-----END CERTIFICATE-----

 Issuer Subject:        MBMxETAPBgNVBAMTCHN3YXJtLWNh
 Issuer Public Key:     MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE/SDVz3Uab0LaRjx30CZFDqw2ApqBEqAyL/obTidHGyLb/eVBupfP06ImK8KfJ3vp3YFOsNoJLYeHubRhx9ZBUA==
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24
  • 25
  • 26
  • 27
  • 28
  • 29
  • 30
  • 31
  • 32
  • 33
  • 34
  • 35
  • 36
  • 37
  • 38

(3)工作节点。

[root@docker01 ~]# docker node inspect docker04 --pretty
ID:                     kuje4tn1zq0onf4z1s0790v9p
Hostname:               docker04
Joined at:              2022-03-04 13:51:29.970941279 +0000 utc
Status:
 State:                 Ready
 Availability:          Active
 Address:               172.16.86.14
Platform:
 Operating System:      linux
 Architecture:          x86_64
Resources:
 CPUs:                  2
 Memory:                1.777GiB
Plugins:
 Log:           awslogs, fluentd, gcplogs, gelf, journald, json-file, local, logentries, splunk, syslog
 Network:               bridge, host, ipvlan, macvlan, null, overlay
 Volume:                local
Engine Version:         20.10.8
TLS Info:
 TrustRoot:
-----BEGIN CERTIFICATE-----
MIIBazCCARCgAwIBAgIUKTcfZTB4XFgTpgIAi454UQ5KSgIwCgYIKoZIzj0EAwIw
EzERMA8GA1UEAxMIc3dhcm0tY2EwHhcNMjIwMzA0MTM0MjAwWhcNNDIwMjI3MTM0
MjAwWjATMREwDwYDVQQDEwhzd2FybS1jYTBZMBMGByqGSM49AgEGCCqGSM49AwEH
A0IABP0g1c91Gm9C2kY8d9AmRQ6sNgKagRKgMi/6G04nRxsi2/3lQbqXz9OiJivC
nyd76d2BTrDaCS2Hh7m0YcfWQVCjQjBAMA4GA1UdDwEB/wQEAwIBBjAPBgNVHRMB
Af8EBTADAQH/MB0GA1UdDgQWBBRN+qybDBzzAbq8aErv9cHSFFP9NzAKBggqhkjO
PQQDAgNJADBGAiEA0sZNhoz3ylDLSDkxGNW+kc4b2qOuhUeySjNc3uIHh0MCIQCj
HJWoJrh8HtZWGs/tNrdHNMWl3Y3rSBOANdQqVDh7cA==
-----END CERTIFICATE-----

 Issuer Subject:        MBMxETAPBgNVBAMTCHN3YXJtLWNh
 Issuer Public Key:     MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE/SDVz3Uab0LaRjx30CZFDqw2ApqBEqAyL/obTidHGyLb/eVBupfP06ImK8KfJ3vp3YFOsNoJLYeHubRhx9ZBUA==
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24
  • 25
  • 26
  • 27
  • 28
  • 29
  • 30
  • 31
  • 32
  • 33
  • 34

2.7 重建集群

2.7.1 重新初始化管理节点

操作设备:管理节点

[root@docker01 ~]# docker node ls
Error response from daemon: rpc error: code = Unknown desc = The swarm does not have a leader. It's possible that too few managers are online. Make sure more than half of the managers are online.

[root@docker01 ~]# docker swarm init --advertise-addr 172.16.86.11:2377
Error response from daemon: This node is already part of a swarm. Use "docker swarm leave" to leave this swarm and join another one.

[root@docker01 ~]# docker swarm leave
Error response from daemon: You are attempting to leave the swarm on a node that is participating as a manager. The only way to restore a swarm that has lost consensus is to reinitialize it with `--force-new-cluster`. Use `--force` to suppress this message.

[root@docker01 ~]# docker swarm leave --force
Node left the swarm.

[root@docker01 ~]# docker swarm init --advertise-addr 172.16.86.11:2377
Swarm initialized: current node (hod82m1r4tykhkg7dugvkr626) is now a manager.

To add a worker to this swarm, run the following command:

    docker swarm join --token SWMTKN-1-22921lkf4gy74u1ovey2cwmaf8s6yh774oqqzcmiswq3z9fpjg-2z2okt46vuensumj8voy4lsya 172.16.86.11:2377

To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.

  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21

2.7.2 重新增加工作节点

操作设备:工作节点

[root@docker02 ~]# docker swarm join --token SWMTKN-1-22921lkf4gy74u1ovey2cwmaf8s6yh774oqqzcmiswq3z9fpjg-6e4mc8y4jfslmzdubg99q0999 172.16.86.11:2377
Error response from daemon: This node is already part of a swarm. Use "docker swarm leave" to leave this swarm and join another one.

[root@docker02 ~]# docker swarm init --advertise-addr 172.16.86.11:2377
Error response from daemon: This node is already part of a swarm. Use "docker swarm leave" to leave this swarm and join another one.

[root@docker02 ~]# docker swarm leave --force
Node left the swarm.

[root@docker02 ~]# docker swarm join --token SWMTKN-1-22921lkf4gy74u1ovey2cwmaf8s6yh774oqqzcmiswq3z9fpjg-6e4mc8y4jfslmzdubg99q0999 172.16.86.11:2377
This node joined a swarm as a manager.
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11

3 添加、更新、删除标签信息

参数:
-label-add list:添加或更新一组标签信息。
-label-remove list:删除一组标签信息。
例:
添加节点标签:

docker node update --label-add role node1      #为node1节点添加标签“role”
  • 1

删除节点标签:

docker node update --label-rm role node1      #为node1节点删除标签“role”
  • 1

操作设备:主管理节点(docker01)
1、为所有节点添加统一标签“all”

[root@docker01 ~]# docker node update --label-add is_all=true docker01
docker01
[root@docker01 ~]# docker node update --label-add is_all=true docker02
docker02
[root@docker01 ~]# docker node update --label-add is_all=true docker03
docker03
[root@docker01 ~]# docker node update --label-add is_all=true docker04
docker04
[root@docker01 ~]# docker node update --label-add is_all=true docker05
docker05
[root@docker01 ~]# docker node update --label-add is_all=true docker06
docker06
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12

2、为管理节点添加标签“manager”

[root@docker01 ~]# docker node update --label-add is_manager=true docker01
docker01
[root@docker01 ~]# docker node update --label-add is_manager=true docker02
docker02
[root@docker01 ~]# docker node update --label-add is_manager=true docker03
docker03
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6

3、为工作节点添加标签“worker”

[root@docker01 ~]# docker node update --label-add is_worker=true docker04
docker04
[root@docker01 ~]# docker node update --label-add is_worker=true docker05
docker05
[root@docker01 ~]# docker node update --label-add is_worker=true docker06
docker06
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6

4、查看节点详细信息

[root@docker01 ~]# docker node inspect docker01 --pretty | head
ID:                     hod82m1r4tykhkg7dugvkr626
Labels:
 - is_all=true
 - is_manager=true
Hostname:               docker01
Joined at:              2022-03-04 13:47:09.655581079 +0000 utc
Status:
 State:                 Ready
 Availability:          Active
 Address:               172.16.86.11

[root@docker01 ~]# docker node inspect docker02 --pretty | head
ID:                     vwsm18thom4kkh9qrtbdy8rpu
Labels:
 - is_all=true
 - is_manager=true
Hostname:               docker02
Joined at:              2022-03-04 13:49:57.555041807 +0000 utc
Status:
 State:                 Ready
 Availability:          Active
 Address:               172.16.86.12

[root@docker01 ~]# docker node inspect docker04 --pretty | head
ID:                     kuje4tn1zq0onf4z1s0790v9p
Labels:
 - is_all=true
 - is_worker=true
Hostname:               docker04
Joined at:              2022-03-04 13:51:29.970941279 +0000 utc
Status:
 State:                 Ready
 Availability:          Active
 Address:               172.16.86.14
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24
  • 25
  • 26
  • 27
  • 28
  • 29
  • 30
  • 31
  • 32
  • 33
  • 34
  • 35