Docker实现跨主机容器实例网络通信(1)——利用LinuxBridge构建多主机Docker网络

时间:2022-11-01 14:17:46

题记

前面我们已经提到了如果我们构建docker集群,肯定会出现跨主机docker实例网络连接的需求,而且为了节省主机网络的IP资源,我们尽量使用docker0自己的网络连接,适当为主节点(容器实例)添加主机网络IP,这样应该是比较理想的业务需求。

--------------------------------------------------------------------------------------

Blog:    http://blog.csdn.net/chinagissoft

QQ群:16403743

宗旨:专注于"GIS+"前沿技术的研究与交流,将云计算技术、大数据技术、容器技术、物联网与GIS进行深度融合,探讨"GIS+"技术和行业解决方案

转载说明:文章允许转载,但必须以链接方式注明源地址,否则追究法律责任!

--------------------------------------------------------------------------------------


为了充分利用docker0网桥的IP资源,我们为每台主机上的Docker daemon指定不同的--fixed-cidr参数,将不同主机上的Docker容器的地址限定在不同的网段中。

Docker实现跨主机容器实例网络通信(1)——利用LinuxBridge构建多主机Docker网络


如上图所示,我有两台主机,分别包含两个网卡 eth0,eth1,我的eth1作为管理网卡,IP如上所示。

默认情况下,每个机器都会创建一个docker0网桥,IP的都是172.17.0.1,不过为了避免IP冲突,将第一台云主机的docker0网桥IP修改为172.17.0.2,确保相互之间可以连通。


1、第一个主机,修改docker0网桥的网络信息

sm@controller:~$ cat /etc/network/interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

auto docker0
iface docker0 inet static
address 172.17.0.1
netmask 255.255.0.0
bridge_ports eth0
bridge_stp off
bridge_fd 0

auto eth1
iface eth1 inet dhcp

修改完毕之后,建议重启服务器

2、为第一台主机添加容器主机IP段设置,添加DOCKER_OPTS="--fixed-cidr=172.17.1.1/24"

sm@controller:~$ cat /etc/default/docker
# Docker Upstart and SysVinit configuration file

# Customize location of Docker binary (especially for development testing).
#DOCKER="/usr/local/bin/docker"

# Use DOCKER_OPTS to modify the daemon startup options.
DOCKER_OPTS="--fixed-cidr=172.17.1.1/24"
# If you need Docker to use an HTTP proxy, it can also be specified here.
#export http_proxy="http://127.0.0.1:3128/"

# This is also a handy place to tweak where Docker's temporary files go.
#export TMPDIR="/mnt/bigdrive/docker-tmp"
#DOCKER_OPTS="-b=br-docker"

添加完毕之后重启docker服务


3、同样的方法,在第二台机器重复上述操作,注意将IP修改为172.17.0.2

另外,添加DOCKER_OPTS="--fixed-cidr=172.17.2.1/24",重启docker服务


简单描述一下,我们在第一台机器上看到docker0的IP是172.17.0.1/16,docker容器也就是从docker0所在网络中获取IP,而且在第一台机器上,将docker容器的IP范围限制在172.17.1.1/24网段。

同理,我们在第二台机器上看到docker0的IP是172.17.0.2/16,docker容器也就是从docker0所在网络中获取IP,而且在第二台机器上,将docker容器的IP范围限制在172.17.2.1/24网段。


测试host1

查看IP,确保能够连通host2的docker0IP

root@controller:~# ifconfig docker0
docker0 Link encap:Ethernet HWaddr 00:0c:29:d3:5a:fe
inet addr:172.17.0.1 Bcast:172.17.255.255 Mask:255.255.0.0
inet6 addr: fe80::20c:29ff:fed3:5afe/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:85 errors:0 dropped:0 overruns:0 frame:0
TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:6599 (6.5 KB) TX bytes:648 (648.0 B)

root@controller:~# ping 172.17.0.2
PING 172.17.0.2 (172.17.0.2) 56(84) bytes of data.
64 bytes from 172.17.0.2: icmp_seq=1 ttl=64 time=0.561 ms
64 bytes from 172.17.0.2: icmp_seq=2 ttl=64 time=0.700 ms

host1创建容器实例,IP为172.17.1.2
root@controller:~# docker run -it --name test1 ubuntu:14.04 /bin/bash
root@54fd72ea7832:/# ifconfig
eth0 Link encap:Ethernet HWaddr 02:42:ac:11:01:02
inet addr:172.17.1.2 Bcast:0.0.0.0 Mask:255.255.0.0
inet6 addr: fe80::42:acff:fe11:102/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:7 errors:0 dropped:0 overruns:0 frame:0
TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:568 (568.0 B) TX bytes:508 (508.0 B)

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

host2查看docker0 IP信息

root@docker2:~# ifconfig docker0
docker0 Link encap:Ethernet HWaddr 00:0c:29:c0:73:8c
inet addr:172.17.0.2 Bcast:172.17.255.255 Mask:255.255.0.0
inet6 addr: fe80::20c:29ff:fec0:738c/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:3697 errors:0 dropped:0 overruns:0 frame:0
TX packets:38 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:327320 (327.3 KB) TX bytes:3420 (3.4 KB)



host2创建容器实例,获得IP :172.17.2.1
root@docker2:~# docker run -it --name test2 ubuntu:14.04 /bin/bash
root@4d049a6397b3:/# ifconfig
eth0 Link encap:Ethernet HWaddr 02:42:ac:11:02:01
inet addr:172.17.2.1 Bcast:0.0.0.0 Mask:255.255.0.0
inet6 addr: fe80::42:acff:fe11:201/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:10 errors:0 dropped:0 overruns:0 frame:0
TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:920 (920.0 B) TX bytes:508 (508.0 B)

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

host2测试连接host1的容器172.17.1.2

root@docker2:~# docker run -it --name test2 ubuntu:14.04 /bin/bash
root@4d049a6397b3:/# ifconfig
eth0 Link encap:Ethernet HWaddr 02:42:ac:11:02:01
inet addr:172.17.2.1 Bcast:0.0.0.0 Mask:255.255.0.0
inet6 addr: fe80::42:acff:fe11:201/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:10 errors:0 dropped:0 overruns:0 frame:0
TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:920 (920.0 B) TX bytes:508 (508.0 B)

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

root@4d049a6397b3:/# ping 172.17.1.2
PING 172.17.1.2 (172.17.1.2) 56(84) bytes of data.
64 bytes from 172.17.1.2: icmp_seq=1 ttl=64 time=0.500 ms
64 bytes from 172.17.1.2: icmp_seq=2 ttl=64 time=0.669 ms
64 bytes from 172.17.1.2: icmp_seq=3 ttl=64 time=0.599 ms
^C
--- 172.17.1.2 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1998ms
rtt min/avg/max/mdev = 0.500/0.589/0.669/0.072 ms