最近在尝试Docker Toolbox+Virtualbox+Rancher搭建Docker集群测试环境,1台RancherServer+3台node,使用Rancher内置的cattle,搭建集群一切正常。但用另外3台node搭建K8S测试环境时,遇到kubelet报错,该服务每没隔几秒重启一下,然后自动停止。日志提示信息中有一行:container_manager_linux.go:205] Running with swap on is not supported, please disable swap! This will be a fatal error by default starting in K8s v1.6!。尝试关闭swap再试,虽然日志中没有该提示信息了,但kubelet服务还是无效。先把禁用启用虚拟内存的方法整理一下,说不定以后用得着。
如果你的服务器内存逐个大,比如32GB/64GB以上,并且想提升运行的速度,也可以考虑禁用虚拟内存(swap交换分区)。
一、不重启电脑,禁用启用swap,立刻生效
# 禁用命令
sudo swapoff -a
# 启用命令
sudo swapon -a
# 查看交换分区的状态
sudo free -m
二、重新启动电脑,永久禁用Swap
# 把根目录文件系统设为可读写
sudo mount -n -o remount,rw /
# 用vi修改/etc/fstab文件,在swap分区这行前加 # 禁用掉,保存退出
vi /etc/fstab
i #进入insert 插入模式
:wq #保存退出
# 重新启动电脑,使用free -m查看分区状态
reboot
sudo free -m
后续工作:使用Docker Toolbox+virtualbox+Docker 1.12.6+Rancher 1.6搭建K8S可能有许多BUG没有解决,要想真正体验K8S,还是直接使用virtualbox创建ubuntu虚拟机,直接在多台Ubuntu虚拟机上组建Rancher+Node的K8S集群,更接近真实环境。
参考链接:
https://askubuntu.com/questions/214805/how-do-i-disable-swap
http://blog.csdn.net/mika001/article/details/72780953
LINUX中SWAP分区设置
http://njs375666635.iteye.com/blog/2241041
http://blog.csdn.net/waterxcfg304/article/details/49766467
Linux 无法修改/etc/fstab内容 (去除fstab文件只读属性)
http://blog.csdn.net/demo_deng/article/details/9429477
参考内容:
sudo swapoff -a
Above command disables swap only for a current session, you need to comment out swap partition in /etc/fstab file. To do this you just need to add "#" (comment line) to the beginning of swap line. Steps are as:
1、Open fstab file, type sudo gedit /etc/fstab in terminal.
File's contents would look like this:
proc /proc proc nodev,noexec,nosuid 0 0
/host/ubuntu/disks/root.disk / ext4 loop,errors=remount-ro 0 1
/host/ubuntu/disks/swap.disk none swap loop,sw 0 0
#/dev/sda10 /media/ASD vfat defaults 0 0
#/dev/sda1 /media/98 vfat defaults 0 0
2、Just add hash (#) to the beginning of the swap partition line, so the line looks as:
#/host/ubuntu/disks/swap.disk none swap loop,sw 0 0
3、Reboot your PC
【kubelet的错误日志】
2017/7/13 上午6:40:18++ awk '{print $2}'
2017/7/13 上午6:40:18++ tac /proc/mounts
2017/7/13 上午6:40:18+ for m in '$(tac /proc/mounts | awk '\''{print $2}'\'' | grep ^${i}/)'
2017/7/13 上午6:40:18+ '[' /run/docker/netns/9acbdfb285d2 '!=' /var/run/nscd ']'
2017/7/13 上午6:40:18+ '[' /run/docker/netns/9acbdfb285d2 '!=' /run/nscd ']'
2017/7/13 上午6:40:18+ umount /run/docker/netns/9acbdfb285d2
2017/7/13 上午6:40:18+ for m in '$(tac /proc/mounts | awk '\''{print $2}'\'' | grep ^${i}/)'
2017/7/13 上午6:40:18+ '[' /run/docker/netns/7109d660bb2c '!=' /var/run/nscd ']'
2017/7/13 上午6:40:18+ '[' /run/docker/netns/7109d660bb2c '!=' /run/nscd ']'
2017/7/13 上午6:40:18+ umount /run/docker/netns/7109d660bb2c
2017/7/13 上午6:40:18+ for m in '$(tac /proc/mounts | awk '\''{print $2}'\'' | grep ^${i}/)'
2017/7/13 上午6:40:18+ '[' /run/docker/netns/8efa050ecb88 '!=' /var/run/nscd ']'
2017/7/13 上午6:40:18+ '[' /run/docker/netns/8efa050ecb88 '!=' /run/nscd ']'
2017/7/13 上午6:40:18+ umount /run/docker/netns/8efa050ecb88
2017/7/13 上午6:40:18+ for m in '$(tac /proc/mounts | awk '\''{print $2}'\'' | grep ^${i}/)'
2017/7/13 上午6:40:18+ '[' /run/docker/netns/b99a87ff2a3c '!=' /var/run/nscd ']'
2017/7/13 上午6:40:18+ '[' /run/docker/netns/b99a87ff2a3c '!=' /run/nscd ']'
2017/7/13 上午6:40:18+ umount /run/docker/netns/b99a87ff2a3c
2017/7/13 上午6:40:18+ for m in '$(tac /proc/mounts | awk '\''{print $2}'\'' | grep ^${i}/)'
2017/7/13 上午6:40:18+ '[' /run/docker/netns/c4ea2298431b '!=' /var/run/nscd ']'
2017/7/13 上午6:40:18+ '[' /run/docker/netns/c4ea2298431b '!=' /run/nscd ']'
2017/7/13 上午6:40:18+ umount /run/docker/netns/c4ea2298431b
2017/7/13 上午6:40:18+ for m in '$(tac /proc/mounts | awk '\''{print $2}'\'' | grep ^${i}/)'
2017/7/13 上午6:40:18+ '[' /run/docker/netns/84d1c94cfb53 '!=' /var/run/nscd ']'
2017/7/13 上午6:40:18+ '[' /run/docker/netns/84d1c94cfb53 '!=' /run/nscd ']'
2017/7/13 上午6:40:18+ umount /run/docker/netns/84d1c94cfb53
2017/7/13 上午6:40:18+ for m in '$(tac /proc/mounts | awk '\''{print $2}'\'' | grep ^${i}/)'
2017/7/13 上午6:40:18+ '[' /run/docker/netns/3868dc64ddc8 '!=' /var/run/nscd ']'
2017/7/13 上午6:40:18+ '[' /run/docker/netns/3868dc64ddc8 '!=' /run/nscd ']'
2017/7/13 上午6:40:18+ umount /run/docker/netns/3868dc64ddc8
2017/7/13 上午6:40:18+ for m in '$(tac /proc/mounts | awk '\''{print $2}'\'' | grep ^${i}/)'
2017/7/13 上午6:40:18+ '[' /run/docker/netns/f859f58003cf '!=' /var/run/nscd ']'
2017/7/13 上午6:40:18+ '[' /run/docker/netns/f859f58003cf '!=' /run/nscd ']'
2017/7/13 上午6:40:18+ umount /run/docker/netns/f859f58003cf
2017/7/13 上午6:40:18+ for m in '$(tac /proc/mounts | awk '\''{print $2}'\'' | grep ^${i}/)'
2017/7/13 上午6:40:18+ '[' /run/docker/netns/07241a32e1d1 '!=' /var/run/nscd ']'
2017/7/13 上午6:40:18+ '[' /run/docker/netns/07241a32e1d1 '!=' /run/nscd ']'
2017/7/13 上午6:40:18+ umount /run/docker/netns/07241a32e1d1
2017/7/13 上午6:40:18+ for m in '$(tac /proc/mounts | awk '\''{print $2}'\'' | grep ^${i}/)'
2017/7/13 上午6:40:18+ '[' /run/docker/netns/da62a990711d '!=' /var/run/nscd ']'
2017/7/13 上午6:40:18+ '[' /run/docker/netns/da62a990711d '!=' /run/nscd ']'
2017/7/13 上午6:40:18+ umount /run/docker/netns/da62a990711d
2017/7/13 上午6:40:18+ for m in '$(tac /proc/mounts | awk '\''{print $2}'\'' | grep ^${i}/)'
2017/7/13 上午6:40:18+ '[' /run/docker/netns/ddb61c229ca1 '!=' /var/run/nscd ']'
2017/7/13 上午6:40:18+ '[' /run/docker/netns/ddb61c229ca1 '!=' /run/nscd ']'
2017/7/13 上午6:40:18+ umount /run/docker/netns/ddb61c229ca1
2017/7/13 上午6:40:18+ for m in '$(tac /proc/mounts | awk '\''{print $2}'\'' | grep ^${i}/)'
2017/7/13 上午6:40:18+ '[' /run/docker/netns/default '!=' /var/run/nscd ']'
2017/7/13 上午6:40:18+ '[' /run/docker/netns/default '!=' /run/nscd ']'
2017/7/13 上午6:40:18+ umount /run/docker/netns/default
2017/7/13 上午6:40:18+ for i in '$(DOCKER_API_VERSION=1.22 ./docker info 2>&1 | grep -i '\''docker root dir'\'' | cut -f2 -d:)' /var/lib/docker /run /var/run
2017/7/13 上午6:40:18++ grep '^/var/run/'
2017/7/13 上午6:40:18++ awk '{print $2}'
2017/7/13 上午6:40:18++ tac /proc/mounts
2017/7/13 上午6:40:18+ mount --rbind /host/dev /dev
2017/7/13 上午6:40:18++ hostname --fqdn
2017/7/13 上午6:40:18+ FQDN=node7
2017/7/13 上午6:40:18+ exec kubelet --kubeconfig=/etc/kubernetes/ssl/kubeconfig --api_servers=https://kubernetes.kubernetes.rancher.internal:6443 --allow-privileged=true --register-node=true --cloud-provider=rancher --healthz-bind-address=0.0.0.0 --cluster-dns=10.43.0.10 --cluster-domain=cluster.local --network-plugin=cni --network-plugin-dir=/etc/cni/managed.d --pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/rancher-cn/pause-amd64:3.0 --hostname-override node7
2017/7/13 上午6:40:18Flag --api-servers has been deprecated, Use --kubeconfig instead. Will be removed in a future version.
2017/7/13 上午6:40:18I0712 22:40:18.782683 13267 feature_gate.go:189] feature gates: map[]
2017/7/13 上午6:40:18I0712 22:40:18.827933 13267 docker.go:356] Connecting to docker on unix:///var/run/docker.sock
2017/7/13 上午6:40:18I0712 22:40:18.828020 13267 docker.go:376] Start docker client with request timeout=2m0s
2017/7/13 上午6:40:18I0712 22:40:18.833149 13267 manager.go:143] cAdvisor running in container: "/docker/7fa09c5ba6ab3f7df5cb7cd791fdae47a83060ebad60ee7bb21ebf44e9d52d0b"
2017/7/13 上午6:40:18W0712 22:40:18.840160 13267 manager.go:151] unable to connect to Rkt api service: rkt: cannot tcp Dial rkt api service: dial tcp 127.0.0.1:15441: getsockopt: connection refused
2017/7/13 上午6:40:18I0712 22:40:18.847451 13267 fs.go:117] Filesystem partitions: map[/dev/sda1:{mountpoint:/etc/resolv.conf major:8 minor:1 fsType:ext4 blockSize:0} none:{mountpoint:/ major:0 minor:82 fsType:aufs blockSize:0}]
2017/7/13 上午6:40:18I0712 22:40:18.848766 13267 manager.go:198] Machine: {NumCores:1 CpuFrequency:3392290 MemoryCapacity:2099503104 MachineID:b07a180a2c8547f7956e9a6f93a452a4 SystemUUID:9E8D277C-FEF4-48E0-9741-9FB1ECC683E3 BootID:e866b615-f8bc-46ee-ac82-8af25d1190c8 Filesystems:[{Device:/dev/sda1 Capacity:19195224064 Type:vfs Inodes:2436448 HasInodes:true} {Device:none Capacity:19195224064 Type:vfs Inodes:2436448 HasInodes:true}] DiskMap:map[8:0:{Name:sda Major:8 Minor:0 Size:20971520000 Scheduler:deadline} 251:0:{Name:zram0 Major:251 Minor:0 Size:465960960 Scheduler:none}] NetworkDevices:[{Name:dummy0 MacAddress:96:71:31:d4:7b:5c Speed:0 Mtu:1500} {Name:eth0 MacAddress:08:00:27:f9:68:a3 Speed:1000 Mtu:1500} {Name:eth1 MacAddress:08:00:27:4b:59:4d Speed:1000 Mtu:1500}] Topology:[{Id:0 Memory:0 Cores:[{Id:0 Threads:[0] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:262144 Type:Unified Level:2}]}] Caches:[{Size:8388608 Type:Unified Level:3}]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
2017/7/13 上午6:40:18I0712 22:40:18.849257 13267 manager.go:204] Version: {KernelVersion:4.4.41-boot2docker ContainerOsVersion:Debian GNU/Linux 8 (jessie) DockerVersion:1.12.6 CadvisorVersion: CadvisorRevision:}
2017/7/13 上午6:40:18W0712 22:40:18.851434 13267 container_manager_linux.go:205] Running with swap on is not supported, please disable swap! This will be a fatal error by default starting in K8s v1.6! In the meantime, you can opt-in to making this a fatal error by enabling --experimental-fail-swap-on.
2017/7/13 上午6:40:18I0712 22:40:18.851598 13267 kubelet.go:252] Watching apiserver
2017/7/13 上午6:40:18W0712 22:40:18.852846 13267 kubelet_network.go:69] Hairpin mode set to "promiscuous-bridge" but kubenet is not enabled, falling back to "hairpin-veth"
2017/7/13 上午6:40:18I0712 22:40:18.852891 13267 kubelet.go:477] Hairpin mode set to "hairpin-veth"
2017/7/13 上午6:40:18I0712 22:40:18.863198 13267 docker_manager.go:256] Setting dockerRoot to /mnt/sda1/var/lib/docker
2017/7/13 上午6:40:18I0712 22:40:18.863250 13267 docker_manager.go:259] Setting cgroupDriver to cgroupfs
2017/7/13 上午6:40:18I0712 22:40:18.865486 13267 server.go:770] Started kubelet v1.5.4-rancher1
2017/7/13 上午6:40:18E0712 22:40:18.867115 13267 kubelet.go:1145] Image garbage collection failed: unable to find data for container /
2017/7/13 上午6:40:18I0712 22:40:18.867272 13267 kubelet_node_status.go:204] Setting node annotation to enable volume controller attach/detach
2017/7/13 上午6:40:18I0712 22:40:18.867295 13267 rancher.go:641] ExternalID [node7]
2017/7/13 上午6:40:18I0712 22:40:18.867347 13267 rancher.go:648] InstanceID [node7]
2017/7/13 上午6:40:18I0712 22:40:18.867566 13267 server.go:123] Starting to listen on 0.0.0.0:10250
2017/7/13 上午6:40:18I0712 22:40:18.905081 13267 rancher.go:648] InstanceID [node7]
2017/7/13 上午6:40:18I0712 22:40:18.939183 13267 rancher.go:648] InstanceID [node7]
2017/7/13 上午6:40:18I0712 22:40:18.951685 13267 kubelet_node_status.go:246] Adding node label from cloud provider: beta.kubernetes.io/instance-type=rancher
2017/7/13 上午6:40:18I0712 22:40:18.952368 13267 kubelet_node_status.go:257] Adding node label from cloud provider: failure-domain.beta.kubernetes.io/zone=FailureDomain1
2017/7/13 上午6:40:18I0712 22:40:18.952534 13267 kubelet_node_status.go:261] Adding node label from cloud provider: failure-domain.beta.kubernetes.io/region=Region1
2017/7/13 上午6:40:18E0712 22:40:18.961547 13267 kubelet.go:1634] Failed to check if disk space is available for the runtime: failed to get fs info for "runtime": unable to find data for container /
2017/7/13 上午6:40:18E0712 22:40:18.961617 13267 kubelet.go:1642] Failed to check if disk space is available on the root partition: failed to get fs info for "root": error trying to get filesystem Device for dir /var/lib/kubelet: err: could not find device with major: 0, minor: 15 in cached partitions map
2017/7/13 上午6:40:18I0712 22:40:18.964391 13267 fs_resource_analyzer.go:66] Starting FS ResourceAnalyzer
2017/7/13 上午6:40:18I0712 22:40:18.964512 13267 status_manager.go:129] Starting to sync pod status with apiserver
2017/7/13 上午6:40:18I0712 22:40:18.964534 13267 kubelet.go:1714] Starting kubelet main sync loop.
2017/7/13 上午6:40:18I0712 22:40:18.964561 13267 kubelet.go:1725] skipping pod synchronization - [container runtime is down]
2017/7/13 上午6:40:18I0712 22:40:18.964920 13267 volume_manager.go:242] Starting Kubelet Volume Manager
2017/7/13 上午6:40:18I0712 22:40:18.979653 13267 factory.go:295] Registering Docker factory
Unknown Date
2017/7/13 上午6:40:18W0712 22:40:18.981216 13267 manager.go:247] Registration of the rkt container factory failed: unable to communicate with Rkt api service: rkt: cannot tcp Dial rkt api service: dial tcp 127.0.0.1:15441: getsockopt: connection refused
2017/7/13 上午6:40:18I0712 22:40:18.981253 13267 factory.go:54] Registering systemd factory
2017/7/13 上午6:40:18I0712 22:40:18.982168 13267 factory.go:86] Registering Raw factory
2017/7/13 上午6:40:18I0712 22:40:18.982701 13267 manager.go:1106] Started watching for new ooms in manager
2017/7/13 上午6:40:18I0712 22:40:18.984232 13267 oomparser.go:185] oomparser using systemd
2017/7/13 上午6:40:18I0712 22:40:18.984778 13267 manager.go:288] Starting recovery of all containers
2017/7/13 上午6:40:18I0712 22:40:18.984859 13267 manager.go:293] Recovery completed
2017/7/13 上午6:40:18F0712 22:40:18.984885 13267 kubelet.go:1210] Failed to start cAdvisor inotify_add_watch /var/lib/docker/aufs/mnt/e23bc5b9c67056aa3f611c92af3f73dadf4512b4c2ce77e7ff7aaa3d5ae23bfb/sys/fs/cgroup/cpuset: no such file or directory