1 k8s kubeadm搭建
1.1 k8s kubeadm搭建步骤
kubeadm init
在使用kubeadm方式安装k8s集群是，可根据初始化配置文件或配置参数选项快速的初始化生成一个k8s的master管理平台
kubeadm join
根据kubadm init初始化的提示信息快速的将一个node节点或其他的master节点加入到k8s集群里
1）所有节点进行初始化，安装容器引擎、kubeadm、kubelet
2）执行 kubeadm config print init-defaults 命令生成K8S集群初始化配置文件并进行修改
3）执行 kubeadm init --config 指定初始化配置文件进行K8S集群的初始化，生成K8S的master管理控制节点
4）在其它节点执行 kubeadm join 命令将node节点或其它master节点加入到K8S集群里
5）安装 cni 网络插件（flannel或calico）
1.2 如何更新kubeadm搭建的K8S的证书有效期
1）备份好证书和kubeconfig配置文件
mkdir /root/k8s.bak cp -r /etc/kubernetes/pki /root/k8s.bak/ cp /etc/kubernetes/*.yaml /root/k8s.bak/
2）更新证书
kubeadm alpha certs renew all --config=/opt/k8s/kubeadm-config.yaml
3）更新kubeconfig配置文件
kubeadm init phase kubeconfig all --config=/opt/k8s/kubeadm-config.yaml
4）重启kubelet进程和其它K8S组件的Pod
systemctl restart kubelet mv /etc/kubernetes/manifests/.yaml /tmp mv /tmp/.yaml /etc/kubernetes/manifests/
5）查看证书有效期
kubeadm alpha certs check-expiration openssl x509 -noout -dates -in /etc/kubernetes/pki/XXX.crt
2 kubeadm搭建高可用k8s实际部署
Kubeadm - K8S1.20 - 高可用集群部署
注意事项：
master节点cpu核心数要求大于2
●最新的版本不一定好，但相对于旧版本，核心功能稳定，但新增功能、接口相对不稳
●学会一个版本的 高可用部署，其他版本操作都差不多
●宿主机尽量升级到CentOS 7.9
●内核kernel升级到 4.19+ 这种稳定的内核
●部署k8s版本时，尽量找 1.xx.5 这种大于5的小版本（这种一般是比较稳定的版本）
2.1 初始化环境准备
所有节点，关闭防火墙规则，关闭selinux，关闭swap交换
systemctl stop firewalld systemctl disable firewalld setenforce 0 sed -i 's/enforcing/disabled/' /etc/selinux/config iptables -F && iptables -t nat -F && iptables -t mangle -F && iptables -X swapoff -a sed -ri 's/.*swap.*/#&/' /etc/fstab
修改主机名，所有节点修改hosts文件
hostnamectl set-hostname master01 hostnamectl set-hostname master02 hostnamectl set-hostname master03 hostnamectl set-hostname node01 hostnamectl set-hostname node02 hostnamectl set-hostname node03 vim /etc/hosts 192.168.111.5 master01 192.168.111.6 master02 192.168.111.7 master03 192.168.111.8 node01 192.168.111.9 node02 192.168.111.55 node03
所有节点时间同步
yum -y install ntpdate ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime echo 'Asia/Shanghai' >/etc/timezone ntpdate time2.aliyun.com systemctl enable --now crond crontab -e */30 * * * * /usr/sbin/ntpdate time2.aliyun.com
所有节点实现Linux的资源限制
vim /etc/security/limits.conf * soft nofile 65536 * hard nofile 131072 * soft nproc 65535 * hard nproc 655350 * soft memlock unlimited * hard memlock unlimited
所有节点升级内核
wget http://193.49.22.109/elrepo/kernel/el7/x86_64/RPMS/kernel-ml-devel-4.19.12-1.el7.elrepo.x86_64.rpm -O /opt/kernel-ml-devel-4.19.12-1.el7.elrepo.x86_64.rpm wget http://193.49.22.109/elrepo/kernel/el7/x86_64/RPMS/kernel-ml-4.19.12-1.el7.elrepo.x86_64.rpm -O /opt/kernel-ml-4.19.12-1.el7.elrepo.x86_64.rpm cd /opt/ yum localinstall -y kernel-ml*
更改内核启动方式
grub2-set-default 0 && grub2-mkconfig -o /etc/grub2.cfg grubby --args="user_namespace.enable=1" --update-kernel="$(grubby --default-kernel)" grubby --default-kernel reboot
调整内核参数
调整内核参数 cat > /etc/sysctl.d/k8s.conf <<EOF net.ipv4.ip_forward = 1 net.bridge.bridge-nf-call-iptables = 1 net.bridge.bridge-nf-call-ip6tables = 1 fs.may_detach_mounts = 1 vm.overcommit_memory=1 vm.panic_on_oom=0 fs.inotify.max_user_watches=89100 fs.file-max=52706963 fs.nr_open=52706963 net.netfilter.nf_conntrack_max=2310720 net.ipv4.tcp_keepalive_time = 600 net.ipv4.tcp_keepalive_probes = 3 net.ipv4.tcp_keepalive_intvl =15 net.ipv4.tcp_max_tw_buckets = 36000 net.ipv4.tcp_tw_reuse = 1 net.ipv4.tcp_max_orphans = 327680 net.ipv4.tcp_orphan_retries = 3 net.ipv4.tcp_syncookies = 1 net.ipv4.tcp_max_syn_backlog = 16384 net.ipv4.ip_conntrack_max = 65536 net.ipv4.tcp_max_syn_backlog = 16384 net.ipv4.tcp_timestamps = 0 net.core.somaxconn = 16384 EOF #生效参数 sysctl --system 加载 ip_vs 模块 for i in $(ls /usr/lib/modules/$(uname -r)/kernel/net/netfilter/ipvs|grep -o "^[^.]*");do echo $i; /sbin/modinfo -F filename $i >/dev/null 2>&1 && /sbin/modprobe $i;done
2.2 所有节点安装docker
yum install -y yum-utils device-mapper-persistent-data lvm2
yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum install -y docker-ce docker-ce-cli containerd.io
mkdir /etc/docker
cat > /etc/docker/daemon.json <<EOF
{
"registry-mirrors": ["https://6ijb8ubo.mirror.aliyuncs.com"],
"exec-opts": ["native.cgroupdriver=systemd"],
"log-driver": "json-file",
"log-opts": {
"max-size": "500m", "max-file": "3"
}
}
EOF
systemctl daemon-reload
systemctl restart docker.service
systemctl enable docker.service
docker info | grep "Cgroup Driver"
Cgroup Driver: systemd
2.3 所有节点安装kubeadm，kubelet和kubectl
定义kubernetes源 cat > /etc/yum.repos.d/kubernetes.repo << EOF [kubernetes] name=Kubernetes baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64 enabled=1 gpgcheck=0 repo_gpgcheck=0 gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg EOF yum install -y kubelet-1.20.15 kubeadm-1.20.15 kubectl-1.20.15 配置Kubelet使用阿里云的pause镜像 cat > /etc/sysconfig/kubelet <<EOF KUBELET_EXTRA_ARGS="--cgroup-driver=systemd --pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google_containers/pause-amd64:3.2" EOF 开机自启kubelet systemctl enable --now kubelet
2.4 使用nginx结合keepalived实现四层代理高可用组件安装、配置
两台高可用节点安装配置nginx四层代理转发
yum -y install nginx
user nginx;
worker_processes auto;
vim /etc/nginx/nginx.conf
events {
worker_connections 1024;
}
stream {
upstream k8s-apiservers {
server 192.168.111.5:6443;
server 192.168.111.6:6443;
server 192.168.111.7:6443;
}
server {
listen 6443;
proxy_pass k8s-apiservers;
}
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
#tcp_nopush on;
keepalive_timeout 65;
#gzip on;
include /etc/nginx/conf.d/*.conf;
}
systemctl enable --now nginx
安装配置keepalived实现高可用转发
yum -y install keepalived
编辑nginx服务健康检查脚本
vim check_nginx.sh
#!/bin/bash
if ! killall -0 nginx
then
systemctl stop keepalived
fi
配置keepalived配置文件
vim keepalived.conf
! Configuration File for keepalived
notification_email {
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id NGINX_02
}
vrrp_script check_nginx {
interval 2
script "/etc/keepalived/check_nginx.sh"
}
vrrp_instance VI_1 {
state MASTER
interface ens33
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.111.100
}
track_script {
check_nginx
}
}
使用ip a检测maser地址是否生成master
ip a
关闭master节点的nginx服务查看vip地址是否会漂移到BACKUP
systemctl stpo nginx
从节点上查看
ip a
2.5 部署K8S集群
在 master01 节点上设置集群初始化配置文件
kubeadm config print init-defaults > /opt/kubeadm-config.yaml
cd /opt/
vim kubeadm-config.yaml
......
11 localAPIEndpoint:
12 advertiseAddress: 192.168.80.10 #指定当前master节点的IP地址
13 bindPort: 6443
21 apiServer:
22 certSANs: #在apiServer属性下面添加一个certsSANs的列表，添加所有master节点的IP地址和集群VIP地址
23 - 192.168.111.100
24 - 192.168.111.5
25 - 192.168.111.6
26 - 192.168.111.7
30 clusterName: kubernetes
31 controlPlaneEndpoint: "192.168.80.100:6443" #指定集群VIP地址
32 controllerManager: {}
38 imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers #指定镜像下载地址
39 kind: ClusterConfiguration
40 kubernetesVersion: v1.20.15 #指定kubernetes版本号
41 networking:
42 dnsDomain: cluster.local
43 podSubnet: "10.244.0.0/16" #指定pod网段，10.244.0.0/16用于匹配flannel默认网段
44 serviceSubnet: 10.96.0.0/16 #指定service网段
45 scheduler: {}
#末尾再添加以下内容
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
mode: ipvs #把默认的kube-proxy调度方式改为ipvs模式
补充：可以配置外置的etcd网段，只能二选一
etcd：
external:
endpoints:
- https://192.168.80.10:2379
- https://192.168.80.11:2379
- https://192.168.80.12:2379
- caFile:/opt/etcd/ssl/ca.pem
- certFile:/opt/etcd/ssl/server.pem
- keyFile:/opt/etcd/ssl/server-key.pem
所有节点拉取镜像
for i in master02 master03; do scp /opt/kubeadm-config.yaml $i:/opt/; done #node节点的镜像加入集群后会自动拉取 kubeadm config images pull --config /opt/kubeadm-config.yaml docker images
master01 节点进行初始化
kubeadm init --config kubeadm-config.yaml --upload-certs | tee kubeadm-init.log
kubeadm init --config kubeadm-config.yaml --upload-certs | tee kubeadm-init.log
[init] Using Kubernetes version: v1.20.15
[preflight] Running pre-flight checks
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 25.0.3. Latest validated version: 19.03
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local master01] and IPs [10.96.0.1 192.168.111.5 192.168.111.100 192.168.111.6 192.168.111.7]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost master01] and IPs [192.168.111.5 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost master01] and IPs [192.168.111.5 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 12.801874 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.20" in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Storing the certificates in Secret "kubeadm-certs" in the "kube-system" Namespace
[upload-certs] Using certificate key:
4e923d767813d1f53185722eca61fb44ee2ba4336c372c9cae4f96b2c3a66e7c
[mark-control-plane] Marking the node master01 as control-plane by adding the labels "node-role.kubernetes.io/master=''" and "node-role.kubernetes.io/control-plane='' (deprecated)"
[mark-control-plane] Marking the node master01 as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
[bootstrap-token] Using token: abcdef.0123456789abcdef
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to get nodes
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Alternatively, if you are the root user, you can run:
export KUBECONFIG=/etc/kubernetes/admin.conf
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
You can now join any number of the control-plane node running the following command on each as root:
kubeadm join 192.168.111.100:6443 --token abcdef.0123456789abcdef \
--discovery-token-ca-cert-hash sha256:4f6e7fe908d8dcda265a26a95a5db9f0c8d64d04bd3a31b54964b3488daa8320 \
--control-plane --certificate-key 4e923d767813d1f53185722eca61fb44ee2ba4336c372c9cae4f96b2c3a66e7c
Please note that the certificate-key gives access to cluster sensitive data, keep it secret!
As a safeguard, uploaded-certs will be deleted in two hours; If necessary, you can use
"kubeadm init phase upload-certs --upload-certs" to reload certs afterward.
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 192.168.111.100:6443 --token abcdef.0123456789abcdef \
--discovery-token-ca-cert-hash sha256:4f6e7fe908d8dcda265a26a95a5db9f0c8d64d04bd3a31b54964b3488daa8320
#若初始化失败，进行的操作
kubeadm reset -f
ipvsadm --clear
rm -rf ~/.kube
再次进行初始化
master01 节点进行环境配置
设定kubectl kubectl需经由API server认证及授权后方能执行相应的管理操作，kubeadm 部署的集群为其生成了一个具有管理员权限的认证配置文件 /etc/kubernetes/admin.conf，它可由 kubectl 通过默认的 “$HOME/.kube/config” 的路径进行加载。 mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config 修改controller-manager和scheduler配置文件 vim /etc/kubernetes/manifests/kube-scheduler.yaml vim /etc/kubernetes/manifests/kube-controller-manager.yaml
所有节点加入集群
master 节点加入集群
kubeadm join 192.168.111.100:6443 --token abcdef.0123456789abcdef \
--discovery-token-ca-cert-hash sha256:4f6e7fe908d8dcda265a26a95a5db9f0c8d64d04bd3a31b54964b3488daa8320 \
--control-plane --certificate-key 4e923d767813d1f53185722eca61fb44ee2ba4336c372c9cae4f96b2c3a66e7c
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
#node 节点加入集群
kubeadm join 192.168.80.100:16443 --token 7t2weq.bjbawausm0jaxury \
--discovery-token-ca-cert-hash sha256:e76e4525ca29a9ccd5c24142a724bdb6ab86512420215242c4313fb830a4eb98
在 master01 查看集群信息
kubectl get nodes
部署cni网络插件flannel
所有节点上传 flannel 镜像 flannel.tar 和网络插件 cni-plugins-linux-amd64-v0.8.6.tgz 到 /opt 目录，master节点上传 kube-flannel.yml 文件 cd /opt docker load < flannel.tar mv /opt/cni /opt/cni_bak mkdir -p /opt/cni/bin tar zxvf cni-plugins-linux-amd64-v0.8.6.tgz -C /opt/cni/bin kubectl apply -f kube-flannel.yml kubectl get pods -A 等所有组件状态都为running再查看nodes节点状态
测试 pod 资源创建
kubectl create deployment nginx --image=nginx kubectl get pods -o wide
暴露端口提供服务
暴露端口提供服务 kubectl expose deployment nginx --port=80 --type=NodePort kubectl get svc 测试访问 curl http://node01:32698
扩展3个副本，测试是否可以负载均衡
kubectl scale deployment nginx --replicas=3 kubectl get pods -o wide
2.6 kubeadm安装补充
初始化kubeadm的其他方法
kubeadm init \ --apiserver-advertise-address=192.168.111.10 \ --image-repository registry.aliyuncs.com/google_containers \ --kubernetes-version=v1.20.15 \ --service-cidr=10.96.0.0/16 \ --pod-network-cidr=10.244.0.0/16 \ --token-ttl=0 -------------------------------------------------------------------------------------------- 初始化集群需使用kubeadm init命令，可以指定具体参数初始化，也可以指定配置文件初始化。 可选参数： --apiserver-advertise-address：apiserver通告给其他组件的IP地址，一般应该为Master节点的用于集群内部通信的IP地址，0.0.0.0表示节点上所有可用地址 --apiserver-bind-port：apiserver的监听端口，默认是6443 --cert-dir：通讯的ssl证书文件，默认/etc/kubernetes/pki --control-plane-endpoint：控制台平面的共享终端，可以是负载均衡的ip地址或者dns域名，高可用集群时需要添加 --image-repository：拉取镜像的镜像仓库，默认是k8s.gcr.io --kubernetes-version：指定kubernetes版本 --pod-network-cidr：pod资源的网段，需与pod网络插件的值设置一致。Flannel网络插件的默认为10.244.0.0/16，Calico插件的默认值为192.168.0.0/16； --service-cidr：service资源的网段 --service-dns-domain：service全域名的后缀，默认是cluster.local --token-ttl：默认token的有效期为24小时，如果不想过期，可以加上 --token-ttl=0 这个参数
方法二初始化后需要修改 kube-proxy 的 configmap，开启 ipvs
kubectl edit configmap kube-proxy -n kube-system
2.7 常见的k8s问题
1）加入集群的 Token 过期
注意：Token值在集群初始化后，有效期为 24小时 ，过了24小时过期。进行重新生成Token，再次加入集群，新生成的Token为 2小时。
1.1、生成Node节点加入集群的 Token
kubeadm token create --print-join-command kubeadm join 192.168.80.100:16443 --token menw99.1hbsurvl5fiz119n --discovery-token-ca-cert-hash sha256:e76e4525ca29a9ccd5c24142a724bdb6ab865 12420215242c4313fb830a4eb98
1.2、生成Master节点加入集群的 --certificate-key
kubeadm init phase upload-certs --upload-certs I1105 12:33:08.201601 93226 version.go:254] remote version is much newer: v1.22.3; falling back to: stable-1.20 [upload-certs] Storing the certificates in Secret "kubeadm-certs" in the "kube-system" Namespace [upload-certs] Using certificate key: 38dba94af7a38700c3698b8acdf8e23f273be07877f5c86f4977dc023e333deb
#master节点加入集群的命令 kubeadm join 192.168.80.100:16443 --token menw99.1hbsurvl5fiz119n --discovery-token-ca-cert-hash sha256:e76e4525ca29a9ccd5c24142a724bdb6ab86512420215242c4313fb830a4eb98 \ --control-plane --certificate-key 38dba94af7a38700c3698b8acdf8e23f273be07877f5c86f4977dc023e333deb
2）master节点 无法部署非系统Pod
解析：主要是因为master节点被加上污点，污点是不允许部署非系统 Pod，在 测试 环境，可以将污点去除，节省资源，可利用率。
2.1、查看污点
kubectl describe node -l node-role.kubernetes.io/master= | grep Taints Taints: node-role.kubernetes.io/master:NoSchedule Taints: node-role.kubernetes.io/master:NoSchedule Taints: node-role.kubernetes.io/master:NoSchedule
2.2、取消污点
kubectl taint node -l node-role.kubernetes.io/master node-role.kubernetes.io/master:NoSchedule- node/master01 untainted node/master02 untainted node/master03 untainted
kubectl describe node -l node-role.kubernetes.io/master= | grep Taints Taints: <none> Taints: <none> Taints: <none>
3）修改NodePort的默认端口
原理：默认k8s的使用端口的范围为30000左右，作为对外部提供的端口。我们也可以通过对配置文件的修改去指定默认的对外端口的范围。
报错
The Service "nginx-svc" is invalid: spec.ports[0].nodePort: Invalid value: 80: provided port is not in the valid range. The range of valid ports is 30000-32767
[root@k8s-master1 ~]# vim /etc/kubernetes/manifests/kube-apiserver.yaml
-
--service-cluster-ip-range=10.96.0.0/16
-
--service-node-port-range=1-65535 #找到后进行添加即可
无需重启，k8s会自动生效
4）外部 etcd 部署配置
kubeadm config print init-defaults > /opt/kubeadm-config.yaml
cd /opt/ vim kubeadm-config.yaml apiVersion: kubeadm.k8s.io/v1beta2 bootstrapTokens:
-
groups:
-
system:bootstrappers:kubeadm:default-node-token token: abcdef.0123456789abcdef ttl: 24h0m0s usages:
-
signing
-
authentication kind: InitConfiguration localAPIEndpoint: advertiseAddress: 192.168.80.14 bindPort: 6443 nodeRegistration: criSocket: /var/run/dockershim.sock name: master01 taints:
-
effect: NoSchedule key: node-role.kubernetes.io/master
-
apiServer: certSANs:
-
10.96.0.1
-
127.0.0.1
-
localhost
-
kubernetes
-
kubernetes.default
-
kubernetes.default.svc
-
kubernetes.default.svc.cluster.local
-
192.168.80.100
-
192.168.80.10
-
192.168.80.11
-
192.168.80.12
-
master01
-
master02
-
master03 timeoutForControlPlane: 4m0s apiVersion: kubeadm.k8s.io/v1beta2 certificatesDir: /etc/kubernetes/pki clusterName: kubernetes controlPlaneEndpoint: 192.168.80.100:16443 controllerManager: {} dns: type: CoreDNS etcd: external: #使用外部etcd的方式 endpoints:
-
https://192.168.80.10:2379
-
https://192.168.80.11:2379
-
https://192.168.80.12:2379 caFile: /opt/etcd/ssl/ca.pem #需要把etcd的证书都复制到所有master节点上 certFile: /opt/etcd/ssl/server.pem keyFile: /opt/etcd/ssl/server-key.pem imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers kind: ClusterConfiguration kubernetesVersion: v1.20.15 networking: dnsDomain: cluster.local podSubnet: "10.244.0.0/16" serviceSubnet: 10.96.0.0/16 scheduler: {}
-
apiVersion: kubeproxy.config.k8s.io/v1alpha1 kind: KubeProxyConfiguration mode: ipvs
3 k8s图形化界面
3.1 Dashboard安装
在 master01 节点上操作
#上传 recommended.yaml 文件到 /opt/k8s 目录中
cd /opt/k8s
vim recommended.yaml
#默认Dashboard只能集群内部访问，修改Service为NodePort类型，暴露到外部：
kind: Service
apiVersion: v1
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kubernetes-dashboard
spec:
ports:
- port: 443
targetPort: 8443
nodePort: 30001 #添加
type: NodePort #添加
selector:
k8s-app: kubernetes-dashboard
kubectl apply -f recommended.yaml
kubectl get pods -A
创建service account并绑定默认cluster-admin管理员集群角色
kubectl create serviceaccount dashboard-admin -n kube-system
kubectl create clusterrolebinding dashboard-admin --clusterrole=cluster-admin --serviceaccount=kube-system:dashboard-admin
kubectl describe secrets -n kube-system $(kubectl -n kube-system get secret | awk '/dashboard-admin/{print $1}')
浏览器访问测试
浏览器打不开的用360浏览器（很多浏览器证书不可用）
