如何将k8s中的某些节点单独、仅给某些应用来使用

时间:2022-06-01 12:38:37

1、概述

在k8s集群的使用场景中有这样的一种情况,某些机器只给某些特殊的应用来使用。那么,这个时候,需要有以下的2个条件来进行保障:

  • 节点不允许其他的pod来使用
  • 应用只允许被调度到该节点上

2、实现方法

我们如果要实现上述的目标,节点不被其他的pod应用来使用,那么将节点增加taints就可以,然后,pod在调度的时候有可能会被调度到其他的节点上,那么要保证pod只会被调度到这些的节点上,那么,在打了taints的节点上,在增加label即可。

下面是具体的实现的过程。

2.1、节点上增加taints和标签

kubectl taint nodes nccztsjb-node-23 role=master:NoSchedule

这样节点上就不允许没有toleration的pod运行

kubectl label nodes nccztsjb-node-23 dedicated=prod

2.2、pod上设置toleration和nodeSelector

apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-taints
namespace: default
spec:
progressDeadlineSeconds: 600
selector:
matchLabels:
app: nginx-taints
replicas: 5
template:
metadata:
labels:
app: nginx-taints
spec:
containers:
- image: 172.20.58.152/middleware/nginx:1.21.4
imagePullPolicy: IfNotPresent
name: nginx
dnsPolicy: ClusterFirst
restartPolicy: Always
tolerations:
- key: "role"
operator: "Exists"
effect: "NoSchedule"
nodeSelector:
dedicated: "prod"

toleration保证pod可以在这个节点上运行,nodeSelector保证pod只在有包含dedicated=prod的标签节点上运行。

运行结果:

kubectl apply -f nginx-taints.yaml

查看pod运行状态

[root@nccztsjb-node-23 ~]# kubectl get pod -l app=nginx-taints -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-taints-78b7978fd5-7sjm5 1/1 Running 0 5s 172.39.209.112 nccztsjb-node-23 <none> <none>
nginx-taints-78b7978fd5-97hg9 1/1 Running 0 3s 172.39.209.116 nccztsjb-node-23 <none> <none>
nginx-taints-78b7978fd5-bswrb 1/1 Running 0 5s 172.39.209.113 nccztsjb-node-23 <none> <none>
nginx-taints-78b7978fd5-lfwzm 1/1 Running 0 5s 172.39.209.114 nccztsjb-node-23 <none> <none>
nginx-taints-78b7978fd5-vxhfq 1/1 Running 0 3s 172.39.209.115 nccztsjb-node-23 <none> <none>
[root@nccztsjb-node-23 ~]#

pod的多个实例都运行在nccztsjb-node-23上了。

OK,以上是基本的配置过程。

如果#1:pod没有设置toleration

apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-taints
namespace: default
spec:
progressDeadlineSeconds: 600
selector:
matchLabels:
app: nginx-taints
replicas: 5
template:
metadata:
labels:
app: nginx-taints
spec:
containers:
- image: 172.20.58.152/middleware/nginx:1.21.4
imagePullPolicy: IfNotPresent
name: nginx
dnsPolicy: ClusterFirst
restartPolicy: Always
#tolerations:
#- key: "role"
# operator: "Exists"
# effect: "NoSchedule"
nodeSelector:
dedicated: "prod"

运行pod及查看结果

[root@nccztsjb-node-23 ~]# kubectl apply -f nginx-taints.yaml
deployment.apps/nginx-taints created
[root@nccztsjb-node-23 ~]# kubectl get pod -l app=nginx-taints -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-taints-7cfdd85578-67smg 0/1 Pending 0 1s <none> <none> <none> <none>
nginx-taints-7cfdd85578-877zb 0/1 Pending 0 1s <none> <none> <none> <none>
nginx-taints-7cfdd85578-nl8p6 0/1 Pending 0 1s <none> <none> <none> <none>
nginx-taints-7cfdd85578-qgf4t 0/1 Pending 0 1s <none> <none> <none> <none>
nginx-taints-7cfdd85578-vw987 0/1 Pending 0 1s <none> <none> <none> <none>
[root@nccztsjb-node-23 ~]#

都未被调度到节点上。

如果#2:节点上未设置nodeSelector

[root@nccztsjb-node-23 ~]# cat nginx-taints.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-taints
namespace: default
spec:
progressDeadlineSeconds: 600
selector:
matchLabels:
app: nginx-taints
replicas: 5
template:
metadata:
labels:
app: nginx-taints
spec:
containers:
- image: 172.20.58.152/middleware/nginx:1.21.4
imagePullPolicy: IfNotPresent
name: nginx
dnsPolicy: ClusterFirst
restartPolicy: Always
tolerations:
- key: "role"
operator: "Exists"
effect: "NoSchedule"
#nodeSelector:
# dedicated: "prod"

运行及查看pod的状态

[root@nccztsjb-node-23 ~]# kubectl apply -f nginx-taints.yaml
deployment.apps/nginx-taints created
[root@nccztsjb-node-23 ~]# kubectl get pod -l app=nginx-taints -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-taints-6cb85bb844-8ggsc 1/1 Running 0 3s 172.39.209.117 nccztsjb-node-23 <none> <none>
nginx-taints-6cb85bb844-flbf2 1/1 Running 0 3s 172.39.21.121 nccztsjb-node-25 <none> <none>
nginx-taints-6cb85bb844-gjlqm 1/1 Running 0 3s 172.39.21.120 nccztsjb-node-25 <none> <none>
nginx-taints-6cb85bb844-hrxfr 1/1 Running 0 3s 172.39.157.206 nccztsjb-node-24 <none> <none>
nginx-taints-6cb85bb844-q9vfk 1/1 Running 0 3s 172.39.157.201 nccztsjb-node-24 <none> <none>
[root@nccztsjb-node-23 ~]#

这样的结果就是pod可以在任意的节点上运行了,不仅仅是在nccztsjb-node-23节点上。