目录
| 服务器版本 | docker软件版本 | Kubernetes(k8s)集群版本 | CPU架构 |
|---|---|---|---|
| CentOS Linux release 7.4.1708 (Core) | Docker version 20.10.12 | v1.21.9 | x86_64 |
Kubernetes集群架构：k8scloude1作为master节点，k8scloude2，k8scloude3作为worker节点
| 服务器 | 操作系统版本 | CPU架构 | 进程 | 功能描述 |
|---|---|---|---|---|
| k8scloude1/192.168.110.130 | CentOS Linux release 7.4.1708 (Core) | x86_64 | docker，kube-apiserver，etcd，kube-scheduler，kube-controller-manager，kubelet，kube-proxy，coredns，calico | k8s master节点 |
| k8scloude2/192.168.110.129 | CentOS Linux release 7.4.1708 (Core) | x86_64 | docker，kubelet，kube-proxy，calico | k8s worker节点 |
| k8scloude3/192.168.110.128 | CentOS Linux release 7.4.1708 (Core) | x86_64 | docker，kubelet，kube-proxy，calico | k8s worker节点 |
二.前言
使用deployment的前提是已经有一套可以正常运行的Kubernetes集群，关于Kubernetes(k8s)集群的安装部署，可以查看博客《Centos7 安装部署Kubernetes(k8s)集群》https://www.cnblogs.com/renshengdezheli/p/16686769.html。
三.Kubernetes 控制器
在 Kubernetes 中，控制器通过监控集群 的公共状态，并致力于将当前状态转变为期望的状态。
一个控制器至少追踪一种类型的 Kubernetes 资源。这些 对象 有一个代表期望状态的 spec 字段。 该资源的控制器负责确保其当前状态接近期望状态。
控制器可能会自行执行操作；在 Kubernetes 中更常见的是一个控制器会发送信息给 API 服务器。
Kubernetes 内置一组控制器，运行在 kube-controller-manager 内。 这些内置的控制器提供了重要的核心功能。
Deployment 控制器和 Job 控制器是 Kubernetes 内置控制器的典型例子。 Kubernetes 允许你运行一个稳定的控制平面，这样即使某些内置控制器失败了， 控制平面的其他部分会接替它们的工作。
四.Deployment概览
一个 Deployment 为 Pod 和 ReplicaSet 提供声明式的更新能力。
你负责描述 Deployment 中的 目标状态，而 Deployment 控制器（Controller） 以受控速率更改实际状态， 使其变为期望状态。你可以定义 Deployment 以创建新的 ReplicaSet，或删除现有 Deployment， 并通过新的 Deployment 收养其资源。
注意：不要管理 Deployment 所拥有的 ReplicaSet 。
以下是 Deployments 的典型用例：
- 创建 Deployment 以将 ReplicaSet 上线。ReplicaSet 在后台创建 Pod。 检查 ReplicaSet 的上线状态，查看其是否成功。
- 通过更新 Deployment 的 PodTemplateSpec，声明 Pod 的新状态 。 新的 ReplicaSet 会被创建，Deployment 以受控速率将 Pod 从旧 ReplicaSet 迁移到新 ReplicaSet。 每个新的 ReplicaSet 都会更新 Deployment 的修订版本。
- 如果 Deployment 的当前状态不稳定，回滚到较早的 Deployment 版本。 每次回滚都会更新 Deployment 的修订版本。
- 扩大 Deployment 规模以承担更多负载。
- 暂停 Deployment 的上线 以应用对 PodTemplateSpec 所作的多项修改， 然后恢复其执行以启动新的上线版本。
- 使用 Deployment 状态来判定上线过程是否出现停滞。
- 清理较旧的不再需要的 ReplicaSet 。
五.创建deployment
创建存放deployment配置文件的目录和namespace
[root@k8scloude1 ~]# mkdir deployment
[root@k8scloude1 ~]# cd deployment/
[root@k8scloude1 deployment]# pwd
/root/deployment
#创建namespace
[root@k8scloude1 deployment]# kubectl create ns deployment
namespace/deployment created
#切换命名空间为deployment
[root@k8scloude1 deployment]# kubens deployment
Context "kubernetes-admin@kubernetes" modified.
Active namespace is "deployment".
[root@k8scloude1 deployment]# kubectl get pod
No resources found in deployment namespace.
Kubernetes里有很多简写，比如deployment简称deploy，pod简称po
查看pod
[root@k8scloude1 deployment]# kubectl get po
No resources found in deployment namespace.
查看对资源的操作有哪些
[root@k8scloude1 deployment]# kubectl --help | grep resource
create Create a resource from a file or from stdin.
get 显示一个或更多 resources
delete Delete resources by filenames, stdin, resources and names, or by resources and label selector
rollout Manage the rollout of a resource
describe 显示一个指定 resource 或者 group 的 resources 详情
patch Update field(s) of a resource
wait Experimental: Wait for a specific condition on one or many resources.
api-resources Print the supported API resources on the server
可以查看k8s支持的简写
[root@k8scloude1 deployment]# kubectl api-resources
NAME SHORTNAMES APIVERSION NAMESPACED KIND
bindings v1 true Binding
componentstatuses cs v1 false ComponentStatus
configmaps cm v1 true ConfigMap
endpoints ep v1 true Endpoints
events ev v1 true Event
limitranges limits v1 true LimitRange
namespaces ns v1 false Namespace
nodes no v1 false Node
persistentvolumeclaims pvc v1 true PersistentVolumeClaim
persistentvolumes pv v1 false PersistentVolume
pods po v1 true Pod
podtemplates v1 true PodTemplate
replicationcontrollers rc v1 true ReplicationController
resourcequotas quota v1 true ResourceQuota
secrets v1 true Secret
serviceaccounts sa v1 true ServiceAccount
services svc v1 true Service
mutatingwebhookconfigurations admissionregistration.k8s.io/v1 false MutatingWebhookConfiguration
validatingwebhookconfigurations admissionregistration.k8s.io/v1 false ValidatingWebhookConfiguration
customresourcedefinitions crd,crds apiextensions.k8s.io/v1 false CustomResourceDefinition
apiservices apiregistration.k8s.io/v1 false APIService
controllerrevisions apps/v1 true ControllerRevision
daemonsets ds apps/v1 true DaemonSet
deployments deploy apps/v1 true Deployment
replicasets rs apps/v1 true ReplicaSet
statefulsets sts apps/v1 true StatefulSet
tokenreviews authentication.k8s.io/v1 false TokenReview
localsubjectaccessreviews authorization.k8s.io/v1 true LocalSubjectAccessReview
selfsubjectaccessreviews authorization.k8s.io/v1 false SelfSubjectAccessReview
selfsubjectrulesreviews authorization.k8s.io/v1 false SelfSubjectRulesReview
subjectaccessreviews authorization.k8s.io/v1 false SubjectAccessReview
horizontalpodautoscalers hpa autoscaling/v1 true HorizontalPodAutoscaler
cronjobs cj batch/v1 true CronJob
jobs batch/v1 true Job
certificatesigningrequests csr certificates.k8s.io/v1 false CertificateSigningRequest
leases coordination.k8s.io/v1 true Lease
bgpconfigurations crd.projectcalico.org/v1 false BGPConfiguration
bgppeers crd.projectcalico.org/v1 false BGPPeer
blockaffinities crd.projectcalico.org/v1 false BlockAffinity
caliconodestatuses crd.projectcalico.org/v1 false CalicoNodeStatus
clusterinformations crd.projectcalico.org/v1 false ClusterInformation
felixconfigurations crd.projectcalico.org/v1 false FelixConfiguration
globalnetworkpolicies crd.projectcalico.org/v1 false GlobalNetworkPolicy
globalnetworksets crd.projectcalico.org/v1 false GlobalNetworkSet
hostendpoints crd.projectcalico.org/v1 false HostEndpoint
ipamblocks crd.projectcalico.org/v1 false IPAMBlock
ipamconfigs crd.projectcalico.org/v1 false IPAMConfig
ipamhandles crd.projectcalico.org/v1 false IPAMHandle
ippools crd.projectcalico.org/v1 false IPPool
ipreservations crd.projectcalico.org/v1 false IPReservation
kubecontrollersconfigurations crd.projectcalico.org/v1 false KubeControllersConfiguration
networkpolicies crd.projectcalico.org/v1 true NetworkPolicy
networksets crd.projectcalico.org/v1 true NetworkSet
endpointslices discovery.k8s.io/v1 true EndpointSlice
events ev events.k8s.io/v1 true Event
ingresses ing extensions/v1beta1 true Ingress
flowschemas flowcontrol.apiserver.k8s.io/v1beta1 false FlowSchema
prioritylevelconfigurations flowcontrol.apiserver.k8s.io/v1beta1 false PriorityLevelConfiguration
nodes metrics.k8s.io/v1beta1 false NodeMetrics
pods metrics.k8s.io/v1beta1 true PodMetrics
ingressclasses networking.k8s.io/v1 false IngressClass
ingresses ing networking.k8s.io/v1 true Ingress
networkpolicies netpol networking.k8s.io/v1 true NetworkPolicy
runtimeclasses node.k8s.io/v1 false RuntimeClass
poddisruptionbudgets pdb policy/v1 true PodDisruptionBudget
podsecuritypolicies psp policy/v1beta1 false PodSecurityPolicy
clusterrolebindings rbac.authorization.k8s.io/v1 false ClusterRoleBinding
clusterroles rbac.authorization.k8s.io/v1 false ClusterRole
rolebindings rbac.authorization.k8s.io/v1 true RoleBinding
roles rbac.authorization.k8s.io/v1 true Role
priorityclasses pc scheduling.k8s.io/v1 false PriorityClass
csidrivers storage.k8s.io/v1 false CSIDriver
csinodes storage.k8s.io/v1 false CSINode
csistoragecapacities storage.k8s.io/v1beta1 true CSIStorageCapacity
storageclasses sc storage.k8s.io/v1 false StorageClass
volumeattachments storage.k8s.io/v1 false VolumeAttachment
查看deployment控制器
[root@k8scloude1 deployment]# kubectl get deploy
No resources found in deployment namespace.
建议使用yaml文件创建deploy,命令行的方式，支持的选项比较少，
生成deployment的yaml文件
#nginx为deploy的名字 --image=nginx使用Nginx镜像
[root@k8scloude1 deployment]# kubectl create deploy nginx --image=nginx --dry-run=client -o yaml >nginx.yaml
[root@k8scloude1 deployment]# ll -h nginx.yaml
-rw-r--r-- 1 root root 384 1月 25 15:39 nginx.yaml
查看生成的deploy yaml配置文件
[root@k8scloude1 deployment]# cat nginx.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
creationTimestamp: null
labels:
app: nginx
name: nginx
spec:
replicas: 1
selector:
matchLabels:
app: nginx
strategy: {}
template:
metadata:
creationTimestamp: null
labels:
app: nginx
spec:
containers:
- image: nginx
name: nginx
resources: {}
status: {}
pod模板可以有多个标签，只要其中一个标签和matchLabels里的标签一致即可
[root@k8scloude1 deployment]# vim nginx.yaml
[root@k8scloude1 deployment]# cat nginx.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
creationTimestamp: null
#这个labels是deployment的标签
labels:
app: nginx
#deployment名字
name: nginx
spec:
#replicas: 1 表示副本数为1，只生成一个pod
replicas: 1
#selector标签选择器
selector:
matchLabels:
appx: xyz
strategy: {}
#template下面是pod的模板
template:
metadata:
creationTimestamp: null
#这个labels表示pod标签
labels:
#标签
app: nginx
appx: xyz
spec:
containers:
- image: nginx
name: nginx
resources: {}
status: {}
创建deploy
[root@k8scloude1 deployment]# kubectl apply -f nginx.yaml
deployment.apps/nginx created
[root@k8scloude1 deployment]# kubectl get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
nginx 0/1 1 0 11s
直接下载nginx镜像太慢，重新修改下yaml文件
[root@k8scloude1 deployment]# kubectl delete deploy nginx
deployment.apps "nginx" deleted
[root@k8scloude1 deployment]# vim nginx.yaml
[root@k8scloude1 deployment]# cat nginx.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
creationTimestamp: null
labels:
app: nginx
name: nginx
spec:
replicas: 1
selector:
matchLabels:
appx: xyz
strategy: {}
template:
metadata:
creationTimestamp: null
labels:
app: nginx
appx: xyz
spec:
#优雅删除pod的期限时间
terminationGracePeriodSeconds: 0
containers:
- image: nginx
name: nginx
#修改镜像下载策略：IfNotPresent表示本地有镜像就不下载
imagePullPolicy: IfNotPresent
resources: {}
status: {}
创建deploy
[root@k8scloude1 deployment]# kubectl apply -f nginx.yaml
deployment.apps/nginx created
查看deployment
[root@k8scloude1 deployment]# kubectl get deploy -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
nginx 1/1 1 1 11s nginx nginx appx=xyz
#--show-labels 显示标签
[root@k8scloude1 deployment]# kubectl get deploy -o wide --show-labels
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR LABELS
nginx 1/1 1 1 60s nginx nginx appx=xyz app=nginx
可以看到pod也被创建出来了
[root@k8scloude1 deployment]# kubectl get pod -o wide --show-labels
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES LABELS
nginx-6fcb7995f5-c8lpc 1/1 Running 0 2m25s 10.244.112.146 k8scloude2 <none> <none> app=nginx,appx=xyz,pod-template-hash=6fcb7995f5
注意：删除pod之后，deploy会根据副本数replicas，创建缺失的pod,所以要想真正删除pod，需要删除deployment控制器
[root@k8scloude1 deployment]# kubectl delete pod nginx-6fcb7995f5-c8lpc --force
warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
pod "nginx-6fcb7995f5-c8lpc" force deleted
#pod又重新创建了
[root@k8scloude1 deployment]# kubectl get pod -o wide --show-labels
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES LABELS
nginx-6fcb7995f5-s6smg 1/1 Running 0 5s 10.244.112.147 k8scloude2 <none> <none> app=nginx,appx=xyz,pod-template-hash=6fcb7995f5
六.修改deploy副本数
修改deployment副本数有三种方法：
- kubectl edit deploy deployname ，直接修改replicas的数目即可，实时生效，注意：可以在线修改deploy但是不能在线修改pod
- 可以直接修改yaml文件： 比如：replicas: 6
- kubectl scale deployment deployname 在线伸缩副本数
6.1 kubectl edit deploy 修改副本数
查看deploy
[root@k8scloude1 deployment]# kubectl get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
nginx 1/1 1 1 10m
第一种：kubectl edit deploy deployname ,直接修改replicas的数目即可，实时生效，注意：可以在线修改deploy但是不能在线修改pod
[root@k8scloude1 deployment]# kubectl edit deploy nginx
deployment.apps/nginx edited
查看deploy，可以发现副本数变了，READY为3/3
[root@k8scloude1 deployment]# kubectl get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
nginx 3/3 3 3 11m
[root@k8scloude1 deployment]# kubectl get deploy -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
nginx 3/3 3 3 11m nginx nginx appx=xyz
6.2 修改yaml文件更改副本数
第二种：可以直接修改yaml文件： replicas: 6，修改为6个副本
[root@k8scloude1 deployment]# vim nginx.yaml
[root@k8scloude1 deployment]# cat nginx.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
creationTimestamp: null
labels:
app: nginx
name: nginx
spec:
replicas: 6
selector:
matchLabels:
appx: xyz
strategy: {}
template:
metadata:
creationTimestamp: null
labels:
app: nginx
appx: xyz
spec:
terminationGracePeriodSeconds: 0
containers:
- image: nginx
name: nginx
imagePullPolicy: IfNotPresent
resources: {}
status: {}
kubectl edit可以立即生效，但是修改yaml文件需要apply才能生效
[root@k8scloude1 deployment]# kubectl get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
nginx 3/3 3 3 14m
[root@k8scloude1 deployment]# kubectl apply -f nginx.yaml
deployment.apps/nginx configured
[root@k8scloude1 deployment]# kubectl get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
nginx 6/6 6 6 15m
可以看到deploy生成了6个pod
[root@k8scloude1 deployment]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-6fcb7995f5-64gm2 1/1 Running 0 22s 10.244.251.204 k8scloude3 <none> <none>
nginx-6fcb7995f5-75jfd 1/1 Running 0 22s 10.244.251.206 k8scloude3 <none> <none>
nginx-6fcb7995f5-8c75n 1/1 Running 0 4m23s 10.244.251.205 k8scloude3 <none> <none>
nginx-6fcb7995f5-cs6w4 1/1 Running 0 4m23s 10.244.112.143 k8scloude2 <none> <none>
nginx-6fcb7995f5-s6smg 1/1 Running 0 11m 10.244.112.147 k8scloude2 <none> <none>
nginx-6fcb7995f5-tpwr4 1/1 Running 0 22s 10.244.112.148 k8scloude2 <none> <none>
6.3 kubectl scale修改副本数
kubectl scale在线伸缩deploy，--replicas=10修改副本数为10个
[root@k8scloude1 deployment]# kubectl scale deployment nginx --replicas=10
deployment.apps/nginx scaled
[root@k8scloude1 deployment]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-6fcb7995f5-5sv29 1/1 Running 0 4s 10.244.112.152 k8scloude2 <none> <none>
nginx-6fcb7995f5-64gm2 1/1 Running 0 2m12s 10.244.251.204 k8scloude3 <none> <none>
nginx-6fcb7995f5-75jfd 1/1 Running 0 2m12s 10.244.251.206 k8scloude3 <none> <none>
nginx-6fcb7995f5-8c75n 1/1 Running 0 6m13s 10.244.251.205 k8scloude3 <none> <none>
nginx-6fcb7995f5-cs6w4 1/1 Running 0 6m13s 10.244.112.143 k8scloude2 <none> <none>
nginx-6fcb7995f5-f6nz9 1/1 Running 0 4s 10.244.112.150 k8scloude2 <none> <none>
nginx-6fcb7995f5-hx224 1/1 Running 0 4s 10.244.112.149 k8scloude2 <none> <none>
nginx-6fcb7995f5-s6smg 1/1 Running 0 13m 10.244.112.147 k8scloude2 <none> <none>
nginx-6fcb7995f5-tpwr4 1/1 Running 0 2m12s 10.244.112.148 k8scloude2 <none> <none>
nginx-6fcb7995f5-zmsfx 1/1 Running 0 4s 10.244.251.208 k8scloude3 <none> <none>
修改pod副本为1
[root@k8scloude1 deployment]# kubectl scale deployment nginx --replicas=1
deployment.apps/nginx scaled
[root@k8scloude1 deployment]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-6fcb7995f5-8c75n 1/1 Running 0 6m33s 10.244.251.205 k8scloude3 <none> <none>
七.HorizontalPodAutoscaler(HPA)
7.1 Pod 水平自动扩缩HorizontalPodAutoscaler概览
在 Kubernetes 中，HorizontalPodAutoscaler 自动更新工作负载资源 （例如 Deployment 或者 StatefulSet）， 目的是自动扩缩工作负载以满足需求。
水平扩缩意味着对增加的负载的响应是部署更多的 Pod。 这与 “垂直（Vertical）” 扩缩不同，对于 Kubernetes， 垂直扩缩意味着将更多资源（例如：内存或 CPU）分配给已经为工作负载运行的 Pod。
如果负载减少，并且 Pod 的数量高于配置的最小值， HorizontalPodAutoscaler 会指示工作负载资源（Deployment、StatefulSet 或其他类似资源）缩减。
水平 Pod 自动扩缩不适用于无法扩缩的对象（例如：DaemonSet。）
HorizontalPodAutoscaler 被实现为 Kubernetes API 资源和控制器。
资源决定了控制器的行为。 在 Kubernetes 控制平面内运行的水平 Pod 自动扩缩控制器会定期调整其目标（例如：Deployment）的所需规模，以匹配观察到的指标， 例如，平均 CPU 利用率、平均内存利用率或你指定的任何其他自定义指标。
7.2 HPA自动伸缩pod副本数实战
HPA（horizontal pod autoscalers）水平自动伸缩 通过检测pod CPU的负载，解决deployment里某pod负 载太重，动态伸缩pod的数量来负载均衡。
现在deploy只生成了一个pod
[root@k8scloude1 deployment]# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-6fcb7995f5-8c75n 1/1 Running 0 45m
查看HPA
[root@k8scloude1 deployment]# kubectl get hpa
No resources found in deployment namespace.
设置HPA：--min=2 最小pod副本数为2，--max=5 最大pod副本数为5
[root@k8scloude1 deployment]# kubectl autoscale deployment nginx --max=5 --min=2
horizontalpodautoscaler.autoscaling/nginx autoscaled
观察pod副本数，发现pod副本数自动变为2了，因为HPA设置了最小pod副本数为2
[root@k8scloude1 deployment]# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-6fcb7995f5-8c75n 1/1 Running 0 49m
[root@k8scloude1 deployment]# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-6fcb7995f5-8c75n 1/1 Running 0 49m
[root@k8scloude1 deployment]# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-6fcb7995f5-8c75n 1/1 Running 0 49m
nginx-6fcb7995f5-zrdpj 1/1 Running 0 22s
[root@k8scloude1 deployment]# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-6fcb7995f5-8c75n 1/1 Running 0 49m
nginx-6fcb7995f5-zrdpj 1/1 Running 0 39s
使用kubectl scale 设置pod副本数为1
[root@k8scloude1 deployment]# kubectl scale deployment nginx --replicas=1
deployment.apps/nginx scaled
因为autoscale是实时扩展的，所以就算scale修改为1，autoscale也会自动把副本数修改为2
[root@k8scloude1 deployment]# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-6fcb7995f5-5jl6h 0/1 ContainerCreating 0 0s
nginx-6fcb7995f5-8c75n 1/1 Running 0 50m
[root@k8scloude1 deployment]# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-6fcb7995f5-5jl6h 1/1 Running 0 4s
nginx-6fcb7995f5-8c75n 1/1 Running 0 50m
[root@k8scloude1 deployment]# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-6fcb7995f5-5jl6h 1/1 Running 0 16s
nginx-6fcb7995f5-8c75n 1/1 Running 0 50m
查看hpa
[root@k8scloude1 deployment]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
nginx Deployment/nginx <unknown>/80% 2 5 2 4m18s
删除HPA
[root@k8scloude1 deployment]# kubectl delete hpa nginx
horizontalpodautoscaler.autoscaling "nginx" deleted
[root@k8scloude1 deployment]# kubectl get hpa
No resources found in deployment namespace.
删除nginx deployment
[root@k8scloude1 deployment]# kubectl delete -f nginx.yaml
deployment.apps "nginx" deleted
[root@k8scloude1 deployment]# kubectl get deploy
No resources found in deployment namespace.
接下来给pod设置资源限制，对pod的资源限制可以通过pod里的resource字段来限制，resources里的requests字段表示容器所在节点资源的最小值，最低要求，满足不了这个要求pod创建不成功。
requests:cpu: 400m 表示pod需要400个微核心才能创建成功，注意：m代表微核心，1个核等于1000个微核心。
[root@k8scloude1 deployment]# vim nginxcpulimit.yaml
[root@k8scloude1 deployment]# cat nginxcpulimit.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
creationTimestamp: null
labels:
app: nginx
name: nginx
spec:
replicas: 1
selector:
matchLabels:
appx: xyz
strategy: {}
template:
metadata:
creationTimestamp: null
labels:
app: nginx
appx: xyz
spec:
terminationGracePeriodSeconds: 0
containers:
- image: nginx
name: nginx
imagePullPolicy: IfNotPresent
resources:
#设置资源限制：CPU：400m #m代表微核心，1个核等于1000个微核心
requests:
cpu: 400m
status: {}
创建deployment
[root@k8scloude1 deployment]# kubectl apply -f nginxcpulimit.yaml
deployment.apps/nginx created
[root@k8scloude1 deployment]# kubectl get deployments -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
nginx 1/1 1 1 13s nginx nginx appx=xyz
查看node节点的负载
[root@k8scloude1 deployment]# kubectl top node
W0125 17:06:20.377474 15650 top_node.go:119] Using json format to get metrics. Next release will switch to protocol-buffers, switch early by passing --use-protocol-buffers flag
NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
k8scloude1 230m 11% 1375Mi 42%
k8scloude2 100m 5% 669Mi 34%
k8scloude3 113m 5% 737Mi 38%
设置HPA：最小pod副本数为1，最大pod副本数为15，--cpu-percent=80表示当deployment资源对象的CPU使用率高达80%时，就会进行扩容，最多扩容到15个
[root@k8scloude1 deployment]# kubectl autoscale deployment nginx --min=1 --max=15 --cpu-percent=80
horizontalpodautoscaler.autoscaling/nginx autoscaled
查看hpa
[root@k8scloude1 deployment]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
nginx Deployment/nginx 0%/80% 1 15 1 69s
安装一个压力测试的包：ab工具用于压力测试
[root@k8scloude1 deployment]# yum -y install httpd-tools
创建一个service服务，服务端口为80，service类型为NodePort
[root@k8scloude1 deployment]# kubectl expose --name=nginxsvc deployment nginx --port=80 --type=NodePort
service/nginxsvc exposed
[root@k8scloude1 deployment]# kubectl get svc -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
nginxsvc NodePort 10.96.231.54 <none> 80:31085/TCP 10s appx=xyz
查看pod
[root@k8scloude1 deployment]# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-79b8d956bd-wbcq5 1/1 Running 0 12m
使用ab进行压力测试，一直给nginx服务发请求，Nginx服务的网址为：http://k8s节点ip:31085/，也就是http://192.168.110.130:31085/index.html。
ab压力测试命令参数解释：
- -t：测试所进行的最大秒数。其内部隐含值是-n 50000，它可以使对服务器的测试限制在一个固定的总时间以内。默认时，没有时间限制。
- -n：在测试会话中所执行的请求个数。默认时，仅执行一个请求。
- -c：一次产生的请求个数。默认是一次一个。
下面一直给Nginx服务发请求
[root@k8scloude1 ~]# ab -t 600 -n 1000000 -c 1000 http://192.168.110.130:31085/index.html
This is ApacheBench, Version 2.3 <$Revision: 1430300 %sSourceCode%gt;
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 192.168.110.130 (be patient)
Completed 100000 requests
^C
Server Software: nginx/1.21.5
Server Hostname: 192.168.110.130
Server Port: 31085
Document Path: /index.html
Document Length: 615 bytes
Concurrency Level: 1000
Time taken for tests: 27.096 seconds
Complete requests: 112145
Failed requests: 2029
(Connect: 0, Receive: 0, Length: 0, Exceptions: 2029)
Write errors: 0
Total transferred: 95814989 bytes
HTML transferred: 69487620 bytes
Requests per second: 4138.87 [#/sec] (mean)
Time per request: 241.612 [ms] (mean)
Time per request: 0.242 [ms] (mean, across all concurrent requests)
Transfer rate: 3453.31 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 121 219.7 73 2693
Processing: 1 113 84.3 96 5111
Waiting: 0 95 78.9 80 5063
Total: 2 235 241.9 178 7568
Percentage of the requests served within a certain time (ms)
50% 178
66% 208
75% 230
80% 247
90% 319
95% 469
98% 1188
99% 1233
100% 7568 (longest request)
ab压力测试之后，查看pod的变化，pod数自动变为了3个
[root@k8scloude1 deployment]# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-79b8d956bd-622fw 1/1 Running 0 14s
nginx-79b8d956bd-8z5h7 1/1 Running 0 14s
nginx-79b8d956bd-wbcq5 1/1 Running 0 14m
查看hpa的变化
[root@k8scloude1 deployment]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
nginx Deployment/nginx 31%/80% 1 15 3 11m
[root@k8scloude1 deployment]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
nginx Deployment/nginx 61%/80% 1 15 3 12m
[root@k8scloude1 deployment]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
nginx Deployment/nginx 61%/80% 1 15 3 12m
[root@k8scloude1 deployment]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
nginx Deployment/nginx 61%/80% 1 15 3 12m
[root@k8scloude1 deployment]# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-79b8d956bd-622fw 1/1 Running 0 101s
nginx-79b8d956bd-8z5h7 1/1 Running 0 101s
nginx-79b8d956bd-wbcq5 1/1 Running 0 15m
删除服务并重新创建一个service服务，这次的service服务类型为ClusterIP。
[root@k8scloude1 deployment]# kubectl delete svc nginxsvc
[root@k8scloude1 deployment]# kubectl get svc
No resources found in deployment namespace.
[root@k8scloude1 deployment]# kubectl expose --name=nginxsvc deployment nginx --port=80
service/nginxsvc exposed
[root@k8scloude1 deployment]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginxsvc ClusterIP 10.109.240.56 <none> 80/TCP 3s
Nginx服务的网址为：http://10.109.240.56:80/index.html，继续进行ab压力测试，给Nginx服务发请求。
#继续ab压力测试
[root@k8scloude1 ~]# ab -t 600 -n 1000000 -c 1000 http://10.109.240.56:80/index.html
This is ApacheBench, Version 2.3 <$Revision: 1430300 %sSourceCode%gt;
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 10.109.240.56 (be patient)
Completed 100000 requests
Completed 200000 requests
Completed 300000 requests
Completed 400000 requests
Completed 500000 requests
Completed 600000 requests
Completed 700000 requests
Completed 800000 requests
Completed 900000 requests
apr_pollset_poll: The timeout specified has expired (70007)
Total of 985106 requests completed
执行ab压力测试的同时观察pod的变化
查看pod的负载
[root@k8scloude1 deployment]# kubectl top pods
W0125 17:25:03.753728 30784 top_pod.go:140] Using json format to get metrics. Next release will switch to protocol-buffers, switch early by passing --use-protocol-buffers flag
NAME CPU(cores) MEMORY(bytes)
nginx-79b8d956bd-5hq5j 179m 4Mi
nginx-79b8d956bd-622fw 366m 5Mi
nginx-79b8d956bd-6wjsl 248m 5Mi
nginx-79b8d956bd-8z5h7 360m 5Mi
nginx-79b8d956bd-wbcq5 254m 5Mi
查看hpa，现在已经有5个pod了
[root@k8scloude1 deployment]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
nginx Deployment/nginx 71%/80% 1 15 5 16m
继续观察pod
[root@k8scloude1 deployment]# kubectl top pods
W0125 17:25:49.908634 31380 top_pod.go:140] Using json format to get metrics. Next release will switch to protocol-buffers, switch early by passing --use-protocol-buffers flag
NAME CPU(cores) MEMORY(bytes)
nginx-79b8d956bd-5hq5j 254m 5Mi
nginx-79b8d956bd-622fw 326m 5Mi
nginx-79b8d956bd-6wjsl 322m 5Mi
nginx-79b8d956bd-8z5h7 255m 5Mi
nginx-79b8d956bd-wbcq5 264m 5Mi
[root@k8scloude1 deployment]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
nginx Deployment/nginx 70%/80% 1 15 5 18m
[root@k8scloude1 deployment]# kubectl top pods
W0125 17:27:46.476364 32773 top_pod.go:140] Using json format to get metrics. Next release will switch to protocol-buffers, switch early by passing --use-protocol-buffers flag
NAME CPU(cores) MEMORY(bytes)
nginx-79b8d956bd-5hq5j 285m 5Mi
nginx-79b8d956bd-622fw 331m 5Mi
nginx-79b8d956bd-6wjsl 351m 5Mi
nginx-79b8d956bd-8z5h7 277m 5Mi
nginx-79b8d956bd-wbcq5 264m 6Mi
[root@k8scloude1 deployment]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
nginx Deployment/nginx 75%/80% 1 15 5 19m
停止给nginx发送请求之后hpa的TARGETS降下来了
[root@k8scloude1 deployment]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
nginx Deployment/nginx 0%/80% 1 15 5 20m
但是pod数没有降下来
[root@k8scloude1 deployment]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
nginx Deployment/nginx 0%/80% 1 15 5 20m
[root@k8scloude1 deployment]# kubectl top pods
W0125 17:29:32.004502 34116 top_pod.go:140] Using json format to get metrics. Next release will switch to protocol-buffers, switch early by passing --use-protocol-buffers flag
NAME CPU(cores) MEMORY(bytes)
nginx-79b8d956bd-5hq5j 0m 5Mi
nginx-79b8d956bd-622fw 0m 5Mi
nginx-79b8d956bd-6wjsl 0m 5Mi
nginx-79b8d956bd-8z5h7 0m 5Mi
nginx-79b8d956bd-wbcq5 0m 6Mi
[root@k8scloude1 deployment]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
nginx Deployment/nginx 0%/80% 1 15 5 21m
注意：pod负载降下来之后，pod副本数不会立即减少，过一段时间pod副本数才会减少，默认时间间隔为5分钟，这样的目的是为了防止pod副本数的抖动。
10分钟之后观察pod，pod副本降下来了。
[root@k8scloude1 deployment]# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
nginx Deployment/nginx 0%/80% 1 15 1 46m
[root@k8scloude1 deployment]# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-79b8d956bd-wbcq5 1/1 Running 0 50m
[root@k8scloude1 deployment]# kubectl delete hpa nginx
horizontalpodautoscaler.autoscaling "nginx" deleted
八.k8s升级镜像
8.1 kubectl edit deployment升级镜像
查看deploy，可以看到镜像为nginx
[root@k8scloude1 deployment]# kubectl get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
nginx 1/1 1 1 179m
[root@k8scloude1 deployment]# kubectl get deploy -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
nginx 1/1 1 1 179m nginx nginx appx=xyz
第一种方法：使用kubectl edit deployment deployname升级镜像。把deploy的镜像由nginx升级为hub.c.163.com/library/nginx:1.12.0，修改 - image: nginx 为 - image: hub.c.163.com/library/nginx:1.12.0
[root@k8scloude1 deployment]# kubectl edit deployment nginx
deployment.apps/nginx edited
修改之后查看deploy，IMAGES变了。
修改deploy的镜像的时候，本质上是删除现有的pod，然后用新镜像创建新pod。
[root@k8scloude1 deployment]# kubectl get deployments -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
nginx 1/1 1 1 3h3m nginx hub.c.163.com/library/nginx:1.12.0 appx=xyz
[root@k8scloude1 deployment]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-66bcf4489b-ztvnc 1/1 Running 0 77s 10.244.251.211 k8scloude3 <none> <none>
8.2 kubectl set image deploy升级镜像
查看deploy
[root@k8scloude1 deployment]# kubectl get deployment -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
nginx 1/1 1 1 6h41m nginx hub.c.163.com/library/nginx:1.12.0 appx=xyz
第二种方式：使用kubectl set image deploy deployname 容器名=镜像名 升级镜像。
[root@k8scloude1 deployment]# kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:1.13.0
deployment.apps/nginx image updated
查看deploy，可以发现镜像升级为hub.c.163.com/library/nginx:1.13.0了
[root@k8scloude1 deployment]# kubectl get deployment -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
nginx 1/1 1 1 6h42m nginx hub.c.163.com/library/nginx:1.13.0 appx=xyz
继续升级镜像
[root@k8scloude1 deployment]# kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:latest
deployment.apps/nginx image updated
[root@k8scloude1 deployment]# kubectl get deployment -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
nginx 1/1 1 1 6h43m nginx hub.c.163.com/library/nginx:latest appx=xyz
[root@k8scloude1 deployment]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-75698d6db5-bzv9d 1/1 Running 0 17s 10.244.251.212 k8scloude3 <none> <none>
8.3 镜像升级历史记录
查看镜像替换的历史记录 kubectl rollout history deployment nginx，现在镜像替换的历史记录是不清楚的，是因为升级镜像的时候没有record记录。
[root@k8scloude1 deployment]# kubectl rollout history deployment nginx
deployment.apps/nginx
REVISION CHANGE-CAUSE
1 <none>
2 <none>
3 <none>
4 <none>
替换镜像并记录
[root@k8scloude1 deployment]# kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:1.12.0 --record
deployment.apps/nginx image updated
可以看到镜像替换成功
[root@k8scloude1 deployment]# kubectl get deploy -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
nginx 1/1 1 1 6h47m nginx hub.c.163.com/library/nginx:1.12.0 appx=xyz
多进行几次镜像替换并记录
[root@k8scloude1 deployment]# kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:1.13.0 --record
deployment.apps/nginx image updated
[root@k8scloude1 deployment]# kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:latest --record
deployment.apps/nginx image updated
[root@k8scloude1 deployment]# kubectl set image deploy nginx nginx=nginx --record
deployment.apps/nginx image updated
查看镜像替换的历史记录
[root@k8scloude1 deployment]# kubectl rollout history deployment nginx
deployment.apps/nginx
REVISION CHANGE-CAUSE
5 kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:1.12.0 --record=true
6 kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:1.13.0 --record=true
7 kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:latest --record=true
8 kubectl set image deploy nginx nginx=nginx --record=true
查看deploy和pod
[root@k8scloude1 deployment]# kubectl get deploy -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
nginx 1/1 1 1 6h48m nginx nginx appx=xyz
[root@k8scloude1 deployment]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-79b8d956bd-xblj9 1/1 Running 0 61s 10.244.251.214 k8scloude3 <none> <none>
kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:1.13.0 --record=true是第6条修改记录，我们可以进行回滚，回滚到镜像是hub.c.163.com/library/nginx:1.13.0的时候。
[root@k8scloude1 deployment]# kubectl rollout undo deployment nginx --to-revision=6
deployment.apps/nginx rolled back
[root@k8scloude1 deployment]# kubectl get deploy -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
nginx 1/1 1 1 6h51m nginx hub.c.163.com/library/nginx:1.13.0 appx=xyz
变更镜像的本质是删除旧版本的pod，然后用新镜像创建pod。
修改deploy副本数为10个
[root@k8scloude1 deployment]# kubectl scale deployment nginx --replicas=10
deployment.apps/nginx scaled
编辑deploy，修改镜像
[root@k8scloude1 deployment]# kubectl edit deployment nginx
deployment.apps/nginx edited
[root@k8scloude1 deployment]# kubectl get deploy -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
nginx 5/5 5 5 7h1m nginx hub.c.163.com/library/nginx:1.13.0 appx=xyz
#滚动更新配置说明：
strategy:
rollingUpdate:
#maxSurge：在升级过程中一次升级几个，可以是数字也可以是百分比
maxSurge: 25%
#maxUnavailable 在升级过程中，只能有几个不可用， 一次性删除多少个pod，可以是数字也可以是百分比
maxUnavailable: 25%
type: RollingUpdate