第22关 RBAC
深入解析K8s中的RBAC角色访问控制策略
RBAC介绍
在k8s我们如何控制访问权限呢,答案就是Role-based access control (RBAC) - 基于角色(Role)的访问控制,(RBAC)是一种基于组织中用户的角色来调节控制对 计算机或网络资源的访问的方法。
在早期的K8s版本,RBAC还未出现的时候,整个K8s的安全是较为薄弱的,有了RBAC后,我们可以对K8s集群的访问人员作非常明细化的控制,控制他们能访问什么资源,以只读还是可以读写的形式来访问,目前RBAC是K8s默认的安全授权标准,所以我们非常有必要来掌握RBAC的使用,这样才有更有力的保障我们的K8s集群的安全使用,下面我们将以生产中的实际使用来大家了解及掌握RBAC的生产应用。
RBAC里面的几种资源关系图,下面将用下面的资源来演示生产中经典的RBAC应用
|--- Role --- RoleBinding 只在指定namespace中生效
ServiceAccount ---|
|--- ClusterRole --- ClusterRoleBinding 不受namespace限制,在整个K8s集群中生效在我看来,RBAC在K8s上的用途主要分为两大类:
第一类是保证在K8s上运行的pod服务具有相应的集群权限,如gitlab的CI/CD,它需要能访问除自身以外其他pod,比如gitlab-runner的pod的权限,再比如gitlab-runner的pod需要拥有创建新的临时pod的权限,用以来构建CI/CD自动化流水线,这里大家没用过不懂没关系,先简单了解下就可以了,在本课程后面基于K8s及gitlab的生产实战CI/CD内容会给大家作详细实战讲解;
第二类是创建能访问K8s相应资源、拥有对应权限的kube-config配置给到使用K8s的人员,来作为连接K8s的授权凭证
第一类的实战这里先暂时以早期的helm2来作下讲解,helm是一个快捷安装K8s各类资源的管理工具,通过之前给大家讲解的,一个较为完整的服务可能会存在deployment,service,configmap,secret,ingress等资源来组合使用,大家在用的过程中可能会觉得配置使用较为麻烦,这时候helm就出现了,它把这些资源都打包封装成它自己能识别的内容,我们在安装一个服务的时候,就只需要作下简单的配置,一条命令即可完成上述众多资源的配置安装,titller相当于helm的服务端,它是需要有权限在K8s中创建各类资源的,在初始安装使用时,如果没有配置RBAC权限,我们会看到如下报错
root@node1:~# helm install stable/mysql
Error: no available release name found这时,我们可以来快速解决这个问题,创建sa关联K8s自带的最高权限的ClusterRole(生产中建议不要这样做,权限太高有安全隐患,这个就和linux的root管理帐号一样,一般都是建议通过sudo来控制帐号权限)
# 在【命名空间kube-system】下创建一个【账号tiller】
kubectl create serviceaccount --namespace kube-system tiller
# 创建一个【集群角色绑定tiller-cluster-rule】,角色为【cluster-admin】,账号为【tiller】
kubectl create clusterrolebinding tiller-cluster-rule --clusterrole=cluster-admin --serviceaccount=kube-system:tiller
kubectl patch deploy --namespace kube-system tiller-deploy -p '{"spec":{"template":{"spec":{"serviceAccount":"tiller"}}}}'
部署一些服务提示权限不足可以这样给他一个管理员权限再绑定一下,慎用
- 这个命令通过修改部署(deployment)资源的配置来指定使用之前创建的
tiller服务账户。tiller-deploy是 Helm Tiller 服务的部署名称。在默认情况下,Helm 安装时会创建一个名为tiller-deploy的部署资源来部署 Tiller 服务。- 通过
kubectl patch命令,我们将部署资源的配置进行了修改,指定了serviceAccount字段为之前创建的tiller服务账户,这样 Helm Tiller 服务就会以tiller服务账户的身份运行,从而拥有了之前绑定的cluster-admin权限。
第二类,我这里就直接以我在生产中实施的完整脚本来做讲解及实战,相信会给大家带来一个全新的学习感受,并能很快掌握它们:
root目录下的kube-config非常重要,一定不能泄露,丢了要重新签证非常麻烦
root@node-1:~/.kube# ls -lh
total 12K
drwxr-x--- 4 root root 4.0K May 19 12:16 cache
-r-------- 1 root root 6.1K May 19 13:00 config创建对所有namespace有只读权限的kube-config
#!/bin/bash
export KUBECONFIG=/root/.kube/config
BASEDIR="$(dirname "$0")"
folder="$BASEDIR/kube_config"
echo -e "All namespaces is here: \n$(kubectl get ns|awk 'NR!=1{print $1}')"
echo "endpoint server if local network you can use $(kubectl cluster-info |awk '/Kubernetes/{print $NF}')"
clustername=$1
endpoint=$(echo "$2" | sed -e 's,https\?://,,g')
if [[ -z "$endpoint" || -z "$clustername" ]]; then
echo "Use "$(basename "$0")" CLUSTERNAME ENDPOINT";
exit 1;
fi
# https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.24.md#urgent-upgrade-notes
echo "---
apiVersion: v1
kind: ServiceAccount
metadata:
name: all-readonly-${clustername}
namespace: kube-system
---
apiVersion: v1
kind: Secret
metadata:
name: all-readonly-secret-sa-$clustername-user
namespace: kube-system
annotations:
kubernetes.io/service-account.name: "all-readonly-${clustername}"
type: kubernetes.io/service-account-token
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: all-readonly-${clustername}
rules:
- apiGroups:
- ''
resources:
- configmaps
- endpoints
- persistentvolumes
- persistentvolumeclaims
- pods
- replicationcontrollers
- replicationcontrollers/scale
- serviceaccounts
- services
- nodes
verbs:
- get
- list
- watch
- apiGroups:
- ''
resources:
- bindings
- events
- limitranges
- namespaces/status
- pods/log
- pods/status
- replicationcontrollers/status
- resourcequotas
- resourcequotas/status
verbs:
- get
- list
- watch
- apiGroups:
- ''
resources:
- namespaces
verbs:
- get
- list
- watch
- apiGroups:
- apps
resources:
- controllerrevisions
- daemonsets
- deployments
- deployments/scale
- replicasets
- replicasets/scale
- statefulsets
- statefulsets/scale
verbs:
- get
- list
- watch
- apiGroups:
- autoscaling
resources:
- horizontalpodautoscalers
verbs:
- get
- list
- watch
- apiGroups:
- batch
resources:
- cronjobs
- jobs
verbs:
- get
- list
- watch
- apiGroups:
- extensions
resources:
- daemonsets
- deployments
- deployments/scale
- ingresses
- networkpolicies
- replicasets
- replicasets/scale
- replicationcontrollers/scale
verbs:
- get
- list
- watch
- apiGroups:
- policy
resources:
- poddisruptionbudgets
verbs:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
resources:
- networkpolicies
verbs:
- get
- list
- watch
- apiGroups:
- metrics.k8s.io
resources:
- pods
verbs:
- get
- list
- watch
- apiGroups:
- storage.k8s.io
resources:
- storageclasses
verbs:
- get
- list
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: all-readonly-${clustername}
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: all-readonly-${clustername}
subjects:
- kind: ServiceAccount
name: all-readonly-${clustername}
namespace: kube-system" | kubectl apply -f -
mkdir -p $folder
#tokenName=$(kubectl get sa all-readonly-${clustername} -n $namespace -o "jsonpath={.secrets[0].name}")
tokenName="all-readonly-secret-sa-$clustername-user"
token=$(kubectl get secret $tokenName -n kube-system -o "jsonpath={.data.token}" | base64 --decode)
certificate=$(kubectl get secret $tokenName -n kube-system -o "jsonpath={.data['ca\.crt']}")
echo "apiVersion: v1
kind: Config
preferences: {}
clusters:
- cluster:
certificate-authority-data: $certificate
server: https://$endpoint
name: all-readonly-${clustername}
users:
- name: all-readonly-${clustername}
user:
as-user-extra: {}
client-key-data: $certificate
token: $token
contexts:
- context:
cluster: all-readonly-${clustername}
user: all-readonly-${clustername}
name: ${clustername}
current-context: ${clustername}" > $folder/${clustername}-all-readonly.conf这段脚本的作用是创建一个 Kubernetes 集群的只读访问权限。脚本需要两个参数:集群名称和集群的 API 地址。
脚本首先检查参数是否为空,然后创建一个具有只读权限的 ServiceAccount,并为该 ServiceAccount 创建一个 Secret,其中包含访问令牌。接着,它定义了一个 ClusterRole,给予该 ServiceAccount 在集群中只读访问权限。最后,它创建了一个 ClusterRoleBinding,将该权限绑定到上述创建的 ServiceAccount 上。
在执行完上述步骤后,脚本会创建一个 Kubernetes 配置文件,其中包含集群的 CA 证书、API 地址和访问令牌等信息。最终,该配置文件会保存在指定目录下,以便后续使用。
# kubectl cluster-info
Kubernetes control plane is running at https://10.0.1.201:6443
CoreDNS is running at https://10.0.1.201:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
KubeDNSUpstream is running at https://10.0.1.201:6443/api/v1/namespaces/kube-system/services/kube-dns-upstream:dns/proxy
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.运行
# grep boge.com /etc/kubeasz/clusters/test-cn/config.yml
- test-cnk8s.boge.com# bash readonly.sh
All namespaces is here:
default
kube-node-lease
kube-public
kube-system
test-ingress-controller
test-nfs-sc
endpoint server if local network you can use https://10.0.1.201:6443
Use readonly.sh CLUSTERNAME ENDPOINT
# 当前集群名称
# kubectl config get-contexts
CURRENT NAME CLUSTER AUTHINFO NAMESPACE
* context-test-cn test-cn admin
# 公网可以换成域名
bash readonly.sh [集群名称] [api server 地址(6443)]
# 不能写本身test-cn一样的名称。相当于创建一个k8s用户?
# bash readonly.sh test https://10.0.1.201:6443
All namespaces is here:
default
kube-node-lease
kube-public
kube-system
test-ingress-controller
test-nfs-sc
endpoint server if local network you can use https://10.0.1.201:6443
serviceaccount/all-readonly-test created # 创建账户
secret/all-readonly-secret-sa-test-user created # 创建secret
clusterrole.rbac.authorization.k8s.io/all-readonly-test created # 创建集群角色
clusterrolebinding.rbac.authorization.k8s.io/all-readonly-test created # 集群角色绑定
查看生成的配置
cat ./kube_config/test-all-readonly.conf
--client-certificate、--client-key、--username、--password 和 --token
kubectl --kubeconfig ./kube_config/hong-all-readonly.conf get pod
携带配置来进行操作
# kubectl --kubeconfig ./kube_config/hong-all-readonly.conf get node
NAME STATUS ROLES AGE VERSION
10.0.1.201 Ready,SchedulingDisabled master 14d v1.27.5
10.0.1.202 Ready,SchedulingDisabled master 14d v1.27.5
10.0.1.203 Ready node 14d v1.27.5
10.0.1.204 Ready node 14d v1.27.5
kubectl --kubeconfig ./kube_config/test-all-readonly.conf get pod
kubectl --kubeconfig ./kube_config/test-all-readonly.conf get pod -n test-ingress
# kubectl --kubeconfig ./kube_config/hong-all-readonly.conf -n test-nfs-sc delete pod nfs-provisioner-01-5cb46dcb99-92s4s
Error from server (Forbidden): pods "nfs-provisioner-01-5cb46dcb99-92s4s" is forbidden: User "system:serviceaccount:kube-system:all-readonly-hong" cannot delete resource "pods" in API group "" in the namespace "test-nfs-sc"想回收配置的话
把对应的创建的资源删除掉
sa、集群角色、集群角色绑定、sercet、
serviceaccount/all-readonly-testt created
secret/all-readonly-secret-sa-testt-user created
clusterrole.rbac.authorization.k8s.io/all-readonly-testt created
clusterrolebinding.rbac.authorization.k8s.io/all-readonly-testt created
# 查看SA账号 kubectl get serviceaccounts -A
# kubectl -n kube-system get serviceaccounts
NAME SECRETS AGE
all-readonly-test 0 11m
all-readonly-test-cn 0 18m
all-readonly-testt 0 8m54s
删除 ServiceAccount:
# kubectl -n kube-system delete serviceaccount all-readonly-test
serviceaccount "all-readonly-test" deleted
# 查看Secret(删除 ServiceAccount后发现查看Secret不见了)
kubectl get secret -A
删除 Secret:
# kubectl -n kube-system delete secret all-readonly-secret-sa-test-user
secret "all-readonly-secret-sa-test-user" deleted
# 查看
# kubectl get clusterrole
NAME CREATED AT
admin 2024-05-19T05:06:11Z
all-readonly-test 2024-06-02T09:54:23Z
all-readonly-test-cn 2024-06-02T09:47:13Z
all-readonly-testt 2024-06-02T09:57:02Z
删除 ClusterRole:
# kubectl delete clusterrole all-readonly-test
clusterrole.rbac.authorization.k8s.io "all-readonly-test" deleted
查看
# kubectl get clusterrolebinding
NAME ROLE AGE
all-readonly-test ClusterRole/all-readonly-test 31m
all-readonly-test-cn ClusterRole/all-readonly-test-cn 38m
all-readonly-testt ClusterRole/all-readonly-testt 28m
删除 ClusterRoleBinding:
kubectl delete clusterrolebinding all-readonly-test下次可以重新签证一个配置
创建对指定已有的namespace有所有权限的kube-config
比如给某项目组去使用
#!/bin/bash
export KUBECONFIG=/root/.kube/config
BASEDIR="$(dirname "$0")"
folder="$BASEDIR/kube_config"
echo -e "All namespaces is here: \n$(kubectl get ns|awk 'NR!=1{print $1}')"
echo "endpoint server if local network you can use $(kubectl cluster-info |awk '/Kubernetes/{print $NF}')"
namespace=$1
endpoint=$(echo "$2" | sed -e 's,https\?://,,g')
if [[ -z "$endpoint" || -z "$namespace" ]]; then
echo "Use "$(basename "$0")" NAMESPACE ENDPOINT";
exit 1;
fi
# https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.24.md#urgent-upgrade-notes
echo "---
apiVersion: v1
kind: ServiceAccount
metadata:
name: $namespace-user
namespace: $namespace
---
apiVersion: v1
kind: Secret
metadata:
name: secret-sa-$namespace-user
namespace: $namespace
annotations:
kubernetes.io/service-account.name: "$namespace-user"
type: kubernetes.io/service-account-token
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: $namespace-user-full-access
namespace: $namespace
rules:
- apiGroups: ['', 'extensions', 'apps', 'metrics.k8s.io', 'networking.k8s.io']
resources: ['*']
verbs: ['*']
- apiGroups: ['batch']
resources:
- jobs
- cronjobs
verbs: ['*']
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: $namespace-user-view
namespace: $namespace
subjects:
- kind: ServiceAccount
name: $namespace-user
namespace: $namespace
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: $namespace-user-full-access" | kubectl apply -f -
mkdir -p $folder
#tokenName=$(kubectl get sa $namespace-user -n $namespace -o "jsonpath={.secrets[0].name}")
tokenName="secret-sa-$namespace-user"
token=$(kubectl get secret $tokenName -n $namespace -o "jsonpath={.data.token}" | base64 --decode)
certificate=$(kubectl get secret $tokenName -n $namespace -o "jsonpath={.data['ca\.crt']}")
echo "apiVersion: v1
kind: Config
preferences: {}
clusters:
- cluster:
certificate-authority-data: $certificate
server: https://$endpoint
name: $namespace-cluster
users:
- name: $namespace-user
user:
as-user-extra: {}
client-key-data: $certificate
token: $token
contexts:
- context:
cluster: $namespace-cluster
namespace: $namespace
user: $namespace-user
name: $namespace
current-context: $namespace" > $folder/$namespace.kube.conf运行
先创建好2个命名空间并且运行一个pod
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: web
name: web
spec:
replicas: 1
selector:
matchLabels:
app: web
template:
metadata:
labels:
app: web
spec:
containers:
- image: nginx:1.21.6
name: nginx
resources:
limits: # 因为我这里是测试环境,所以这里CPU只分配50毫核(0.05核CPU)和20M的内存
cpu: "50m"
memory: 20Mi
requests: # 保证这个pod初始就能分配这么多资源
cpu: "50m"
memory: 20Mi
---
# SVC
kind: Service
apiVersion: v1
metadata:
name: web
spec:
selector:
app: web
ports:
- name: http-port
# 服务端口
port: 80
protocol: TCP
# pod端口
targetPort: 80
---kubectl create ns test-kubeconfig-a
kubectl -n test-kubeconfig-a apply -f test-kubeconfig-nginx.yaml
kubectl -n test-kubeconfig-a autoscale deployment web --max=3 --min=1 --cpu-percent=30kubectl create ns test-kubeconfig-b
kubectl -n test-kubeconfig-b apply -f test-kubeconfig-nginx.yaml
kubectl -n test-kubeconfig-b autoscale deployment web --max=3 --min=1 --cpu-percent=30bash ns-all.sh [命名空间] [api server地址]
# bash ns-all.sh test-kubeconfig-a https://10.0.1.201:6443
All namespaces is here:
default
kube-node-lease
kube-public
kube-system
test-ingress-controller
test-kubeconfig-a
test-kubeconfig-b
test-nfs-sc
endpoint server if local network you can use https://10.0.1.201:6443
serviceaccount/test-kubeconfig-a-user created
secret/secret-sa-test-kubeconfig-a-user created
role.rbac.authorization.k8s.io/test-kubeconfig-a-user-full-access created
rolebinding.rbac.authorization.k8s.io/test-kubeconfig-a-user-view created
secret 提取token用
使用test-a的配置查看节点
# kubectl --kubeconfig ./kube_config/test-kubeconfig-a.kube.conf get node
Error from server (Forbidden): nodes is forbidden: User "system:serviceaccount:test-kubeconfig-a:test-kubeconfig-a-user" cannot list resource "nodes" in API group "" at the cluster scope
可以看自己命名空间下的
# kubectl --kubeconfig ./kube_config/test-kubeconfig-a.kube.conf get pod
NAME READY STATUS RESTARTS AGE
web-67f5d7fc55-l4wlv 1/1 Running 0 3m56s
不可以看其他命名空间
# kubectl --kubeconfig ./kube_config/test-kubeconfig-a.kube.conf get pod -n default
Error from server (Forbidden): pods is forbidden: User "system:serviceaccount:test-kubeconfig-a:test-kubeconfig-a-user" cannot list resource "pods" in API group "" in the namespace "default"
# 查看命名空间下全部资源
# kubectl --kubeconfig ./kube_config/test-kubeconfig-a.kube.conf get all
NAME READY STATUS RESTARTS AGE
pod/web-67f5d7fc55-l4wlv 1/1 Running 0 4m57s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/web ClusterIP 10.68.65.142 <none> 80/TCP 4m59s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/web 1/1 1 1 4m59s
NAME DESIRED CURRENT READY AGE
replicaset.apps/web-67f5d7fc55 1 1 1 4m58s
Error from server (Forbidden): horizontalpodautoscalers.autoscaling is forbidden: User "system:serviceaccount:test-kubeconfig-a:test-kubeconfig-a-user" cannot list resource "horizontalpodautoscalers" in API group "autoscaling" in the namespace "test-kubeconfig-a"
提示hpa自动伸缩的看不了
根据提示,编辑角色权限加上去
# kubectl -n test-kubeconfig-a edit role test-kubeconfig-a-user-full-access
根据提示 cannot list resource "horizontalpodautoscalers" in API group "autoscaling"
API group "autoscaling"
resource "horizontalpodautoscalers"
加上autoscaling,resource是*可以不管了
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
annotations:
name: test-kubeconfig-a-user-full-access
namespace: test-kubeconfig-a
rules:
- apiGroups:
- ""
- extensions
- apps
- metrics.k8s.io
- networking.k8s.io
- autoscaling
resources:
- '*'
verbs:
- '*'
- apiGroups:
- batch
resources:
- jobs
- cronjobs
verbs:
- '*'
再查看全部资源
# kubectl --kubeconfig ./kube_config/test-kubeconfig-a.kube.conf get all
NAME READY STATUS RESTARTS AGE
pod/web-67f5d7fc55-l4wlv 1/1 Running 0 9m53s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/web ClusterIP 10.68.65.142 <none> 80/TCP 9m55s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/web 1/1 1 1 9m55s
NAME DESIRED CURRENT READY AGE
replicaset.apps/web-67f5d7fc55 1 1 1 9m54s
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
horizontalpodautoscaler.autoscaling/web Deployment/web 0%/30% 1 3 1 9m47s在已有sa上附加其他命名空间的权限
在上面的【命名空间test-kubeconfig-a】的【账号test-kubeconfig-a-user】的权限基础上给他【命名空间test-kubeconfig-b】的操作权限
一个SA关联多个角色
# same ServiceAccount:" test-a-user " default can contorl my own namespace:" test-a " and config later to contorl other namespace: "test-b"
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: test-b-user-full-access
namespace: test-kubeconfig-b
rules:
- apiGroups: ['', 'extensions', 'apps', 'metrics.k8s.io', 'networking.k8s.io', 'autoscaling']
resources: ['*']
verbs: ['*']
- apiGroups: ['batch']
resources:
- jobs
- cronjobs
verbs: ['*']
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: test-b-user-full-access-both-test-a-user
namespace: test-kubeconfig-b
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
# 那个角色
name: test-b-user-full-access
subjects:
- kind: ServiceAccount
# 绑给谁
name: test-kubeconfig-a-user
namespace: test-kubeconfig-a运行
# kubectl -n test-kubeconfig-b apply -f test-kubeconfig-b-role.yaml
role.rbac.authorization.k8s.io/test-b-user-full-access created
rolebinding.rbac.authorization.k8s.io/test-b-user-full-access-both-test-a-user created
测试
# kubectl --kubeconfig ./kube_config/test-kubeconfig-a.kube.conf get pod -n test-kubeconfig-a
NAME READY STATUS RESTARTS AGE
web-67f5d7fc55-l4wlv 1/1 Running 0 29m
root@node-1:~/boge/0602/1743# kubectl --kubeconfig ./kube_config/test-kubeconfig-a.kube.conf get pod -n test-kubeconfig-b
NAME READY STATUS RESTARTS AGE
web-67f5d7fc55-gnttp 1/1 Running 0 29m
# kubectl --kubeconfig ./kube_config/test-kubeconfig-a.kube.conf get all -n test-kubeconfig-a
NAME READY STATUS RESTARTS AGE
pod/web-67f5d7fc55-l4wlv 1/1 Running 0 30m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/web ClusterIP 10.68.65.142 <none> 80/TCP 30m
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/web 1/1 1 1 30m
NAME DESIRED CURRENT READY AGE
replicaset.apps/web-67f5d7fc55 1 1 1 30m
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
horizontalpodautoscaler.autoscaling/web Deployment/web 0%/30% 1 3 1 30m
删除b pod