How to set a pod to run on a specified node

1. When creating a load , adopt nodeSelector Appoint Node

• Label nodes

1	kubectl label node node2 project=A

• Appoint nodeSelector Create a workload

1 2 3 4 5 6 7 8 910111213141516171819202122

cat <<EOF | kubectl apply -f -apiVersion: apps/v1kind: Deploymentmetadata: name: nginx-nodeselectorspec: replicas: 1 selector: matchLabels: app: nginx-nodeselector template: metadata: labels: app: nginx-nodeselector spec: nodeSelector: project: A containers: - name: nginx image: nginxEOF

• View workload

1234	kubectl get pod -o wideNAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATESnginx-nodeselector-7bb75b7687-7r5xk 1/1 Running 0 19s 10.233.96.60 node2 <none> <none>

In line with expectations ,Pod Run on the specified node node2 On .

• Clean up the environment

12	kubectl delete deployments nginx-nodeselector kubectl label node node2 project-

actually , There is another node selection parameter nodeName, Specify the node name directly . But this setting is too blunt , And beyond Kubernetes Its own scheduling mechanism , Very little is used in actual production .

2. Bind namespaces to nodes through admission control

Specify... When creating a load nodeSelector, You can set Pod Running nodes . But if you want to bind all under the namespace Pod Run under the specified node , It seems to be out of hand . While using kube-apiserver Access control can achieve this goal , This is a Kubernetes 1.5 Enter when alpha Characteristics of the stage .

2.1 modify kube-apiserver Parameters

edit kube-apiserver file :

1	vim /etc/kubernetes/manifests/kube-apiserver.yaml

stay admission-plugins Newly added PodNodeSelector:

1	- --enable-admission-plugins=NodeRestriction,PodNodeSelector

there NodeRestriction Is turned on by default . If it is a highly available cluster , Then you need to modify each kube-apiserver. After the modification is completed , Just a moment kube-apiserver Will complete the restart process .

2.2 to Namespace adding annotations

Edit namespace , Add notes :

1	kubectl edit ns default

123456

apiVersion: v1kind: Namespacemetadata: name: default annotations: scheduler.alpha.kubernetes.io/node-selector: project=A

scheduler.alpha.kubernetes.io/node-selector It can be the node name , It can also be label Key value pair .

2.3 Add the specified for the node label

to node3 The node is marked with project=A The label of :

1	kubectl label node node3 project=A

Here the namespace default Load on , Bound to node node3 On .

2.4 Create a payload

• Create a load to test

1 2 3 4 5 6 7 8 91011121314151617181920

cat <<EOF | kubectl apply -f -apiVersion: apps/v1kind: Deploymentmetadata: name: nginx-schedulerspec: replicas: 3 selector: matchLabels: app: nginx-scheduler template: metadata: labels: app: nginx-scheduler spec: containers: - name: nginx image: nginxEOF

• View load distribution

123456

kubectl get pod -o wideNAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATESnginx-scheduler-6478998698-brkzn 1/1 Running 0 84s 10.233.92.52 node3 <none> <none>nginx-scheduler-6478998698-m422x 1/1 Running 0 84s 10.233.92.51 node3 <none> <none>nginx-scheduler-6478998698-mnf4d 1/1 Running 0 84s 10.233.92.50 node3 <none> <none>

You can see , Although there are 4 Available nodes , however default The loads in the space are running in node3 Under node .

2.5 Clean up the environment

• clear label

1	kubectl label node node3 project-

• Clean load

1	kubectl delete deployments nginx-scheduler

• Clean up annotations

kubectl edit ns default

It should be noted that , If the namespace is already open scheduler.alpha.kubernetes.io/node-selector, The node has no associated label , here ,Pod Will always be in Pending state , Unable to get scheduled , Until there is one that meets the label Node appear .

3. Use topology domain to group nodes

Here's the picture , adopt kube-apiserver Access control plug-ins for , We can build models , One namespace per project , Each namespace contains the specified node . This will satisfy , Business isolation 、 Requirements for cost accounting . But as the cluster gets bigger , The project needs to divide several availability zones under the cluster , It is used to ensure the availability of the business .

The topology domain is mainly used to solve Pod In the cluster distribution problem , Can be used to implement Pod The need for directional selection of nodes .Kubernetes The topology domain characteristics of the cluster scheduler are 1.16 Get into Alpha Stage , stay 1.18 Get into Beta Stage . Let's do some experiments :

• Divide nodes into different topological domains

Here, the nodes node2 into zone a, take node3、node4 into zone b.

1	kubectl label node node2 zone=a

1	kubectl label node node3 node4 zone=b

• Create a payload

1 2 3 4 5 6 7 8 9101112131415161718192021222324252627

cat <<EOF | kubectl apply -f -apiVersion: apps/v1kind: Deploymentmetadata: name: nginx-topologyspec: replicas: 20 selector: matchLabels: app: nginx-topology template: metadata: labels: app: nginx-topology spec: topologySpreadConstraints: - maxSkew: 1 topologyKey: zone whenUnsatisfiable: DoNotSchedule labelSelector: matchLabels: app: nginx-topology containers: - name: nginx image: nginxEOF

here topologyKey Used to specify the partition of the topology domain Key,maxSkew It means that zone=a、zone=b in Pod The quantity difference cannot exceed 1, whenUnsatisfiable: DoNotSchedule It means that the condition is not satisfied , No scheduling .

• see Pod Distribution

1 2 3 4 5 6 7 8 91011121314151617181920212223

kubectl get pod -o wideNAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATESnginx-topology-7d8698544d-2srcj 1/1 Running 0 3m3s 10.233.92.63 node3 <none> <none>nginx-topology-7d8698544d-2wxkp 1/1 Running 0 3m3s 10.233.96.53 node2 <none> <none>nginx-topology-7d8698544d-4db5b 1/1 Running 0 3m3s 10.233.105.43 node4 <none> <none>nginx-topology-7d8698544d-9tqvn 1/1 Running 0 3m3s 10.233.96.58 node2 <none> <none>nginx-topology-7d8698544d-9zll5 1/1 Running 0 3m3s 10.233.105.45 node4 <none> <none>nginx-topology-7d8698544d-d6nbm 1/1 Running 0 3m3s 10.233.105.44 node4 <none> <none>nginx-topology-7d8698544d-f4nw9 1/1 Running 0 3m3s 10.233.96.54 node2 <none> <none>nginx-topology-7d8698544d-ggfgv 1/1 Running 0 3m3s 10.233.92.66 node3 <none> <none>nginx-topology-7d8698544d-gj4pg 1/1 Running 0 3m3s 10.233.92.61 node3 <none> <none>nginx-topology-7d8698544d-jc2xt 1/1 Running 0 3m3s 10.233.92.62 node3 <none> <none>nginx-topology-7d8698544d-jmmcx 1/1 Running 0 3m3s 10.233.96.56 node2 <none> <none>nginx-topology-7d8698544d-l45qj 1/1 Running 0 3m3s 10.233.92.65 node3 <none> <none>nginx-topology-7d8698544d-lwp8m 1/1 Running 0 3m3s 10.233.92.64 node3 <none> <none>nginx-topology-7d8698544d-m65rx 1/1 Running 0 3m3s 10.233.96.57 node2 <none> <none>nginx-topology-7d8698544d-pzrzs 1/1 Running 0 3m3s 10.233.96.55 node2 <none> <none>nginx-topology-7d8698544d-tslxx 1/1 Running 0 3m3s 10.233.92.60 node3 <none> <none>nginx-topology-7d8698544d-v4cqx 1/1 Running 0 3m3s 10.233.96.50 node2 <none> <none>nginx-topology-7d8698544d-w4r86 1/1 Running 0 3m3s 10.233.96.52 node2 <none> <none>nginx-topology-7d8698544d-wwn95 1/1 Running 0 3m3s 10.233.96.51 node2 <none> <none>nginx-topology-7d8698544d-xffpx 1/1 Running 0 3m3s 10.233.96.59 node2 <none> <none>

Among them in node2 node 10 individual Pod、node3 node 7 Nodes 、node4 node 3 Nodes . You can see ,Pod Evenly distributed in zone=a、zone=b On .

• Clean up the environment

12	kubectl label node node2 node3 node4 zone-kubectl delete deployments nginx-topology

4. summary

As clusters get bigger , Isolation between businesses 、 Problems such as the monopolization of business to nodes will emerge . Usually , Each business has a separate namespace , therefore , We can bind namespaces to nodes .

This paper mainly gives two methods , One is when creating a load , Set up directly nodeSelector, The trick is to use the namespace value as value; Another way is , With the help of kube-apiserver Provided access control plug-ins , By way of annotation , When creating a payload under a namespace , Specify nodes by label filtering , Complete the binding between namespace and node .

Think about it further , If the number of nodes is very large , It is necessary to divide the available zones to distribute the load , Then we can use the topological domain to realize . Through the topology domain , We can make the load , It is evenly distributed in the specified availability zone according to the configured policy 、 On the cabinet .

5. Reference resources

• https://kubernetes.io/zh-cn/docs/reference/scheduling/config/
• https://kubernetes.io/zh-cn/docs/reference/access-authn-authz/admission-controllers/