Detailed upgrade process of AWS eks

Upgrade Overview

AWS EKS As K8S Cloud platform hosting service , Also need to follow K8S Version iteration for , Update regularly . How to upgrade , And the detailed steps of upgrading , There are detailed records and descriptions in this article .

EKS The updated timeline is as follows ：

Version update differences , Please refer to the documentation ：

​ ​https://docs.aws.amazon.com/zh_cn/eks/latest/userguide/kubernetes-versions.html​​

1.20 An important tip of version

· Docker Container running time has gradually stopped .Kubernetes The community has written a detailed ​ ​ Blog posts ​​ And provide special ​ ​ FAQ page ​​.Docker The generated image can continue to be used , And will continue to function . You can safely ignore kubelet Start the printing in the log dockershim Discard warning messages .EKS Will eventually migrate to Containerd, The latter will serve as EKS Optimized version Amazon Linux 2 AMI Running time of . You can refer to the container roadmap ​ ​ problem ​​, Learn more about .

Kubernetes Planned for the upcoming 1.24 Deprecated in version dockershim

​ ​http://dockone.io/article/2434814​​

Upgrade mode ：

1.   Upgrade in place , Just click AWS Upgrade the console , Control level upgrade , Data plane upgrade , Plug in component upgrade

2.  By switching node group Way to upgrade , Control level upgrade , Data level upgrade （ Add new node group, Upgrade and switch ）, Plug-in upgrade

3.    adopt ALB, Nginx Flow control of , upgrade

Test environment preparation

1.   Install first AWS Cli and kubectl, The reference link is as follows

​ ​https://docs.aws.amazon.com/zh_cn/zh_cn/eks/latest/userguide/install-kubectl.html​​​​

The test content of this article is ： From version 1.18 Upgrade to 1.19

execution environment , You need to install kubectl 1.18

2. install EKS

2.1 To configure kubectl,

Log in and copy

      
      

       
       aws eks --region us-east-2 update-kubeconfig --name eks1118
      
      
      
      

       
       
1.
      
      

2.2 Create command

Log in and copy

      
      

       
       AWS_REGION=us-east-2
       
       
AWS_DEFAULT_REGION=us-east-2
       
       
CLUSTER_NAME=eks11811
       
       
eksctl create cluster --name=${CLUSTER_NAME} --version 1.18 --nodes=10
       
       
--node-type c5.large --managed --alb-ingress-access --region=${AWS_REGION}
      
      
      
      

       
       
1.
       
       
2.
       
       
3.
       
       
4.
       
       
5.
      
      

notes ：

Created 1.18 After cluster version or higher , You can deploy with the cluster Amazon VPC CNI、CoreDNS and  kube-proxy  Add ons migrated to Amazon EKS Add ons . For more information , see also  ​ ​Amazon EKS Add ons ​​.

3.  created EKS 1.18 edition

Upgrade test

In this paper, upgrade methods are divided into three categories ： Upgrade in place , Upgrade by updating the node group ,ALB/Nginx Upgrade the traffic distribution method

Method 1 ： Upgrade in place

1． Official documentation of the upgrade process ：

Amazon EKS Perform standard infrastructure and readiness health checks on network traffic on these new nodes , To confirm whether they work as expected . If any check fails ,Amazon EKS Will resume infrastructure deployment , And your cluster remains the previous Kubernetes edition . Running applications will not be affected , And your cluster will never be in an uncertain or unrecoverable state .Amazon EKS All managed clusters will be backed up regularly , And it has a mechanism to restore the cluster when necessary . We will constantly evaluate and improve our Kubernetes Infrastructure management process .

2． Upgrading cannot be rolled back

3． It can only be upgraded one by one

4.  The control level needs to be upgraded first , The upgrade time is 30 About minutes , The upgrade process is smooth .

5.  Upgrade node group

5.1.   notes ：

​ ​https://docs.aws.amazon.com/zh_cn/eks/latest/userguide/update-workers.html​​​​

There are two basic ways to update self managed node groups in a cluster to use the new AMI：

• ​ ​ Migrate to a new node group ​​ – Create a new node group and put your Pod Migrate to this group . Migrating to a new node group is easier than simply updating an existing AWS CloudFormation In the stack AMI ID More natural , Because the migration process will take  NoSchedule  Form to execute on the old node group Taint operation , And be ready to accept the existing on the new stack Pod These nodes are exhausted after the workload .


• ​ ​ Update existing self managed node groups ​​​ – Update the existing node group AWS CloudFormation

Stack to use new AMI. Use  eksctl  The created node group does not support this method .


6. Node group upgrade process — Update existing self managed node groups

6.1. Node group upgrade , There are two upgrade options ,Rolling and Force, Suggest Rolling The way to upgrade

6.2 During the upgrade , Will the new Node, For transfer pod

6.3. In the startup process ,node Nodes begin to be replaced step by step

6.4. Managed node update behavior

​ ​https://docs.aws.amazon.com/zh_cn/eks/latest/userguide/managed-node-update-behavior.html​​

6.5 The upgrade of the new managed node group is complete , Old managed node groups can be deleted

​ ​https://docs.aws.amazon.com/zh_cn/eks/latest/userguide/delete-managed-node-group.html​​

The whole upgrade process is 17 Minutes to complete . The upgrade time will increase with node The increase in number , And increase .

Method 2 ： Upgrade by updating the node group

1. Add a new node group to upgrade

​ ​https://docs.aws.amazon.com/zh_cn/eks/latest/userguide/update-workers.html​​​​

There are two basic ways to update self managed node groups in a cluster to use the new AMI：

​ ​ Migrate to a new node group ​​ – Create a new node group and put your Pod Migrate to this group . Migrating to a new node group is easier than simply updating an existing AWS CloudFormation In the stack AMI ID More natural , Because the migration process will take  NoSchedule  Form to execute on the old node group Taint operation , And be ready to accept the existing on the new stack Pod These nodes are exhausted after the workload .

2.  Add a node group

3. You can select node groups of different models

4.  View node groups

5.    

matters needing attention

This method , It is also necessary to update the control level first , Then update the node group .

You can try it from the command line , Update the node group before updating the control layer , Will report a mistake . The control level needs to be upgraded first , And ensure that the version of the node group is not higher than the control level . You can upgrade the node group to the control level Kubernetes The same version . for example , If your cluster is running Kubernetes 1.19, Then you can use the following command to run Kubernetes 1.18 Upgrade the node of to version 1.19.

Examples ：

Log in and copy

      
      

       
       eksctl upgrade nodegroup \
       
       
    --name=ng119 \
       
       
    --cluster=eks11811 \
       
       
    --kubernetes-version=1.19
       
       

       
       
Error: InvalidParameterException: Nodegroup Kubernetes version should be
       
       
equal to Cluster kubernetes version 1.18 or NodeGroup kubernetes version 1.18{
       
       
  RespMetadata: {
       
       
    StatusCode: 400,
       
       
    RequestID:
       
       
"c9133322-2fe0-4fa2-94aa-792caa95f29d"  },
       
       
  ClusterName: "eks11811",
       
       
  Message_: "Nodegroup
       
       
Kubernetes version should be equal to Cluster kubernetes version 1.18 or
       
       
NodeGroup kubernetes version 1.18",  NodegroupName: "ng119"
       
       
}
      
      
      
      

       
       
1.
       
       
2.
       
       
3.
       
       
4.
       
       
5.
       
       
6.
       
       
7.
       
       
8.
       
       
9.
       
       
10.
       
       
11.
       
       
12.
       
       
13.
       
       
14.
       
       
15.
       
       
16.
      
      

6.  Update node group , Two node groups , Yes, you can upgrade separately .

6.1. You can upgrade on the console , You can also upgrade through the command line

6.2. Update complete

7.   Now there are two different versions of node groups , Can pass eksctl and kubectl, take pod Do the migration operation .

8. determine pod In the new version node After running normally on , You can delete the old version node group .

Migrate to a new node group

​ ​https://docs.aws.amazon.com/zh_cn/eks/latest/userguide/migrate-stack.html​​

How to check 、 Expand 、 Delete or exhaust Amazon EKS Worker thread node in

​ ​https://aws.amazon.com/cn/premiumsupport/knowledge-center/eks-worker-node-actions/​​​​

Method 3 ：ALB/Nginx Upgrade the traffic distribution method

1.  adopt ALB,Nginx Methods such as , By weight Traffic allocation , This method does not involve EKS upgrade , But test more , Provide the following two document references .

2.  How for my Application Load
Balancer Set weighted target group

​ ​https://aws.amazon.com/cn/premiumsupport/knowledge-center/elb-make-weighted-target-groups-for-alb/​​​​

3. Nginx Weight distribution

​ ​https://blog.csdn.net/u012156116/article/details/81510978​​​​

EKS Add-ons Add on upgrade

1.  After the control level and data level are upgraded , Components need to be checked , And upgrade according to the corresponding version . Corresponding version , Please refer to the following document ：

management  kube-proxy  Add ons

​ ​https://docs.amazonaws.cn/eks/latest/userguide/managing-kube-proxy.html​​

management CoreDNS Add ons

​ ​https://docs.amazonaws.cn/eks/latest/userguide/managing-coredns.html​​

2.  Through the console , Click on update now

3.  Select the corresponding version

4.  Upgrade is complete

Reference documents ：

Amazon EKS Cluster upgrade Guide

​ ​https://aws.amazon.com/cn/blogs/china/amazon-eks-cluster-upgrade-guide/​​

Amazon EKS Kubernetes edition

​ ​https://docs.aws.amazon.com/zh_cn/eks/latest/userguide/kubernetes-versions.html​​

How to use it Managed Node Group Efficient management Amazon EKS colony

​ ​https://aws.amazon.com/cn/blogs/china/how-to-manage-amazon-eks-clusters-with-managed-node-group-preface/​​​​

How to check 、 Expand 、 Delete or exhaust Amazon EKS Worker thread node in

​ ​https://aws.amazon.com/cn/premiumsupport/knowledge-center/eks-worker-node-actions/​​​​

How for my Application Load Balancer Set weighted target group ？

​ ​https://aws.amazon.com/cn/premiumsupport/knowledge-center/elb-make-weighted-target-groups-for-alb/​​​​

EKS Migrate to a new node group

​ ​https://docs.aws.amazon.com/zh_cn/eks/latest/userguide/migrate-stack.html​​

Amazon EKS Add ons

​ ​https://docs.aws.amazon.com/zh_cn/eks/latest/userguide/eks-add-ons.html​​

management  kube-proxy  Add ons

​ ​https://docs.amazonaws.cn/eks/latest/userguide/managing-kube-proxy.html​​

management CoreDNS Add ons

​ ​https://docs.amazonaws.cn/eks/latest/userguide/managing-coredns.html​​