Kubernetes 1.24:statefulset introduces maxunavailable copies

author ：Mayank Kumar（Salesforce）

Kubernetes StatefulSets[1] Since the 1.5 Introduction in , And in 1.9 Since it became stable , It has been widely used to run stateful applications . It provides stable unit identity 、 Persistent unit storage , And orderly deployment 、 Expansion and rolling updates . You can take StatefulSet Think of it as an atomic building block for running complex stateful applications . With Kubernetes More and more , need StatefulSets More and more scenes . In your right StatefulSets Use OrderedReady pod Managing policies , Many of these scenarios require more than one at a time than currently supported Pod Update faster scrolling updates .

Here are some examples ：

• I use StatefulSet To orchestrate a multi instance 、 Cache based applications , The size of the cache is very large . Cache cold start , It takes a long time to start the container . More initial startup tasks may be required . Before the application is completely updated , Regarding this StatefulSet Conduct RollingUpdate It will take a long time . If StatefulSet Support updating multiple at a time pod, Then the update speed will be much faster .
• My stateful application is led by （leader） And followers （follower） form , Or by a writer And multiple reader form . I have more than one reader Or followers , My application can tolerate multiple pod Simultaneous closure . I want to update multiple applications at once , So I can get new updates quickly , Especially when there are a large number of instances of my application . Please note that , My application still needs every pod The unique identifier of the .

To support such scenarios ,Kubernetes 1.24 Contains a new alpha Features to help . Before using new features , You have to enable MaxUnavailableStatefulSet Functional logo . Once enabled , You can specify a named maxUnavailable New fields for , This is a StatefulSet Part of the norm . for example ：

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: web
  namespace: default
spec:
  podManagementPolicy: OrderedReady  # you must set OrderedReady
  replicas: 5
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - image: k8s.gcr.io/nginx-slim:0.8
        imagePullPolicy: IfNotPresent
        name: nginx
  updateStrategy:
    rollingUpdate:
      maxUnavailable: 2 # this is the new alpha field, whose default value is 1
      partition: 0
    type: RollingUpdate

Copy

If new features are enabled , And not in StatefulSet It is specified in maxUnavailable Value ,Kubernetes The default... Will be applied maxUnavailable: 1. This matches the behavior you see when you don't enable new features .

I will run a scenario based on the sample manifest to demonstrate how this feature works . I will deploy one with 5 Copies of StatefulSet, among maxUnavailable Set to 2,partition Set to 0.

I can change the image to k8s.gcr.io/nginx-slim:0.9 To trigger a rollover . Once you start rolling updates , I can check it all at once 2 individual pod to update , because maxUnavailable The current value of is 2. The output below shows a period of time , And incomplete .maxUnavailable It can be an absolute number （ for example ,2） Or what you need pod Percent of （ for example ,10%）. Absolute numbers are calculated by rounding down the percentage .

kubectl get pods --watch

Copy

NAME    READY   STATUS    RESTARTS   AGE
web-0   1/1     Running   0          85s
web-1   1/1     Running   0          2m6s
web-2   1/1     Running   0          106s
web-3   1/1     Running   0          2m47s
web-4   1/1     Running   0          2m27s
web-4   1/1     Terminating   0          5m43s ----> start terminating 4
web-3   1/1     Terminating   0          6m3s  ----> start terminating 3
web-3   0/1     Terminating   0          6m7s
web-3   0/1     Pending       0          0s
web-3   0/1     Pending       0          0s
web-4   0/1     Terminating   0          5m48s
web-4   0/1     Terminating   0          5m48s
web-3   0/1     ContainerCreating   0          2s
web-3   1/1     Running             0          2s
web-4   0/1     Pending             0          0s
web-4   0/1     Pending             0          0s
web-4   0/1     ContainerCreating   0          0s
web-4   1/1     Running             0          1s
web-2   1/1     Terminating         0          5m46s ----> start terminating 2 (only after both 4 and 3 are running)
web-1   1/1     Terminating         0          6m6s  ----> start terminating 1
web-2   0/1     Terminating         0          5m47s
web-1   0/1     Terminating         0          6m7s
web-1   0/1     Pending             0          0s
web-1   0/1     Pending             0          0s
web-1   0/1     ContainerCreating   0          1s
web-1   1/1     Running             0          2s
web-2   0/1     Pending             0          0s
web-2   0/1     Pending             0          0s
web-2   0/1     ContainerCreating   0          0s
web-2   1/1     Running             0          1s
web-0   1/1     Terminating         0          6m6s ----> start terminating 0 (only after 2 and 1 are running)
web-0   0/1     Terminating         0          6m7s
web-0   0/1     Pending             0          0s
web-0   0/1     Pending             0          0s
web-0   0/1     ContainerCreating   0          0s
web-0   1/1     Running             0          1s

Copy

Please note that , Once the rolling update starts ,4 and 3（ The two highest serial numbers pod） Will terminate at the same time . Serial number is 4 and 3 Of pod You can prepare at your own pace . once 4 Number and 3 Number pod Be ready ,2 Number pod and 1 Number pod Start and end at the same time . When pod 2 and 1 Are running and ready ,pod 0 Start and end .

stay Kubernetes in , to update Pod when , Yes StatefulSets Updates for follow a strict sequence . In this example , Update from replica 4 Start , Then there is the copy 3, Then there is the copy 2, And so on , One at a time pod. When processing one at a time pod when ,3 It can't be in 4 Before running and ready . When maxUnavailable Greater than 1 when （ In the example scenario , I will maxUnavailable Set to 2）, copy 3 Maybe in the copy 4 It is ready and running before it is ready , That's ok . If you're a developer , And will maxUnavailable Set to greater than 1, Then you should know that this result is possible , And you have to make sure that your application can handle possible sorting problems . When you will maxUnavailable Set to greater than 1 when , Each batch will be guaranteed pod The order between updates . This guarantee means updating the batch 2 Medium pod（ copy 2 and 1） Cannot start update , Until batch 0 Medium pod（ copy 4 and 3） Be ready .

Even though Kubernetes Call these copies （replica）, But your stateful application may have different views , also StatefulSet Each pod May hold with others pod Completely different data . What is important here is to StatefulSets The update of is in batch , You can now have more than 1 Batch size for （ As alpha characteristic ）.

Attention, please. , The above behavior is aimed at podManagementPolicy: OrderedReady Of . If you will StatefulSet Defined as podManagementPolicy: Parallel, Not only maxUnavailable The number of copies is terminated at the same time ;maxUnavailable The number of copies is also equal to ContainerCreating Phase start . This is called bursting.

therefore , Now you may have a lot of questions about ：

• Set up podManagementPolicy: Parallel What is the system condition at ？
• When partition by 0 What happens when a value other than ？

It might be better if you try it yourself . This is a alpha characteristic ,Kubernetes Our contributors are looking for feedback on this feature . Does this help you implement your stateful scenarios ？ Did you find a bug, Or do you think the behavior of the implementation is not intuitive , Or it might break applications or take them by surprise ？ please Submit questions [2] Let us know .

Extended reading and next step

• Maximum unavailable Pod[3]
• KEP[4]
• Realization [5]
• enhance [6]

Reference material

[1]

StatefulSets: https://kubernetes.io/zh/docs/concepts/workloads/controllers/statefulset/

[2]

Submit questions : https://github.com/kubernetes/kubernetes/issues

[3]

Maximum unavailable Pod: https://kubernetes.io/zh/docs/concepts/workloads/controllers/statefulset/#%E6%9C%80%E5%A4%A7%E4%B8%8D%E5%8F%AF%E7%94%A8-pod

[4]

KEP: https://github.com/kubernetes/enhancements/tree/master/keps/sig-apps/961-maxunavailable-for-statefulset

[5]

Realization : https://github.com/kubernetes/kubernetes/pull/82162/files

[6]

enhance : https://github.com/kubernetes/enhancements/issues/961

---

CNCF (Cloud Native Computing Foundation) Founded on 2015 year 12 month , Affiliated to the Linux Foundation, It's a non-profit organization .

CNCF（ Cloud native Computing Foundation ） Committed to fostering and maintaining a vendor neutral open source ecosystem , To promote cloud native technology . By democratizing the most cutting-edge model , Let these innovations be used by the public