Root cause analysis ｜ inventory of nine scenarios with abnormal status of kubernetes pod

Kubernetes Pod As Kubernetes Core resource object , Not only Service、Controller、Workload All work around it . It is the smallest scheduling unit , It also serves as a tradition IT Responsibilities of the environment host , Including scheduling , The Internet , Storage , Safety and other capabilities .

Precisely because Pod With complex lifecycles and dependencies , most Kubernetes The problem will eventually be Pod Show up on . therefore , We will introduce the problems we will encounter in practical work 9 A typical scene , And how to use it Kubernetes Monitoring to handle these scenarios , Quickly locate and find problems .

A container is a user process ,Pod Like a machine , So scheduling , The Internet , Storage , Machine level exceptions such as security and process running exceptions will be found in Pod It is reflected in the above . Around Pod Come on , There are several key points that are very prone to problems ：

• Dispatch

• Mirror pull

• Disk mount

• Liveless/Readiness probe

• postStart/preStop handler

• To configure

• Runtime  

that , Next, let's take stock of related common problem scenarios .

Problem scenario 1： Ready failed , namely Pod Has been unable to reach Ready state , Unable to receive request for business processing .

Common root causes are as follows ：

• Insufficient resources , Unable to schedule （Pending）, That is to say, cluster Node There are no reserved resources to meet Pod Of Request resources ;

• Image pull failed （ ImagePullBackoff ）, The warehouse address of the image ,tag Problems arise ;

• Disk mount failed （Pending）, The container holds PVC No, bound;

• Liveless probe Probe failed , Frequent restart ;

• Readiness probe Probe failed , Unable to reach Ready state ;

• postStart Execution failure , Has been unable to enter the running state ;

• Runtime program crash （ CrashLoopBackOff ）, Frequent restart ;

• Configuration error , Such as mounted Volume non-existent （RunContainerError）.