当前位置:网站首页>Kubernetes - problems and Solutions
Kubernetes - problems and Solutions
2022-07-06 10:39:00 【Zhuang Xiaoyan】
Abstract
Kubernetes All kinds of problems in the process of operation , So I sort out what I met about k8s Related problems and solutions of
One 、k8s Restart error :The connection to the server 192.168.102.149:6443 was refused
1.1 The phenomenon
k8s Restart error
# kubectl get pods
The connection to the server xxx:6443 was refused - did you specify the right host or port?
1.2 Troubleshoot problems
According to the error report description , Connect kubelet Of 6443 Port rejected :
View the status of the port
The display port is not started :
ss -antulp | grep :6443
The port is kubelet Of api Listening port , Should be kubelet Boot failure , Try to restart , see kubelet state , Sure enough, the startup failed , Analysis log
systemctl status kubelet
journalctl -xefu kubelet
It is possible that some components failed to start , View container status , I found that none of the components started up , restart docker And related containers , Report errors
[[email protected] ~]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
56f463b5684b 9b60aca1d818 "kube-controller-man…" 40 hours ago Exited (2) 39 hours ago k8s_kube-controller-manager_kube-controller-manager-master_kube-system_8f99a56fb3eeae0c61283d6071bfb1f4_5
5043f1103f1f aaefbfa906bd "kube-scheduler --au…" 40 hours ago Exited (2) 39 hours ago k8s_kube-scheduler_kube-scheduler-master_kube-system_285062c53852ebaf796eba8548d69e43_5
2d707069ab22 bfe3a36ebd25 "/coredns -conf /etc…" 41 hours ago Exited (0) 39 hours ago k8s_coredns_coredns-6d56c8448f-mt7vz_kube-system_abc65488-0a54-4a1a-8e23-339f3f23f6d2_0
0dadfca20cb7 bfe3a36ebd25 "/coredns -conf /etc…" 41 hours ago Exited (0) 39 hours ago k8s_coredns_coredns-6d56c8448f-hdtlf_kube-system_e1f90d02-77d0-4529-bea5-b4a72cdb4cf5_0
f25051c775cf registry.aliyuncs.com/google_containers/pause:3.2 "/pause" 41 hours ago Exited (0) 39 hours ago k8s_POD_coredns-6d56c8448f-mt7vz_kube-system_abc65488-0a54-4a1a-8e23-339f3f23f6d2_0
b24a10712152 registry.aliyuncs.com/google_containers/pause:3.2 "/pause" 41 hours ago Exited (0) 39 hours ago k8s_POD_coredns-6d56c8448f-hdtlf_kube-system_e1f90d02-77d0-4529-bea5-b4a72cdb4cf5_0
fed8e33864c1 e708f4bb69e3 "/opt/bin/flanneld -…" 41 hours ago Exited (137) 39 hours a
[[email protected] ~]# docker start $(docker ps -a | awk '{ print $1}' | tail -n +2)
Error response from daemon: cgroup-parent for systemd cgroup should be a valid slice named as "xxx.slice"
Error response from daemon: cgroup-parent for systemd cgroup should be a valid slice named as "xxx.slice"
Error response from daemon: cgroup-parent for systemd cgroup should be a valid slice named as "xxx.slice"
Error response from daemon: cgroup-parent for systemd cgroup should be a valid slice named as "xxx.slice"
1.3 Solution
According to the error description , yes docker Configuration file configuration driver configuration error , You can comment out , restart docker, restart kubelet( Do not restart the container manually , Because there is a start-up sequence between containers , If you don't know , Manual restart is not recommended )
Two 、kubectl Command execution error :(Unable to connect to the server: x509: certificate signed by unknown authority )
2.1 The phenomenon
kubectl get nodes Error report in execution :
kubectl get nodes
Unable to connect to the server: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubernetes")
2.2 Question why
/root/.kube/config Certificate authentication error , Use the correct certificate , May be kubeadm reset Certificate not deleted
2.3 Solution
Delete the original certificate and cache file
#rm -rf /root/.kube/*
Copy master Just connect the certificate on the node to the directory
k8s Single node deployment (master ,node part ) - A confidant - Blog Garden
3、 ... and 、k8s Cluster replacement runc:docker->containerd
3.1 be based on kubeadm Installed kubelet Solution
# Use kubeadm View default configuration :
kubeadm config print init-defaults --component-configs KubeletConfiguration
If you want to change the runtime from the default docker Switch to containerd, Then you need to modify the file :
vim /var/lib/kubelet/kubeadm-flags.env
stay KUBELET_KUBEADM_ARGS Add the following parameters :
--container-runtime=remote --container-runtime-endpoint=/run/containerd/containerd.sock
example
KUBELET_KUBEADM_ARGS="--container-runtime=remote --container-runtime-endpoint=/run/containerd/containerd.sock --pod-infra-container-image=registry.aliyuncs.com/google_containers/pause:3.5"
3.2 Based on... Deployed directly using executable files kubelet Solution
modify /usr/lib/systemd/system/kubelet.service file , Add startup parameters :
--container-runtime=remote --container-runtime-endpoint=unix:///run/containerd/containerd.sock
#systemctl daemon-reload && systemctl restart kubelet
kubeadm Used drop-in How to manage kubelet service , This modification kubelet Launch parameters , Directly modifying /usr/lib/systemd/system/kubelet.service The file will not work ,
Four 、coredns Access certificate error
4.1 The phenomenon
kubectl describe pod -n kube-system coredns-757569d647-qj8ts
Failed to create pod sandbox: rpc error: code = Unknown desc = [failed to set up sandbox container "b7ea16c5b21e06069d1418b322e04bd2da482acdf21f863f47c96a80c551eab5" network for pod "coredns-757569d647-qj8ts": networkPlugin cni failed to set up pod "coredns-757569d647-qj8ts_kube-system" network: error getting ClusterInformation: Get https://[10.31.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubernetes"), failed to clean up sandbox container "b7ea16c5b21e06069d1418b322e04bd2da482acdf21f863f47c96a80c551eab5" network for pod "coredns-757569d647-qj8ts": networkPlugin cni failed to teardown pod "coredns-757569d647-qj8ts_kube-system" network: error getting ClusterInformation: Get https://[10.31.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubernetes")]
Various methods are used to compare coredns Generated ciphertext
kubectl get secrets -n kube-system coredns-token-xc8kc -o yaml
Discover and host /etc/kubernetes/admin.conf Recorded in the document ca Ciphertext is as like as two peas . Unable to access kube-apiserver Service for .
Use ipvsadm -Ln The command check didn't find anything wrong .
The final solution is , hold admin.conf Medium ca Ciphertext decryption .certificate-authority-data: The following is copied into a text . such as ca.txt, And then use base64 -d ./ca.txt Command restore Certificate . Then save the certificate to /etc/pki/ca-trust/source/anchors/kube.pem in . modify coredns Of deploy Mount Directory . add to pki mount
4.2 Solution
volumeMounts:
- name: config-volume
mountPath: /etc/coredns
readOnly: true
- name: etc-pki
mountPath: /etc/pki
readOnly: true
volumes:
- name: config-volume
configMap:
name: coredns
items:
- key: Corefile
path: Corefile
- hostPath:
path: /etc/pki
type: DirectoryOrCreate
name: etc-pki
边栏推荐
- [Li Kou 387] the first unique character in the string
- [Julia] exit notes - Serial
- Super detailed steps for pushing wechat official account H5 messages
- Download and installation of QT Creator
- 使用OVF Tool工具从Esxi 6.7中导出虚拟机
- MySQL 29 other database tuning strategies
- Yum prompt another app is currently holding the yum lock; waiting for it to exit...
- Not registered via @enableconfigurationproperties, marked (@configurationproperties use)
- CSDN博文摘要(一) —— 一个简单的初版实现
- MySQL的存储引擎
猜你喜欢
Record the first JDBC
MySQL23-存储引擎
Bytetrack: multi object tracking by associating every detection box paper reading notes ()
Mysql32 lock
实现微信公众号H5消息推送的超级详细步骤
MySQL23-存儲引擎
高并发系统的限流方案研究,其实限流实现也不复杂
Super detailed steps to implement Wechat public number H5 Message push
Mysql34 other database logs
Security design verification of API interface: ticket, signature, timestamp
随机推荐
Bytetrack: multi object tracking by associating every detection box paper reading notes ()
UnicodeDecodeError: ‘utf-8‘ codec can‘t decode byte 0xd0 in position 0成功解决
MySQL combat optimization expert 10 production experience: how to deploy visual reporting system for database monitoring system?
MySQL combat optimization expert 09 production experience: how to deploy a monitoring system for a database in a production environment?
MySQL real battle optimization expert 11 starts with the addition, deletion and modification of data. Review the status of buffer pool in the database
MySQL combat optimization expert 05 production experience: how to plan the database machine configuration in the real production environment?
百度百科数据爬取及内容分类识别
Opencv uses freetype to display Chinese
Good blog good material record link
ZABBIX introduction and installation
Transactions have four characteristics?
MySQL combat optimization expert 12 what does the memory data structure buffer pool look like?
MySQL23-存儲引擎
Solution to the problem of cross domain inaccessibility of Chrome browser
MySQL21-用户与权限管理
MySQL combat optimization expert 04 uses the execution process of update statements in the InnoDB storage engine to talk about what binlog is?
[leectode 2022.2.13] maximum number of "balloons"
API learning of OpenGL (2004) gl_ TEXTURE_ MIN_ FILTER GL_ TEXTURE_ MAG_ FILTER
Solve the problem of remote connection to MySQL under Linux in Windows
What is the current situation of the game industry in the Internet world?