Kubedm builds kubernetes cluster

One . Cluster introduction

1.1 Cluster building method

The current production deployment Kubernetes There are two main ways to cluster ：

1. Kubeadm
   Kubeadm It's a K8s Deployment tools , Provide kubeadm init and kubeadm join, For rapid deployment Kubernetes colony .
2. Binary package
   from github Download the distribution's binary package , Manually deploy each component , form Kubernetes colony .

Kubeadm Lower deployment threshold , But it's a lot of detail , It's hard to troubleshoot problems . If you want to be more controllable , Binary package deployment is recommended Kubernetes colony , Although manual deployment is troublesome , You can learn a lot about how it works , It's also good for later maintenance .

1.2 Cluster architecture

At present, one master and one slave are generally used to build clusters 、 Multi master and multi slave cluster architecture （ High availability ）

Two . Cluster deployment

This article uses one master and two slaves , A total of three servers are built k8s colony

2.1 Environment configuration

The following operations are required for all three servers

1. close swap

# Temporarily Closed  swapoff -a 
# Permanent ban , Need to restart  
sed -i 's/.*swap.*/#&/' /etc/fstab 
# see ,swap It has to be for 0 
free -g

2. Add host mapping

echo "192.168.48.14 k8s" >>/etc/hosts
echo "192.168.48.13 k8s-node1" >>/etc/hosts
echo "192.168.48.11 k8s-node2" >>/etc/hosts

3. To be bridged IPv4 Flow to iptables Chain

cat > /etc/sysctl.d/k8s.conf << EOF net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 net.ipv4.ip_forward=1 EOF
# Refresh 
sysctl --system

4. docker install

# Turn off the firewall and selinux
systemctl stop firewalld
systemctl disable firewalld
setenforce 0
sed -i '/SELINUX/s/enforcing/disabled/' /etc/selinux/config
# Install dependency packages 
yum install -y yum-utils device-mapper-persistent-data lvm2
# Configure alisource 
yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo 
# install docker
yum install -y docker-ce

systemctl start docker
systemctl enable docker
# modify docker Resource management 
vim /etc/docker/daemon.json
{
    
  "exec-opts": ["native.cgroupdriver=systemd"]
}
systemctl daemon-reload
systemctl restart docker

5. To configure K8s Source

cat > /etc/yum.repos.d/kubernetes.repo << EOF [kubernetes] name=Kubernetes baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64 enabled=1 gpgcheck=0 repo_gpgcheck=0 gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg EOF

# Install the software 
yum install -y kubelet-1.21.3 kubeadm-1.21.3 kubectl-1.21.3

systemctl enable kubelet
systemctl start kubelet

2.2 master Node making

stay master Master server operation , Perform cluster initialization , Pay attention to changing your own ip Information

kubeadm init --apiserver-advertise-address=192.168.48.14 --image-repository registry.cn-hangzhou.aliyuncs.com/google_containers --kubernetes-version v1.21.3 --service-cidr=10.125.0.0/16 --pod-network-cidr=10.150.0.0/16

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

ps： In the command to join the cluster token== Valid for 24 Hours ==, If overdue , Need to reapply token

# Generate token
kubeadm token create
# see token
kubeadm token list  | awk -F" " '{print $1}'
# obtain CA Public key hash value 
openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed  's/^ .* //'

2.3 node Nodes join the cluster

kubeadm join 192.168.48.14:6443 --token brc1q1.tnuas794b21c6l7a \
	--discovery-token-ca-cert-hash sha256:c3b8b3b770b60af4bc884b4034cc9ab922ae49700f41628b83a25b651e72111d 

2.4 Add network plug-ins

###################### Network add ############################
# add to 
kubectl apply -f \
https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
# Delete 
kubectl delete -f \
https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
######################weget download ##########################
wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
# Changing configuration files 
vim kube-flannel.yml
# take IP Change to master When making nodes IP Address 
 84       "Network": "10.150.0.0/16"
# Add the plug-in 
kubectl apply -f + kube-flannel.yml The path of 

to node Add tags

kubectl label node k8s-node1 node-role.kubernetes.io/node=node
kubectl label node k8s-node2 node-role.kubernetes.io/node=node
# Get node information 
kubectl get nodes

master Check

# Inquire about master If there is something wrong 
kubectl get cs
# if unhealthy
vim /etc/kubernetes/manifests/kube-scheduler.yaml 
vim /etc/kubernetes/manifests/kube-controller-manager.yaml
- --port=0       # Comment out , Each file has a line  

kube-controller-manager.yaml File modification ： Comment out 27 That's ok

kube-scheduler.yaml Configuration modification ： Comment out 19 That's ok ,- --port=0

2.5 Service deployment

testing K8s Whether the cluster is normal , Then run the service deployment

# Query all pod Is it working 
kubectl get pods -A
# Inquire about master If there is something wrong 
kubectl get cs
# Inquire about node Is the node ready
kubectl get nodes

# Deployment Services 
kubectl create deployment nginx --image=nginx:1.14
# Exposed port 
kubectl expose deployment nginx --port=80 --type=NodePort
kubectl get pods
kubectl get svc
# Delete pod And svc
kubectl delete deploy/nginx
kubectl delete svc/nginx

Access test ：

2.6 Error message

If the following error occurs ：

Failed to run kubelet" err="failed to run Kubelet: misconfiguration: kubelet cgroup driver: \"cgroupfs\" is diffelet: misconfiguration: kubelet cgroup driver: \"cgroupfs\" is different from docker cgroup driver: \"systemd\""

Error reporting fault ：docker And k8s Different resource managers kubelet cgroup driver: “cgroupfs” is different from docker cgroup driver: “systemd”"
resolvent ： modify docker And k8s The resource manager of is systemd

Inquire about docker Of cgroup

#docker modify 
vim /etc/docker/daemon.json
{
    
  "exec-opts": ["native.cgroupdriver=systemd"]
}
# Restart the service 
systemctl daemon-reload
systemctl restart docker
#k8s Modification method 
vim /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
# or vim /usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf
--cgroup-driver=systemd

# Restart the service 
systemctl daemon-reload
systemctl restart kubelet