Kubernetes data persistence scheme

At the beginning k8s Before persistent storage , It is necessary for us to understand k8s Of emptydir and hostpath、configmap as well as secret The mechanism and use of .

1、Emptydir EmptyDir It's an empty directory , His life cycle and what he belongs to Pod It's exactly the same ,EmptyDir The main functions can be in the same Pod The files generated in the working process are shared between different containers in . If Pod Configured with emptyDir type Volume, Pod Be assigned to Node Last time , Will create emptyDir, as long as Pod Running on the Node On ,emptyDir Will exist （ Hanging the container will not cause emptyDir Lost data ）, But if Pod from Node Was deleted on （Pod Be deleted , perhaps Pod There's a migration ）,emptyDir It's also deleted , And lost forever .

# cat emptydir.yaml apiVersion: v1 kind: Pod metadata:   name: busybox spec:   containers:   - name : busybox     image: registry.fjhb.cn/busybox     imagePullPolicy: IfNotPresent     command:       - sleep       - "3600"     volumeMounts:       - mountPath: /busybox-data         name: data   volumes:   - name: data     emptyDir: {}

2、Hostpath Hostpath The specified volume on the host will be loaded into the container , If Pod Cross host reconstruction occurs , Its content is difficult to guarantee . This kind of roll is generally similar to DaemonSet Use it with .hostPath Allow the mount Node On the file system to Pod Go inside . If Pod Need to use Node Things on , have access to hostPath, However, it is recommended to use , Because in theory Pod Should not perceive Node Of .

# cat hostpath.yaml apiVersion: v1 kind: Pod metadata:   name: busybox spec:   containers:   - name : busybox     image: registry.fjhb.cn/busybox     imagePullPolicy: IfNotPresent     command:       - sleep       - "3600"     volumeMounts:       - mountPath: /busybox-data         name: data   volumes:   - hostPath:       path: /tmp     name: data

emptyDir and hostPat Many scenarios cannot meet the requirements of persistence , Because in Pod When migration occurs , Data cannot be transferred , This requires Distributed File system support .

3、Configmap During the use of image , Configuration files are often needed 、 Start scripts and other ways to affect the operation of the container , If there are only a few configurations , We can use environment variables to configure . However, for some more complex configurations ,k8s Provides configmap Solution .　 ConfigMap API Resource storage key / Value pair configuration data , The data can be found in pods Use in . ConfigMap Follow Secrets similar , however ConfigMap It is more convenient to deal with strings that do not contain sensitive information . When ConfigMap Mount in the form of a data volume Pod When the , Now update ConfigMap（ Or delete and rebuild ConfigMap）,Pod The configuration information mounted inside will be updated . At this time, you can add some scripts to monitor configuration file changes , then reload Corresponding services ConfigMap Of API Conceptually speaking, it is very simple . From a data point of view ,ConfigMap The type of is just a key value group . Applications can be configured from different perspectives . In a pod It uses ConfigMap There are roughly three ways ： 1、 Command line arguments 2、 environment variable 3、 Data volume file

Make variables configmap

take nginx The configuration file is made configmap

# cat nginx.conf    user nginx; worker_processes auto; error_log /etc/nginx/error.log; pid /run/nginx.pid;

# Load dynamic modules. See /usr/share/nginx/README.dynamic. include /usr/share/nginx/modules/*.conf;

events {     worker_connections 1024; }

http {     log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '                       '$status $body_bytes_sent "$http_referer" '                       '"$http_user_agent" "$http_x_forwarded_for"';     server_tokens    off;     access_log        /usr/share/nginx/html/access.log  main;

    sendfile            on;     tcp_nopush          on;     tcp_nodelay        on;     keepalive_timeout  65;     types_hash_max_size 2048;

    include            /etc/nginx/mime.types;     default_type        application/octet-stream;

    include /etc/nginx/conf.d/*.conf;

    server { [[email protected] ~]# cat nginx.conf  user nginx; worker_processes auto; error_log /etc/nginx/error.log; pid /run/nginx.pid;

# Load dynamic modules. See /usr/share/nginx/README.dynamic. include /usr/share/nginx/modules/*.conf;

events {     worker_connections 1024; }

http {     log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '                       '$status $body_bytes_sent "$http_referer" '                       '"$http_user_agent" "$http_x_forwarded_for"';     server_tokens    off;     access_log        /usr/share/nginx/html/access.log  main;

    sendfile            on;     tcp_nopush          on;     tcp_nodelay        on;     keepalive_timeout  65;     types_hash_max_size 2048;

    include            /etc/nginx/mime.types;     default_type        application/octet-stream;

    include /etc/nginx/conf.d/*.conf;

    server {         listen      80 default_server;         listen      [::]:80 default_server;         server_name  _;         root        /usr/share/nginx/html;

        include /etc/nginx/default.d/*.conf;

        location / {         }

        error_page 404 /404.html;             location = /40x.html {         }

        error_page 500 502 503 504 /50x.html;             location = /50x.html {         }     }

}# kubectl create configmap nginxconfig --from-file nginx.conf  # kubectl get configmap # kubectl get configmap -o yaml

stay rc Use in profile configmap

# cat nginx-rc-configmap.yaml  apiVersion: v1 kind: ReplicationController metadata:   name: nginx   labels:     name: nginx spec:   replicas: 2   selector:     name: nginx   template:     metadata:       labels:       name: nginx     spec:       containers:       - name: nginx         image: docker.io/nginx         volumeMounts:         - name: nginx-etc           mountPath: /etc/nginx/nginx.conf           subPath: nginx.conf         ports:         - containerPort: 80       volumes:       - name: nginx-etc         configMap:         name: nginxconfig         items:           - key: nginx.conf             path: nginx.conf# kubectl create -f nginx-rc-configmap.yaml

configmap The information is actually stored in etcd Medium , have access to kubectl edit configmap xxx Come on configmap Make changes

# etcdctl ls /registry/configmaps/default # etcdctl get /registry/configmaps/default/nginxconfig

4、Secret Kubemetes Provides Secret To process sensitive data , Like passwords 、Token And the key , Compared with directly configuring sensitive data in Pod In the definition or image of ,Secret Provides a more secure mechanism （Base64 encryption ）, Prevent data leakage .Secret Is created independently of Pod Of , Mount to in the form of data volume Pod in ,Secret The data of will be saved in the form of file , The container can get the required data by reading the file . at present Secret There are types of 3 Kind of ：  Opaque(default): Any string   kubernetes.io/service-account-token: Act on ServiceAccount kubernetes.io/dockercfg: Act on Docker registry, User download docker Image authentication uses secert The specific configuration of is mentioned above serviceaccount Has been introduced in , This article will not be repeated .

Now let's introduce k8s The persistent storage scheme of , at present k8s The supported storage schemes are mainly as follows ： Distributed file system ：NFS/GlusterFS/CephFS Public cloud storage solutions ：AWS/GCE/Auzre

Nfs Storage plan NFS yes Network File System Abbreviation , Network file system .Kubernetes It can be mounted through simple configuration NFS To Pod in , and NFS The data in , meanwhile NFS Supports simultaneous write operations .

1、 First installation nfs

# yum -y install nfs-util* # cat /etc/exports /home 192.168.115.0/24(rw,sync,no_root_squash) # systemctl start rpcbind # systemctl start nfs # showmount -e 127.0.0.1 Export list for 127.0.0.1: /home 192.168.115.0/24

2、 Use pod Mount directly nfs Make sure that colony All of the inside node Nodes can be mounted nfs

# cat nfs.yaml apiVersion: v1 kind: Pod metadata:   name: busybox spec:   containers:   - name : busybox     image: registry.fjhb.cn/busybox     imagePullPolicy: IfNotPresent     command:       - sleep       - "3600"     volumeMounts:       - mountPath: /busybox-nfsdata         name: nfsdata   volumes:   - name: nfsdata     nfs:       server: 192.168.115.6       path: /home

3、 Use PV and PVC In actual use , We usually divide the storage into PV, And then PVC Bind to pod Use . PV：PersistentVolume PVC：PersistentVolumeClaim

PV and PVC Life cycle of ： Ready to supply ： adopt colony External storage systems or public cloud storage solutions to provide storage persistence support . Static provide ： Administrators manually create multiple PV, for PVC Use . Providing dynamic ： Dynamically create PVC specific PV, And bind the .

binding ： The user to create pvc And specify the required resources and access patterns . After finding available pv Before ,pvc It will remain unbound .

Use ： The user can be in pod The Chinese image uses volume The use of pvc.

Release ： User deletion pvc To reclaim the storage resources ,pv Will become “released” state . Because we still have the previous data , These data need to be processed according to different strategies , Otherwise these storage resources cannot be otherwise pvc Use .

Recycling (Reclaiming)：pv You can set three recovery strategies ： Retain （Retain）, Recycling （Recycle） And delete （Delete） Retention policy ： Allows manual processing of retained data . Deletion policy ： Will delete pv The storage resource associated with the external , Need plug-in support . Recovery strategy ： A clear operation is performed , And then it can be renewed pvc Use , Need plug-in support .

PV Volume phase state ： Available – Resources have not yet been PVC Use Bound – The volume has been bound to PVC 了 Released – PVC Be deleted ,PV The volume is in the release state , But not by colony Recycling . Failed – PV Automatic volume recovery failed

PV The access mode of the volume ReadWriteOnce – single node Read and write   ReadOnlyMany – many node Read only   ReadWriteMany – many node Read and write

establish pv And pvc

# cat nfs-pv.yaml apiVersion: v1 kind: PersistentVolume metadata:   name: pv-nfs-001 spec:   capacity:     storage: 5Gi   accessModes:   - ReadWriteMany   nfs:     path: /home     server: 192.168.115.6   persistentVolumeReclaimPolicy: Recycle

# cat nfs-pvc.yaml                  kind: PersistentVolumeClaim apiVersion: v1 metadata:   name: nfs-data spec:   accessModes:     - ReadWriteMany   resources:     requests:       storage: 5Gi

stay PVC binding PV It is usually bound according to two conditions , One is the size of the storage , The other is access mode .

stay rc Use... In the document PVC

# cat nginx-rc-configmap.yaml  apiVersion: v1 kind: ReplicationController metadata:   name: nginx   labels:     name: nginx spec:   replicas: 2   selector:     name: nginx   template:     metadata:       labels:       name: nginx     spec:       containers:       - name: nginx         image: docker.io/nginx         volumeMounts:         - name: nginx-data           mountPath: /usr/share/nginx/html         - name: nginx-etc           mountPath: /etc/nginx/nginx.conf           subPath: nginx.conf         ports:         - containerPort: 80       volumes:       - name: nginx-data         persistentVolumeClaim:         claimName: nfs-data       - name: nginx-etc         configMap:         name: nginxconfig         items:           - key: nginx.conf             path: nginx.conf