Kubernetes version docking object storage

Object storage

Want to create  PersistentVolume（PV）/PersistentVolumeClaim（PVC）, And use cloud object storage to mount data volumes for workloads  COS

For object storage , stay k8s There is no need for PV/PVC To do resource abstraction , Applications can directly access and use , If you need to k8s Support access , It's going to take a corresponding csi plug-in unit , Support object storage to file storage, such as ：  The way of several big factories, Ali ：https://help.aliyun.com/document_detail/130911.html tencent ：https://cloud.tencent.com/document/product/457/44232 Huawei ：https://support.huaweicloud.com/intl/zh-cn/usermanual-cce/cce_01_0267.html

If the docking file is stored , You can use S3fs You can put Bucket Mount as a folder to Linux Inside the system , Use as a system folder .

Community programs

programme 1：Object Storage API (COSI)

Currently in development , immature

Official documents

https://container-object-storage-interface.github.io/docs/

https://container-object-storage-interface.github.io/

https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/1979-object-storage-support#provisionergetinfo

Kubernetes v1.23  Official release , What are the enhancements ？

Kubernetes  Now it has better extended support for file and block storage  (CSI), But these do not support object storage very well , Here's why ：

• oss  In barrels  (bucket)  To organize and allocate storage units instead of file system mounts or block devices

• oss  Services are accessed through network calls, not locally  POSIX  call

• oss  No trial  csi  Defined  Attach/Detach  Logic  ( No need to mount / uninstall )

COSI  It's about how to container workloads  (Pod)  Provide object storage services  (oss)  Standard agreement for . And  csi  and  cni  similar ,Kubernetes  It aims to communicate with third parties by defining some standard interfaces  oss  Service provider decoupling .

[ Failed to transfer the external chain picture , The origin station may have anti-theft chain mechanism , It is suggested to save the pictures and upload them directly (img-JJJmj72m-1651055590510)(https://cdn.jsdelivr.net/gh/Fly0905/[email protected]/img/202204252104317.jpeg)]

programme 2：csi-s3

have access to , But more testing is needed

Official documents

https://github.com/CTrox/csi-s3

https://github.com/majst01/csi-driver-s3

Use s3(minio) by kubernetes Provide pv Storage

Kubernetes  requirement

• Kubernetes 1.13+（CSI v1.0.0  Compatibility ）

• <font color=red>
  Kubernetes  Privilege containers must be allowed
  </font>

• Docker  Daemons must allow shared mounts （systemd  sign
  MountFlags=shared
  ）

• <font color=red>
  It's very experimental , Not yet used in any production environment . According to the mounting procedure and  S3  Storage back end , Unexpected data loss may occur .
  </font>

MountFlags

stay  18.09  Previous  Docker  In the version ,containerd  from  Docker  Engine daemon management .

stay  Docker Engine 18.09  in ,containerd  from  systemd  management . because  containerd  from  systemd  management , So anything

docker.service

Change the of Mount settings  systemd  Custom configuration （ for example ,

MountFlags=slave

） Will destroy  Docker Engine  Daemons and  containerd  Interaction between , And you will not be able to start the container .

Run the following command to get   Of

MountFlags

Current value of property

docker.service

：

$ sudo systemctl show --property=MountFlags docker.service
MountFlags=


If this command is   Print non null values , Please update your configuration

MountFlags

, And then restart it  docker  service .

Why would you S3  Mount to... In the form of file storage  Kubernetes  platform ?

For applications that used to use local directories to access data , For example, an application system that uses a local disk or a network shared disk to save data , If the user wants to put the data in S3 On , You need to modify the data access method , For example, modify to use SDK  or CLI visit S3 Data stored in .

At the same time to realize kubernetes Cluster is the demand of many users , Whether using managed services or self built kubernetes colony , Storage is kubernetes The focus of cluster construction . also docker The deployment method also allows clients to reduce their dependence on the underlying environment . In order to enable users to use the original application system directly without modification S3 service ,

Need to put S3 The bucket is mounted to the user as a directory kubernetes In the cluster worker Node

.

utilize S3fs take S3 The storage bucket is kubernetes On the platform

sidecar

How to mount to kubernetes Clustered worker Example of pod in , After mounting, you need to read and write the contents of this bucket pod Can read and write to this bucket , To realize the shared storage function .

What is?  S3FS ？

S3fs Is based on FUSE File system

, allow Linux and Mac Os X  mount S3 Of the buckets in the local file system ,S3fs Be able to keep the original format of the object ,S3FS yes POSIX A large subset of , Including reading / Writing documents 、 Catalog 、 A symbolic link 、 Pattern 、uid/gid And extended properties ,

And

AmazonS3、Google Cloud storage and others

be based on S3 Object storage compatibility

. About S3fs Detailed introduction , Please see the ：https://github.com/s3fs-fuse/s3fs-fuse

follow-up

• Verify the privilege container , How many privileges have been upgraded

Reference link

• Use s3(minio) by kubernetes Provide pv Storage

• be based on openshift+ Huawei objects store CSI Development

• utilize  S3FS  take  S3  Mount as shared storage to  Kubernetes Pod

• Use S3fs stay Linux Mount on instance Bucket

• S3FS： File system based on object storage

• https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.23/#podsecuritycontext-v1-core

• https://github.com/FLY-Open-K8s/csi-driver-s3.git