Network file storage system (II) practical operation of Minio distributed file system

introduction

minio summary

MinIO It's based on Go High performance object storage services for languages . It USES a Apache License v2.0 Open source licenses , Amazon compatible S3 Cloud storage service interface , It is very suitable for storing large capacity unstructured data .
MinIO Can be used as the primary storage for cloud native applications , Compared with traditional object storage , Cloud native applications need higher throughput and lower latency . And these are all MinIO Performance indicators that can be achieved .

minio framework

MinIO With local drive （JBOD / JBOF） On the product server . All servers in the cluster have the same functions （ Completely symmetrical architecture ）. No name node or metadata server .

MinIO Write data and metadata together as objects , Thus, there is no need to use metadata database . Besides ,MinIO Inline , Strictly and consistently perform all functions （ Erase the code , position rotrot Check , encryption ）. The result is MinIO remarkable dexterity .

Every MinIO Clusters are distributed MinIO A collection of servers , One process per node . MinIO Run in user space as a single process , And use lightweight collaborative routines to achieve high concurrency . Group drives into erase sets （ By default , Each group 16 A drive ）, Then the object is placed on these erasure sets using a deterministic hash algorithm .

MinIO Designed for large scale , Designed for cloud storage services in multiple data centers . Each tenant runs its own MinIO to cluster around , The cluster is completely isolated from other tenants , So that they can protect them from escalation , Any interference with updates and security events . Each tenant expands independently by combining clusters across geographical regions .

Usage method

minio download , Just go to the mirror website , Here the little ape will not post the connection , The following is mainly about how we can centos To use minio

step1 Upload and modify permissions

# Upload files 
rz  xxxx
#  Modify the permissions 
chmod +x minio

step2 Prepare data file directories and log files

mkdir -p /data/minio
touch minio.log

step3 Set the console password

vim /etc/profile
# Set console account （ least 3 position ）
export MINIO_ACCESS_KEY=feitian
export MINIO_SECRET_KEY=feitian_minio
source /etc/profile

step4 start-up

# Background start 
nohup /usr/local/minio server --address :9800 --console-address :9889 /data/minio >/data/minio/minio.log 2>&1 &

explain

  nohup： Background start 
   ./minio server： Start command 
   --address :9800： Appoint API port 
   --console-address :9889： Specify the console port 
  /home/minio/data： Specify the storage directory 
  >/home/minio/minio.log 2>&1 ： Redirect console logs to /home/minio/minio.log In file 
  &： Background operation 

step5 Sign in

step6 close minio

## see minio process 
ps -ef|grep minio 
##  obtain pid xxx  by pid
kill -9 xxx

Client side usage

stay pom Add the following dependencies to

     <!-- Minio -->
     <dependency>
         <groupId>io.minio</groupId>
         <artifactId>minio</artifactId>
         <version>8.2.2</version>
     </dependency>

Configuration class

@Configuration
@ConfigurationProperties(prefix = "minio")
public class MinioConfig
{
    
    /** *  Service address  */
    private String url;

    /** *  user name  */
    private String accessKey;

    /** *  password  */
    private String secretKey;

    /** *  Bucket name  */
    private String bucketName;

    public String getUrl()
    {
    
        return url;
    }

    public void setUrl(String url)
    {
    
        this.url = url;
    }

    public String getAccessKey()
    {
    
        return accessKey;
    }

    public void setAccessKey(String accessKey)
    {
    
        this.accessKey = accessKey;
    }

    public String getSecretKey()
    {
    
        return secretKey;
    }

    public void setSecretKey(String secretKey)
    {
    
        this.secretKey = secretKey;
    }

    public String getBucketName()
    {
    
        return bucketName;
    }

    public void setBucketName(String bucketName)
    {
    
        this.bucketName = bucketName;
    }

    @Bean
    public MinioClient getMinioClient()
    {
    
        return MinioClient.builder().endpoint(url).credentials(accessKey, secretKey).build();
    }
}

Data access class

@Primary
@Service
public class MinioSysFileServiceImpl implements ISysFileService
{
    
    @Autowired
    private MinioConfig minioConfig;

    @Autowired
    private MinioClient client;

    /** *  Local file upload interface  * * @param file  Uploaded files  * @return  Access address  * @throws Exception */
    @Override
    public String uploadFile(MultipartFile file) throws Exception
    {
    
        String fileName = FileUploadUtils.extractFilename(file);
        PutObjectArgs args = PutObjectArgs.builder()
                .bucket(minioConfig.getBucketName())
                .object(fileName)
                .stream(file.getInputStream(), file.getSize(), -1)
                .contentType(file.getContentType())
                .build();
        client.putObject(args);
        return minioConfig.getUrl() + "/" + minioConfig.getBucketName() + "/" + fileName;
    }
}

FileUploadUtils class

..........
    /** *  Encoded file name  */
    public static final String extractFilename(MultipartFile file)
    {
    
        return StringUtils.format("{}/{}_{}.{}", DateUtils.datePath(),
                FilenameUtils.getBaseName(file.getOriginalFilename()), Seq.getId(Seq.uploadSeqType), FileTypeUtils.getExtension(file));
    }
 ..........