当前位置：网站首页>Deployment and use of Minio distributed object storage

Deployment and use of Minio distributed object storage

2022-07-28 11:47:00 【Say it, say it】

One 、 Deploy

download Minio direct Github Search ,minio/minio: Multi-Cloud Object Storage

1.1 Single deployment

Stand alone deployment command , Specify the static port 9001.

Generally, it will occupy two ports ,9000 And specified 9001, The actual visit is 9001.

mkdir data
#  Grant Execution Authority 
chmod +x minio
#  Start command 
MINIO_ROOT_USER=minioadmin MINIO_ROOT_PASSWORD=minioadmin nohup ./minio server data/ --console-address=:9001 >log 2>&1 &

It is recommended to create the startup command as bash Script , Easy to carry out

My machine ip yes 10.0.0.10, So go straight to 10.0.0.10:9000 that will do , Will automatically jump to 9001.

1.2 Cluster deployment

Cluster deployment , Multiple machines can use the same set of startup commands .

Hard requirements ： At least 4 individual driver, Here I understand it as partition .

At least 4 individual driver, Because of distributed Minio To enable the Erasure code algorithm To ensure data security , This algorithm guarantees , As long as it doesn't exceed N/2 The block partition is broken , Data will not be lost .

Deployment cluster , I can choose 2 Nodes , Every node 2 individual driver, perhaps 4 Nodes , Every node 1 individual driver

The way I use is the latter .

Get ready 4 Servers

10.0.0.10
10.0.0.11
10.0.0.12
10.0.0.13

Each machine executes the following commands .

#  establish minio Data storage directory 
mkdir /minio
#  View all partitions 
fdisk -l
#  belt * The primary partition of , My is /dev/sda1, Mount the data storage directory to the primary partition 
mount /dev/sda1 /minio
#  Temporary environment variables , To configure root The username and password of 
export MINIO_ROOT_USER=xiangwan 
export MINIO_ROOT_PASSWORD=xiangwan
#  Start cluster 
nohup /root/minio server http://10.0.0.10/minio http://10.0.0.11/minio http://10.0.0.12/minio http://10.0.0.13/minio --console-address=:9001 >log 2>&1 &

After all four machines are executed , It can be done by cat log Check the log , Here's the picture .

Visit any one of them , I will visit directly 10.0.0.10:9000, You can view node monitoring information

Two 、 Official use SDK

Example demo Is pure maven project ,meethigher/minio-usage: Object storage minio Use

2.1 Basic configuration

pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>top.meethigher</groupId>
    <artifactId>minio-usage</artifactId>
    <version>1.0.0</version>

    <properties>
        <maven.compiler.source>8</maven.compiler.source>
        <maven.compiler.target>8</maven.compiler.target>
    </properties>

    <dependencies>
        <dependency>
            <groupId>io.minio</groupId>
            <artifactId>minio</artifactId>
            <version>7.1.0</version>
        </dependency>
        
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.13.2</version>
            <scope>test</scope>
        </dependency>
    </dependencies>
</project>

create profile application.properties

endpoint=http://10.0.0.10:9000
accessKey=xiangwan
secretKey=xiangwan

Create configuration class MinioConfig

package top.meethigher.config;

import java.io.IOException;
import java.util.Properties;

/** * minio To configure  * * @author chenchuancheng * @since 2022/7/8 14:41 */
public class MinioConfig {
    
    // Connect url
    public static String endpoint;
    // Public key 
    public static String accessKey;
    // Private key 
    public static String secretKey;

    static {
    
        Properties prop = new Properties();
        try {
    
            prop.load(MinioConfig.class.getClassLoader().getResourceAsStream("application.properties"));
            endpoint=prop.getProperty("endpoint");
            accessKey=prop.getProperty("accessKey");
            secretKey=prop.getProperty("secretKey");
        } catch (IOException e) {
    
            e.printStackTrace();
        }
    }
}

establish MinioClientBuilder

package top.meethigher.utils;

import io.minio.MinioClient;
import top.meethigher.config.MinioConfig;

/** *  obtain MinioClientBuilder * * @author chenchuancheng * @since 2022/7/8 14:42 */
public class MinioClientBuilder {
    

    private static MinioClient minioClient = null;

    private MinioClientBuilder() {
    
        minioClient = MinioClient.builder()
                .endpoint(MinioConfig.endpoint)
                .credentials(MinioConfig.accessKey, MinioConfig.secretKey)
                .build();
    }

    private static final class MinioClientBuilderHolder {
    
        static final MinioClientBuilder minioClientBuilder = new MinioClientBuilder();
    }

    public static MinioClient build() {
    
        MinioClientBuilder minioClientBuilder = MinioClientBuilderHolder.minioClientBuilder;
        return minioClientBuilder.getMinioClient();
    }

    private MinioClient getMinioClient() {
    
        return minioClient;
    }
}

2.2 api Use

Upload function

MinioClient minioClient = MinioClientBuilder.build();
UploadObjectArgs args = UploadObjectArgs.builder()
        .bucket("xiangwan")
        .object("wuhuang.jpg")
        // Create and upload to test Folder 
        //.object("test/wuhuang.jpg")
        .filename("C:\\Users\\14251\\Desktop\\ My Emperor .png")
        .build();
minioClient.uploadObject(args);

Download function

MinioClient minioClient = MinioClientBuilder.build();
GetObjectArgs args = GetObjectArgs.builder()
        .bucket("xiangwan")
        .object("wuhuang.jpg")
        .build();
InputStream is = minioClient.getObject(args);
FileOutputStream fos = new FileOutputStream("xiangwan.jpg");
int len;
byte[] b=new byte[1024];
while((len=is.read(b))!=-1) {
    
    fos.write(b,0,len);
}
fos.close();
is.close();

Query function

MinioClient minioClient = MinioClientBuilder.build();
ListObjectsArgs args= ListObjectsArgs.builder()
        .bucket("xiangwan")
        // Specify folders 
        //.prefix("test/")
        .build();
Iterable<Result<Item>> results = minioClient.listObjects(args);
for (Result<Item> next : results) {
    
    Item item = next.get();
    boolean dir = item.isDir();
    if (dir) {
    
        System.out.printf(" file name :%s, File modification time :0, file size :0, file type :%s\n", item.objectName(),
                item.isDir());
    } else {
    
        System.out.printf(" file name :%s, File modification time :%s-%s-%s %s:%s:%s, file size :%s, file type :%s\n", URLDecoder.decode(item.objectName(), "utf-8"),
                item.lastModified().getYear(), item.lastModified().getMonthValue(), item.lastModified().getDayOfMonth(),
                item.lastModified().getHour(), item.lastModified().getMinute(), item.lastModified().getSecond(),
                item.size(),
                item.isDir());
    }
}

The operation result is as shown in the figure

Delete function

MinioClient minioClient = MinioClientBuilder.build();
// Removing a single 
RemoveObjectArgs args= RemoveObjectArgs.builder()
        .bucket("xiangwan")
        .object("log")
        .build();
minioClient.removeObject(args);
        
// To delete multiple 
Iterable<Result<Item>> results = minioClient.listObjects(ListObjectsArgs.builder()
        .bucket("xiangwan")
        .prefix("test/")
        .build());
List<DeleteObject> deleteObjects=new LinkedList<>();
for (Result<Item> result : results) {
    
    Item item = result.get();
    deleteObjects.add(new DeleteObject(URLDecoder.decode(item.objectName(),"utf-8")));
}
RemoveObjectsArgs argss= RemoveObjectsArgs.builder()
        .bucket("xiangwan")
        .objects(deleteObjects)
        .build();
// This method is lazy loading , Be similar to Feature, You need to get the return value to execute 
Iterable<Result<DeleteError>> removeObjects = minioClient.removeObjects(argss);
for (Result<DeleteError> removeObject : removeObjects) {
    
    DeleteError error = removeObject.get();
    System.out.printf("minio Delete error ->bucketName=%s,objectName=%s,message=%s\n", error.bucketName(), error.objectName(), error.message());
}

Deleting a folder does not provide a direct way
Delete multiple objects if necessary

3、 ... and 、 Configure permissions

3.1 Users and groups

Directly create users or groups , End user permissions = User permissions + Group permission .

3.2 secret key

Specific reference to MinIO Multi user quick start guide | Minio Chinese document

{
    
	"Version": "2012-10-17",
	"Statement": [

		{
    
			"Effect": "Allow",
			"Action": [
				"s3:GetObject",
				"s3:ListAllMyBuckets",
				"s3:ListBucket",
				"s3:PutObject",
				"s3:DeleteObject",
				"s3:GetBucketLocation"
			],
			"Resource": [
				"arn:aws:s3:::*"
			]
		}
	]
}