目录
概述
MinIO集群部署
准备docker-compose.yml
测试启动
MinIO用户管理
Buckets管理
创建Buckets
MinIO客户端
引入依赖
文件上传下载Demo
调用API碰到的问题
概述
MinIO | 高性能, Kubernetes 原生对象存储
MinIO是全球领先的对象存储先锋，目前在全世界有数百万的用户。
-
高性能 ，在标准硬件上，读/写速度上高达183GB/秒和171GB/秒，拥有更高的吞吐量和更低的延迟
-
可扩展性 ，为对象存储带来了简单的缩放模型，通过添加更多集群可以扩展空间
-
简单 ，极简主义是MinIO的指导性设计原则，即可在几分钟内安装和配置
-
与Amazon S3兼容 ，亚马逊云的
S3 API（接口协议）是在全球范围内达到共识的对象存储的协议，是全世界内大家都认可的标准 -
数据安全 ，使用纠删码来保护数据免受硬件故障和无声数据损坏
MinIO集群部署
基于docker-compose部署入门案例
准备docker-compose.yml
version: '3.7'
# 所有容器通用的设置和配置
x-minio-common: &minio-common
image: minio/minio
command: server --console-address ":9001" http://minio{1...4}/data
expose:
- "9000"
# environment:
# MINIO_ROOT_USER: minioadmin
# MINIO_ROOT_PASSWORD: minioadmin
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
interval: 30s
timeout: 20s
retries: 3
# 启动4个docker容器运行minio服务器实例
# 使用nginx反向代理9000端口，负载均衡, 你可以通过9001、9002、9003、9004端口访问它们的web console
services:
minio1:
<<: *minio-common
hostname: minio1
ports:
- "9001:9001"
volumes:
- ./data/data1:/data
minio2:
<<: *minio-common
hostname: minio2
ports:
- "9002:9001"
volumes:
- ./data/data2:/data
minio3:
<<: *minio-common
hostname: minio3
ports:
- "9003:9001"
volumes:
- ./data/data3:/data
minio4:
<<: *minio-common
hostname: minio4
ports:
- "9004:9001"
volumes:
- ./data/data4:/data
nginx:
image: nginx:1.19.2-alpine
hostname: nginx
volumes:
- ./config/nginx.conf:/etc/nginx/nginx.conf:ro
ports:
- "9000:9000"
depends_on:
- minio1
- minio2
- minio3
- minio4在当前目录下，创建config，创建nginx.conf
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 4096;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
keepalive_timeout 65;
# include /etc/nginx/conf.d/*.conf;
upstream minio {
server minio1:9000;
server minio2:9000;
server minio3:9000;
server minio4:9000;
}
server {
listen 9000;
listen [::]:9000;
server_name localhost;
# To allow special characters in headers
ignore_invalid_headers off;
# Allow any size file to be uploaded.
# Set to a value such as 1000m; to restrict file size to a specific value
client_max_body_size 0;
# To disable buffering
proxy_buffering off;
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 300;
# Default is HTTP/1, keepalive is only enabled in HTTP/1.1
proxy_http_version 1.1;
proxy_set_header Connection "";
chunked_transfer_encoding off;
proxy_pass http://minio;
}
}
}测试启动
docker-compose up -d
访问任意节点
默认账户密码：minioadmin minioadmin
MinIO用户管理
需要创建一个用户，客户端用户可上传下载权限。
注意：长度最好是32位，否则后续调用API会报错
Buckets管理
MinIO uses buckets to organize objects. A bucket is similar to a folder or directory in a filesystem, where each bucket can hold an arbitrary number of objects.
MinIO使用bucket来组织对象。bucket类似于文件系统中的文件夹或目录，其中每个bucket可以容纳任意数量的对象。
创建Buckets
注意桶名的命名规范，否则后续调用api创建桶会报错
可以在控制台管理文件，上传下载预览删除等
Simple Storage Service（Amazon S3）存储桶命名要求 - AWS CloudTrail
MinIO客户端
官方接口API文档
Java Client API Reference — MinIO Object Storage for Linux
MinIO支持多种客户端，本文以java为例
引入依赖
<dependency>
<groupId>io.minio</groupId>
<artifactId>minio</artifactId>
<version>8.5.3</version>
</dependency>文件上传下载Demo
package pers.kw.minio.demo;
import com.alibaba.fastjson.JSON;
import io.minio.*;
import io.minio.errors.*;
import org.apache.commons.io.FileUtils;
import org.apache.commons.io.IOUtils;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;
import java.util.List;
public class MinIODemo {
private static final String accessKey = "PPeCeQbHtFsEcGNxnlmSpbYasxELHoGy";
private static final String secretKey = "GEMQDMUhQSHaIJCqmxperpESNYCJXvjJ";
private static final String endpoint = "http://192.168.8.88:9000";
public static void main(String[] args)
throws ServerException, InsufficientDataException, ErrorResponseException,
IOException, NoSuchAlgorithmException, InvalidKeyException, InvalidResponseException, XmlParserException, InternalException {
String bucket = "test01";
String objectName = "small125750JkEMk1625374670.jpg";
downloadFile(bucket, objectName);
}
public static void downloadFile(String bucket, String objectName)
throws ServerException, InsufficientDataException, ErrorResponseException, IOException, NoSuchAlgorithmException, InvalidKeyException, InvalidResponseException, XmlParserException, InternalException {
MinioClient minioClient =
MinioClient.builder()
.endpoint(endpoint)
.credentials(accessKey, secretKey)
.build();
//todo 需要关闭流
try (InputStream inputStream = minioClient.getObject(
GetObjectArgs.builder()
.bucket(bucket)
.object(objectName)
.build())) {
// Read data from stream
byte[] bytes = IOUtils.toByteArray(inputStream);
String fileName = "22.jpg";
System.out.println("下载文件：" + bytes.length);
FileUtils.writeByteArrayToFile(new File("C:\\Users\\pc\\Desktop\\" + fileName), bytes);
System.out.println("文件下载成功");
IOUtils.closeQuietly(inputStream);
}
}
public static void uploadFile(String bucket) throws IOException, NoSuchAlgorithmException, InvalidKeyException {
try {
// Create a minioClient with the MinIO server playground, its access key and secret key.
MinioClient minioClient =
MinioClient.builder()
.endpoint(endpoint)
.credentials(accessKey, secretKey)
.build();
// Make 'asiatrip' bucket if not exist.
boolean found =
minioClient.bucketExists(BucketExistsArgs.builder().bucket(bucket).build());
if (!found) {
// Make a new bucket called bucket.
minioClient.makeBucket(MakeBucketArgs.builder().bucket(bucket).build());
} else {
System.out.println("Bucket " + bucket + " already exists.");
}
// Upload '/home/user/Photos/asiaphotos.zip' as object name 'asiaphotos-2015.zip' to bucket
// 'asiatrip'.
String fileName = "C:\\Users\\pc\\Desktop\\gateway-server.rar";
String objectName = "gateway-server_2023058.rar";
minioClient.uploadObject(
UploadObjectArgs.builder()
.bucket(bucket)
.object(objectName)
.filename(fileName)
.build());
System.out.println(
fileName + " is successfully uploaded as"
+ objectName + " to bucket " + bucket + ".");
} catch (MinioException e) {
System.out.println("Error occurred: " + e);
System.out.println("HTTP trace: " + e.httpTrace());
}
}
}调用API碰到的问题
问题一：
java.lang.NoSuchMethodError: okhttp3.RequestBody.create ( BLokhttp3/Medialype:)Lokhttp3/RequestBody;
解决: 引入如下依赖
<dependency>
<groupId>com.squareup.okhttp3</groupId>
<artifactId>okhttp</artifactId>
<version>4.9.0</version>
</dependency>问题二：
bucket name does not follow Amazon S3 standards.
命名不符合规则换个名字，最好在代码中增加正则校验
问题三：
RequestTimeTooSkewed
客户端时间与服务器时间相差过大导致
设置服务器时间
ntpdate time.windows.com问题4：
InvalidAccessKeyId
秘钥信息太短
