MINIO Provider

The MINIO provider uses MinIO, a high-performance S3-compatible object storage system, for storing files. MinIO is ideal for cluster deployments where multiple webPDF Server nodes need to share file storage.

Overview

• Provider Name: MINIO
• Supports: File Storage only
• Configuration File: provider-minio.yaml
• Use Case: Cluster deployments, scalable file storage, S3-compatible storage
• Protocol: S3 API compatible

Features

• ✓ S3-compatible API
• ✓ High-performance object storage
• ✓ Cluster-ready with shared file access
• ✓ Scalable and distributed
• ✓ Support for versioning and encryption
• ✓ Multi-tenant with bucket isolation
• ✓ ETag-based local caching for improved performance
• ✓ Automatic retry with exponential backoff
• ✓ Batch deletion for efficient cleanup
• ✓ Configurable connection pooling
• ✓ Event Manager support for notifications
• ✗ Requires external MinIO server or S3-compatible service
• ✗ Higher latency compared to local filesystem

Prerequisites

• MinIO server or S3-compatible service
• Network connectivity from webPDF Server to MinIO
• MinIO access credentials (access key, secret key)
• Bucket created or automatic bucket creation enabled

Configuration

provider.json

{
  "fileStorage": {
    "name": "MINIO",
    "checks": {
      "enabled": true,
      "interval": 10000
    }
  }
}

provider-minio.yaml

Configuration file for MinIO file storage (YAML format):

url: http://minio.example.com:9000
username: minioadmin
password: minioadmin-secret
bucketName: webpdf-files
settings:
  region: us-east-1

Configuration Attributes

Required Attributes

Attribute	Type	Description	Example
url	string	MinIO server endpoint URL	http://minio.example.com:9000
username	string	MinIO access key	minioadmin
password	string	MinIO secret key	minioadmin-secret

Optional Attributes

Attribute	Type	Description	Default
bucketName	string	S3 bucket name for file storage	Auto-generated
settings	object	Additional MinIO/S3 client settings	{}

Settings Object

The settings object supports S3 client configuration options:

Setting	Type	Description	Default
region	string	AWS/MinIO region	-

Bucket Naming

If bucketName is not specified, it is automatically generated using the namespace configuration:

{org}-{app}-{env}-{version}

Example: webpdf-webpdf-prod-v10-0

tip

For better organization, explicitly set bucketName to a meaningful value like webpdf-files-prod.

Environment Variables

# MinIO file storage
WEBPDF_MINIO_FILE_STORAGE_SETTINGS_URL=http://minio.example.com:9000
WEBPDF_MINIO_FILE_STORAGE_SETTINGS_USERNAME=minioadmin
WEBPDF_MINIO_FILE_STORAGE_SETTINGS_PASSWORD=minioadmin-secret
WEBPDF_MINIO_FILE_STORAGE_SETTINGS_BUCKET_NAME=webpdf-files

Health Checks

Health checks verify connectivity to MinIO and bucket accessibility:

{
  "fileStorage": {
    "name": "MINIO",
    "checks": {
      "enabled": true,
      "interval": 10000
    }
  }
}

• enabled: Enable health checks (recommended: true)
• interval: Check interval in milliseconds (recommended: 10000 - 30000)

Health Check Implementation

The health check performs a lightweight connectivity test:

1. Bucket Existence Check - Uses a HEAD request to check if the bucket exists
2. Connection Validation - Verifies network connectivity to MinIO endpoint
3. SSL Validation - Ensures SSL/TLS configuration is correct (for HTTPS endpoints)

What is checked:

• Network connectivity to MinIO service
• Bucket accessibility
• SSL/TLS configuration validity

Health Status Results:

• UP - Connection established, bucket accessible
• DOWN - Connection failed, bucket not accessible, or SSL error

Note: The health check only verifies connectivity, not bucket creation. If the bucket doesn't exist, automatic creation is attempted during storage initialization.

Security Considerations

Authentication

MinIO uses access key and secret key authentication:

username: access-key-id
password: secret-access-key

TLS/SSL Configuration

For HTTPS connections:

url: https://minio.example.com:9000
username: access-key
password: secret-key
settings:
  region: us-east-1

IAM Policies

Configure MinIO IAM policies for least privilege access:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject",
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::webpdf-files/*",
        "arn:aws:s3:::webpdf-files"
      ]
    }
  ]
}

warning

Never store MinIO credentials in configuration files. Use environment variables or secure configuration management.

Bucket Management

Automatic Bucket Creation

If the specified bucket doesn't exist, webPDF will attempt to create it automatically during storage initialization. The bucket name is automatically converted to lowercase and trimmed.

Bucket Naming Rules:

• Must be lowercase (automatically converted)
• Can contain letters, numbers, and hyphens
• Must be between 3 and 63 characters
• Must start and end with a letter or number
• Follows AWS S3 bucket naming rules

Manual Bucket Creation

Using MinIO Client (mc):

mc alias set myminio http://minio.example.com:9000 accesskey secretkey
mc mb myminio/webpdf-files
mc policy set download myminio/webpdf-files

Bucket Lifecycle Policies

Configure lifecycle policies for automatic cleanup:

<LifecycleConfiguration>
  <Rule>
    <ID>DeleteOldFiles</ID>
    <Status>Enabled</Status>
    <Expiration>
      <Days>30</Days>
    </Expiration>
  </Rule>
</LifecycleConfiguration>

High Availability

For production deployments:

1. Distributed MinIO – Multi-node MinIO cluster with erasure coding
2. Load Balancing – Distribute requests across MinIO nodes
3. Replication – Site-to-site bucket replication
4. Backup – Regular bucket snapshots

Distributed Setup Example

# 4-node distributed MinIO
minio server http://minio{1...4}.example.com/data{1...4}

Performance Tuning

MinIO Server Optimization

# Environment variables
export MINIO_CACHE_SIZE=10GB
export MINIO_CACHE_DRIVES="/mnt/cache1,/mnt/cache2"

Built-in Connection Management

The MinIO provider includes two optimized HTTP client configurations:

Default Client (Standard Operations)

• Connection Pool: 50 connections, 5-minute keep-alive
• Connect Timeout: 5 seconds
• Read/Write Timeout: 10 seconds
• Call Timeout: 15 seconds
• Retry on Failure: Enabled
• Use Case: Regular file operations (get, put, delete)

Streaming Client (Large File Operations)

• Connection Pool: Isolated (no connection reuse)
• Connect Timeout: 10 seconds
• Read/Write Timeout: 5 minutes
• Call Timeout: 10 minutes
• Retry on Failure: Disabled (handled by retry policy)
• Use Case: Long-running uploads/downloads

Automatic Retry Policy

Retry configuration for transient failures:

• Max Attempts: 5 retries
• Strategy: Exponential backoff
• Initial Delay: 500ms
• Max Delay: 30 seconds
• Jitter: Enabled (randomized delays)

This handles network hiccups and temporary MinIO unavailability automatically.

Batch Processing

For efficient bulk operations:

• Batch Size: 1000 objects
• Use Case: Deleting multiple files in a single request
• Benefit: Reduces API calls and improves performance

File Caching

ETag-based caching mechanism:

1. Local Cache - Files are cached locally after download
2. ETag Validation - ETags and file sizes are compared before re-downloading
3. Automatic Invalidation - Cache is updated when ETags don't match or file size differs
4. Consistency Check - File size verification ensures complete uploads/downloads

Network Optimization

1. Use dedicated network for storage traffic
2. Enable jumbo frames (MTU 9000) if supported
3. Co-locate webPDF and MinIO in same data center/region

Monitoring

MinIO Metrics

Monitor MinIO health and performance:

# MinIO metrics endpoint
curl http://minio.example.com:9000/minio/v2/metrics/cluster

Key Metrics

• Request rate (GET/PUT operations)
• Response time (latency)
• Bandwidth utilization
• Error rate
• Storage capacity

Troubleshooting

Cannot Connect to MinIO

1. Verify MinIO is running: curl http://minio.example.com:9000/minio/health/live
2. Check network connectivity
3. Verify access credentials
4. Check firewall rules (port 9000)

Bucket Not Found

1. Verify bucket exists: mc ls myminio/
2. Check bucket name matches configuration
3. Verify access permissions
4. Enable automatic bucket creation

Slow Upload/Download

1. Check network bandwidth
2. Review MinIO server performance
3. Consider distributed MinIO setup
4. Check file size - large files automatically use streaming client
5. Review retry attempts in logs

Access Denied Errors

1. Verify access key and secret key
2. Check IAM policy permissions
3. Review bucket policies
4. Verify bucket ownership

AWS S3 Compatibility

MinIO provider works with AWS S3 and S3-compatible services:

AWS S3 Configuration

url: https://s3.amazonaws.com
username: AWS_ACCESS_KEY_ID
password: AWS_SECRET_ACCESS_KEY
bucketName: webpdf-files
settings:
  region: us-east-1

Examples

Basic MinIO Setup

url: http://192.168.1.100:9000
username: webpdf-user
password: secure-password
bucketName: webpdf-files

MinIO with TLS

url: https://minio.example.com:9000
username: webpdf-user
password: secure-password
bucketName: webpdf-files-prod
settings:
  region: us-east-1

AWS S3

url: https://s3.eu-central-1.amazonaws.com
username: AKIAIOSFODNN7EXAMPLE
password: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
bucketName: my-webpdf-bucket
settings:
  region: eu-central-1