S3 has won. You can argue about it, but the reality is every backup tool, every media server, every modern web app speaks S3. Restic, Rclone, Nextcloud, Immich, Authentik — they all expect an S3 endpoint. So if you’re running infrastructure at home or for a small team, you eventually need to answer the question: which S3-compatible storage do I run?
The three serious contenders for self-hosters are MinIO, SeaweedFS, and Garage. They’re all open-source, all claim S3 compatibility, and all solve the problem differently. This article breaks down each one honestly — how they work, where they shine, where they’ll bite you, and which one belongs on your server.
I’ll skip the marketing fluff. Let’s look at what actually matters when you’re the one running this thing at 2 AM.
The Short Answer (Before the Deep Dive)
- MinIO — maximum S3 compatibility, maximum resource usage, maximum enterprise tooling. If you need it to "just work" with everything and have the RAM to spare.
- SeaweedFS — the specialist. Billions of small files, complex replication topologies, serious distributed workloads. High reward, high setup cost.
- Garage — the quiet achiever. Low footprint, built for geo-distribution, trivial to operate. The right answer for most homelabs and small multi-site setups.
Now the longer answer.
MinIO: The Reference Implementation
GitHub: github.com/minio/minio
MinIO is what most people mean when they say "S3-compatible storage." It was built to be a drop-in replacement for AWS S3, and it succeeds at that mission more completely than any other project in this space. The API compatibility is excellent — if something claims S3 support, it almost certainly works with MinIO.
Architecture
MinIO runs as a single Go binary. In single-node mode, you point it at one or more local directories and you’re done. In distributed mode (which MinIO calls "distributed MinIO" or DMNIO), you run the same binary across multiple nodes and it handles erasure coding across them.
The minimum viable distributed cluster is 4 nodes with 4 drives each, giving you EC(2+2) — two data shards, two parity shards. That’s a hard floor. You can’t run a 2-node MinIO cluster with proper erasure coding. MinIO will let you run single-node, but that’s not HA — one drive dies, your data might not be recoverable depending on configuration.
# docker-compose.yml — single-node MinIO (dev/backup use)
services:
minio:
image: quay.io/minio/minio:latest
command: server /data --console-address ":9001"
environment:
MINIO_ROOT_USER: admin
MINIO_ROOT_PASSWORD: changeme-use-a-real-secret
ports:
- "9000:9000" # S3 API
- "9001:9001" # Web console
volumes:
- minio_data:/data
restart: unless-stopped
volumes:
minio_data:
The web console is genuinely good. Object browser, IAM, bucket policies, lifecycle rules — it’s all there and usable by someone who isn’t a CLI person.
What MinIO Does Well
S3 feature completeness is the headline. Multipart uploads, object locking, versioning, lifecycle policies, bucket notifications, server-side encryption, STS — MinIO supports all of it. If your application does something exotic with the S3 API, MinIO is your safest bet.
Performance on large objects is excellent. MinIO was benchmarked at pushing past 300 GB/s in some enterprise configurations. That’s not homelab territory, but it tells you the architecture isn’t a bottleneck.
Tooling around MinIO is mature. The mc (MinIO Client) CLI is excellent — better than the AWS CLI for many workflows. The Operator for Kubernetes is production-grade. There are Terraform providers, Ansible roles, and everything else you’d expect.
The Gotchas
Licensing is the big one. MinIO uses AGPL-3.0. If you’re using it internally for personal projects or open-source services, you’re fine. The moment you’re running a commercial service built on top of MinIO without releasing your application’s source code under AGPL, you need a commercial license. This has bitten companies before. Read the license.
Memory usage is real. A single-node MinIO instance serving modest load will eat 1-2 GB of RAM baseline. Under load with large numbers of concurrent requests, it climbs. On a machine with 4 GB total RAM running ten other services, MinIO is a bruiser.
Distributed mode is all-or-nothing. Unlike Ceph or HDFS, you can’t add a single node to a MinIO cluster. You add sets of nodes (a server pool), and the new pool must have the same erasure coding configuration. This makes horizontal scaling less flexible than you might expect.
Single-node MinIO has no redundancy. This surprises a lot of people. MinIO in single-node mode with multiple drives uses erasure coding across those drives, which protects against individual drive failures. But the node itself is a single point of failure. It’s fine for backups, not fine if uptime matters.
SeaweedFS: Built for Scale, Demands Respect
GitHub: github.com/seaweedfs/seaweedfs
SeaweedFS started as a research project inspired by Facebook’s Haystack paper on efficient blob storage at scale. The core insight: traditional filesystems are terrible at storing millions of small files because each file requires multiple filesystem operations just to find it. SeaweedFS sidesteps this by packing many small files ("needles") into large volume files and maintaining its own index.
The S3 interface is a layer on top of this architecture, not the foundation. That’s an important distinction.
Architecture
SeaweedFS has three main components:
Master server — tracks which volume servers exist and which volumes have space. Handles volume assignment and replication decisions. You typically run 3 masters with Raft consensus for HA.
Volume server — stores the actual data as "needles" in volume files. Each volume is a fixed-size file (default 30 GB). Volume servers register with the master and report their capacity.
Filer — the translation layer that maps file paths to needle locations in volumes. The S3 API runs on top of the Filer. The Filer needs its own metadata store — SeaweedFS supports LevelDB, RocksDB, Redis, Cassandra, PostgreSQL, MySQL, and others.
# docker-compose.yml — minimal SeaweedFS with S3
services:
master:
image: chrislusf/seaweedfs:latest
command: master -mdir=/data
volumes:
- master_data:/data
ports:
- "9333:9333"
- "19333:19333"
volume:
image: chrislusf/seaweedfs:latest
command: volume -mserver=master:9333 -dir=/data -max=5
volumes:
- volume_data:/data
depends_on:
- master
filer:
image: chrislusf/seaweedfs:latest
command: filer -master=master:9333
ports:
- "8888:8888"
depends_on:
- master
- volume
s3:
image: chrislusf/seaweedfs:latest
command: s3 -filer=filer:8888 -config=/etc/seaweedfs/s3.json
ports:
- "8333:8333"
volumes:
- ./s3.json:/etc/seaweedfs/s3.json
depends_on:
- filer
volumes:
master_data:
volume_data:
What SeaweedFS Does Well
Massive file counts. If you need to store billions of small objects — think photo storage, IoT sensor data, generated thumbnails — SeaweedFS handles this efficiently where other systems start to struggle. The needle architecture means lookup is O(1) via the in-memory index.
Flexible replication. SeaweedFS’s replication model is sophisticated. You specify a replication strategy like 001 (1 replica in the same rack), 010 (1 replica in a different data center), 100 (1 replica across data centers). This maps to real physical topology if you label your nodes correctly.
Data center awareness is baked in, not bolted on. You can tell SeaweedFS which rack and data center each volume server belongs to, and it will place replicas accordingly.
Tiered storage. SeaweedFS supports moving cold data from local volumes to cloud storage (S3, GCS, Azure) automatically. This is genuinely useful if you have limited local SSD but want to keep older data accessible.
The Gotchas
Complexity is real. You’re running at minimum 3 processes. For production, 3 masters, 3+ volume servers, 2+ filers, and an S3 gateway. That’s a lot of moving parts for a homelab. Debugging when something goes wrong requires understanding all these layers.
The Filer’s metadata backend is another dependency. The default LevelDB is fine for small deployments, but for anything serious you’ll want an external database. Now you’re also operating PostgreSQL or Cassandra for your object storage. This is fine for people running databases anyway; it’s extra weight for everyone else.
S3 compatibility isn’t perfect. Some S3 features — particularly around object tagging, some lifecycle behaviors, and certain multipart edge cases — have historically had gaps in SeaweedFS’s S3 layer. Most common use cases work fine, but if your application does non-standard S3 things, test it thoroughly.
No built-in web UI worth mentioning. There’s a basic status page but nothing like MinIO’s console.
Garage: The One Built for Your Situation
Official repo: git.deuxfleurs.fr/Deuxfleurs/garage
Garage is what happens when someone sits down and thinks carefully about what self-hosters and small organizations actually need from distributed object storage, then builds exactly that. It’s written in Rust, it’s small, it’s fast to start, and it handles the one scenario nobody else solves well: you have a few machines in different physical locations and you want consistent, replicated object storage across them.
The project comes out of Deuxfleurs, a French non-profit running federated internet infrastructure. They built Garage because nothing else worked for their deployment model.
Architecture
Garage uses a distributed hash table with Rendezvous hashing to decide where objects live. There’s no separate metadata server and no separate filer — every Garage node participates equally. You configure a cluster layout by assigning each node a zone, capacity weight, and optional tags. Garage distributes objects across zones according to your replication factor.
The config is a single TOML file. The whole system is one binary.
# garage.toml
metadata_dir = "/var/lib/garage/meta"
data_dir = "/var/lib/garage/data"
db_engine = "lmdb"
replication_factor = 3
[rpc_bind_addr]
# This node listens for cluster RPC on this address
rpc_bind_addr = "0.0.0.0:3901"
[s3_api]
s3_region = "garage"
api_bind_addr = "0.0.0.0:3900"
root_domain = ".s3.example.com" # optional, for virtual-hosted style
[s3_web]
bind_addr = "0.0.0.0:3902"
root_domain = ".web.example.com"
index = "index.html"
[admin]
api_bind_addr = "0.0.0.0:3903"
# docker-compose.yml — single Garage node (repeat on each machine)
services:
garage:
image: dxflrs/garage:v1.0.1
network_mode: host # needed for cluster RPC to work cleanly
volumes:
- ./garage.toml:/etc/garage.toml
- garage_meta:/var/lib/garage/meta
- garage_data:/var/lib/garage/data
environment:
GARAGE_CONFIG_FILE: /etc/garage.toml
restart: unless-stopped
volumes:
garage_meta:
garage_data:
Cluster bootstrap is explicit and simple:
# On any node, after all nodes are running:
garage layout assign -z dc1 -c 10G <node-id-1>
garage layout assign -z dc2 -c 10G <node-id-2>
garage layout assign -z dc3 -c 10G <node-id-3>
garage layout apply --version 1
# Create a bucket and keys
garage bucket create my-backups
garage key create backup-key
garage bucket allow --read --write --key backup-key my-backups
garage key info backup-key # shows access key and secret
That’s it. You have a 3-node, 3-zone distributed object store.
What Garage Does Well
Geo-distribution is first-class. The zone concept means Garage won’t place all replicas in the same physical location. With replication_factor = 3 and three zones, one full zone can go offline and you still have two copies. This is the scenario that’s surprisingly hard to achieve with the alternatives.
Resource usage is embarrassing (in a good way). A Garage node under light load uses around 30-50 MB of RAM. You can run it on a Raspberry Pi. You can run it on a $5 VPS. The Rust implementation is genuinely lean.
Zero-dependency clustering. No external Raft service, no ZooKeeper, no etcd. Garage handles its own cluster state. Adding a node is garage node connect <address>, assign it a zone and capacity, apply the layout.
Static website hosting is built in. The s3_web listener serves bucket contents as a website, including index documents and custom error pages. Useful if you’re hosting static sites.
Honest about what it doesn’t have. Garage doesn’t support object versioning in older releases (it’s been added more recently), doesn’t have bucket notifications, and has a simpler IAM model than MinIO. The documentation says so clearly instead of burying the gaps.
The Gotchas
No web UI. Manage it via the CLI or the admin API. There are third-party UIs like Cyberduck and S3-compatible clients that work fine, but if your users expect a MinIO-style browser, Garage doesn’t provide one.
S3 compatibility is good but not perfect. Garage targets the common subset: PutObject, GetObject, DeleteObject, multipart uploads, presigned URLs, bucket policies. Server-side encryption, object locking, lifecycle rules, and Intelligent-Tiering are not supported. Check the compatibility matrix in the docs before committing.
Rebalancing on layout change takes time. When you add a node or change capacity weights, Garage rebalances data across the new layout. For large amounts of data, this takes hours and generates significant network and disk I/O. Plan layout changes during low-traffic windows.
The cluster needs an odd number of nodes for consensus. 1, 3, or 5 nodes. A 2-node Garage cluster can store data but can’t make layout decisions if one node is unreachable. Plan your topology accordingly.
Head-to-Head: What Actually Matters
| Dimension | MinIO | SeaweedFS | Garage |
|---|---|---|---|
| RAM baseline | 1-2 GB | 500 MB+ (per component) | 30-50 MB |
| Setup complexity | Low | High | Low |
| S3 compatibility | Excellent | Good | Good |
| Clustering model | Erasure coding pools | Master + Volumes + Filer | DHT, zone-aware |
| Min nodes for HA | 4 | 3 (masters) + 3+ volumes | 3 |
| Geo-distribution | Manual/difficult | Data center tags | Native |
| License | AGPL-3.0 | Apache-2.0 | AGPL-3.0 |
| Web UI | Excellent | Minimal | None |
| Written in | Go | Go | Rust |
Production-Ready Practices
For MinIO in production: Always run TLS. MinIO can manage its own certs or you can front it with Nginx/Caddy. Never expose the S3 API or console on a public interface without TLS. Set MINIO_BROWSER_REDIRECT_URL if you’re behind a reverse proxy — a lot of console issues come from misconfigured redirects.
For SeaweedFS in production: Run your Filer metadata on a real database (PostgreSQL is a safe choice), not LevelDB. Back up the Filer metadata separately from the volume data — the volumes are useless without it. Monitor volume server disk usage; a volume server that fills up completely causes write failures cluster-wide.
For Garage in production: Put the admin API behind a firewall — it has no authentication by default. Set admin_token in the config for any internet-facing deployment. If you’re running across real network links (not just LAN), test your RPC latency — Garage tolerates high latency but prefers low. Consider using lmdb over sqlite for the metadata engine on write-heavy workloads.
The Decision
Run Garage if: You have 3+ machines across different locations, you want to minimize operational overhead, and you don’t need the full S3 feature set. This is the right call for most homelabs and small organizations. The RAM savings alone make it worth it for multi-service setups.
Run MinIO if: You need complete S3 compatibility for compatibility-sensitive applications, you have RAM to spare, and you want the best tooling around your storage. Also the right call if your team is already familiar with it and that familiarity has value.
Run SeaweedFS if: You’re storing billions of small objects, you need sophisticated topology-aware replication, or you need tiered storage to the cloud. Not worth the complexity unless you specifically need what it’s good at.
The trap to avoid is picking MinIO by default because it’s the most famous. Famous doesn’t mean right for your situation. If you’re running three Raspberry Pis or three cheap VPSes and want replicated S3 storage across them, Garage is the answer and MinIO would be overengineered misery.
A Note on Ceph
People always ask: what about Ceph? Ceph is extraordinarily powerful and extraordinarily complex. Running Ceph properly requires dedicated ops time that most homelabs don’t have. It’s the right tool for organizations with full-time infrastructure people. For everyone else, the three projects above cover the realistic use cases without requiring a PhD in distributed systems to debug at 2 AM.
Summary
S3-compatible self-hosted storage is a solved problem — you just need to pick the right solution for your scale and complexity tolerance. Garage earns its spot as the default recommendation for most self-hosters: it’s small, it handles multi-site replication better than anything else in this class, and the operational burden is minimal. MinIO is the professional choice when you need the full API surface or want enterprise tooling. SeaweedFS is specialized; use it when you’ve hit the limits of the other two.
All three are worth having in your toolkit. Knowing when to reach for each one is the actual skill.