Why bother building your own CDN?
Because Cloudflare Pro costs $20/month, their cache rules are limited on the free tier, and you have zero visibility into what’s actually happening at the edge. If you’re running a homelab blog, a portfolio, or even a small business site off your own iron, your origin is in one datacenter — or worse, your living room. Users in Tokyo loading assets from your Hetzner box in Frankfurt are waiting 200ms just for the connection to establish.
A CDN solves this by putting cache nodes geographically close to your users. When someone in Singapore hits your site, they get the cached response from a VPS in Singapore, not a round-trip to Germany.
This is not complicated. Nginx’s proxy_cache directive has been production-grade for years. A €4/month Hetzner CAX11 (ARM, 2 vCPU, 4GB RAM) running nginx can handle tens of thousands of requests per second for static content. You can build a 3-continent CDN for less than a single Cloudflare Pro subscription.
Here’s what we’re building: an origin server (your main server or homelab box), two or more edge nodes in different regions, nginx on every node, GeoDNS to route users to the nearest edge, and a cache invalidation strategy that doesn’t make you want to quit the industry.
Architecture overview
┌─────────────────┐
│ GeoDNS │
│ (routes by IP) │
└────────┬────────┘
│
┌───────────────┼───────────────┐
│ │ │
┌────────▼──────┐ ┌─────▼──────┐ ┌────▼──────────┐
│ Edge: EU │ │ Edge: US │ │ Edge: APAC │
│ (Hetzner FRA)│ │(Vultr NYC) │ │ (Oracle SIN) │
│ nginx cache │ │nginx cache │ │ nginx cache │
└────────┬──────┘ └─────┬──────┘ └────┬──────────┘
└───────────────┼───────────────┘
│ cache miss → origin
┌────────▼────────┐
│ Origin server │
│ (your homelab/ │
│ main VPS) │
└─────────────────┘
Every edge node acts as a caching reverse proxy. On a cache hit, users get the response from the edge’s local disk. On a cache miss, the edge pulls from origin, caches it, and serves it — subsequent requests are hits. Your origin barely sees traffic for static assets once the cache is warm.
Setting up the origin
Your origin needs nothing special beyond being a normal nginx server. The only thing that matters is that it sends proper cache headers so the edges know what to cache and for how long.
# /etc/nginx/sites-available/origin.conf
server {
listen 80;
server_name origin.yourdomain.com;
# Static assets — cache for a long time
location ~* \.(jpg|jpeg|png|gif|webp|ico|svg|woff2|woff|ttf|css|js)$ {
root /var/www/html;
expires 30d;
add_header Cache-Control "public, max-age=2592000, immutable";
add_header Vary "Accept-Encoding";
}
# HTML — shorter TTL, edges will revalidate
location / {
root /var/www/html;
try_files $uri $uri/ /index.html;
expires 1h;
add_header Cache-Control "public, max-age=3600";
}
}
Lock the origin down — it should only accept connections from your edge nodes, not the public internet. The edges are your public surface.
# Add this inside the server block on origin
allow 10.0.0.1; # edge EU
allow 10.0.0.2; # edge US
allow 10.0.0.3; # edge APAC
deny all;
Use the actual public IPs of your edge VPS nodes, not private IPs (unless you’ve set up a WireGuard mesh — more on that in the Gotchas section).
The edge node setup
Every edge node is identical. This is the beauty of the setup: you deploy the same Docker Compose stack everywhere and change one environment variable.
Directory structure on each edge:
/opt/cdn-edge/
├── docker-compose.yml
├── nginx/
│ ├── nginx.conf
│ └── conf.d/
│ └── edge.conf
└── cache/ # nginx writes here, Docker volume
docker-compose.yml:
version: "3.9"
services:
nginx:
image: nginx:1.27-alpine
container_name: cdn-edge
restart: unless-stopped
ports:
- "80:80"
- "443:443"
volumes:
- ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
- ./nginx/conf.d:/etc/nginx/conf.d:ro
- cache_data:/var/cache/nginx
- ./certs:/etc/nginx/certs:ro # your TLS certs
environment:
- ORIGIN_HOST=${ORIGIN_HOST:-origin.yourdomain.com}
# nginx doesn't expand env vars in config natively,
# so we use envsubst at startup
entrypoint: >
/bin/sh -c "envsubst '$$ORIGIN_HOST' < /etc/nginx/conf.d/edge.conf.tmpl
> /etc/nginx/conf.d/edge.conf && nginx -g 'daemon off;'"
volumes:
cache_data:
driver: local
driver_opts:
type: none
o: bind
device: /opt/cdn-edge/cache
nginx/nginx.conf:
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 4096;
use epoll;
multi_accept on;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
# Cache zone: 512MB metadata in RAM, 20GB on disk
# Adjust max_size to your available disk space
proxy_cache_path /var/cache/nginx
levels=1:2
keys_zone=edge_cache:128m
max_size=20g
inactive=7d
use_temp_path=off;
# Logging with cache status
log_format cache_log '$remote_addr - $remote_user [$time_local] '
'"$request" $status $body_bytes_sent '
'"$http_referer" "$http_user_agent" '
'cache=$upstream_cache_status '
'upstream_time=$upstream_response_time';
access_log /var/log/nginx/access.log cache_log;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
gzip on;
gzip_vary on;
gzip_proxied any;
gzip_comp_level 6;
gzip_types text/plain text/css application/json application/javascript
text/xml application/xml text/javascript image/svg+xml;
include /etc/nginx/conf.d/*.conf;
}
nginx/conf.d/edge.conf.tmpl (the template, envsubst fills in $ORIGIN_HOST):
# Upstream origin
upstream origin {
server ${ORIGIN_HOST}:80;
keepalive 32;
}
# Redirect HTTP to HTTPS
server {
listen 80;
server_name _;
return 301 https://$host$request_uri;
}
server {
listen 443 ssl http2;
server_name cdn.yourdomain.com;
ssl_certificate /etc/nginx/certs/fullchain.pem;
ssl_certificate_key /etc/nginx/certs/privkey.pem;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384;
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 1d;
# Security headers
add_header X-Cache-Status $upstream_cache_status always;
add_header X-Edge-Node $hostname always;
add_header Strict-Transport-Security "max-age=63072000" always;
# Cache bypass for authenticated requests (cookies, auth header)
# This prevents serving cached responses to logged-in users
proxy_cache_bypass $http_authorization $cookie_session;
proxy_no_cache $http_authorization $cookie_session;
# Use the cache zone defined in nginx.conf
proxy_cache edge_cache;
# Cache successful responses for 24h, redirects for 1h
proxy_cache_valid 200 24h;
proxy_cache_valid 301 302 1h;
proxy_cache_valid 404 1m;
# Serve stale content if origin is down (up to 1h)
proxy_cache_use_stale error timeout http_500 http_502 http_503 http_504;
proxy_cache_background_update on;
proxy_cache_lock on;
location / {
proxy_pass http://origin;
proxy_http_version 1.1;
proxy_set_header Connection ""; # keepalive to upstream
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# How long to wait for origin before returning error
proxy_connect_timeout 5s;
proxy_read_timeout 30s;
proxy_send_timeout 30s;
# Don't cache if origin says not to
proxy_ignore_headers Cache-Control Expires Set-Cookie;
# ^ Remove this line if you WANT to respect origin cache headers
}
# Static assets get longer cache and immutable hint
location ~* \.(jpg|jpeg|png|gif|webp|ico|svg|woff2|woff|ttf|css|js|gz|zip|mp4)$ {
proxy_pass http://origin;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header Host $host;
proxy_cache edge_cache;
proxy_cache_valid 200 7d;
proxy_cache_use_stale error timeout http_500 http_502 http_503 http_504;
# Cache key doesn't include query string for assets
# This means ?v=1234 and ?v=5678 hit the same cache entry
# Good for versioned assets, bad if you use query params for variants
proxy_cache_key "$scheme$request_method$host$uri";
add_header Cache-Control "public, max-age=604800, immutable";
add_header X-Cache-Status $upstream_cache_status always;
}
# Cache purge endpoint — protect with IP whitelist
location ~ /purge(/.*) {
allow 127.0.0.1;
allow YOUR_ADMIN_IP;
deny all;
proxy_cache_purge edge_cache "$scheme$request_method$host$1";
}
}
Deploy this stack on each VPS. Set ORIGIN_HOST in a .env file next to docker-compose.yml:
ORIGIN_HOST=origin.yourdomain.com
Then:
mkdir -p /opt/cdn-edge/cache
chmod 777 /opt/cdn-edge/cache # nginx user inside container needs write access
cd /opt/cdn-edge
docker compose up -d
GeoDNS: routing users to the nearest edge
This is where most tutorials fall apart. GeoDNS is the piece that makes it a geographic CDN, not just a caching proxy.
Option 1: Cloudflare Load Balancer with Geo Steering (paid, $5/month)
If you’re already using Cloudflare for DNS, this is the easiest path. Add your edge IPs as pool origins, enable geo steering, done. Their network figures out the rest.
Option 2: PowerDNS with GeoIP (free, self-hosted, more work)
This is the homelab-pure approach. Run PowerDNS with the geoip backend on your authoritative nameserver.
# /etc/powerdns/pdns.conf additions
launch=geoip
geoip-database-files=/usr/share/GeoIP/GeoLite2-Country.mmdb
geoip-zones-file=/etc/powerdns/geoip.yaml
# /etc/powerdns/geoip.yaml
domains:
- domain: cdn.yourdomain.com
ttl: 30 # Low TTL so changes propagate fast
records:
cdn.yourdomain.com:
- content: "203.0.113.1" # EU edge — default
weight: 1
services:
cdn.yourdomain.com:
default: [203.0.113.1] # EU edge
"!EU": [203.0.113.1]
US: [198.51.100.1] # US edge
AS: [192.0.2.1] # APAC edge
OC: [192.0.2.1] # Oceania → APAC
Option 3: Multiple CNAME records with round-robin and low TTL
Honest opinion: this doesn’t do geographic routing. It’s just load balancing. Don’t call it a CDN if you’re doing this — call it what it is.
Cache invalidation
Cache invalidation is famously one of the two hard problems in computer science. Here’s a pragmatic approach.
The proxy_cache_purge location in the config above uses the ngx_cache_purge module. It’s not in the standard nginx Docker image, so use nginxinc/nginx-unprivileged or build a custom image:
FROM nginx:1.27-alpine AS builder
RUN apk add --no-cache build-base pcre-dev openssl-dev zlib-dev git
RUN git clone --depth=1 https://github.com/FRiCKLE/ngx_cache_purge.git /tmp/ngx_cache_purge
RUN nginx_version=$(nginx -v 2>&1 | sed 's/.*\///') && \
wget http://nginx.org/download/nginx-${nginx_version}.tar.gz -O /tmp/nginx.tar.gz && \
tar -C /tmp -xzf /tmp/nginx.tar.gz
RUN cd /tmp/nginx-$(nginx -v 2>&1 | sed 's/.*\///') && \
./configure --with-compat --add-dynamic-module=/tmp/ngx_cache_purge && \
make modules
RUN cp /tmp/nginx-*/objs/ngx_http_cache_purge_module.so /usr/lib/nginx/modules/
FROM nginx:1.27-alpine
COPY --from=builder /usr/lib/nginx/modules/ngx_http_cache_purge_module.so /usr/lib/nginx/modules/
RUN echo "load_module modules/ngx_http_cache_purge_module.so;" > /etc/nginx/modules-enabled/50-cache-purge.conf
Purge a specific URL from all edges:
#!/bin/bash
# purge.sh — run on your management machine
URL_PATH=$1
EDGES=("203.0.113.1" "198.51.100.1" "192.0.2.1")
for edge in "${EDGES[@]}"; do
curl -s -o /dev/null -w "%{http_code}" \
-X PURGE "https://cdn.yourdomain.com${URL_PATH}" \
--resolve "cdn.yourdomain.com:443:${edge}"
echo " purged from $edge"
done
For bulk invalidation (e.g., after a site deploy), the nuclear option is to just clear the cache directory:
# On each edge node — wipe the cache, nginx rebuilds it on demand
docker exec cdn-edge sh -c "rm -rf /var/cache/nginx/* && nginx -s reload"
TLS certificates
You need one wildcard cert that works on all edges, or individual certs per edge with the same SAN. Using Let’s Encrypt with DNS-01 challenge is the cleanest approach — you don’t need port 80 open on each edge during issuance.
# On your management machine
certbot certonly \
--dns-cloudflare \
--dns-cloudflare-credentials ~/.secrets/certbot/cloudflare.ini \
-d cdn.yourdomain.com \
--preferred-challenges dns-01
# Then copy the cert to each edge
for edge in edge-eu edge-us edge-apac; do
rsync -avz /etc/letsencrypt/live/cdn.yourdomain.com/ \
root@${edge}:/opt/cdn-edge/certs/
done
Set up a cron job to renew and redistribute. Or use Caddy as the edge server if you want automatic cert management — Caddy’s reverse_proxy directive with caching via a third-party plugin is a viable alternative, though nginx’s proxy_cache is more battle-tested.
Gotchas
Cache key collisions. By default nginx uses $scheme$proxy_host$request_uri as the cache key, which includes the query string. If your URLs have tracking params like ?utm_source=newsletter, every variant is a separate cache entry. Either strip those params before they hit nginx (using map directives) or normalize the cache key explicitly with proxy_cache_key.
Vary headers will wreck you. If your origin sends Vary: Accept-Encoding, nginx caches separate versions for gzip and identity. That’s usually fine. But if origin sends Vary: Accept, Vary: User-Agent, or any other header that has high cardinality, your cache hit rate drops to near zero. Strip or override the Vary header on the edge if this is happening.
Set-Cookie and caching don’t mix. If any response from origin includes Set-Cookie, nginx won’t cache it by default. This is correct behavior — you don’t want to serve one user’s session cookie to another. But if your origin is naively setting cookies on static asset responses (it happens), add proxy_ignore_headers Set-Cookie; carefully, and only for the specific locations that you know are truly uncacheable.
Cache disk filling up silently. The max_size parameter evicts the least recently used entries when the limit is hit, but nginx doesn’t alert you. Set up a cron that checks df -h and alerts when /var/cache/nginx is above 80%.
The thundering herd. When your cache is cold (e.g., after a purge or node restart), every request for an uncached URL goes to origin simultaneously. proxy_cache_lock on in the config above helps — it serializes concurrent requests for the same cache key so only one goes to origin. The others wait. Make sure proxy_cache_lock_timeout is long enough for your origin’s slowest response.
Origin IP exposure. If someone resolves your origin’s DNS directly, they bypass the edge. Put the origin behind a firewall that only allows traffic from edge IPs. On Hetzner Cloud, use the Firewall rules in the UI. On bare metal, use ufw:
ufw default deny incoming
ufw allow from 203.0.113.1 to any port 80 # EU edge
ufw allow from 198.51.100.1 to any port 80 # US edge
ufw allow from 192.0.2.1 to any port 80 # APAC edge
ufw allow 22 # SSH
ufw enable
Monitoring cache health
Check cache status on any edge:
# See hit/miss ratio from the last 1000 requests
docker exec cdn-edge sh -c \
"tail -1000 /var/log/nginx/access.log | grep -oP 'cache=\K\w+' | sort | uniq -c | sort -rn"
Output looks like:
831 HIT
143 MISS
26 EXPIRED
Anything above 80% HIT for static assets is healthy. If you’re seeing a lot of EXPIRED, your proxy_cache_valid TTL might be shorter than your cache inactive timeout — both need to be checked.
Add a /cache-status endpoint on each edge that returns a 200 with basic stats. Wire it into Uptime Kuma or whatever monitoring you’re running.
Production-ready extras
WireGuard mesh between nodes. Instead of exposing the origin’s port 80 to the public internet (even with IP allowlisting), set up a WireGuard mesh. Origin gets a private address 10.10.0.1, edges get 10.10.0.2, .3, .4. Traffic between nodes is encrypted and the origin firewall accepts nothing from the public internet on port 80. This is the right architecture for anything you care about.
Cache warming after deploy. Write a script that crawls your sitemap.xml and hits each URL through each edge after deploying new content. This fills the cache before real users arrive.
#!/bin/bash
# warm-cache.sh
SITEMAP="https://yourdomain.com/sitemap.xml"
EDGES=("203.0.113.1" "198.51.100.1" "192.0.2.1")
urls=$(curl -s "$SITEMAP" | grep -oP '(?<=<loc>)[^<]+')
for url in $urls; do
for edge in "${EDGES[@]}"; do
path=$(echo "$url" | sed 's|https://yourdomain.com||')
curl -s -o /dev/null \
--resolve "cdn.yourdomain.com:443:${edge}" \
"https://cdn.yourdomain.com${path}" &
done
done
wait
echo "Cache warmed on all edges"
Stale-while-revalidate behavior. The combination of proxy_cache_use_stale and proxy_cache_background_update on means your edges serve the old cached version while fetching the fresh one from origin in the background. Users never see a slow response during cache revalidation. This is the single most impactful setting for perceived performance.
The final result is a CDN that costs roughly what one Cloudflare Pro subscription costs, gives you full visibility into every cache decision, and runs on infrastructure you control. The X-Cache-Status header tells you exactly what happened on every request — HIT, MISS, EXPIRED, BYPASS, STALE. No black box.
For a single-region homelab blog, the EU edge alone (your home server proxied through a Hetzner VPS) will cut your page load times for European visitors significantly just by avoiding the TTFB from residential upload bandwidth. Add the US and APAC nodes and you’ve got something that would have cost serious money to build even five years ago.