You just tuned your Nginx config, bumped your PHP-FPM workers, or swapped out your reverse proxy. Now you want to know: can this thing actually handle traffic, or will it fold like a cheap lawn chair the moment a real spike hits?
This is where most people reach for ab, run one command, see big numbers, and call it a day. That’s a mistake. The tool you pick, and how you use it, determines whether your benchmark means anything at all.
Here’s a no-nonsense breakdown of three CLI benchmarking tools that cover 95% of what you’ll ever need: hey, ab, and siege. Each has a distinct personality. Picking the wrong one gives you numbers that look great but don’t reflect reality.
The Contenders
ab (Apache Bench) has been around since roughly 1996. It ships with Apache HTTP Server (apache2-utils on Debian/Ubuntu) and has been the default "quick sanity check" tool for decades. It’s ancient, limited, and still surprisingly useful for a 10-second gut check.
hey is a modern Go rewrite that fixes most of ab’s pain points. It was written by Jaana Dogan (formerly at Google). Single binary, sane defaults, histogram output out of the box. GitHub: https://github.com/rakyll/hey
siege is the heavy hitter for realistic scenario simulation. It supports URL lists, delays between requests, basic auth, and cookies — things that actually model how users behave. GitHub: https://github.com/JoeDog/siege
Installation
All three run on Linux and macOS. Here’s the fastest path to each.
# ab — part of apache2-utils, no separate install needed
sudo apt install apache2-utils # Debian/Ubuntu
sudo dnf install httpd-tools # RHEL/Fedora
# hey — grab the binary directly
wget -qO /usr/local/bin/hey https://hey-release.s3.us-east-2.amazonaws.com/hey_linux_amd64
chmod +x /usr/local/bin/hey
# siege
sudo apt install siege # Debian/Ubuntu
sudo dnf install siege # RHEL/Fedora
# or build from source if you need the latest version
hey also has a go install path if you’re on a Go dev machine:
go install github.com/rakyll/hey@latest
Your First Benchmark: Same Target, Three Tools
Let’s hit the same endpoint with all three and compare what we get back. I’ll use a local Nginx serving a static file as the target.
ab
ab -n 10000 -c 100 https://cd-linux.club/index.html
-n is total requests, -c is concurrency. That’s basically the whole interface.
Output (trimmed):
Server Software: nginx/1.24.0
Document Length: 612 bytes
Concurrency Level: 100
Time taken for tests: 1.423 seconds
Complete requests: 10000
Failed requests: 0
Requests per second: 7026.84 [#/sec] (mean)
Time per request: 14.232 [ms] (mean)
Time per request: 0.142 [ms] (mean, across all concurrent requests)
Percentage of the requests served within a certain time (ms)
50% 9
66% 11
75% 12
80% 13
90% 17
95% 22
98% 29
99% 34
100% 89 (longest request)
Clean, readable. The percentile table at the bottom is genuinely useful — that p99 number (34ms here) tells you more about tail latency than the mean ever will.
hey
hey -n 10000 -c 100 https://cd-linux.club/index.html
Output:
Summary:
Total: 1.4021 secs
Slowest: 0.0891 secs
Fastest: 0.0001 secs
Average: 0.0138 secs
Requests/sec: 7132.56
Response time histogram:
0.000 [1] |
0.009 [4823] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
0.018 [3641] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
0.027 [387] |■■■
...
Latency distribution:
10% in 0.0062 secs
25% in 0.0094 secs
50% in 0.0126 secs
75% in 0.0170 secs
90% in 0.0223 secs
95% in 0.0268 secs
99% in 0.0389 secs
Status code distribution:
[200] 10000 responses
The histogram is what sells hey for me. You immediately see the shape of your latency distribution — is it a clean bell curve or does it have a long tail? That visual tells you more than percentiles alone.
siege
siege -c 100 -r 100 https://cd-linux.club/index.html
-c is concurrent users, -r is repetitions per user (so total requests = c × r = 10,000).
Output:
Transactions: 10000 hits
Availability: 100.00 %
Elapsed time: 1.44 secs
Data transferred: 5.84 MB
Response time: 0.01 secs
Transaction rate: 6944.44 trans/sec
Throughput: 4.06 MB/sec
Concurrency: 99.63
Successful transactions: 10000
Failed transactions: 0
Longest transaction: 0.09
Shortest transaction: 0.00
Siege gives you data transfer metrics that ab and hey skip by default. The Availability percentage and Failed transactions count are nice safety nets when you’re pushing a server that starts throwing errors under load.
Where Each Tool Pulls Ahead
ab: The 10-Second Sanity Check
ab is for moments when you just need a number, fast. You changed something in your config and want to know if it helped or hurt. That’s it. The interface hasn’t changed in 20 years, so you can run it on any server without thinking.
One genuinely useful flag that most people miss: -k for HTTP keep-alive.
# Without keep-alive (default, creates a new TCP connection per request)
ab -n 5000 -c 50 https://cd-linux.club/api/status
# With keep-alive (much closer to real browser behavior)
ab -n 5000 -c 50 -k https://cd-linux.club/api/status
The difference is dramatic. Forgetting -k when your app uses keep-alive connections is one of the most common ways people benchmark a feature and not their actual server.
Custom headers and POST bodies work too:
# POST with JSON body
ab -n 1000 -c 20 -m POST \
-T "application/json" \
-p payload.json \
https://cd-linux.club/api/users
Where payload.json contains your request body. ab will send the exact same body to every request — fine for read-heavy tests, useless for anything that needs unique inputs.
hey: The Daily Driver
hey is what ab should have been. Same mental model (total requests + concurrency), but saner output and a few extra tricks.
Rate limiting is the killer feature for controlled tests:
# 50 requests per second, unlimited total (run for 30 seconds)
hey -q 50 -z 30s https://cd-linux.club/api/endpoint
-q sets the rate, -z sets duration instead of request count. This is how you simulate sustained load at a specific throughput level — much more realistic than "hammer it as fast as possible."
Custom headers and POST:
hey -n 2000 -c 50 \
-H "Authorization: Bearer your-token" \
-H "Content-Type: application/json" \
-m POST \
-d '{"query": "test"}' \
https://cd-linux.club/api/search
One thing hey does that ab doesn’t: it respects HTTP/2 if you use an HTTPS endpoint and the server supports it. Use -http1 to force HTTP/1.1 if you want to isolate that variable.
siege: The Realistic Traffic Simulator
This is where siege separates itself. Real users don’t hammer a single URL. They browse — hit the homepage, click a product, add to cart, check out. If you only benchmark /index.html, you’re testing your cache and missing everything that actually matters.
Create a URL list file:
# urls.txt
https://cd-linux.club/
https://cd-linux.club/products/
https://cd-linux.club/products/widget-42
https://cd-linux.club/cart
https://cd-linux.club/api/inventory?sku=42
Then run siege against it:
siege -c 50 -t 60S -f urls.txt
-t 60S runs for 60 seconds. Siege will randomly cycle through the URL list per virtual user, which is a much closer approximation of real traffic patterns.
Add a delay between requests to simulate human think time:
siege -c 50 -t 60S -d 2 -f urls.txt
-d 2 adds a random delay of 0 to 2 seconds between requests per user. Drop this and you’re modeling bots, not people.
Siege also supports a config file at ~/.siege/siege.conf for persistent settings:
# ~/.siege/siege.conf
verbose = false
color = on
quiet = false
json_output = false
show-logfile = true
log = /var/log/siege.log
logfile = /var/log/siege.log
protocol = HTTP/1.1
chunked = true
cache = false
connection = close
concurrent = 25
delay = 1
timeout = 30
expire-session = false
failures = 512
Gotchas
ab’s HTTP/1.0 default. By default, ab speaks HTTP/1.0. Your server speaks HTTP/1.1 or HTTP/2. The benchmark you just ran is measuring protocol overhead that your real users never hit. Always use -k and confirm with -v 2 (verbose) that the server is actually sending Connection: Keep-Alive back.
The loopback isn’t production. Benchmarking from localhost to localhost removes network latency, TCP handshake overhead, and TLS from the equation. Your numbers will be 5-10x better than what users see. Benchmark from a separate machine on the same network — ideally a cheap VM in the same datacenter.
Single URL benchmarks miss your bottlenecks. If your app has a slow database query on /api/products/search but your benchmark only hits /health, you’ve learned nothing useful. Map your actual traffic patterns first.
concurrency ≠ users. -c 100 in ab/hey means 100 simultaneous in-flight requests, not 100 users. A single user might fire off 10 parallel requests if they have 10 browser tabs open. Think about what your -c value actually represents for your use case.
siege’s default concurrency is 15. Run siege --help and you’ll see it. If you set it in ~/.siege/siege.conf, that overrides the CLI default. Check your config file before trusting any numbers.
TLS certificate validation. All three tools verify SSL certificates by default. If you’re testing a staging server with a self-signed cert, use -k (ab), --disable-keepalive doesn’t help here — for hey use -allow-insecure, for siege add ssl-cert-file to the config or use --ye=insecure depending on version. Know your flags before assuming the tool is broken.
Server-side connection limits. Hit Nginx with 500 concurrent connections and you’ll see ECONNREFUSED errors if worker_connections is set too low. Those failed requests in your benchmark output are telling you about a config limit, not the real throughput ceiling. Check worker_connections in your Nginx config and ulimit -n on the server.
Production-Ready Patterns
Warm-up before measuring. Cold starts skew results — JIT compilation, DNS caches, database connection pools spinning up. Run a small warmup before your actual benchmark:
# Warmup
hey -n 500 -c 10 http://yourserver/api/endpoint > /dev/null
# Actual measurement
hey -n 10000 -c 100 -z 60s http://yourserver/api/endpoint
Capture baselines in CI. Store benchmark results in a file and compare on every deploy. A 20% regression in p99 latency is a bug.
#!/bin/bash
# benchmark.sh — run in CI after deploy
RESULT=$(hey -n 5000 -c 50 -q 100 \
-H "Authorization: Bearer $API_TOKEN" \
https://staging.yourapp.com/api/health 2>&1)
RPS=$(echo "$RESULT" | grep "Requests/sec" | awk '{print $2}')
P99=$(echo "$RESULT" | grep "99%" | awk '{print $2}')
echo "RPS: $RPS"
echo "P99: $P99"
# Fail CI if p99 exceeds 500ms
if (( $(echo "$P99 > 0.500" | bc -l) )); then
echo "ERROR: p99 latency ${P99}s exceeds threshold (0.500s)"
exit 1
fi
Use siege for soak testing. Short bursts reveal peak throughput. Long soak tests (30+ minutes at moderate load) reveal memory leaks, connection pool exhaustion, and garbage collection pauses that a 60-second test will never surface.
# 2-hour soak at 30 concurrent users
siege -c 30 -t 2H -d 3 -f production-urls.txt \
--log=/var/log/siege-soak-$(date +%Y%m%d).log
Check your server metrics during the test, not just after. CPU, memory, open file descriptors, active connections — these tell you why the numbers are what they are. A benchmark result without server metrics is just a guess. Use vmstat 1, ss -s, or a proper metrics stack if you have one.
Head-to-Head: When to Reach for Each
| Situation | Tool |
|---|---|
| Quick config change sanity check | ab |
| Daily development benchmarking | hey |
| Testing a specific throughput target | hey -q |
| Simulating real user browsing patterns | siege -f urls.txt |
| Soak test over hours | siege -t |
| HTTP/2 endpoint testing | hey |
| Need the tool on any random server | ab (already installed) |
| Scripting benchmark output for CI | hey (clean parseable output) |
ab is the pocket knife — always there, good enough for most jobs, nobody’s first choice for serious work. hey is the daily driver for developers who actually care about what the numbers mean. siege is what you pull out when you need to simulate real traffic or run a test overnight.
Most teams should have both hey and siege installed and use them for different phases: hey during development for fast iteration feedback, siege in staging for pre-release soak testing against realistic URL patterns.
The benchmark that matters is the one that reflects your actual traffic. Everything else is just watching a number go up.