Every team has that one API endpoint that quietly breaks in staging, gets deployed anyway, and causes a 2 AM incident because nobody checked the response shape. Not a missing field — just a field that changed from a string to a number, or an array that started returning null instead of []. Tests were green. No schema validation existed.
This article fixes that.
We’re going to use HTTPie for clean, readable HTTP requests, wire up JSON Schema to validate every response, and shove the whole thing into CI so that a broken API contract fails the pipeline before it ever hits staging.
No heavy frameworks. No Postman GUI exports. Just shell scripts, a JSON schema file, and a GitHub Actions workflow you can steal and adapt in 20 minutes.
Why HTTPie and Not curl
If you’re still writing curl -H "Content-Type: application/json" -d '{"key":"value"}' -X POST https://api.example.com/endpoint — stop. That’s punishment, not tooling.
HTTPie (github.com/httpie/cli) is a modern HTTP client built for humans. Same power as curl, but the syntax is sane, the output is colored and formatted by default, and JSON requests don’t require escaping nightmares. It also has --offline mode for testing your own request construction, session support, and plugin hooks.
Install it:
# On Debian/Ubuntu
sudo apt install httpie
# With pip (latest version)
pip install httpie
# On macOS
brew install httpie
The basic difference in daily use:
# curl — acceptable if you enjoy suffering
curl -s -X POST https://api.example.com/users \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $TOKEN" \
-d '{"name":"alice","role":"admin"}'
# HTTPie — actual human syntax
http POST https://api.example.com/users \
"Authorization: Bearer $TOKEN" \
name=alice role=admin
HTTPie auto-sets Content-Type: application/json when you pass key=value pairs, formats the response, and exits non-zero on HTTP errors when you pass --check-status. That last bit is critical for scripting.
The Problem With "It Returns 200"
Most ad-hoc API tests just check the status code. That’s necessary but nowhere near sufficient.
APIs break their consumers in quieter ways:
- A field gets renamed (
user_id→userId) - A nullable field becomes
nullwhen it used to always have a value - An array endpoint starts returning a single object on empty results instead of
[] - A timestamp field changes from ISO 8601 to Unix epoch
None of these change the HTTP status code. All of them break downstream consumers.
JSON Schema lets you describe the exact shape of a valid response and fail loudly when the shape changes. Combined with HTTPie’s clean output and a tiny validation script, you get API contract testing without a heavyweight framework.
Setting Up JSON Schema Validation
We’ll use Python’s jsonschema library. It’s mature, well-documented, and available everywhere CI runs.
pip install jsonschema
For Node.js users, ajv is the gold standard, but the Python path integrates more cleanly into shell scripts without a package.json ceremony.
Writing Your First Schema
Here’s a realistic schema for a paginated user list endpoint:
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"required": ["data", "meta"],
"additionalProperties": false,
"properties": {
"data": {
"type": "array",
"items": {
"type": "object",
"required": ["id", "name", "email", "created_at"],
"additionalProperties": false,
"properties": {
"id": { "type": "integer" },
"name": { "type": "string", "minLength": 1 },
"email": { "type": "string", "format": "email" },
"role": {
"type": "string",
"enum": ["admin", "user", "viewer"]
},
"created_at": {
"type": "string",
"format": "date-time"
}
}
}
},
"meta": {
"type": "object",
"required": ["total", "page", "per_page"],
"properties": {
"total": { "type": "integer", "minimum": 0 },
"page": { "type": "integer", "minimum": 1 },
"per_page": { "type": "integer", "minimum": 1, "maximum": 100 }
}
}
}
}
Notice "additionalProperties": false. This is the strict mode that makes schemas actually useful — it means any field added to the response that isn’t declared in the schema will cause a validation failure. That’s intentional. You want to know when the API surface changes unexpectedly.
The Validation Script
Save this as scripts/validate.py:
#!/usr/bin/env python3
"""
Validate a JSON file against a JSON Schema.
Usage: validate.py <schema_file> <json_file>
Exit: 0 on success, 1 on validation failure, 2 on usage/parse error.
"""
import sys
import json
import jsonschema
from jsonschema import validate, ValidationError, SchemaError
def main():
if len(sys.argv) != 3:
print(f"Usage: {sys.argv[0]} <schema.json> <data.json>", file=sys.stderr)
sys.exit(2)
schema_path, data_path = sys.argv[1], sys.argv[2]
try:
with open(schema_path) as f:
schema = json.load(f)
with open(data_path) as f:
data = json.load(f)
except (FileNotFoundError, json.JSONDecodeError) as e:
print(f"ERROR: {e}", file=sys.stderr)
sys.exit(2)
try:
validate(instance=data, schema=schema)
print(f"OK: {data_path} is valid against {schema_path}")
sys.exit(0)
except ValidationError as e:
# Print the full path to the failing field, not just the message
path = " -> ".join(str(p) for p in e.absolute_path) or "(root)"
print(f"FAIL [{path}]: {e.message}", file=sys.stderr)
sys.exit(1)
except SchemaError as e:
print(f"SCHEMA ERROR: {e.message}", file=sys.stderr)
sys.exit(2)
if __name__ == "__main__":
main()
The path reporting matters. When validation fails, knowing it failed at data -> 3 -> email is actually useful. Knowing it failed "somewhere" is not.
Writing the Test Script
Now glue it together. Save this as scripts/api_tests.sh:
#!/usr/bin/env bash
# API contract tests — run locally or in CI
# Requires: httpie, python3, jsonschema (pip install jsonschema)
set -euo pipefail
BASE_URL="${API_BASE_URL:-https://cd-linux.club:8000}"
SCHEMA_DIR="$(dirname "$0")/../schemas"
VALIDATE="$(dirname "$0")/validate.py"
TMP_DIR=$(mktemp -d)
FAILURES=0
cleanup() { rm -rf "$TMP_DIR"; }
trap cleanup EXIT
# Helper: run a test case
# Usage: run_test <name> <schema_file> <http_args...>
run_test() {
local name="$1"
local schema="$2"
shift 2
local out="$TMP_DIR/${name}.json"
echo "=> $name"
# --check-status: non-2xx exits non-zero
# --ignore-stdin: prevents hanging when piped
# -b: body only, no headers
if ! http --check-status --ignore-stdin -b "$@" > "$out" 2>/dev/null; then
echo " FAIL: HTTP request returned non-2xx status"
(( FAILURES++ )) || true
return
fi
if ! python3 "$VALIDATE" "$SCHEMA_DIR/$schema" "$out"; then
(( FAILURES++ )) || true
fi
}
# -------------------------------------------------------
# Test cases
# -------------------------------------------------------
run_test "list_users" \
"users_list.json" \
GET "${BASE_URL}/api/v1/users" \
"Authorization: Bearer ${API_TOKEN:-test-token}"
run_test "get_user" \
"user_single.json" \
GET "${BASE_URL}/api/v1/users/1" \
"Authorization: Bearer ${API_TOKEN:-test-token}"
run_test "create_user" \
"user_single.json" \
POST "${BASE_URL}/api/v1/users" \
"Authorization: Bearer ${API_TOKEN:-test-token}" \
name="ci-test-user" \
email="[email protected]" \
role="viewer"
# -------------------------------------------------------
echo ""
if [ "$FAILURES" -gt 0 ]; then
echo "FAILED: $FAILURES test(s) did not pass."
exit 1
else
echo "ALL TESTS PASSED."
exit 0
fi
The structure is intentional. run_test handles the boilerplate — status check, output capture, schema validation, failure counting. Adding a new test case is one function call. When tests fail, the script collects all failures instead of stopping at the first one, which is more useful in CI.
Gotchas
1. additionalProperties: false will bite you during development.
The moment a developer adds a field to the response (even a useful one), your schema tests fail. That’s correct behavior — the schema is a contract and contracts need updating when the API changes. Some teams flip this to true while iterating and only lock it down pre-release. Pick a policy and document it.
2. HTTPie sessions and auth tokens in CI.
Don’t store tokens in the script. Pass them as environment variables and let CI inject secrets. The ${API_TOKEN:-test-token} pattern in the script above uses a safe default for local runs and expects the real secret in CI.
3. format validators require an extra package.
jsonschema doesn’t validate "format": "email" or "format": "date-time" by default — it just ignores them. To actually validate formats:
pip install jsonschema[format]
# then in Python:
from jsonschema import FormatChecker
validate(instance=data, schema=schema, format_checker=FormatChecker())
Update the validate.py script to use format_checker=FormatChecker() in the validate() call if you care about format enforcement.
4. Test data isolation.
The create_user test above creates real data in whatever environment it runs against. In CI, either point at a test environment that gets reset between runs, or add teardown logic to delete the created resource. Leaving orphaned test records in a shared environment will eventually cause problems.
5. HTTPie’s JSON output isn’t always stable.
It pretty-prints JSON, which is fine for validate.py. But if you’re grepping the output or parsing it with jq, be aware that http -b returns formatted JSON. Use jq -c if you need compact output downstream.
Wiring into GitHub Actions
Save this as .github/workflows/api-tests.yml:
name: API Contract Tests
on:
push:
branches: [main, develop]
pull_request:
branches: [main]
# Allow manual runs against non-default environments
workflow_dispatch:
inputs:
environment:
description: "Target environment (staging/production)"
required: false
default: "staging"
jobs:
api-tests:
runs-on: ubuntu-latest
environment: staging # GitHub environment with secrets
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
cache: pip
- name: Install dependencies
run: |
pip install httpie "jsonschema[format]"
- name: Wait for service (if testing a PR deployment)
if: github.event_name == 'pull_request'
run: |
# Adjust URL and timeout to your deploy time
timeout 120 bash -c \
'until curl -sf "${API_BASE_URL}/health"; do sleep 3; done'
env:
API_BASE_URL: ${{ vars.API_BASE_URL }}
- name: Run API contract tests
run: bash scripts/api_tests.sh
env:
API_BASE_URL: ${{ vars.API_BASE_URL }}
API_TOKEN: ${{ secrets.API_TOKEN }}
- name: Upload failure artifacts
if: failure()
uses: actions/upload-artifact@v4
with:
name: api-test-responses
path: /tmp/tmp*/ # captures the TMP_DIR output files
retention-days: 3
A few things worth calling out here:
The wait for service step is for PR-based deployments where the test environment spins up as part of the PR pipeline. If you’re testing a stable staging environment, drop that step.
The artifact upload on failure captures the raw JSON responses that failed validation. Being able to download and inspect the actual malformed response is far more useful than staring at a log line that says "validation failed."
vars.API_BASE_URL uses GitHub’s non-secret environment variables (available since 2023) so the URL is visible in logs without exposing credentials.
Production-Ready Additions
Once the basics work, here’s what teams running this in anger add next:
Response time assertions. HTTPie has --timeout but won’t fail on slow responses. Wrap your http call with time and check the duration:
START=$(date +%s%3N)
http --ignore-stdin -b "$@" > "$out"
END=$(date +%s%3N)
ELAPSED=$(( END - START ))
if [ "$ELAPSED" -gt 2000 ]; then
echo " WARN: response took ${ELAPSED}ms (threshold: 2000ms)"
fi
Schema versioning. Store schemas in a schemas/v1/ directory structure. When the API version changes, add schemas/v2/ without deleting v1. Your CI tests both versions simultaneously during any transition period.
Negative tests. Test that your API correctly rejects bad input. A 422 on an invalid payload is a feature, and you should test that it stays a 422:
run_negative_test() {
local name="$1"
local expected_status="$2"
shift 2
if http --ignore-stdin "$@" &>/dev/null; then
echo " FAIL: expected non-2xx but got success"
(( FAILURES++ )) || true
fi
}
run_negative_test "create_user_no_email" 422 \
POST "${BASE_URL}/api/v1/users" \
"Authorization: Bearer ${API_TOKEN}" \
name="bad-user"
Multiple environments in one run. Parameterize api_tests.sh to accept a base URL argument and call it twice in CI — once for staging, once for production (read-only tests only in prod). Drift between environments catches a lot of configuration bugs that unit tests never will.
Structuring Your Schema Directory
Keep schemas next to the tests, not buried somewhere in docs/:
project/
├── schemas/
│ ├── users_list.json
│ ├── user_single.json
│ └── health.json
├── scripts/
│ ├── api_tests.sh
│ └── validate.py
└── .github/
└── workflows/
└── api-tests.yml
Check schemas into version control. When an API change is merged, the schema update goes in the same PR. Reviewers see both the code change and the contract change side by side — that’s the point.
Where This Fits in Your Test Pyramid
This isn’t a replacement for unit tests or integration tests. It sits at the top of the pyramid: end-to-end contract verification against a running service. Run it:
- On every PR against a staging deployment
- After every production deploy as a smoke test
- On a schedule (every 15 minutes) against production with read-only tests, so you catch environmental drift before users do
The script as written takes under 10 seconds for a handful of endpoints. It’s fast enough to run on every push and cheap enough to run continuously.
A broken API response shape is a bug. Treat it like one, test for it automatically, and stop finding out about it from your users.