ripgrep Advanced: –type, –json, –hidden, –pre, and Smart-Case Done Right

If you’re still reaching for grep -r out of muscle memory, I’m not going to judge you — but I will show you what you’re missing. ripgrep (rg) isn’t just a faster grep. Its real power is in the flags that let you treat search like a first-class engineering operation: typed, structured, composable, and scriptable.

This article covers the five features that separate casual rg users from people who can grep a 50GB monorepo and get back a clean JSON feed in under a second. We’re talking --type, --json, --hidden, --pre, and smart-case. No hand-waving — concrete flags, real examples, and the gotchas that will bite you if you skip this.

The official repo is at https://github.com/BurntSushi/ripgrep. The README is excellent. Read it after this.

---

Why ripgrep Is Not Just "Faster grep"

The performance story is real — ripgrep uses Rust’s regex engine, parallelizes file I/O automatically, and respects .gitignore by default. But that’s table stakes.

What actually matters in day-to-day work is the semantic layer. ripgrep knows about file types. It can emit machine-parseable output. It can pipe every file through an arbitrary preprocessor before searching. These aren’t niceties — they change what problems you can solve from the command line.

---

Smart-Case: The Flag You Didn’t Know You Needed

By default, rg is case-sensitive. Pass -i and it’s case-insensitive. Neither is always right.

Smart-case (-S or --smart-case) does something better: if your pattern has no uppercase letters, it searches case-insensitively. The moment you add an uppercase character, it flips to case-sensitive.

# Matches "error", "Error", "ERROR"
rg -S "error" logs/

# Matches ONLY "ApiError" — uppercase triggers case-sensitive mode
rg -S "ApiError" src/

This mirrors how most people actually think when searching code. You type in lowercase when you’re fishing; you capitalize when you know exactly what you want.

Set it as your default in ~/.config/ripgrep/config (or $RIPGREP_CONFIG_PATH):

# ~/.config/ripgrep/config
--smart-case
--hidden

Gotcha: smart-case is pattern-scoped, not flag-scoped. If you use multiple patterns with -e, each pattern is evaluated independently. rg -S -e "error" -e "ApiError" will match "ERROR" (from the first pattern) but only exact "ApiError" from the second. Exactly right behavior, but worth knowing.

---

--type and --type-add: Search With Surgical Precision

Every project has noise: generated files, vendored dependencies, minified JS, compiled artifacts. --type lets you declare exactly which file categories you care about.

# Search only Python files
rg --type py "def authenticate"

# Short form
rg -t py "def authenticate"

# Exclude a type
rg --type-not js "TODO" src/

# Short form
rg -T js "TODO" src/

List all built-in types with:

rg --type-list

You’ll get 200+ entries. It covers everything from agda to zsh. For most projects, you’ll use py, js, ts, go, rust, yaml, toml, html, css, sql.

Defining Custom Types

Built-in types won’t cover your stack perfectly. --type-add lets you define your own:

# Add a custom type "proto" for protobuf files
rg --type-add 'proto:*.proto' -t proto "service Auth"

# Stack multiple globs
rg --type-add 'config:{*.yaml,*.yml,*.toml,*.json}' -t config "database_url"

For types you use constantly, make them permanent in your config:

# ~/.config/ripgrep/config
--type-add=tf:*.tf
--type-add=tf:*.tfvars
--type-add=helm:{Chart.yaml,values.yaml,templates/**/*.yaml}

Now rg -t tf "aws_s3_bucket" just works everywhere.

Gotcha: --type applies to file extensions and names via glob matching. It does not look inside files to determine type (no shebang detection). A shell script named deploy with no extension won’t be caught by -t sh. Use globs directly in those cases: rg -g 'deploy*' "kubectl".

Production practice: In CI scripts, always use -t or -g when grepping source code. Without it, you’ll eventually hit a build artifact or generated file that triggers a false positive and breaks your pipeline at 2 AM.

---

--hidden: When the Answer Is in the Dotfiles

ripgrep skips hidden files and directories (anything starting with .) by default. This is sensible for interactive use — you rarely want to grep inside .git/ or .DS_Store. But plenty of real config lives in hidden paths: .env, .github/workflows, .eslintrc, .docker/, ~/.config/.

# Include hidden files
rg --hidden "POSTGRES_URL" .

# Short form
rg -. "POSTGRES_URL" .

Note that even with --hidden, ripgrep still respects .gitignore. To also override ignore rules:

# Ignore nothing — search absolutely everything
rg --hidden --no-ignore "secret_key" .

# Nuclear option: --unrestricted (same as -uuu, disables all filtering)
rg -uuu "secret_key" .

The granularity matters. --hidden alone is usually enough. --no-ignore is for auditing purposes — checking if your .gitignore is actually covering what you think it covers. --unrestricted (-u stacks: one -u disables .gitignore, two disables hidden filtering, three disables binary filtering) is for forensic work.

Practical Example: Audit Exposed Secrets

# Check if any secrets are accidentally committed or left in dotfiles
rg --hidden -t yaml -t toml -t env \
   -e "password\s*=" \
   -e "secret\s*=" \
   -e "api_key\s*=" \
   --glob '!.git' .

Gotcha: If you add --hidden to your global config, searching inside project directories becomes slower because ripgrep now descends into .git/ unless you exclude it. Always pair global --hidden with --glob '!.git' in config, or accept the perf hit:

# ~/.config/ripgrep/config
--hidden
--glob=!.git

---

--json: Turn Search Results Into a Data Pipeline

This is where ripgrep stops being a terminal tool and starts being a building block.

--json emits newline-delimited JSON (NDJSON). Each line is a JSON object describing one event: a Begin (file opened), Match (line matched), End (file closed), or Summary (total stats).

rg --json "panic!" src/

Output looks like:

{"type":"begin","data":{"path":{"text":"src/main.rs"}}}
{"type":"match","data":{"path":{"text":"src/main.rs"},"lines":{"text":"    panic!(\"index out of bounds\");\n"},"line_number":42,"absolute_offset":1204,"submatches":[{"match":{"text":"panic!"},"start":4,"end":10}]}}
{"type":"end","data":{"path":{"text":"src/main.rs"},"binary_offset":null,"stats":{"elapsed":{"secs":0,"nanos":46231},"searches":1,"searches_with_match":1,"bytes_searched":4096,"bytes_printed":312,"matched_lines":1,"matches":1}}}
{"type":"summary","data":{"elapsed_total":{"secs":0,"nanos":183210},"stats":{"elapsed":{"secs":0,"nanos":183210},"searches":1,"searches_with_match":1,"bytes_searched":4096,"bytes_printed":312,"matched_lines":1,"matches":1}}}

Piping Into jq

# Get just the file paths that contain matches
rg --json "TODO" src/ | jq -r 'select(.type=="match") | .data.path.text' | sort -u

# Get match text with line numbers as a TSV
rg --json "TODO" src/ \
  | jq -r 'select(.type=="match") | [.data.path.text, (.data.line_number | tostring), .data.lines.text] | @tsv'

# Count matches per file
rg --json "deprecated" src/ \
  | jq -r 'select(.type=="end") | [.data.path.text, (.data.stats.matches | tostring)] | @csv'

Build a Match Report in Bash

#!/bin/bash
# audit-todos.sh — generate a CSV of all TODOs with owner hints

OUTPUT="todos_$(date +%Y%m%d).csv"
echo "file,line,text" > "$OUTPUT"

rg --json -i "TODO|FIXME|HACK" src/ \
  | jq -r '
      select(.type == "match") |
      [
        .data.path.text,
        (.data.line_number | tostring),
        (.data.lines.text | gsub("\n";"") | gsub(",";" "))
      ] | @csv
    ' >> "$OUTPUT"

echo "Written to $OUTPUT ($(wc -l < "$OUTPUT") entries)"

Gotcha: The JSON output uses {"text": "..."} wrappers around paths and line content because ripgrep handles non-UTF-8 paths. Those get a bytes field instead of text. If you’re piping to jq and hitting null values, check for bytes vs text in the path object:

# Robust path extraction
jq -r 'select(.type=="match") | .data.path | (.text // (.bytes | @base64d))'

Production practice: --json plus jq is the right way to integrate ripgrep into build tooling, pre-commit hooks, and audit scripts. It’s stable, machine-readable, and doesn’t depend on parsing colored terminal output. Never parse rg‘s human-readable output in scripts — the format can shift between versions, and color codes will ruin your day.

---

--pre: Search Inside Anything

--pre is ripgrep’s wildcard extension mechanism. It lets you specify a preprocessor: a program that takes a file path as its argument, outputs the file’s content to stdout, and lets ripgrep search that output instead of the raw file.

This means you can search inside compressed files, binary formats, encrypted vaults, or anything else you can pipe through a converter.

# Search inside gzipped log files
rg --pre 'zcat' "OOM killed" /var/log/app/*.gz

# Search inside zip archives (requires custom wrapper)
rg --pre ~/bin/unzip-to-stdout "database_url" deploy/artifacts/*.zip

Real-World Example: Search Git History

This is where --pre gets genuinely powerful. Search across all commits without checking anything out:

#!/bin/bash
# git-grep-history.sh
# Usage: rg --pre ./git-grep-history.sh "pattern" <path-to-git-repo>

# ripgrep passes the path as $1
git -C "$1" log --all --format="%H %s" 2>/dev/null | while read hash subject; do
  echo "=== commit: $hash — $subject ==="
  git -C "$1" show "$hash" 2>/dev/null
done

rg --pre ./git-grep-history.sh "dropped_column" /path/to/repo/.git

You can search for when a column was dropped from a migration without doing git log -p and hunting manually.

Decompress Multiple Formats on the Fly

#!/bin/bash
# decompress.sh — universal decompressor for rg --pre

FILE="$1"
case "$FILE" in
  *.gz)   zcat "$FILE" ;;
  *.bz2)  bzcat "$FILE" ;;
  *.xz)   xzcat "$FILE" ;;
  *.zst)  zstdcat "$FILE" ;;
  *.lz4)  lz4cat "$FILE" ;;
  *)      cat "$FILE" ;;  # pass-through for normal files
esac

rg --pre ./decompress.sh "error" /var/log/archive/

Gotcha: --pre disables all of ripgrep’s own binary detection and encoding handling on that file. Your preprocessor is responsible for producing valid UTF-8 (or the encoding ripgrep expects). If your preprocessor crashes or outputs garbage, ripgrep will silently skip the match — no error, just nothing. Always test your preprocessor script standalone first:

./decompress.sh some-file.gz | head -20

Gotcha #2: --pre runs once per file, not once globally. If you’re processing thousands of files, make sure your preprocessor is fast. Spinning up a Python interpreter for each of 10,000 files will undo every performance advantage ripgrep has.

Production practice: Combine --pre with --pre-glob to apply preprocessing only to matching filenames:

# Only run zcat on .gz files; let everything else pass through natively
rg --pre-glob '*.gz' --pre 'zcat' "error" logs/

This is far more efficient than a pass-through wrapper.

---

Putting It All Together

Here’s a realistic scenario: you’re auditing a microservices repo for hardcoded credentials in config files, including archived configs in compressed backups, dotfiles, and environment files that might be lurking in git history.

#!/bin/bash
# security-audit.sh

# 1. Live filesystem — config files including dotfiles
rg \
  --hidden \
  --glob '!.git' \
  --smart-case \
  --type-add 'cfg:{*.yaml,*.yml,*.toml,*.env,*.ini,.env*}' \
  -t cfg \
  --json \
  -e "password\s*[:=]" \
  -e "secret\s*[:=]" \
  -e "api[_-]key\s*[:=]" \
  . \
  | jq -r 'select(.type=="match") | "\(.data.path.text):\(.data.line_number)  \(.data.lines.text | rtrimstr("\n"))"'

echo "---"

# 2. Compressed log archives
rg \
  --pre-glob '*.gz' \
  --pre 'zcat' \
  --json \
  -e "Authorization: Bearer" \
  -e "X-Api-Key:" \
  logs/archive/ \
  | jq -r 'select(.type=="match") | "\(.data.path.text):\(.data.line_number)"'

One script, two passes, clean output. The --json pipeline means you can redirect this into a reporting tool, feed it into a SIEM, or dump it to CSV without touching the output format.

---

Config File: Your Permanent Setup

Stop retyping flags. Put your preferred defaults in ~/.config/ripgrep/config:

# Smart case by default
--smart-case

# Always search hidden files, skip .git
--hidden
--glob=!.git

# Custom type definitions
--type-add=tf:{*.tf,*.tfvars}
--type-add=k8s:{*.yaml,*.yml}
--type-add=env:{.env,.env.*,*.env}

# Follow symlinks
--follow

# Show some context around matches
--context=2

Set the config path explicitly if you want multiple profiles:

export RIPGREP_CONFIG_PATH=~/.config/ripgrep/config

---

One Honest Caveat

ripgrep is not a replacement for find. It searches file contents — traversal logic like "files modified in the last 24 hours" or "files larger than 10MB" belongs to find or fd. The right pattern is often fd for discovery, rg for content:

# Find recently modified Python files containing "deprecated"
fd --changed-within 1d -e py | xargs rg "deprecated"

These tools compose cleanly. Use them that way.

---

Summary Cheat Sheet

Flag	What it does	When to use
-S / --smart-case	Case-insensitive unless pattern has uppercase	Default for interactive search
-t / --type	Restrict to file type	Anywhere you don’t want noise
--type-add	Define custom file types	Project-specific extensions
-. / --hidden	Include dotfiles	Configs, workflows, .env files
--no-ignore	Skip .gitignore rules	Audits, debugging ignore rules
--json	NDJSON output	Scripts, CI, data pipelines
--pre	Run preprocessor per file	Compressed files, binary formats, git history
--pre-glob	Apply --pre only to matching filenames	Performance — avoid spawning procs for all files

The flags are composable and orthogonal. That’s the design philosophy of the whole tool — small, well-defined behaviors that stack cleanly. Once you internalize that, you stop thinking "what grep command do I need" and start thinking "what do I actually want to find."