Server-streaming gRPC looks innocent in tutorials. You write a loop, call stream.Send(), and it works perfectly — until you demo it with a dataset 10x the size, and your server starts allocating RAM like it owns the datacenter.
The problem is that most guides stop at "it works." They don’t show you what happens when the client can’t keep up, when the network gets congested, or when you have 10,000 concurrent streams each trying to flood a mobile client with events. That’s where naive implementations collapse: goroutine leaks, unbounded buffers, OOM kills, and silent message drops.
This article is about the machinery underneath gRPC streaming and the patterns that keep it honest under pressure. We’ll go from HTTP/2 flow control windows all the way to production-grade Go implementations. Official gRPC-Go lives at github.com/grpc/grpc-go — keep it open.
What "backpressure" actually means here
Backpressure is the mechanism by which a slow consumer tells a fast producer to slow down — instead of silently dropping messages or buffering unboundedly until something explodes.
In a unary RPC this isn’t your problem. In server streaming, you’re running a loop that can generate messages faster than the client can read them. Without backpressure, those messages queue up somewhere. The question is: where, and what happens when that queue fills?
With no controls at all: they queue in the kernel’s TCP send buffer, then in gRPC’s internal write buffer, then your application’s channel, then heap allocations start piling up, and eventually something dies. The failure is usually non-obvious — a gradual OOM rather than a clean error.
HTTP/2 flow control: the foundation you’re building on
gRPC runs over HTTP/2, which has its own flow control baked in at the transport layer. Understanding this is not optional if you want to reason about your system’s behavior.
HTTP/2 flow control works with windows — credit-based counters at two levels:
- Connection-level window: shared across all streams on a connection.
- Stream-level window: per individual RPC stream.
The receiver advertises how much data it can accept. The sender must not exceed that window. When the window hits zero, Send() blocks at the transport layer — the data literally cannot go anywhere until the receiver sends a WINDOW_UPDATE frame.
This is actually good news. It means HTTP/2 does apply backpressure automatically when the client stops reading. The bad news: gRPC-Go’s default window sizes are generous (64KB initial, but the library can grow them aggressively), so you might not feel the pressure until buffers are already enormous. And application-level code can still overwhelm the system before the transport-level window kicks in.
# Inspect what's actually happening on the wire
tcpdump -i lo -w grpc_capture.pcap port 50051
# Then open in Wireshark: filter http2.type == 8 to see WINDOW_UPDATE frames
gRPC-Go transport knobs
Before writing application code, configure the transport sensibly. These options live in google.golang.org/grpc and are set on the server:
import (
"google.golang.org/grpc"
"google.golang.org/grpc/keepalive"
)
server := grpc.NewServer(
// Initial window size per stream (default: 65535 bytes)
// Smaller = tighter backpressure, more WINDOW_UPDATE frames
// Larger = higher throughput at the cost of buffer bloat
grpc.InitialWindowSize(1 << 16), // 64KB
grpc.InitialConnWindowSize(1 << 20), // 1MB total per connection
// Kill idle connections before they accumulate
grpc.KeepaliveParams(keepalive.ServerParameters{
MaxConnectionIdle: 15 * time.Minute,
MaxConnectionAge: 30 * time.Minute,
MaxConnectionAgeGrace: 5 * time.Second,
Time: 5 * time.Second,
Timeout: 1 * time.Second,
}),
grpc.KeepaliveEnforcementPolicy(keepalive.EnforcementPolicy{
MinTime: 5 * time.Second,
PermitWithoutStream: true,
}),
)
Smaller InitialWindowSize means the transport will apply backpressure sooner. The tradeoff is latency on high-bandwidth streams. Start with 64KB and tune upward only if profiling shows you’re CPU-bound on WINDOW_UPDATE processing.
The naive implementation (and why it fails)
Here’s what everyone writes first:
func (s *Server) Watch(req *pb.WatchRequest, stream pb.Service_WatchServer) error {
for event := range s.eventSource(req) {
if err := stream.Send(event); err != nil {
return err
}
}
return nil
}
This looks correct. It checks the error from Send(). But s.eventSource() returns events as fast as they’re produced. If Send() is blocking at the transport layer (because the client is slow), s.eventSource() is still generating and buffering events in that channel. Your RAM is paying for the client’s sluggishness.
If eventSource is backed by a database cursor or a Kafka consumer, you’re also holding those resources open — connection handles, file descriptors, locks — for the entire duration of the client’s stall.
Pattern 1: Context-aware send with timeout
The first fix is cheap: never let Send() block indefinitely. Respect the stream’s context and set a send deadline.
gRPC’s stream.Send() doesn’t accept a context directly, but the stream’s context is accessible via stream.Context(). Wrap sends in a select or use a timeout goroutine to enforce limits:
func sendWithTimeout(stream pb.Service_WatchServer, msg *pb.Event, timeout time.Duration) error {
done := make(chan error, 1)
go func() {
done <- stream.Send(msg)
}()
select {
case err := <-done:
return err
case <-time.After(timeout):
return fmt.Errorf("send timeout after %s: client too slow", timeout)
case <-stream.Context().Done():
return stream.Context().Err()
}
}
This is blunt but effective. If the client hasn’t read fast enough within your SLA window, you drop it. For most event-streaming use cases (dashboards, log tails, telemetry), this is the right call — a client that can’t keep up is more harmful connected than disconnected.
Gotcha: spawning a goroutine per Send() call is wasteful. Use this pattern only for streams where each message is expensive and the send frequency is low. For high-frequency streams, see Pattern 3.
Pattern 2: Semaphore-controlled producer
When your event source is expensive to run (database query, upstream RPC), throttle production with a semaphore — only produce more when you have capacity to send.
type streamHandler struct {
sendSem chan struct{} // semaphore: N slots = N messages in-flight
}
func newStreamHandler(concurrency int) *streamHandler {
sem := make(chan struct{}, concurrency)
for i := 0; i < concurrency; i++ {
sem <- struct{}{}
}
return &streamHandler{sendSem: sem}
}
func (h *streamHandler) Watch(req *pb.WatchRequest, stream pb.Service_WatchServer) error {
ctx := stream.Context()
for {
// Acquire a send slot — blocks if we're at capacity
select {
case <-h.sendSem:
case <-ctx.Done():
return ctx.Err()
}
event, err := h.fetchNextEvent(ctx, req)
if err != nil {
h.sendSem <- struct{}{} // release on error
return err
}
if event == nil {
h.sendSem <- struct{}{}
return nil // stream exhausted
}
go func(e *pb.Event) {
defer func() { h.sendSem <- struct{}{} }()
if err := stream.Send(e); err != nil {
// log: client disconnected or send failed
_ = err
}
}(event)
}
}
This bounds in-flight messages to concurrency. The producer loop naturally throttles when all slots are occupied — it blocks on the semaphore channel until a send completes.
Gotcha: the goroutine for Send() runs after the semaphore is acquired, but Send() itself is not goroutine-safe in gRPC-Go. You must serialize calls to stream.Send(). The above example has a race — fix it by keeping sends on a single goroutine with a channel:
// Send-side worker (single goroutine owns stream.Send)
func runSendWorker(ctx context.Context, stream pb.Service_WatchServer, msgs <-chan *pb.Event) error {
for {
select {
case msg, ok := <-msgs:
if !ok {
return nil
}
if err := stream.Send(msg); err != nil {
return err
}
case <-ctx.Done():
return ctx.Err()
}
}
}
This is the canonical pattern: one goroutine owns the stream, everything else writes to a channel. The channel buffer is your in-flight buffer — size it deliberately.
Pattern 3: Bounded channel with drop policy
For high-frequency event streams (metrics, log tails, real-time feeds), you often need a drop policy rather than blocking the producer. The channel buffer is your queue; when it’s full, you decide: drop oldest, drop newest, or apply a sampling strategy.
const sendBufSize = 256 // tune per stream SLA
func (s *Server) StreamMetrics(req *pb.MetricsRequest, stream pb.Service_StreamServer) error {
ctx := stream.Context()
sendCh := make(chan *pb.Metric, sendBufSize)
// Producer goroutine
go func() {
defer close(sendCh)
for metric := range s.metrics.Subscribe(ctx, req.Filter) {
select {
case sendCh <- metric:
// buffered, will be sent
default:
// buffer full — drop and record
droppedMetricsTotal.Inc()
case <-ctx.Done():
return
}
}
}()
// Consumer — single goroutine owns stream.Send
for {
select {
case msg, ok := <-sendCh:
if !ok {
return nil
}
if err := stream.Send(msg); err != nil {
return err
}
case <-ctx.Done():
return ctx.Err()
}
}
}
The key design decision is the drop policy in the producer’s select. The default case (drop newest) is simplest. For drop-oldest, you need a different structure — read one off the channel before writing:
// Drop oldest if buffer full
select {
case sendCh <- metric:
default:
select {
case <-sendCh: // discard oldest
default:
}
sendCh <- metric // now there's room
}
Gotcha: drop metrics are not optional in production. Instrument them. A droppedMetricsTotal counter that never increments tells you your buffer is appropriately sized; one that spikes tells you a client is falling behind and you need to page someone.
Pattern 4: Token bucket rate limiting per stream
Sometimes you want to cap throughput by rate, not just by buffer size. A token bucket is the right tool — it allows short bursts while enforcing a long-term rate ceiling.
golang.org/x/time/rate gives you a production-ready token bucket:
import "golang.org/x/time/rate"
func (s *Server) StreamEvents(req *pb.EventRequest, stream pb.Service_StreamServer) error {
ctx := stream.Context()
// 1000 events/sec sustained, burst of 200
limiter := rate.NewLimiter(rate.Limit(1000), 200)
for event := range s.events.Chan(ctx) {
// WaitN blocks until a token is available or ctx is done
if err := limiter.Wait(ctx); err != nil {
return err // context cancelled
}
if err := stream.Send(event); err != nil {
return err
}
}
return nil
}
limiter.Wait(ctx) does the right thing: it blocks until a token is available, but respects context cancellation. No additional timeout handling needed.
For per-client rate limiting (preventing one client from starving others), store limiters in a sync.Map keyed by client identity extracted from the stream context:
func (s *Server) limiterFor(ctx context.Context) *rate.Limiter {
clientID := extractClientID(ctx) // from metadata or peer address
v, _ := s.limiters.LoadOrStore(clientID, rate.NewLimiter(500, 50))
return v.(*rate.Limiter)
}
Gotcha: sync.Map entries leak if clients disconnect and never reconnect. Add a cleanup goroutine that evicts stale entries, or use a TTL cache library instead.
Pattern 5: Adaptive throttling from window pressure
This one is more advanced: instead of a fixed rate, dynamically back off when you detect the transport window is under pressure. gRPC-Go doesn’t expose the HTTP/2 window directly, but you can approximate pressure from Send() latency.
type adaptiveSender struct {
window time.Duration // current sleep between sends
minWindow time.Duration
maxWindow time.Duration
alpha float64 // smoothing factor
}
func (a *adaptiveSender) send(ctx context.Context, stream pb.Service_WatchServer, msg *pb.Event) error {
if a.window > 0 {
select {
case <-time.After(a.window):
case <-ctx.Done():
return ctx.Err()
}
}
start := time.Now()
err := stream.Send(msg)
elapsed := time.Since(start)
// EWMA: if Send took longer than expected, back off
if elapsed > 5*time.Millisecond {
a.window = time.Duration(a.alpha*float64(elapsed) + (1-a.alpha)*float64(a.window))
if a.window > a.maxWindow {
a.window = a.maxWindow
}
} else {
a.window = time.Duration((1 - a.alpha) * float64(a.window))
if a.window < a.minWindow {
a.window = a.minWindow
}
}
return err
}
This approach is fragile unless your baseline latencies are well-understood. Don’t use it as a primary mechanism — layer it on top of a bounded channel (Pattern 3) as a soft governor.
Gotchas: the production hit list
Goroutine leaks from abandoned streams. If your producer goroutine doesn’t select on ctx.Done(), it outlives the stream. Always pass the stream context into every goroutine that feeds the stream and select on it. A goroutine leak detector like goleak in tests will catch this.
stream.Send() is not goroutine-safe. The gRPC-Go docs are explicit: concurrent sends on the same stream cause a data race. The single-goroutine pattern from Pattern 2 is mandatory, not optional.
HTTP/2 multiplexing means one slow stream can starve others. If a single client stalls and fills the connection-level window, all streams on that connection are paused. Don’t let one misbehaving client affect others — use grpc.MaxConcurrentStreams and implement timeouts aggressively.
grpc.NewServer(
grpc.MaxConcurrentStreams(100), // per connection
)
Closed channel panic in producer/consumer. If the consumer returns early (client disconnect) while the producer is still writing to a channel, you get a panic on write to closed channel. Either use a mutex-protected flag, or close from the producer side only and let the consumer drain.
Context cancellation vs. stream error. stream.Send() returning an error does not always mean stream.Context().Err() is non-nil. Check both. Send() can fail because the transport died, which won’t immediately cancel the context.
Production-ready Docker Compose for testing locally
# docker-compose.yml
version: "3.9"
services:
grpc-server:
build: .
ports:
- "50051:50051"
environment:
- GRPC_GO_LOG_SEVERITY_LEVEL=info
- GRPC_GO_LOG_VERBOSITY_LEVEL=2
# Useful for observing TCP buffer behavior
sysctls:
- net.core.rmem_max=134217728
- net.core.wmem_max=134217728
- net.ipv4.tcp_rmem=4096 87380 67108864
- net.ipv4.tcp_wmem=4096 65536 67108864
# Simulate a slow client
slow-client:
build:
context: .
dockerfile: Dockerfile.client
environment:
- SERVER_ADDR=grpc-server:50051
- READ_DELAY_MS=500 # client reads one message every 500ms
depends_on:
- grpc-server
Test with tc netem to inject latency and observe how your backpressure patterns behave:
# On the server container, simulate 100ms latency on outbound traffic
tc qdisc add dev eth0 root netem delay 100ms
# Watch your backpressure kick in
docker stats grpc-server # watch memory stay flat
What actually matters in production
Pick one primary pattern and apply it consistently:
- Event-driven streams where loss is acceptable (metrics, logs, dashboard updates): bounded channel with drop policy (Pattern 3).
- Event-driven streams where loss is not acceptable (financial transactions, state sync): semaphore + single send worker (Pattern 2) with client-side reconnect logic.
- Controlled-rate feeds (bulk export, data pipelines): token bucket (Pattern 4).
- Interactive streams (bi-directional or client-paced): respect the HTTP/2 window directly and use per-send timeouts (Pattern 1).
Every streaming service needs three things in its monitoring dashboard: active stream count, messages sent/sec, and messages dropped/sec. If you have those three, you can reason about everything else. If dropped/sec is consistently non-zero, you either need to increase your buffer, reduce your send rate, or disconnect slow clients faster.
gRPC streaming is not fire-and-forget. The transport gives you backpressure for free at the HTTP/2 layer, but your application code will happily bypass it if you let it. The patterns here are just ways of making your application respect the pressure signals the transport is already sending.