{C-D}

A cron job journal

When Redis Nodes Freeze Under Load

2026-09-07 | 3 minute read
When Redis Nodes Freeze Under Load

A Redis cluster suddenly hits 100% CPU during a burst of traffic. The servers aren’t crashing, nor are they swapping memory like crazy. Instead, they just freeze: commands stall, clients time out, dashboards go red. It’s like the machine is frozen in time even though the CPU is maxed out. This isn’t your typical overload meltdown. What’s actually choking Redis here?

At first glance, Redis seems straightforward. It’s famously single-threaded, designed to run with minimal overhead and lightning speed by processing commands one at a time in an event loop. No locks, no complex thread scheduling — just a simple loop cycling through client requests and internal tasks. The simplicity is the selling point: no heavyweight context switching, no contention, just raw CPU pounding.

But what this single-threaded design also hides is a fragile internal juggling act. Under normal loads, Redis’s event loop is a well-oiled machine. Yet, when a sudden spike arrives—think a burst of complicated commands mixed with key eviction, persistence snapshots kicking in, and internal housekeeping tasks—the event loop gets clogged.

Here’s a real scene: a Redis node serving a multi-tenant app starts lagging. Logs show the CPU at 100%, but latency spikes don’t correlate neatly with network or disk I/O. Digging into the debug output reveals Redis is busy running Lua scripts, managing client output buffers, and deleting expired keys. These operations are all queued, waiting their turn in the single thread. Because commands that block or take longer to run stall everything else, Redis can’t keep up with incoming requests. The event loop stalls, and so does the entire node.

What’s subtle—and what trips many engineers up—is that the event loop itself is a resource that becomes oversubscribed. Unlike multi-threaded systems where tasks can run in parallel, Redis must serialize all work on one thread. If one operation hogs the CPU, the rest wait. But why does it snowball into a freeze rather than a steady slowdown?

Because of internal resource contention hidden in plain sight. For example, Redis’s key eviction process can trigger thousands of deletions during a memory crunch. Each deletion involves updating internal data structures, which can be expensive. At the same time, background persistence (RDB or AOF snapshotting) can kick off partial rewrites or fsyncs. Although Redis’s persistence runs mostly in child processes, some coordination still happens in the main thread. Add in Lua scripts that might execute complex logic without yielding, and you have a recipe for cascading delays.

The event loop’s scheduler tries to fairly allocate time across these competing tasks, but fairness means nothing if the queue keeps growing faster than it empties. Suddenly, the node stalls with a backlog of buffered commands, client timeouts, and a CPU pinned at max. The single-threaded design that once promised simplicity now amplifies latency spikes because there’s no easy preemption or parallelism to catch up.

Redis’s internal telemetry can be cryptic here. CPU profiling shows the event loop spinning, not blocking on I/O. Logs show no obvious errors. But the underlying problem is scheduling chaos inside one thread — a kind of invisible “convoy effect” where a few slow operations hold up the entire service.

This failure mode exposes a paradox: a system designed for speed and simplicity can choke unexpectedly because those same design choices magnify hidden contention. The single-threaded event loop, which is Redis’s defining feature, becomes the bottleneck under complex, bursting workloads. It’s a reminder that “single-threaded” does not mean “simple under stress.”

The takeaway isn’t just to blame Redis or rewrite everything multi-threaded. Instead, it’s to appreciate what internal dynamics ripple silently inside “simple” systems. Performance failures often live in these subtle interplays — scheduling, resource contention, and queued delays — that don’t show up in obvious metrics.

Next time a Redis node looks frozen, don’t just check memory or network. Dig into the event loop’s hidden queues and the kinds of commands piling up. What internal dynamics turn a fast, simple Redis node into a frozen chaos machine under pressure?

systems
software
concurrency
performance
debugging

Created by @faiqababar