Module P-12·38 min read

Cache-aside vs write-through vs write-behind, and the two failure modes that take the database down anyway: the stampede and the thundering herd.

JJS

Written by Jatin Jain Saraf · Senior Software Engineer

Module P-12 — Distributed Caching with Redis

Q: A homepage feed key is read 3,000 times a second at a 99.9% hit ratio; the query behind it takes about 400 ms. Every few hours the service has a two-minute outage in which the database saturates and the cache stays empty. TTLs across the codebase already have jitter applied. What is happening and what fixes it?

The hot key's single expiry causes a stampede: all in-flight requests miss simultaneously, so roughly 1,200 concurrent identical 400 ms queries hit an origin sized for 0.1% of read traffic, and the fills cannot complete because the origin they need is now saturated — so it does not self-heal. The fix is to collapse the fills: in-process single-flight plus a per-key lock, or probabilistic early expiry so the refresh happens while the value is still being served. — This is the module's seven-step stampede, including the self-reinforcing step: the recovery action requires the origin, and the origin is exactly what has been saturated, so waiting does not help. The module also makes the hit-ratio point — at 99.9% the origin is provisioned for a thousandth of read traffic, so losing the key is a 1000× event — and names single-flight and probabilistic early expiry as the collapse mechanisms. Option B is the genuine near-miss: jitter is a real and necessary technique from Module F-12, but it spreads a *fleet* of expiries and does nothing for one hot key, which has exactly one expiry time that all 3,000 req/s hit together. Option A mistakes a symptom for the cause; a faster query reduces but does not remove a 1,200-way concurrent pile-up. Option C describes a plausible different cause, but eviction and expiry produce the same stampede — the mitigation still has to be fill collapsing.

Q: At 09:00 every day, p99 latency across every endpoint — including ones with no relationship to each other — jumps for about 300 ms. Database metrics, application CPU, and Redis memory are all flat. A reporting job runs at 09:00 and reads a Redis set that has been accumulating members for eight months. What is the mechanism?

Redis executes commands on a single thread, so the O(n) `SMEMBERS` on a set that grew unbounded holds the execution loop for its entire duration and every other client — on every endpoint — waits behind it. The fix is to bound the collection and read it incrementally with `SSCAN` rather than in one command. — The module's central Redis point: command execution is serialised, so there is no such thing as a slow command that only affects its caller, and the signature is a service-wide p99 spike with no database or CPU signal — which is exactly why teams spend an hour looking in the wrong place. It also names this specific bug shape: the original code was fine at 100 elements, and the set quietly grew for months. `SLOWLOG GET` and `INFO commandstats` confirm it. Option B is the strongest near-miss, because a multi-megabyte reply genuinely does consume bandwidth and serialisation time — but that is a contributing cost, not the mechanism; the blocking happens in the single execution thread and would occur on an idle network. Option C invents a reconnection storm the scenario gives no evidence for. Option D misdescribes expiry, which is lazy plus an incremental background cycle deliberately bounded so it cannot monopolise the loop.

Q: A Redis instance used purely as a cache reaches its memory limit. Reads continue to succeed and the hit ratio dashboard looks normal, but over the following twenty minutes origin load climbs steadily and cache fills stop taking effect. `INFO` shows `evicted_keys` at zero. What is the most likely cause?

The eviction policy is not evicting: either the `noeviction` default, or a `volatile-*` policy on an instance where some code path writes keys without a TTL, leaving them un-evictable. At the limit Redis serves reads normally and returns errors on writes, so fills fail silently in a `catch` block while the miss rate climbs as TTLs expire with nothing replacing them. `evicted_keys` at zero is the confirming signal. — The module calls this the most common way a Redis instance behaves unexpectedly, and names both routes to it: `noeviction` is the default, and a `volatile-*` policy can only evict keys that carry an expiry, so a single `SET` without `EX` repeated enough times fills the instance with un-evictable data and produces identical behaviour. The symptom set is distinctive — reads fine, hit ratio initially normal, every fill failing quietly — and `evicted_keys` at zero on an instance at its limit is the giveaway. Option C is the real near-miss: a working set larger than the instance is a genuine failure mode with a similar rising-miss-rate shape, but it produces a large and climbing `evicted_keys` count, which the scenario explicitly rules out. Option D describes a real concern the module raises about `mem_fragmentation_ratio`, but fragmentation causes eviction or an OOM kill, not silent write refusal. Option A invents a failover the evidence does not support.

What this module covers: Module F-12 established what a cache is and how to size one. This module is about running a shared, networked cache and surviving it. The four read/write strategies and the specific data loss or latency each one buys, the Redis behaviours that page you — single-threaded command execution, maxmemory-policy, RDB versus AOF — cache stampede as the failure mode that turns a cache into an origin-killer, the hot key that sharding cannot fix, cluster mode's hash slots and multi-key restrictions, and the rule that a cache outage must degrade rather than fail.

Moving the Cache Off the Box Changes What Can Go Wrong

An in-process cache is a hash map. It cannot be unreachable, it has no serialisation cost, and its consistency question is trivial because there is one copy on one heap. It also cannot be shared, dies on deploy, and multiplies memory by your instance count.

A shared Redis instance fixes all three of those and introduces an entirely new set of problems, all of which follow from one fact: the cache is now a network service with its own capacity, its own availability, and its own failure modes. A cache lookup is no longer ~100 ns of DRAM (Module F-2) — it's a round trip, so roughly 0.5 ms in-datacentre plus serialisation, which is still 10–100× faster than the query it replaces but is no longer free. More importantly, it can now time out, refuse writes, evict your working set, stall on someone else's command, and go down entirely while your database is sized on the assumption that it never would.

That last point is the whole reason this module exists. Module F-12's closing observation — that caching a slow query converts a performance problem into an availability problem — is a statement about distributed caches specifically. A local cache cannot take your service down. A shared one can.

Four Strategies, Four Different Things You Agree to Lose

The strategy names get used loosely, so pin them to who does the work and when.

Cache-aside (lazy loading). The application owns the cache. On read: check cache, miss, query the origin, write the result back, return it. On write: write the origin, then delete (or update) the key.

async function getUser(id: string): Promise<User | null> {
  const cached = await redis.get(`user:v3:${id}`);
  if (cached !== null) return JSON.parse(cached);

  const row = await db.users.findById(id);
  if (row === null) {
    // Cache the negative result, briefly, or a missing-ID scan becomes
    // an unthrottled origin scan. Short TTL, because the row may appear.
    await redis.set(`user:v3:${id}`, EMPTY, 'EX', 30);
    return null;
  }
  await redis.set(`user:v3:${id}`, JSON.stringify(row), 'EX', 300);
  return row;
}

Cache-aside is the default and should be, because it is explicit, it tolerates cache unavailability trivially (a failed get is just a miss), and it caches only what is actually asked for. Its costs: every miss path is written by hand in every call site until you abstract it, the first request after any eviction pays full origin latency, and there is a window between the origin write and the cache delete where readers see stale data (Module P-13 is entirely about that window).

Read-through. Same read shape, but the cache library or a sidecar owns the fill. The application calls cache.get(key, loaderFn) and never sees the miss. Cleaner call sites and a natural place to hang single-flight logic. The cost is that the fill path is now inside a layer you did not write, so its timeout, retry, and error semantics are its opinions rather than yours — and a loader that throws inside a shared cache library often turns into a cached error or an unbounded retry.

Write-through. Every write goes to the cache and the origin synchronously, in the same request, before acknowledging. The cache is never stale for data written through it.

The cost is stated plainly: write-through doubles your write latency and doubles the number of things that must be up for a write to succeed. You have added the cache to the critical write path, so a Redis failover of two seconds is two seconds of failed writes — on a system where the cache was supposedly optional. It also caches data nobody has read yet, which for a write-heavy, read-sparse table means you are paying memory for keys that will expire unused.

Write-behind (write-back). Write to the cache, acknowledge the client, flush to the origin asynchronously in batches.

This is the fastest possible write and the one with a genuinely dangerous failure mode: write-behind loses acknowledged data when the cache process dies. You told the client the write succeeded; the only copy was in a memory buffer; the process was OOM-killed or failed over and the buffer went with it. Batching also means the origin is behind by up to a flush interval, so anything reading the origin directly — a report, a CDC pipeline (Module P-23), a second service — sees data that does not match what the API just confirmed. Write-behind is correct for high-volume, individually-worthless writes where you have decided out loud that losing the last few seconds is acceptable: view counters, last-seen timestamps, rate limit buckets (Module P-18). It is never correct for a payment (Module P-24).

Strategy	Read latency on miss	Write latency	Staleness	Failure mode you are buying
Cache-aside	Full origin cost	Origin only	Window between origin write and invalidation	Stampede on a hot miss; every miss path hand-written
Read-through	Full origin cost	Origin only	Same as cache-aside	Fill semantics owned by a library, not by you
Write-through	Full origin cost	Origin + cache	None for written keys	Cache is now on the write path; its downtime is your downtime
Write-behind	Full origin cost	Cache only (fastest)	Origin lags the cache	Acknowledged writes lost on crash; origin readers disagree with the API

The honest default for almost every system: cache-aside for reads, synchronous invalidation on write, and write-behind reserved for named counters where the loss is priced in.

Redis Executes One Command at a Time, So One Bad Command Stalls Everyone

Redis processes commands from a single thread. Networking is threaded in modern versions, but command execution is serialised, and that single design choice explains most Redis incidents.

The consequence: there is no such thing as a slow command that only affects the caller. Every command holds the execution loop for its whole duration, and everyone else waits. Redis does 100K+ simple operations per second precisely because each one is O(1) and takes microseconds. Introduce one command that takes 200 ms and you have just added up to 200 ms to the p99 of every other client — including the health check, including the session lookup on the login path, including services that have nothing to do with the offending key.

The commands that do this:

KEYS pattern — scans the entire keyspace. On a few million keys this is tens to hundreds of milliseconds of total blockage. Use SCAN, which returns a cursor and lets other commands interleave. KEYS in a running application is close to a guaranteed incident; it is usually introduced by an admin script or a "clear the cache for this tenant" feature.
SMEMBERS / HGETALL / LRANGE key 0 -1 on a collection that grew. These are fine at 100 elements and catastrophic at 2 million: O(n) to gather, plus a multi-megabyte reply to serialise and push through the socket. The bug is almost never the original code — it is that the set had no bound and quietly grew for eight months.
DEL on a huge collection. Freeing a million-element structure is O(n) work in the same thread. UNLINK hands the reclamation to a background thread; prefer it for anything large.
Lua scripts and FUNCTION calls, which are atomic because nothing else runs while they execute. That atomicity is genuinely useful (it is how you build a correct rate limiter or a compare-and-delete), and the cost is that a loop inside a script is a stop-the-world pause.
FLUSHALL synchronously, and anything that touches every key.

Analogy: the hot cache key is the printed timetable taped to the station wall. Two hundred commuters a minute read it and never trouble the station master. The moment a cleaner peels it off, those same two hundred people queue at his one window in thirty seconds asking the identical question, and he can answer three. The paper was never the load-bearing part — his throughput was, and the paper was hiding that from everyone, including him.

Why this matters in production: a Redis latency spike presents as a service-wide p99 spike with no corresponding database or CPU signal, which sends teams hunting in the wrong place for an hour. The diagnostic is SLOWLOG GET, plus INFO commandstats for the aggregate time per command type, plus the latencystats percentiles. Set slowlog-log-slower-than low enough to actually catch things (the default threshold hides the 5–10 ms commands that matter when your budget is sub-millisecond). And remember Module F-4: this is fan-out tail amplification with extra steps, because a request that makes eight Redis calls has eight chances to land behind someone else's HGETALL.

The design rules that follow: bound every collection, never write an unbounded pattern scan, keep values small enough that serialisation is not the cost (a 5 MB JSON blob is a bad cache value regardless of how fast the lookup was), and pipeline or MGET batches of small keys rather than issuing them serially — not for Redis's sake but for the round trips (Module F-5).

`maxmemory-policy` Decides Whether a Full Cache Evicts or Starts Refusing Writes

This is the single most common way a Redis instance behaves in a way nobody expected, because the default is not what a cache wants.

Redis needs two settings to behave like a cache: maxmemory (a byte limit) and maxmemory-policy (what to do at the limit). If maxmemory is unset, Redis grows until the kernel OOM-kills it. If maxmemory-policy is noeviction — the default — Redis at the limit does not evict anything. It starts returning errors on every write command while continuing to serve reads perfectly.

That is a uniquely nasty symptom set: reads fine, hit ratio looks normal, and every cache fill fails silently in a catch block somewhere. The cache stops accepting new entries, so the miss rate climbs as TTLs expire and nothing replaces them, and the origin load rises over minutes while the cache dashboard looks healthy.

Policy	Evicts	Use when
`noeviction`	Nothing; writes error at the limit	Redis is a datastore, not a cache, and losing a key is a correctness bug
`allkeys-lru`	Approximated LRU across all keys	General-purpose cache — the usual right answer
`allkeys-lfu`	Approximated LFU across all keys	Skewed access with a stable hot set; resists the sequential-flood problem from Module F-12
`volatile-lru` / `volatile-lfu` / `volatile-ttl`	Only keys that have a TTL set	Mixed instance: some keys are cache, some must persist
`allkeys-random` / `volatile-random`	Random victims	Rarely; cheaper than LRU sampling, much worse hit ratio

The trap in that table is the volatile-* family. A volatile-* policy can only evict keys that have an expiry, so if a code path writes keys without a TTL, those keys are un-evictable — and once they fill the instance, volatile-lru behaves exactly like noeviction: writes start failing even though gigabytes of the instance are occupied by keys nobody has read since last quarter. One forgotten SET without EX, repeated a few million times, is enough.

conf

# Cache instance: bound the memory and always evict rather than refuse writes.
maxmemory 12gb
maxmemory-policy allkeys-lfu
# LRU/LFU here are approximate: Redis samples N candidates and evicts the
# best of them. Raising this trades CPU for eviction accuracy.
maxmemory-samples 7
# Reclaim large structures off-thread so a big DEL cannot stall the loop.
lazyfree-lazy-eviction yes
lazyfree-lazy-expire yes

Two further details worth knowing before they surprise you. Eviction is sampled, not exact — Redis picks a handful of candidates and evicts the best of that sample, so it approximates LRU/LFU rather than implementing it. And expiry is lazy plus a background sweep: a key past its TTL is removed when touched, or when the active expiry cycle finds it. Memory does not free at the instant the TTL passes, which matters when you sized the instance assuming it would.

Also mind mem_fragmentation_ratio in INFO memory. The allocator's RSS can meaningfully exceed the logical data size on a churn-heavy workload, so an instance that looks 70% full by dataset size can be at its real memory ceiling.

Persistence Is Module F-2's `fsync` Trade in Different Clothing

Redis is in-memory, but it can write to disk, and the options are the same durability-versus-latency dial as any database WAL.

RDB takes a point-in-time snapshot by forking and writing the dataset out. Cheap during steady state, compact on disk, fast to load on restart. It loses everything written since the last snapshot. The fork also transiently increases memory via copy-on-write — on a write-heavy instance, the copied pages can approach a significant fraction of the dataset, which is how a snapshot triggers an OOM kill on an instance that was "only" 60% full.
AOF appends every write command to a log and replays it on restart. Durability is set by appendfsync: always calls fsync per write (slowest, smallest loss window), everysec flushes once a second (the usual choice, losing up to a second), no leaves it to the kernel (fastest, loses whatever the page cache held). This is exactly the trade from Module F-2 — durability costs you a synchronous device round trip, and everysec is the same "batch the fsync" compromise a database group-commit makes.

For a pure cache, the correct answer is frequently neither. Persistence on a cache buys faster warm-up after a restart and costs you fork pauses, disk I/O contention, and a class of memory incidents. If the cache is genuinely disposable, disable it and solve warm-up with pre-warming instead.

The exception is warm-up cost at scale. If a cold cache means your origin cannot survive the miss storm, then the persistence is not a durability feature — it is a stampede mitigation, and it should be reasoned about as one. Which is the real subject of this module.

Cache Stampede: One Expiry, N× Origin Load

Here is the failure mode. It is worth being able to draw from memory, because it takes down more services than cache misses ever do.

text

1. Key `homepage:feed` is hot: 3,000 req/s all read it. Hit ratio 99.9%.
   The origin query behind it takes 400ms and is never actually run.

2. The TTL expires. Or the key is evicted. Or a deploy bumped the key
   version. Or the instance failed over and came back empty.

3. Every in-flight request now misses. At 3,000 req/s with a 400ms
   origin query, Little's Law (F-4) says 1,200 concurrent requests are
   executing the identical expensive query at the same instant.

4. Postgres does 5K–15K simple indexed reads/s. It does not do 1,200
   concurrent 400ms aggregations. Connection pool exhausted (Module P-4).

5. Now the origin is slower, so the queries take 4s instead of 400ms,
   so *more* requests pile in behind them. Nobody finishes, so nobody
   repopulates the cache.

6. Clients time out and retry (A-6), adding load. The cache stays empty
   because every filler died before it could write.

7. Total outage — from a TTL expiring on one key.

Step 5 is what makes this self-reinforcing rather than a spike: the recovery action (fill the cache) requires the origin, and the origin is exactly what has been saturated. This is why a stampede tends not to resolve on its own, and why "just wait for it to fill" is not a plan.

Note what a high hit ratio has done here. Module F-12 argued that hit ratio's marginal value increases near the top — going 99% → 99.9% removes 90% of the remaining origin load. The other side of the same coin: at 99.9% your origin is provisioned for 0.1% of read traffic, so the cache going away is not a 0.1% problem, it is a 1000× problem. The higher the hit ratio, the more violent the stampede.

The fixes, and what each one costs

Per-key locking / single-flight. On a miss, one request wins a lock and does the fill; the others either wait briefly for it or serve stale. The origin sees exactly one query.

async function getWithSingleFlight<T>(
  key: string, ttl: number, load: () => Promise<T>,
): Promise<T> {
  const hit = await redis.get(key);
  if (hit) return JSON.parse(hit);

  // NX makes this the lock: exactly one caller gets 'OK'. The EX is
  // mandatory — a lock without a TTL is a permanent outage if the holder dies.
  const token = crypto.randomUUID();
  const won = await redis.set(`lock:${key}`, token, 'NX', 'PX', 5_000);

  if (!won) {
    // Loser: serve the stale copy if we kept one, else wait-and-retry once.
    const stale = await redis.get(`stale:${key}`);
    if (stale) return JSON.parse(stale);
    await sleep(50 + Math.random() * 50);        // jitter, or they resynchronise
    const filled = await redis.get(key);
    if (filled) return JSON.parse(filled);
    return load();                                // last resort: let it through
  }

  try {
    const fresh = await load();
    const body = JSON.stringify(fresh);
    await redis.set(key, body, 'EX', ttl);
    await redis.set(`stale:${key}`, body, 'EX', ttl * 4); // outlives the fresh copy
    return fresh;
  } finally {
    await releaseIfOwner(`lock:${key}`, token);    // compare-and-delete via Lua
  }
}

Costs, and they are real: this is a distributed lock, with everything Module A-2 says about them. If the holder dies mid-fill, everyone waits for the lock TTL. If the lock TTL is shorter than the fill, two fills run anyway. Losers either add latency (waiting) or add complexity (serving stale). And it is two extra round trips on the miss path. A cheaper variant that removes most of the distributed-lock risk: single-flight in-process first (one promise per key per Node instance — see below), so the fan-in is your instance count rather than your request count. 40 instances hitting the origin is survivable; 1,200 is not.

Probabilistic early expiry. Rather than every reader treating the key as valid until an instant and invalid after it, readers near expiry recompute with rising probability. The refresh happens while the cached value is still being served, so there is never a moment with no valid entry.

// Store the fill duration with the value. A reader refreshes early with a
// probability that grows as expiry approaches, scaled by how expensive the
// fill is — costly keys start protecting themselves sooner.
const shouldRefreshEarly =
  Date.now() - entry.deltaMs * BETA * Math.log(Math.random()) >= entry.expiresAtMs;

The cost is a small amount of redundant recomputation in steady state (you refresh before you strictly had to) and a tuning parameter nobody will remember the meaning of in six months. It composes well with locking and is the best single defence for a genuinely hot key.

Stale-while-revalidate in the app tier. Module F-12 and F-15 covered this at the HTTP layer; the same pattern belongs inside the service. Keep two horizons per entry — a fresh deadline and a usable deadline. Past fresh, serve the cached value immediately and refresh in the background. The user never waits for a fill, and the origin sees one refresh per key per interval regardless of read rate. The cost is explicit: you have chosen to serve data you know is stale, so this is unavailable for anything where staleness is a correctness problem (balances, permissions, inventory at checkout), and it needs a bound so a persistently failing origin cannot serve week-old data forever. Note also that a background refresh violates Module F-10's rule about no work after the response returns — on serverless it may simply never run.

Pre-warming. Populate the cache before it takes traffic: a job that fills the top-N keys after a deploy or failover, or a new instance that refuses health checks until warm (a legitimate use of a deep health check, unlike the pattern Module F-11 warns about). This is what makes replacing a cache cluster survivable. The cost is that you must know your top keys — and pre-warming the wrong set is a large amount of origin load for no benefit.

Do not fix a stampede with TTL jitter alone. Jitter (Module F-12) is correct and necessary — it stops ten thousand keys written in the same deploy from expiring in the same second. But jitter spreads a fleet of expiries; it does nothing for a single hot key, because that key has one expiry time and all 3,000 req/s hit it at once. The two problems look alike on a graph and have different fixes.

The Hot Key Is a Problem Sharding Cannot Solve

Add nodes to a Redis cluster and total capacity rises. The capacity for product:12345 does not: that key lives in one hash slot on one node, and no amount of horizontal scaling gives it a second home. When a celebrity post, a flash sale item, or a global feature flag is read 200,000 times a second, one node absorbs all of it while its peers idle. You will see it as a single node at high CPU with the cluster at 15% overall.

The fixes:

Two-tier caching. An in-process L1 (a bounded LRU on the Node heap, TTL measured in single-digit seconds) in front of the shared Redis L2. The L1 absorbs the hot key entirely — a heap hit is ~100 ns and costs no round trip and no Redis CPU — and the L2 continues to serve the long tail.

// L1: tiny, per-instance, deliberately short-lived. The TTL is the
// staleness contract — this value can be up to 3s behind Redis.
const l1 = new LRUCache<string, unknown>({ max: 10_000, ttl: 3_000 });
const inFlight = new Map<string, Promise<unknown>>();

async function get<T>(key: string, load: () => Promise<T>): Promise<T> {
  const local = l1.get(key) as T | undefined;
  if (local !== undefined) return local;

  // Per-process single-flight: 500 concurrent requests for the same key
  // produce one Redis call, not 500.
  let p = inFlight.get(key) as Promise<T> | undefined;
  if (!p) {
    p = fetchFromRedisOrOrigin(key, load).finally(() => inFlight.delete(key));
    inFlight.set(key, p);
  }
  const value = await p;
  l1.set(key, value);
  return value;
}

The cost is stated by the TTL: every instance now holds an independently stale copy, and the staleness window is the L1 TTL. Invalidation becomes genuinely hard, because a delete in Redis does not reach 40 heaps — you need a pub/sub invalidation broadcast (which is best-effort and lost during a partition) or the acceptance that L1 entries live out their TTL. Keep the L1 TTL short and only for data where seconds of skew are harmless. Module P-13 takes this apart properly.

Key splitting. Write N copies — flag:dark_mode:{0..15} — and have each reader pick one at random. Load divides across slots. The cost is that a write now has to update sixteen keys, non-atomically, so readers can disagree with each other during the update. Reasonable for near-static config, wrong for anything that changes often.

Read replicas for the hot slot. Redis replicas can serve reads (READONLY in cluster mode). This adds read capacity for a hot key at the cost of asynchronous replication lag — a read-your-own-writes violation (Module P-11) on top of a cache that was already eventually consistent.

Cluster Mode Turns Multi-Key Operations Into a Data Layout Problem

Redis Cluster shards the keyspace into 16,384 hash slots, assigns slot ranges to nodes, and routes by CRC16(key) mod 16384. It is the consistent-hashing idea from Module P-7 with a fixed virtual-node count: resharding moves slots rather than rehashing everything, and clients cache the slot map and follow MOVED / ASK redirects when it changes.

The constraint that reshapes your code: a multi-key command must have all its keys in one slot. MGET a b c, SINTER, a transaction, a Lua script touching several keys — if the keys hash to different slots you get CROSSSLOT Keys in request don't hash to the same slot. This is not a configuration you can turn off; it exists because there is no cross-node atomicity.

Hash tags give you control. Only the substring inside {...} is hashed, so keys sharing a tag share a slot:

text

user:1001:profile        -> slot A     }  three round trips,
user:1001:prefs          -> slot B     }  no MGET, no atomic
user:1001:sessions       -> slot C     }  multi-key script

user:{1001}:profile      -> same slot  }  one MGET, one script,
user:{1001}:prefs        -> same slot  }  one round trip
user:{1001}:sessions     -> same slot  }

The cost is direct and frequently missed: a hash tag is a deliberate co-location, so it is also a deliberate hot spot. Tag by tenant and your largest tenant's entire working set lands on one node — Module P-8's noisy-neighbour problem, reproduced in the cache tier. Tag by something with low cardinality and you have manufactured an unbalanced cluster you cannot rebalance, because slots move but a tag's keys always hash together. Tag at the granularity you actually need atomicity or batching at, and no wider.

Two more cluster realities worth carrying: pipelining across slots requires a client that splits the batch by node and reassembles (good clients do; verify yours), and SCAN is per-node, so any keyspace sweep must iterate every master. And failover is not instant — during the failure-detection and promotion window, the slots on the affected node are unavailable, which for your application means a burst of errors on a subset of keys. That is not a scenario most cache code has been tested against.

A Cache Outage Must Degrade, Not Fail

This is the rule that everything above serves. At a 99% hit ratio your origin is sized for 1% of read traffic. If the cache disappearing means the origin receives 100× its provisioned load, then you do not have a cache with a database behind it — you have a database with a single point of failure in front of it.

What degrading actually requires:

Every cache call is wrapped in a short timeout and treated as a miss on failure. A Redis timeout must never propagate as a 500. And the timeout must be tight: a 2-second cache timeout in front of a 50-millisecond query means the cache being slow is worse than the cache being absent. Single-digit to low tens of milliseconds.
A circuit breaker in front of the cache (Module A-7). Once Redis is failing, stop calling it — otherwise every request pays the timeout before doing the work it was always going to do, and the added latency exhausts your own pool by the mechanism in Module F-13.
Load shedding on the origin, not just retries (Module A-8). If the origin cannot serve 100× traffic — and it cannot — then the correct behaviour is to serve a subset of users correctly and reject the rest quickly, rather than serve everyone slowly and nobody successfully. Concurrency limits on the fill path are the concrete form of this.
A degraded response defined in advance. The generic feed instead of the personalised one. Cached-at-the-CDN HTML with stale-if-error (Module F-15). A read-only mode. The feature that is expensive to compute switched off by a flag. All of these are product decisions that must be made before the incident, because they cannot be invented during it.
The failure drilled deliberately (Module O-6). Killing the cache in staging and watching what the origin does is the only way to know whether any of the above works. This is close to the highest-value chaos experiment available, because the cache is the dependency most likely to be treated as optional in the code and critical in reality.

Why this matters in production: the graceful-degradation path is the one that has never been exercised. Cache-miss handling gets written on day one when the origin was fast and the traffic was small, then the hit ratio climbs to 99.5% over two years and nobody re-checks the arithmetic. The question to ask in review is not "is the cache reliable" — it is "what is our request rate against the origin if the hit ratio goes to zero for five minutes, and does that number fit."

Where This Shows Up in Your Stack

Redis: set maxmemory and maxmemory-policy explicitly on every instance — the noeviction default silently converts a full cache into one that refuses writes. Watch INFO for evicted_keys, keyspace_misses, blocked_clients, mem_fragmentation_ratio, and latencystats; keep SLOWLOG threshold low; ban KEYS and unbounded LRANGE/SMEMBERS in code review; prefer UNLINK over DEL for large structures.
Node.js: an in-process Map<string, Promise<T>> is the cheapest single-flight you will ever deploy and removes most stampede risk before any distributed lock is involved. Beware that ioredis/node-redis clients pipeline on one connection per node, so a slow command blocks your own subsequent commands too — cache calls made while holding a Postgres connection tie two pools together.
PostgreSQL: the origin's max_connections (default 100) and pool size are the real ceiling a stampede hits, not CPU. pg_stat_statements after an incident will show the stampeding query with a call count that jumped by three orders of magnitude in one minute — a useful confirmation signal.
CDN / reverse proxy: origin shield and request collapsing (Module F-15) are the same single-flight idea one layer up, and the cheapest place to solve a stampede for anything cacheable by URL. Solve it there first; the application-tier lock is for what the CDN cannot cache.

Summary

A distributed cache is a network service with its own capacity, availability, and failure modes — the cost of moving the cache off the box is that it can now be unreachable, refuse writes, evict your working set, and stall on another client's command.
Cache-aside is the right default because a failed cache call is just a miss; its costs are hand-written fill paths and an invalidation window (Module P-13).
Write-through doubles write latency and puts the cache on the write path, so the cache's downtime becomes your downtime. Write-behind loses acknowledged writes when the process dies — acceptable only for data whose loss you have priced in.
Redis executes commands one at a time, so one KEYS, one SMEMBERS on a set that grew, or one Lua loop adds its full duration to every other client's p99. Bound every collection; use SCAN and UNLINK; keep SLOWLOG honest.
maxmemory-policy decides whether a full cache evicts or errors. The noeviction default refuses writes while reads look healthy; volatile-* can only evict keys with a TTL, so one code path writing without EX recreates the same silent failure.
RDB versus AOF appendfsync is Module F-2's fsync trade, plus fork copy-on-write memory. For a disposable cache, no persistence plus pre-warming is often the better answer.
Cache stampede is the centrepiece failure: a hot key expires, N concurrent requests all miss, the origin takes N× load, and the fill cannot complete because the origin is saturated — so it does not self-heal. Fix with single-flight (costs a distributed lock's failure modes), probabilistic early expiry (costs some redundant recomputation), stale-while-revalidate (costs knowingly serving stale data), and pre-warming (costs knowing your top keys). TTL jitter fixes fleet-wide expiry, not one hot key.
A hot key cannot be sharded away. Two-tier caching moves it to the process heap at the cost of one independently stale copy per instance; key splitting costs write atomicity; replicas cost replication lag.
Cluster mode's 16,384 hash slots (Module P-7) confine multi-key commands to a single slot. Hash tags fix that, and a hash tag is a deliberate hot spot — tag no wider than the atomicity you need.
A cache outage must degrade, not fail: tight timeouts, a circuit breaker, load shedding on the origin, a degraded response chosen in advance, and the whole path drilled before it is needed.

The next module, Cache Invalidation, takes the one problem this module kept deferring: the window between the origin write and the cache delete, why write-then-delete and delete-then-write both have a losing interleaving, and what versioned keys, TTL-only designs, and CDC-driven invalidation each cost to close it.

Knowledge Check

A homepage feed key is read 3,000 times a second at a 99.9% hit ratio; the query behind it takes about 400 ms. Every few hours the service has a two-minute outage in which the database saturates and the cache stays empty. TTLs across the codebase already have jitter applied. What is happening and what fixes it?

At 09:00 every day, p99 latency across every endpoint — including ones with no relationship to each other — jumps for about 300 ms. Database metrics, application CPU, and Redis memory are all flat. A reporting job runs at 09:00 and reads a Redis set that has been accumulating members for eight months. What is the mechanism?

A Redis instance used purely as a cache reaches its memory limit. Reads continue to succeed and the hit ratio dashboard looks normal, but over the following twenty minutes origin load climbs steadily and cache fills stop taking effect. INFO shows evicted_keys at zero. What is the most likely cause?

Test your knowledge with more question sets

Sign in to keep reading

The rest of this module is free — sign in with Google to unlock it and track your progress.

PreviousModule P-11: Consistency Models Next Module P-13: Cache Invalidation

Discussion

Join the discussion

Loading comments...