Module F-12·22 min read

The cache layers from browser to CDN to app to database, hit ratio as the only number that matters, TTL choice, and sizing a cache to the working set instead of the dataset.

JJS
Written by Jatin Jain Saraf · Senior Software Engineer

Module F-12 — Caching Fundamentals

What this module covers: Caching is the highest-leverage technique in system design and the one with the most easily-forgotten cost. This module covers the full cache hierarchy from browser to disk, why hit ratio is the only number that matters and why the last percentage point is worth the most, sizing to the working set instead of the dataset, choosing a TTL as an explicit staleness budget, eviction policies and how a single analytics scan can destroy a cache, cache key design and the key-explosion failure that silently drops your hit rate to zero, and the cases where caching is the wrong answer. It closes with the four metrics that tell you whether a cache is working.


Caching Trades Staleness for Latency and Cost

That's the whole deal, and it deserves to be stated as a trade rather than an improvement. You get faster reads and less load on the origin; you pay by serving data that may be out of date, plus the memory it occupies and the invalidation logic you now maintain.

The important consequence: "how stale can this be?" is a product question, not an infrastructure question. Module F-1 listed consistency tolerance as one of the four numbers that change every answer, and a cache is where that number gets spent. If nobody has said it out loud, you've made a product decision inside an infrastructure change — and the bug report ("I changed my name and it still shows the old one") will arrive as a mystery instead of an expected consequence.

Analogy: a cache is a photocopy on your desk instead of the original in the archive. Enormously faster to consult, and the entire risk is that someone edited the original since you copied it. Every question about caching reduces to: how out of date can this photocopy be before it does damage?

Why this matters in production: most cache incidents aren't "the cache was slow." They're "the cache served something wrong for longer than anyone expected," or "the cache disappeared and the origin couldn't take the load it had been shielded from." Both are consequences of the trade being made implicitly.


The Hierarchy: Every Layer Is a Cache

A single read can be answered at any of these, and each hit means every layer below it does nothing at all:

LayerTypical latencyScopeWho controls it
Browser memory / HTTP cache~0 msOne userCache-Control headers
CDN / edge5–50 msAll users near a PoPHeaders + purge API
Reverse proxy (nginx, Varnish)1–5 msAll usersProxy config
Application in-process~0 msOne instanceYour code
Distributed cache (Redis)0.5–2 msAll instancesYour code
Database buffer pool~0 ms (RAM hit)All queriesshared_buffers
OS page cache~0 msAll file readsThe kernel
Disk50 µs – 8 ms

Two principles fall out of the table:

Cache as far out as you can tolerate. A CDN hit never touches your infrastructure — no compute, no database, no egress from your origin. An in-process hit is faster than Redis but is per-instance (Module F-9: N instances means N caches, N× memory, lower combined hit rate, and instances that can disagree). Pushing a cache outward multiplies its value and reduces your control, which is exactly the tension.

You already have caches you didn't configure. Postgres's shared_buffers and the OS page cache are doing enormous work invisibly. This is why a query is 2ms on the second run and 200ms on the first, and why a database restart produces a slow period that looks like a regression. Before adding a cache layer, check whether the existing ones are simply undersized.


Hit Ratio Is the Only Number

Effective latency is a weighted average:

effective = (hit_ratio × cache_latency) + (1 − hit_ratio) × origin_latency

With a 1ms cache and a 50ms origin:

Hit ratioEffective latencyRequests reaching origin
0%50.0 ms100%
50%25.5 ms50%
80%10.8 ms20%
90%5.9 ms10%
95%3.5 ms5%
99%1.5 ms1%
99.9%1.05 ms0.1%

Now the non-obvious part, and it's the most useful idea in the module: the marginal value of hit ratio increases as hit ratio increases.

  • Going 50% → 80% removes 60% of origin load.
  • Going 90% → 95% removes half the remaining origin load.
  • Going 98% → 99% removes half again.

Each additional percentage point near the top halves the traffic your database sees. So a cache at 90% is not "nearly optimal" — the work to reach 98% would reduce database load five-fold. This inverts the usual diminishing-returns intuition, and it's why tuning an already-good cache is often better value than optimising the origin.

The mirror image is the risk: at 99% hit ratio, your origin is sized for 1% of traffic. Lose the cache — an eviction wave, a restart, a failover, a bad deploy that changes key formats — and the origin instantly receives 100× its normal load. This is why the cache is a critical dependency and not an optimisation, and it's the entire subject of Module P-12's stampede section.


Sizing: Working Set, Not Dataset

You don't need to cache everything; you need to cache what's actually being read (Module F-3's fourth calculation). Access in real systems is heavily skewed — a small fraction of items absorbs most requests.

The right approach is to measure rather than assume:

sql

If the top 1,000 items serve 85% of reads, caching 1,000 items buys an 85% hit ratio for a trivial amount of memory. That's the number to design around — and it also tells you when caching won't help, because a flat access distribution (every item read about equally, rarely) has no working set to exploit.

Sign in to keep reading

The rest of this module is free — sign in with Google to unlock it and track your progress.

Sign in & Register

Discussion

0

Join the discussion

Loading comments...

© 2026 Jatin Jain Saraf (JJS). All rights reserved.