Module A-3·32 min read

Request Memoization mechanics, Data Cache tag registry and purge API, use cache directive internals, cacheLife profiles, cacheTag/updateTag, use cache: private and remote, custom cacheHandlers, and cache stampede prevention.

JJS

Written by Jatin Jain Saraf · Senior Software Engineer

A-3 — The Caching Architecture: Deep Internals

Q: In Next.js, how does Request Memoization handle deduplication for `fetch` calls?

It uses a per-request in-memory Map scoped to a single React render tree, keying entries by URL and init options. — Request Memoization patches the global `fetch` and uses an in-memory Map that is created when a render begins and cleared when the response is sent. It is scoped strictly per-request to a single React render tree and does not persist across different requests or automatically handle database queries.

Q: What is the primary cause of a "cache stampede" (thundering herd) in Next.js, and how can it be mitigated using `cacheLife`?

It happens when a popular cached entry expires and multiple concurrent requests try to revalidate simultaneously. It can be mitigated by configuring a `stale` window in `cacheLife` so stale responses are served while one background revalidation occurs. — A cache stampede happens when many requests concurrently hit an expired cache entry, causing multiple identical expensive database queries to run at once. Using `cacheLife` with a `stale` window allows the application to keep serving the stale response while a single background revalidation takes place, preventing the database from being overwhelmed.

Q: How does Next.js derive the cache key for a function marked with the `'use cache'` directive?

It uses a combination of the function's module path, export name, and the serialized arguments passed to it. — During static analysis, Next.js determines the cache key for a `'use cache'` function based on the function's code address (module path + export name) and the serialized arguments passed during the call. This is why using closure variables directly inside the function without passing them as arguments can lead to caching bugs.

Who this is for: Architects who've used the four caching layers from P-7 and need to understand how they actually work internally — the tag registry mechanics, cache stampede and its prevention, how use cache derives its keys, custom cache handlers for distributed deployments, and the failure modes that only appear at scale.

Request Memoization Internals

Request Memoization is implemented inside React's fetch patch. When Next.js starts a request, it patches the global fetch with a version that checks an in-memory Map before making a network call. The map key is a hash of the URL and the init options object.

The Map lives on the React async context — it's scoped to a single React render tree. It's created when rendering begins and cleared when the response is sent. Two components calling fetch('https://api.example.com/user/123') in the same render share the same map entry.

What Request Memoization does not deduplicate:

Database calls through Prisma or any ORM (they don't go through fetch)
fetch calls with different init options (different headers, different methods)
fetch calls across different requests (the Map is per-request, not persistent)

For database queries, React.cache() wraps a function in its own per-request memoisation map. The mechanics are the same — in-memory Map, per-request scope, keyed by the function reference and serialised arguments.

Data Cache Tag Registry

The Data Cache stores fetch results and use cache results keyed by a cache key. Every entry can have zero or more tags associated with it. Tags are how you group cache entries for batch invalidation.

The tag registry is a separate data structure: a Map from tag string to Set of cache keys that carry that tag. When you call revalidateTag('products'), Next.js:

Looks up 'products' in the tag registry → gets the Set of cache keys
Marks each of those cache keys as stale in the Data Cache
The next request for any of those cache keys re-fetches from origin and stores the fresh result

The registry and cache are stored on disk in production (.next/cache/fetch-cache by default) so they survive serverless function restarts. In development, they're in memory only.

The tag registry with multiple caches:

In a multi-instance deployment (multiple Node.js servers), each server has its own Data Cache on its own filesystem. Calling revalidateTag('products') on server A invalidates the cache on server A — but server B still has the stale entry. If the next request for that data lands on server B, it serves stale.

This is the distributed cache invalidation problem. The solution is a custom cache handler backed by shared storage (Redis, Upstash).

Custom Cache Handler for Distributed Deployments

// lib/cache-handler.ts
import Redis from 'ioredis';

const redis = new Redis(process.env.REDIS_URL!);

// The cache handler interface Next.js expects
export default class CacheHandler {
  async get(key: string) {
    const data = await redis.get(key);
    if (!data) return null;
    return JSON.parse(data);
  }

  async set(key: string, data: object, { revalidate }: { revalidate?: number | false }) {
    if (revalidate === false) {
      await redis.set(key, JSON.stringify(data));
    } else if (typeof revalidate === 'number') {
      await redis.setex(key, revalidate, JSON.stringify(data));
    }
  }

  async revalidateTag(tag: string) {
    // Find all keys associated with this tag and delete them
    const keys = await redis.smembers(`tag:${tag}`);
    if (keys.length > 0) {
      await redis.del(...keys);
      await redis.del(`tag:${tag}`);
    }
  }
}

// next.config.ts
const config: NextConfig = {
  cacheHandler: require.resolve('./lib/cache-handler'),
  cacheMaxMemorySize: 0, // disable in-memory cache when using Redis
};

With a Redis-backed cache handler, all server instances share the same cache. revalidateTag('products') on any server invalidates the cache for all servers. ISR works correctly in a multi-instance deployment.

When you need a custom cache handler:

More than one server instance (horizontal scaling, multiple regions)
Kubernetes deployments with rolling updates (pods have independent filesystems)
Any deployment where filesystem state can't be shared

Vercel handles this automatically — their infrastructure uses a shared cache backend. Self-hosted single-instance deployments with a persistent filesystem (a single Docker container) work fine with the default file-based cache.

Cache Stampede — The Production Failure Mode

Cache stampede (also called thundering herd) is what happens when a popular cached entry expires and many concurrent requests all try to revalidate at the same time.

Timeline:

text

t=0     Products page cached with revalidate=300 (5 minutes)
t=300   Cache expires. 500 concurrent users hit the page simultaneously.
t=300   All 500 requests: cache MISS. All 500 start a background revalidation.
t=300   500 concurrent database queries fire for the same products list.
t=300   Database CPU spikes to 100%. Query times go from 5ms to 800ms.
t=301   Database returns errors. 500 requests serve stale/error responses.

Prevention strategies:

Staggered TTLs — add random jitter to revalidation times so not all entries expire simultaneously:

// Instead of a fixed revalidate time
const revalidate = 300;

// Add jitter: 5 minutes ± 30 seconds
const revalidate = 300 + Math.floor(Math.random() * 60) - 30;

cacheLife with separate stale and revalidate windows:

async function getProducts() {
  'use cache';
  cacheLife({
    stale: 60,      // serve stale for up to 60 seconds
    revalidate: 300, // attempt revalidation every 5 minutes
    expire: 3600,   // hard expiry at 1 hour
  });
  return db.products.findMany();
}

The stale window means requests that arrive while revalidation is in progress still get the cached result — they don't pile up waiting for the revalidation to complete.

Prisma Accelerate and Upstash have stampede protection built in — they use a single-flight pattern where only one revalidation runs at a time and other requests wait for it.

`use cache` Static Analysis and Key Derivation

When Next.js encounters a 'use cache' directive, it does static analysis at build time to determine what constitutes the cache key for that function.

The key is derived from:

The function's module path + export name (the "address" in the codebase)
The serialised arguments passed to the function

// lib/queries.ts
async function getProduct(id: string) {
  'use cache';
  return db.products.findUnique({ where: { id } });
}

// Call 1: getProduct('abc123') → key = hash('lib/queries:getProduct:["abc123"]')
// Call 2: getProduct('xyz789') → key = hash('lib/queries:getProduct:["xyz789"]')
// Call 3: getProduct('abc123') → hits the same cache entry as Call 1

The closure variable trap:

// ❌ This creates a bug — the category from closure is NOT part of the key
const category = 'shoes';

async function getProducts() {
  'use cache';
  return db.products.findMany({ where: { category } }); // uses closure variable
}
// All categories produce the same cache key! First call wins.

// ✅ Make the input an explicit argument
async function getProductsByCategory(category: string) {
  'use cache';
  return db.products.findMany({ where: { category } });
}
// Different category = different argument = different cache key

Next.js 15 shows a build warning when it detects that a use cache function has closure variables that could affect its output. Treat these warnings as errors.

`use cache: 'remote'` and Private Caching

Two scope modifiers control where the use cache result is stored:

// Default — stored in the server-side Data Cache
async function getProducts() {
  'use cache';
  return db.products.findMany();
}

// 'remote' — stored in a remote cache (Vercel Data Cache, or your custom handler)
async function getExpensiveComputation(input: string) {
  'use cache: remote';
  cacheLife('days');
  return heavyComputation(input);
}

'use cache: remote' is for entries that should survive server restarts and be shared across instances — more persistent than the default in-process/filesystem cache.

For sensitive data that should never be shared between users:

// 'use cache: private' — per-user cache, never shared
async function getUserDashboard(userId: string) {
  'use cache: private';
  cacheTag(`user-${userId}`);
  cacheLife('minutes');
  return db.users.findUnique({ where: { id: userId }, include: { ... } });
}

Private cache entries are keyed on the combination of the function arguments AND the current user's session — they're never served to a different user. Use this for personalised data that still benefits from caching within a session.

Full Route Cache on Disk

For static routes, Next.js stores rendered output in two formats:

.next/server/app/[route]/page.html — the full HTML document
.next/server/app/[route]/page.rsc — the React Flight payload (for App Router navigations)

Both files are created at build time for ○ routes. For ISR routes, they're updated during background revalidation. The RSC file is what the client receives during App Router navigations (not the HTML) — this is why the browser's network tab shows text/x-component responses during navigation rather than text/html.

When revalidatePath('/blog') fires, Next.js deletes the .html and .rsc files for /blog and schedules a re-render on the next request. On Vercel, this deletion propagates to all CDN edge nodes globally. On self-hosted, it only affects the local filesystem.

Router Cache Layout in Memory

The Router Cache is a WeakRef-based Map stored in the React Router's internal state on the client. It's keyed by pathname + serialised searchParams.

Each entry stores:

The RSC payload (the Flight payload for that route)
A createdAt timestamp
The entry type: static (from generateStaticParams) or dynamic

The staleTimes configuration maps to two values per entry:

staleTimes: {
  dynamic: 30,   // dynamic routes: stale after 30 seconds
  static: 300,   // static routes: stale after 5 minutes
}

When you call router.refresh(), it deletes the current route's entry from the Router Cache Map, forcing the next navigation to that route to fetch a fresh RSC payload. It does not affect other routes.

The Router Cache is completely client-side — it lives in browser memory and is reset on full page reload. This is why router.refresh() is the correct answer for "I just mutated data and the page isn't showing the update."

Where We Go From Here

A-4 covers Partial Prerendering — the architecture that combines the static shell concept from the Full Route Cache with dynamic streaming from the server, serving the best of both rendering models in a single request. With the caching internals from A-3, you'll understand exactly why PPR works and what its limits are.

Self-Hosted Cache Handler with Redis

On Vercel, the Data Cache is managed by Vercel's infrastructure — shared across all instances, propagated globally on revalidation, persisted between deploys. You don't think about it.

On self-hosted Next.js — whether on a single Node.js server, Docker Compose, or Kubernetes — the Data Cache is stored on the local filesystem at .next/cache/fetch-cache/. This works correctly for a single instance. It breaks silently for multiple instances.

The failure mode: You have two pods (Pod A and Pod B) behind a load balancer. A user hits Pod A, which warms the cache for /api/products. Another user hits Pod B — cache miss, fetches from the database. A product update fires revalidatePath('/products') on Pod A. Pod A's cache is invalidated and re-populated on the next request. Pod B's cache is untouched — it still serves the old data. Users get inconsistent results based on which pod answers their request. You debug this for hours before realising the cache is pod-local.

Configuring a Shared Redis Cache Handler

// next.config.ts
import type { NextConfig } from 'next'

const config: NextConfig = {
  cacheHandler: require.resolve('./lib/cache-handler.js'),
  // Disable in-memory cache on top of the handler
  // Without this, each pod still has an in-memory layer that diverges
  cacheMaxMemorySize: 0,
}

export default config

The cache handler module must implement the CacheHandler interface:

// lib/cache-handler.ts
import { createClient } from 'redis'

const client = createClient({
  url: process.env.REDIS_URL,
  socket: { reconnectStrategy: (retries) => Math.min(retries * 50, 2000) },
})

client.connect()

// Next.js CacheHandler interface
export default class RedisCacheHandler {
  async get(key: string) {
    const data = await client.get(key)
    if (!data) return null
    
    const entry = JSON.parse(data)
    // Check TTL — Next.js expects null for expired entries
    if (entry.expiresAt && Date.now() > entry.expiresAt) {
      await client.del(key)
      return null
    }
    
    return entry
  }

  async set(key: string, data: unknown, ctx: { revalidate?: number | false }) {
    const revalidate = ctx.revalidate
    
    const entry = {
      value: data,
      lastModified: Date.now(),
      expiresAt: revalidate ? Date.now() + revalidate * 1000 : null,
    }
    
    if (revalidate) {
      // Set with TTL — Redis evicts automatically
      await client.set(key, JSON.stringify(entry), { EX: revalidate + 60 }) // +60s buffer
    } else {
      await client.set(key, JSON.stringify(entry))
    }
  }

  async revalidateTag(tags: string | string[]) {
    const tagArray = Array.isArray(tags) ? tags : [tags]
    
    for (const tag of tagArray) {
      // Scan for all cache keys associated with this tag
      // Tags are stored as a Redis Set: tag → Set<cacheKey>
      const keys = await client.sMembers(`tag:${tag}`)
      if (keys.length > 0) {
        await client.del(...keys)
        await client.del(`tag:${tag}`)
      }
    }
  }
}

Production note: This is a simplified implementation. The community-maintained @neshca/cache-handler package provides a battle-tested Redis implementation with proper tag indexing, LRU eviction, and serialisation. Use it rather than rolling your own:
bash
npm install @neshca/cache-handler @neshca/json-replacer-reviver ioredis

Tag-to-Key Index

The most non-obvious part of a custom cache handler: revalidateTag('products') needs to know which cache keys are tagged with 'products'. Next.js calls revalidateTag with the tag string — the handler must maintain a reverse index from tag to cache keys.

Every time set() is called with tagged data, store the tag→key mapping:

async set(key: string, data: unknown, ctx: { revalidate?: number; tags?: string[] }) {
  // ... store the entry ...
  
  // Maintain tag index
  if (ctx.tags?.length) {
    for (const tag of ctx.tags) {
      await client.sAdd(`tag:${tag}`, key)
      // Expire the tag index when the cache entry expires
      if (ctx.revalidate) {
        await client.expire(`tag:${tag}`, ctx.revalidate + 60)
      }
    }
  }
}

Without this index, revalidateTag becomes a full cache scan — O(n) across all cache keys. On a large cache, this is too slow.

The Rolling Deploy Cache Stampede

One failure mode specific to self-hosted with a shared cache: rolling deploys.

During a rolling deploy, new pods come up while old pods are still serving traffic. The new pods have a new build ID. Next.js includes the build ID in cache keys. The new pods see 100% cache misses — every request they handle is a cold cache fetch. If your origin database can't handle the burst, it falls over.

Mitigation:

Set cacheMaxMemorySize to a non-zero value — the in-memory cache warms from the Redis cache, reducing the cold-start database load
Pre-warm the cache by making requests to critical routes in your readiness probe before the pod starts receiving traffic
Use canary deployments — bring up one new pod, let it warm, then roll the rest

yaml

# k8s readiness probe — don't receive traffic until critical routes are warm
readinessProbe:
  httpGet:
    path: /api/health?warm=true  # health handler pre-fetches critical data
    port: 3000
  initialDelaySeconds: 15
  periodSeconds: 5
  failureThreshold: 3

The health endpoint:

// app/api/health/route.ts
export async function GET(req: NextRequest) {
  const url = new URL(req.url)
  
  if (url.searchParams.get('warm') === 'true') {
    // Pre-warm critical cached routes
    await Promise.all([
      fetch(`${process.env.NEXT_PUBLIC_BASE_URL}/api/products`),
      fetch(`${process.env.NEXT_PUBLIC_BASE_URL}/api/categories`),
    ])
  }
  
  return Response.json({ status: 'ok', timestamp: Date.now() })
}

This ensures the pod's cache is warm before Kubernetes routes production traffic to it.

Knowledge Check

In Next.js, how does Request Memoization handle deduplication for fetch calls?

What is the primary cause of a "cache stampede" (thundering herd) in Next.js, and how can it be mitigated using cacheLife?

How does Next.js derive the cache key for a function marked with the 'use cache' directive?

Test your knowledge with more question sets

Sign in to keep reading

The rest of this module is free — sign in with Google to unlock it and track your progress.

PreviousModule A-2: The Full Request Lifecycle and Edge Infrastructure Next Module A-4: Partial Prerendering — Static Shell, Dynamic Holes

Discussion

Join the discussion

Loading comments...