F-4 — TTL, Expiry, and Eviction

Who this module is for: You have set expiry on keys using EX but never thought about what happens when Redis runs out of memory, how it decides which keys to delete, or why your cached data sometimes disappears before the TTL expires. This module covers the full lifecycle of a key from creation to deletion — including the seven eviction policies, lazy vs active expiry, and how to design your keyspace so you never get surprised by silent data loss.

The Key Lifecycle

Every Redis key passes through the same lifecycle:

Understanding each transition is the difference between a cache that behaves predictably and one that silently drops data at 3 AM under load.

Setting Expiry

You already know the inline EX option on SET. Redis gives you several ways to set expiry — on creation or after the fact.

On Creation (SET options)

After Creation (standalone commands)

As of Redis 7.0, EXPIRE and PEXPIRE accept option flags:

These are useful for conditional TTL management — for example, extending a session TTL on activity (GT), or ensuring a key's TTL only decreases (LT) for rate-limit windows.

Querying TTL

-1 and -2 are easy to confuse. -1 means the key exists but has no expiry (persistent). -2 means the key does not exist at all — either it was never set or it already expired.

Removing TTL

This is useful when you want to "promote" a cached object to permanent storage without deleting and re-creating it.

How Expiry Works Internally

Redis does not run a background thread that continuously checks every key for expiry. That would be prohibitively expensive with millions of keys. Instead, Redis uses two complementary mechanisms:

1. Lazy Expiry

When a client accesses a key — via GET, SET, EXISTS, or any other command — Redis checks whether the key has an expired TTL before returning the value. If it has expired, Redis deletes it on the spot and returns (nil).

This is "lazy" because the deletion only happens when the key is accessed. A key that expires at 14:00 but is never accessed again after 13:59 will still occupy memory at 15:00.

2. Active Expiry (Periodic Sampling)

To handle keys that expire but are never accessed again, Redis runs an active expiry cycle on a timer (by default 10 times per second, controlled by hz config — default 10). Each cycle:

1. Takes a random sample of 20 keys from the set of all keys with a TTL
2. Deletes any that have expired
3. If more than 25% of the sampled keys were expired, repeats the cycle immediately (keeps going until < 25% are expired)

This adaptive approach means Redis spends more time on expiry cleanup when there are many expired keys, and less time when the keyspace is healthy. The trade-off: a key that expires may remain in memory for up to 100ms after expiration (one hz cycle) if it is never accessed.

In practice: Your application should always handle (nil) from GET even if a key should still be live — clock skew, replication lag, and the expiry timing mean you cannot rely on exact expiry moments.

Eviction: When Redis Runs Out of Memory

Expiry handles keys that have a TTL. What about keys with no TTL when Redis hits its memory limit?

When Redis reaches its configured maxmemory limit, it must decide what to do with new write commands. The behavior is controlled by maxmemory-policy.

Setting maxmemory

In redis.conf or at runtime:

At runtime:

If maxmemory is 0 (the default), Redis uses all available RAM on the machine. Always set maxmemory in production. Letting Redis exhaust system RAM will cause the OS to start swapping (millisecond latencies become second latencies) or OOM-kill the Redis process.

The Seven Eviction Policies

LRU vs LFU:

• LRU (Least Recently Used) — evicts the key that was accessed longest ago. Good for working-set caches where recent access predicts future access.
• LFU (Least Frequently Used) — evicts the key that was accessed fewest times overall. Better when some keys are accessed heavily and others rarely — LRU might evict a key accessed once an hour but recently, while LFU would correctly identify it as low-value.

Redis's LRU implementation is an approximation: instead of maintaining a true LRU linked list (which would require O(1) updates to a doubly-linked list on every access), Redis uses a random sample of maxmemory-samples keys (default 5) and evicts the least recently used among them. Increasing maxmemory-samples to 10 gives better approximation at slightly higher CPU cost.

The volatile-* Policies and Why They're Dangerous

The volatile-* policies only evict keys that have a TTL. If your memory is full and all your keys are persistent (no TTL), volatile-lru and volatile-ttl fall back to noeviction behavior — new writes will error.

This is a common production trap: engineers set volatile-lru thinking "it will evict cached keys," but their cached keys don't have TTLs. Under memory pressure, Redis starts rejecting writes.

Rule of thumb:

• Use allkeys-lru if Redis is purely a cache.
• Use noeviction if Redis holds authoritative data (sessions, queues, rate limit counters).
• Use volatile-lru only if you deliberately have both persistent keys (no TTL) and cache keys (with TTL) in the same instance, and you want cache keys to be evicted before persistent ones.

Eviction in Practice

When Redis needs to evict:

1. A write command arrives and Redis is at maxmemory
2. Redis samples maxmemory-samples keys (from the appropriate pool based on policy)
3. It evicts the best candidate
4. The write proceeds

This happens synchronously on the write path. Under heavy memory pressure with a busy write workload, eviction adds latency to write commands. This is another reason to size your maxmemory correctly — you want Redis operating at 70-80% of its limit, not 99%.

Monitoring Expiry and Eviction

Key fields:

Key fields:

Cache hit rate = keyspace_hits / (keyspace_hits + keyspace_misses)

A well-tuned cache should have > 95% hit rate. Below 80% suggests either your TTLs are too short, your maxmemory is too small, or your cache keys are not matching your access patterns.

evicted_keys > 0 means Redis is actively evicting. A small number is acceptable for a cache. A large and growing number means your cache is under-provisioned — add memory or reduce your dataset.

expires = number of keys with a TTL. If expires << keys, most of your keys are persistent — check that your cache keys are setting TTLs.

Key Design for TTL Management

Use Consistent TTLs per Key Pattern

Document your TTL decisions alongside your key naming conventions. "Why does this key expire in 300 seconds?" should have an answer in your team's Redis design doc.

Sliding Window TTLs

For session-like keys where activity extends the lifetime:

This gives you a session that expires 1 hour after the last activity, not 1 hour after creation. Use EXPIRE key seconds GT (Redis 7.0+) to only extend, never shorten:

Avoid Short TTLs on Expensive Computations

A 1-second TTL on a key that takes 200ms to recompute means under load, you could have 50 concurrent requests all racing to recompute when the key expires — the "thundering herd" or cache stampede problem. Solutions:

1. Longer TTL — obvious but effective
2. Probabilistic early recomputation — when TTL < threshold, recompute with probability proportional to how close you are to expiry (jitter-based)
3. Lock-based recomputation — use SET lock:key ... NX EX 5 to ensure only one request recomputes; others serve stale data or wait

The lock pattern (covered in full in A-1) is the correct solution for expensive recomputations.

Key Expiry and Replication

When a key expires on the primary, the DEL command is propagated to replicas. Replicas do not expire keys independently — they wait for the primary's expiry propagation. This means:

• A replica that is behind in replication may serve an expired key for a moment
• If you read from a replica directly (via read-from-replica config), you may get values that have expired on the primary but not yet been propagated as deleted

For strict expiry correctness, always read from the primary. For cache use cases where stale-for-a-few-milliseconds is acceptable, replica reads are fine.

The OBJECT Command and Key Metadata

OBJECT IDLETIME is particularly useful for understanding which keys in your keyspace are "cold" — not accessed recently. Combine with SCAN to find cold keys and either expire them or investigate why they are not being hit.

Summary

• Expiry is set with EX/PX inline or EXPIRE/PEXPIRE after creation; removed with PERSIST
• Redis expires keys via lazy expiry (on access) and active expiry (periodic 10x/sec sampling of 20 keys)
• Expired keys may linger in memory for up to one hz cycle — always handle (nil) from GET
• Eviction triggers when Redis hits maxmemory; controlled by maxmemory-policy
• allkeys-lru is the right default for pure caches; noeviction for authoritative data stores
• volatile-* policies only evict keys with TTLs — dangerous if your cache keys have no TTLs
• Monitor with INFO stats (evicted_keys, keyspace_hits/misses) and INFO memory
• Keep evicted_keys near zero; cache hit rate above 95%; maxmemory at 70-80% of available RAM

Next: F-5 — HyperLogLog, Bitmaps, and Geospatial — the three specialized types that solve counting, tracking, and location problems at scale without the memory cost of storing raw data.