F-4 — TTL, Expiry, and Eviction

The Key Lifecycle

Every Redis key passes through the same lifecycle:

Created → [Optional TTL set] → Accessed / Modified → Expired or Evicted → Deleted

Understanding each transition is the difference between a cache that behaves predictably and one that silently drops data at 3 AM under load.

Setting Expiry

You already know the inline EX option on SET. Redis gives you several ways to set expiry — on creation or after the fact.

On Creation (SET options)

SET key value EX seconds          → expire in N seconds
SET key value PX milliseconds     → expire in N milliseconds
SET key value EXAT unix-seconds   → expire at Unix timestamp (seconds)
SET key value PXAT unix-ms        → expire at Unix timestamp (milliseconds)

After Creation (standalone commands)

EXPIRE   key seconds              → set TTL in seconds (overwrites existing TTL)
PEXPIRE  key milliseconds         → set TTL in milliseconds
EXPIREAT key unix-seconds         → set expiry at absolute Unix timestamp
PEXPIREAT key unix-ms             → set expiry at absolute Unix timestamp (ms)

As of Redis 7.0, EXPIRE and PEXPIRE accept option flags:

EXPIRE key seconds NX    → set TTL only if the key has NO existing TTL
EXPIRE key seconds XX    → set TTL only if the key ALREADY has a TTL
EXPIRE key seconds GT    → set TTL only if new TTL > current TTL
EXPIRE key seconds LT    → set TTL only if new TTL < current TTL

These are useful for conditional TTL management — for example, extending a session TTL on activity (GT), or ensuring a key's TTL only decreases (LT) for rate-limit windows.

Querying TTL

TTL  key    → remaining time in seconds (-1 = no TTL, -2 = key does not exist)
PTTL key    → remaining time in milliseconds

-1 and -2 are easy to confuse. -1 means the key exists but has no expiry (persistent). -2 means the key does not exist at all — either it was never set or it already expired.

Removing TTL

PERSIST key   → remove the TTL, making the key persistent again

This is useful when you want to "promote" a cached object to permanent storage without deleting and re-creating it.

127.0.0.1:6379> SET token:abc123 "payload" EX 3600
OK
127.0.0.1:6379> TTL token:abc123
(integer) 3599
127.0.0.1:6379> PERSIST token:abc123
(integer) 1
127.0.0.1:6379> TTL token:abc123
(integer) -1

How Expiry Works Internally

Redis does not run a background thread that continuously checks every key for expiry. That would be prohibitively expensive with millions of keys. Instead, Redis uses two complementary mechanisms:

1. Lazy Expiry

When a client accesses a key — via GET, SET, EXISTS, or any other command — Redis checks whether the key has an expired TTL before returning the value. If it has expired, Redis deletes it on the spot and returns (nil).

This is "lazy" because the deletion only happens when the key is accessed. A key that expires at 14:00 but is never accessed again after 13:59 will still occupy memory at 15:00.

2. Active Expiry (Periodic Sampling)

To handle keys that expire but are never accessed again, Redis runs an active expiry cycle on a timer (by default 10 times per second, controlled by hz config — default 10). Each cycle:

1. Takes a random sample of 20 keys from the set of all keys with a TTL
2. Deletes any that have expired
3. If more than 25% of the sampled keys were expired, repeats the cycle immediately (keeps going until < 25% are expired)

This adaptive approach means Redis spends more time on expiry cleanup when there are many expired keys, and less time when the keyspace is healthy. The trade-off: a key that expires may remain in memory for up to 100ms after expiration (one hz cycle) if it is never accessed.

In practice: Your application should always handle (nil) from GET even if a key should still be live — clock skew, replication lag, and the expiry timing mean you cannot rely on exact expiry moments.

Eviction: When Redis Runs Out of Memory

Expiry handles keys that have a TTL. What about keys with no TTL when Redis hits its memory limit?

When Redis reaches its configured maxmemory limit, it must decide what to do with new write commands. The behavior is controlled by maxmemory-policy.

Setting maxmemory

In redis.conf or at runtime:

maxmemory 2gb           → limit Redis to 2GB of RAM
maxmemory-policy allkeys-lru   → eviction policy (see below)

At runtime:

CONFIG SET maxmemory 2gb
CONFIG SET maxmemory-policy allkeys-lru
CONFIG GET maxmemory

If maxmemory is 0 (the default), Redis uses all available RAM on the machine. Always set maxmemory in production. Letting Redis exhaust system RAM will cause the OS to start swapping (millisecond latencies become second latencies) or OOM-kill the Redis process.

The Seven Eviction Policies

LRU vs LFU:

• LRU (Least Recently Used) — evicts the key that was accessed longest ago. Good for working-set caches where recent access predicts future access.
• LFU (Least Frequently Used) — evicts the key that was accessed fewest times overall. Better when some keys are accessed heavily and others rarely — LRU might evict a key accessed once an hour but recently, while LFU would correctly identify it as low-value.

Redis's LRU implementation is an approximation: instead of maintaining a true LRU linked list (which would require O(1) updates to a doubly-linked list on every access), Redis uses a random sample of maxmemory-samples keys (default 5) and evicts the least recently used among them. Increasing maxmemory-samples to 10 gives better approximation at slightly higher CPU cost.

The volatile-* Policies and Why They're Dangerous

The volatile-* policies only evict keys that have a TTL. If your memory is full and all your keys are persistent (no TTL), volatile-lru and volatile-ttl fall back to noeviction behavior — new writes will error.

This is a common production trap: engineers set volatile-lru thinking "it will evict cached keys," but their cached keys don't have TTLs. Under memory pressure, Redis starts rejecting writes.

Rule of thumb:

• Use allkeys-lru if Redis is purely a cache.
• Use noeviction if Redis holds authoritative data (sessions, queues, rate limit counters).
• Use volatile-lru only if you deliberately have both persistent keys (no TTL) and cache keys (with TTL) in the same instance, and you want cache keys to be evicted before persistent ones.

Eviction in Practice

When Redis needs to evict:

1. A write command arrives and Redis is at maxmemory
2. Redis samples maxmemory-samples keys (from the appropriate pool based on policy)
3. It evicts the best candidate
4. The write proceeds

This happens synchronously on the write path. Under heavy memory pressure with a busy write workload, eviction adds latency to write commands. This is another reason to size your maxmemory correctly — you want Redis operating at 70-80% of its limit, not 99%.

Monitoring Expiry and Eviction

INFO memory

Key fields:

used_memory_human: 1.45G      → current memory usage
maxmemory_human: 2.00G        → configured limit
mem_fragmentation_ratio: 1.12 → ratio of RSS to used_memory; > 1.5 suggests fragmentation

INFO stats

Key fields:

expired_keys: 1482931         → total keys expired since server start
evicted_keys: 0               → total keys evicted (0 is ideal for a well-sized cache)
keyspace_hits: 9823441        → GET/HGET etc that found a key
keyspace_misses: 341829       → GET/HGET etc that returned nil

Cache hit rate = keyspace_hits / (keyspace_hits + keyspace_misses)

A well-tuned cache should have > 95% hit rate. Below 80% suggests either your TTLs are too short, your maxmemory is too small, or your cache keys are not matching your access patterns.

evicted_keys > 0 means Redis is actively evicting. A small number is acceptable for a cache. A large and growing number means your cache is under-provisioned — add memory or reduce your dataset.

INFO keyspace

db0:keys=142883,expires=141204,avg_ttl=3591847

expires = number of keys with a TTL. If expires << keys, most of your keys are persistent — check that your cache keys are setting TTLs.

Key Design for TTL Management

Use Consistent TTLs per Key Pattern

# Session tokens: 1 hour
SET session:{token} {payload} EX 3600

# Rate limit windows: 60 seconds
SET rate:{user_id}:{minute} 0 EX 60

# Expensive query cache: 5 minutes
SET cache:user:{id}:dashboard {json} EX 300

# Static reference data: 24 hours
SET ref:currency:rates {json} EX 86400

Document your TTL decisions alongside your key naming conventions. "Why does this key expire in 300 seconds?" should have an answer in your team's Redis design doc.

Sliding Window TTLs

For session-like keys where activity extends the lifetime:

# On each request, reset the TTL
EXPIRE session:{token} 3600

This gives you a session that expires 1 hour after the last activity, not 1 hour after creation. Use EXPIRE key seconds GT (Redis 7.0+) to only extend, never shorten:

EXPIRE session:{token} 3600 GT

Avoid Short TTLs on Expensive Computations

A 1-second TTL on a key that takes 200ms to recompute means under load, you could have 50 concurrent requests all racing to recompute when the key expires — the "thundering herd" or cache stampede problem. Solutions:

1. Longer TTL — obvious but effective
2. Probabilistic early recomputation — when TTL < threshold, recompute with probability proportional to how close you are to expiry (jitter-based)
3. Lock-based recomputation — use SET lock:key ... NX EX 5 to ensure only one request recomputes; others serve stale data or wait

The lock pattern (covered in full in A-1) is the correct solution for expensive recomputations.

Key Expiry and Replication

When a key expires on the primary, the DEL command is propagated to replicas. Replicas do not expire keys independently — they wait for the primary's expiry propagation. This means:

• A replica that is behind in replication may serve an expired key for a moment
• If you read from a replica directly (via read-from-replica config), you may get values that have expired on the primary but not yet been propagated as deleted

For strict expiry correctness, always read from the primary. For cache use cases where stale-for-a-few-milliseconds is acceptable, replica reads are fine.

The OBJECT Command and Key Metadata

OBJECT ENCODING key     → internal encoding
OBJECT REFCOUNT key     → reference count (usually 1; some integers are shared objects)
OBJECT IDLETIME key     → seconds since the key was last accessed
OBJECT FREQ key         → LFU access frequency log counter (requires LFU policy)
OBJECT HELP             → list all OBJECT subcommands

OBJECT IDLETIME is particularly useful for understanding which keys in your keyspace are "cold" — not accessed recently. Combine with SCAN to find cold keys and either expire them or investigate why they are not being hit.

# Find keys idle for more than 1 hour (3600 seconds)
SCAN 0 COUNT 100
# For each key returned:
OBJECT IDLETIME key

Summary

• Expiry is set with EX/PX inline or EXPIRE/PEXPIRE after creation; removed with PERSIST
• Redis expires keys via lazy expiry (on access) and active expiry (periodic 10x/sec sampling of 20 keys)
• Expired keys may linger in memory for up to one hz cycle — always handle (nil) from GET
• Eviction triggers when Redis hits maxmemory; controlled by maxmemory-policy
• allkeys-lru is the right default for pure caches; noeviction for authoritative data stores
• volatile-* policies only evict keys with TTLs — dangerous if your cache keys have no TTLs
• Monitor with INFO stats (evicted_keys, keyspace_hits/misses) and INFO memory
• Keep evicted_keys near zero; cache hit rate above 95%; maxmemory at 70-80% of available RAM

Next: F-5 — HyperLogLog, Bitmaps, and Geospatial — the three specialized types that solve counting, tracking, and location problems at scale without the memory cost of storing raw data.