A-9 — Redis Cluster: Hash Slots and Data Distribution

Who this module is for: Your Redis dataset has grown beyond what a single node can hold in RAM, or your write throughput exceeds what a single node can handle. Redis Cluster is the answer — but it introduces constraints around multi-key operations, key routing, and command support that you must understand before adopting it. This module covers the Cluster model end-to-end.

Why Cluster?

A single Redis node has two hard limits:

1. RAM — the entire dataset must fit in one machine's memory. A 1TB dataset cannot fit on a 256GB node.
2. Write throughput — all writes go through one single-threaded event loop. At some point, you hit the wall (~1M ops/sec on modern hardware).

Redis Cluster solves both by distributing data across multiple primary nodes, each responsible for a subset of the keyspace.

The Hash Slot Model

Redis Cluster partitions the keyspace into 16,384 hash slots (numbered 0–16383). Every key belongs to exactly one slot:

Hash slots are assigned to primary nodes. A 3-node cluster might assign:

When a client issues GET user:1001, Redis computes CRC16("user:1001") % 16384 = 7638. Slot 7638 belongs to Node 2. The client sends the GET to Node 2.

MOVED and ASK Redirections

Cluster clients must track which node owns which slots. When a client sends a command to the wrong node, the node responds with a MOVED redirection:

The client updates its local slot-to-node map and resends the command to the correct node. A well-implemented Cluster client handles this automatically and transparently — application code does not see MOVED errors.

ASK redirections are similar but temporary — used during slot migration (when a slot is being moved from one node to another). ASK means "ask this specific node for this specific command, but do not update your slot map."

Hash Tags: Controlling Key Placement

Multi-key commands (MGET, MSET, pipelines, Lua scripts, transactions) require all keys to be on the same node. In Cluster mode, this means all keys must be in the same hash slot.

Hash tags force a specific portion of the key to be used for slot computation:

With hash tags, all keys for a specific user share the same slot — enabling multi-key operations:

Design rule: For related keys that must be accessed together, use hash tags from the start. Retrofitting hash tags requires migrating all existing keys.

Multi-Key Command Constraints

Commands that operate on multiple keys fail in Cluster if the keys span different slots:

Workarounds:

1. Hash tags — co-locate related keys on the same slot
2. Application-side batching — split multi-key operations by slot, execute per-node, merge results in the application
3. Single-key alternatives — replace MGET with a pipeline of GET commands (each routed to the correct node automatically by the Cluster client)

Cluster Setup

A minimal production Cluster requires 6 nodes: 3 primaries + 3 replicas (one replica per primary).

Cluster-Aware Client: ioredis.Cluster

Cluster Node Commands

cluster_state:fail means at least one slot has no reachable primary — the cluster is refusing some or all write commands.

Automatic Failover in Cluster

Each primary in a Cluster has one or more replicas. When a primary fails, its replicas detect the failure via gossip (covered in A-11) and one replica promotes itself after a timeout:

After failover, the new primary owns all slots previously owned by the failed node. If the failed primary had multiple replicas, only one promotes — the others continue replicating from the new primary.

If a primary fails with no available replica (the replica also failed), the cluster enters a cluster_state:fail state for those slots. No writes are accepted for the missing slots until the node recovers.

Limitations of Redis Cluster

No cross-slot operations: Multi-key commands must use hash tags. This is a design-time constraint — it requires forethought about which keys need to be co-located.

Limited database support: Cluster only uses database 0. The SELECT command is not available in Cluster.

Lua script constraints: EVAL scripts in Cluster must have all KEYS in the same slot. Multi-slot Lua scripts fail with CROSSSLOT. This limits some patterns that work fine on standalone Redis.

No blocking cross-node operations: BLPOP works per-node. If a producer pushes to a list on Node 1 and a consumer blocks on Node 2 (different slot), the consumer never wakes.

Complexity: Cluster adds operational complexity — monitoring multiple nodes, handling resharding, managing replica assignments. For most workloads that fit within a single node's capacity, Sentinel (with a single large primary) is operationally simpler.

When to Use Cluster

Use Cluster when:

• Dataset size exceeds available RAM on a single node
• Write throughput exceeds ~500K ops/second on a single node
• You need geographic distribution of data (Cluster with geo-aware routing)

Do not use Cluster when:

• Your dataset fits on a single node — Sentinel is simpler
• You rely heavily on multi-key commands without hash tag planning
• Your application uses SELECT to separate databases (not supported in Cluster)
• You use complex Lua scripts accessing multiple keys across slots

Summary

• Redis Cluster distributes 16,384 hash slots across primary nodes; slot = CRC16(key) % 16384
• MOVED redirections: permanent reroute to correct node; ASK redirections: temporary reroute during slot migration
• Hash tags {tag} force slot computation on the tag — co-locate related keys: {user:1001}:profile and {user:1001}:session share a slot
• Multi-key commands (MGET, MSET, Lua EVAL) require all keys on the same slot — use hash tags or application-side batching
• Minimal production Cluster: 3 primaries + 3 replicas (one replica per primary)
• Automatic failover when a primary fails — replica promotes after cluster-node-timeout; cluster enters fail state if no replica available
• ioredis.Cluster handles MOVED/ASK transparently; scaleReads: 'slave' routes reads to replicas
• Cluster limitations: no cross-slot ops without hash tags, only database 0, higher operational complexity than Sentinel

Next: A-10 — Resharding, Node Addition, and Live Slot Migration — adding nodes to a running cluster, migrating slots with zero downtime, and the ASKING redirection that enables live traffic during migration.