Lists, Hashes, Sets, and Sorted Sets — Internal Encodings24 min read
Module F-3·24 min read
The full data structure surface and how each uses listpack vs ziplist vs skiplist vs hashtable encoding under the hood — what triggers encoding upgrades and how encoding determines memory usage.
Who this module is for: You know Redis has data structures beyond Strings — you have seen them in documentation — but you have never understood when to reach for each one, why they exist, or what is happening under the hood when you use them. This module teaches the full command surface and internal mechanics of Redis's four most important composite types. By the end, you will not just know the commands; you will know which structure to reach for before you start writing code.
Why Data Structures Matter in Redis
In F-2, you learned that a Redis String is not a string — it is a binary-safe byte array with encoding-aware storage. The same design philosophy applies to every Redis type: the shape of your data and the operations you need should determine which type you use.
This is different from how you might think about SQL. In PostgreSQL, you store everything in tables and columns, then join and aggregate at query time. In Redis, you choose the structure that makes your operations atomic and your memory efficient. The wrong choice does not cause a crash — it causes unnecessary complexity, non-atomic operations, and wasted RAM.
The four structures in this module — Lists, Hashes, Sets, and Sorted Sets — cover the vast majority of Redis use cases. Master these four and you can implement caching, queues, counters, leaderboards, sessions, pub/sub fan-out, social graphs, and rate limiters without reaching for anything else.
Lists
What a Redis List Is
A Redis List is an ordered sequence of strings, linked as a doubly-linked list (or, for small sizes, a listpack — more on this in the encoding section below). You can push elements to either end and pop elements from either end. This makes it both a stack (LIFO) and a queue (FIFO) depending on how you use it.
text
Core Commands
Pushing and popping:
text
Multi-element pop (count argument) was added in Redis 6.2. Before that, you had to call LPOP in a loop.
Inspecting without consuming:
text
LRANGE key 0 -1 returns all elements. Redis list indexes are zero-based, and negative indexes count from the tail: -1 is the last element, -2 is the second-to-last, and so on.
text
Blocking pop — the pattern that powers job queues:
text
BLPOP myqueue 0 blocks the client indefinitely until another client pushes to myqueue. The moment a push occurs, the blocked client is woken and the element is returned. This is how you build a job queue without polling:
text
The timeout argument is in seconds. 0 means block forever (until an element arrives or the connection is closed). A non-zero timeout returns (nil) if nothing arrives within the timeout — useful for workers that need to periodically check for shutdown signals.
BLPOP accepts multiple keys and returns from the first one that has data — this lets a single worker consume from multiple queues with a single connection.
Moving between lists atomically:
LMOVE source destination LEFT|RIGHT LEFT|RIGHT
LMOVE source destination RIGHT LEFT pops from the right of source and pushes to the left of destination atomically. This is the correct way to implement a reliable queue: move a job from pending to processing, and if the worker crashes, the job is still in processing and can be recovered.
The non-atomic alternative — RPOP from one list then LPUSH to another — has a window where the job is in neither list if your application crashes between the two commands.
Trimming a list to a fixed size:
LTRIM key start stop
After LTRIM mylog 0 999, only the first 1,000 elements remain. Use this after every LPUSH to cap a capped log or recent-activity feed.
When to Use a List
Job queue / task queue — RPUSH to enqueue, BLPOP to consume (FIFO)
Stack — LPUSH to push, LPOP to pop (LIFO)
Activity feed / recent events — LPUSH new events, LTRIM to keep the last N, LRANGE to read
Reliable queue — LMOVE from pending to processing, acknowledge by deleting from processing
Do not use a List when:
You need to look up elements by a specific value (no indexed lookup by content — use a Hash or Set)
You need uniqueness (use a Set)
You need ordering by score (use a Sorted Set)
The list grows to millions of elements and you iterate it frequently (consider Streams)
Hashes
What a Redis Hash Is
A Redis Hash is a map of field names to values — exactly what you might call a dictionary, object, or associative array in your programming language. Both fields and values are strings.
text
Core Commands
Setting fields:
text
In Redis 4.0+, HSET accepts multiple field-value pairs. The old HMSET command (which did the same thing) is deprecated. Use HSET.
Getting fields:
text
HGETALL returns a flat alternating list: field1, value1, field2, value2, ... Client libraries typically convert this to a map for you. For large hashes, be careful with HGETALL — it returns everything in one call and can be slow for hashes with thousands of fields.
Checking and deleting:
text
Incrementing numeric fields:
text
This is the right way to track per-user counters inside a Hash object:
text
Scanning large hashes:
HSCAN key cursor [MATCH pattern] [COUNT count]
Like SCAN for top-level keys, HSCAN iterates over hash fields in batches. Use this instead of HGETALL for hashes with thousands of fields.
The Hash vs String JSON Trade-off
A common pattern is to store a user object as a JSON string:
SET user:1001 '{"name":"Jatin","email":"j@example.com","role":"engineer","posts":42}'
This works. But it has a critical flaw: every update requires reading the entire JSON, deserializing, modifying, reserializing, and writing it back. If two processes do this concurrently, one will overwrite the other's changes. You cannot atomically update a single field.
With a Hash:
text
Use a Hash when your object has fields you update independently. Use a String JSON blob only when you always read and write the entire object atomically (rare) or when the object is too complex for the flat Hash model.
When to Use a Hash
Object/entity storage — user profiles, product metadata, session data
Configuration maps — feature flags, settings with multiple keys
Grouping related counters — instead of 10 separate String keys, one Hash with 10 fields
Do not use a Hash when:
You need TTL on individual fields (not supported — TTL is per key, not per field)
The number of fields varies wildly and you need range queries (consider Sorted Set)
You need to search by value (no indexed lookup by value in a Hash)
Sets
What a Redis Set Is
A Redis Set is an unordered collection of unique strings. It behaves like a mathematical set: no duplicates, no guaranteed order, and support for set-theoretic operations (union, intersection, difference).
text
Core Commands
Adding and removing:
text
Querying:
text
Set operations:
text
The STORE variants write the result to a new key and return the count. Use these when you want to cache a set operation result rather than computing it repeatedly.
Scanning large sets:
SSCAN key cursor [MATCH pattern] [COUNT count]
text
When to Use a Set
Tracking unique items — unique visitors today, users who have seen a notification
Tags and categories — per-entity tag sets with intersection ("users who follow both X and Y")
Social graph — followers, following (intersection = mutual followers)
Deduplication — "have I processed this job ID before?" (SADD returns 0 if already exists)
Random sampling — SPOP for lottery, SRANDMEMBER for random recommendations
Do not use a Set when:
You need ordered results (use Sorted Set)
You need to store values with associated data beyond the member string (use Sorted Set with score, or Hash)
You need approximate uniqueness at extreme scale (use HyperLogLog, covered in F-5)
Sorted Sets
What a Redis Sorted Set Is
A Redis Sorted Set (also called a ZSet) is like a Set with an added requirement: every member has a score (a double-precision float), and members are ordered by score from lowest to highest. Within the same score, members are ordered lexicographically.
This single addition — the score — makes Sorted Sets one of the most versatile Redis types. You can implement leaderboards, rate limiters, priority queues, time-series indexes, and autocomplete with a Sorted Set.
text
Core Commands
Adding and updating:
ZADD key [NX|XX] [GT|LT] [CH] [INCR] score member [score member ...]
The options make ZADD flexible:
NX — add only; never update existing members
XX — update only; never add new members
GT — only update if new score > current score
LT — only update if new score < current score
CH — return count of members changed (added + updated); default returns only count added
INCR — instead of setting the score, increment it; equivalent to ZINCRBY
text
Querying by rank (position):
text
As of Redis 6.2, ZRANGE is the unified range query command. Before 6.2, you used separate commands:
ZRANGE key start stop — by rank, ascending
ZREVRANGE key start stop — by rank, descending
ZRANGEBYSCORE key min max — by score, ascending
ZREVRANGEBYSCORE key max min — by score, descending
All of these still work in Redis 7, but ZRANGE with REV and BYSCORE options is the modern approach.
text
Querying scores and cardinality:
text
Removing:
text
Incrementing scores:
ZINCRBY key increment member → atomically add increment to member's score, return new score
This is the safe way to update a leaderboard score without a read-modify-write cycle:
ZINCRBY leaderboard 150 "player:alice" → alice's score is now 1650 (was 1500)
Scanning:
ZSCAN key cursor [MATCH pattern] [COUNT count]
Building a Real-Time Leaderboard
The leaderboard is the canonical Sorted Set use case. Here is how it works in practice:
text
All operations are O(log N). The Sorted Set's skip list data structure makes rank queries, range queries, and insertions all logarithmic — the same asymptotic complexity as a balanced binary tree, but with better real-world constant factors for this access pattern.
Using Sorted Sets as Time-Series Indexes
Store the Unix timestamp as the score:
text
This is a common pattern for activity feeds, audit logs, and sliding-window rate limiting.
When to Use a Sorted Set
Leaderboards — score as float, member as user ID
Rate limiters — timestamp as score, request ID as member (count members in recent window)
Priority queues — priority as score, job ID as member (ZPOPMIN = get highest priority)
Time-series indexes — Unix timestamp as score, event ID as member
Autocomplete — lex-ordered members with same score for prefix queries
Scheduling — scheduled-at timestamp as score, task ID as member (poll with ZRANGE ... BYSCORE 0 now)
Internal Encodings
Understanding what Redis stores internally explains memory usage and performance cliffs.
Lists
Size
Encoding
Description
≤ 128 elements, each ≤ 64 bytes
listpack
Compact byte array, no pointers, cache-friendly
> 128 elements or any element > 64 bytes
quicklist
Doubly-linked list of listpack nodes
Before Redis 7.0, the small-list encoding was ziplist. Redis 7.0 replaced it with listpack (more compact, same concept). The thresholds are controlled by list-max-listpack-size and list-max-ziplist-size config.
A quicklist is a doubly-linked list where each node contains a listpack (a compact byte array). This gives you the cache efficiency of contiguous memory for small sublists and the O(1) push/pop of a linked list.
Hashes
Size
Encoding
≤ 128 fields, each ≤ 64 bytes
listpack
> 128 fields or any field/value > 64 bytes
hashtable
The listpack encoding stores field-value pairs sequentially in a byte array. For a small user profile object (< 128 fields), Redis uses essentially zero pointer overhead. The moment you add the 129th field — or a field with a value longer than 64 bytes — the entire Hash is converted to a hashtable with pointer-based buckets.
intset is a sorted array of integers stored in minimal bytes (2, 4, or 8 bytes per element depending on range). It is remarkably compact and fast to search (binary search). The moment you add a non-integer member, the entire set converts to listpack or hashtable.
The skiplist encoding uses a probabilistic skip list for the ordered-by-score structure and a hashtable for O(1) score lookup by member. Both structures point to the same member/score pairs in memory — the memory cost is one set of data with two indexes.
The listpack encoding is a sorted byte array: fast for small sets (no pointer overhead), but linear-time lookup. This is fine for ≤ 128 members.
The default thresholds are good starting points, but they are not sacred. If your use case has objects that always stay small, you can raise the listpack thresholds to keep more objects in the compact encoding:
text
The trade-off: larger listpack means more scanning per operation (linear time within the listpack). For objects you always HGETALL (read all fields), a larger listpack is fine. For objects where you HGET specific fields frequently, raising the threshold too high can make individual field lookups slower.
In practice: profile with OBJECT ENCODING and OBJECT IDLETIME. If your hashes are at hashtable encoding with only 10 fields, your encoding threshold is too low.
Choosing the Right Structure
Here is a decision framework you can internalize:
Does your data have a natural ordering by importance/priority/time?
→ Yes → Sorted Set (the score is the ordering)
Does your data need uniqueness with no associated metadata?
→ Yes, and order doesn't matter → Set
→ Yes, and order matters (by score) → Sorted Set
Does your data have named fields that you update independently?
→ Yes → Hash
Does your data need FIFO/LIFO access or blocking consumers?
→ Yes → List
Is your data a simple value that you cache, count, or atomically update?
→ String
One entity with multiple attributes + per-attribute counters + access by ID:
→ Hash for attributes, separate Sorted Set for ranking that entity
Summary
Lists — ordered sequences with head/tail access. Use for queues (RPUSH/BLPOP), stacks (LPUSH/LPOP), activity feeds (LPUSH + LTRIM). LMOVE for reliable queues.
Hashes — field-value maps. Use for entity objects (user profiles, sessions). HINCRBY for atomic per-field counters. Prefer over JSON strings when you update individual fields.
Sets — unordered unique collections. Use for uniqueness tracking, social graphs (SINTER for mutual follows), deduplication. Set operations (SINTER/SUNION/SDIFF) are atomic.
Sorted Sets — unique members ordered by float score. Use for leaderboards (ZINCRBY + ZREVRANK), time-series indexes (timestamp as score), priority queues (ZPOPMIN), rate limiters.
Encodings — all four types start in a compact listpack/intset encoding and upgrade to pointer-based structures when they exceed configured thresholds. Know your thresholds; inspect with OBJECT ENCODING.
Next: F-4 — Redis's Single-Threaded Event Loop — Speed, Limits, and Multithreading — how the event loop handles 100K+ RPS, what Redis 6+ multithreading parallelises, and the one failure mode that brings a Redis instance to its knees.
Knowledge Check
A team needs to build a reliable background job processing system. A worker process must claim a job, process it, and acknowledge completion. If the worker crashes mid-processing, the job must not be lost. Which combination of Redis List commands provides this reliability guarantee without race conditions?
You are tasked with building a real-time global leaderboard for a multiplayer game. The leaderboard must rank millions of players by their score (which updates constantly), allowing you to quickly fetch the top 10 players and query an individual player's current rank. Which Redis data structure is designed precisely for this use case?
An engineer creates a Redis Hash to store user profile data using HSET user:123 name "Alice" role "Admin". They then notice via OBJECT ENCODING user:123 that the Hash is encoded internally as a listpack. They add a new field: HSET user:123 bio "..." where the biography string is 5,000 bytes long. What happens to the internal encoding of this Hash?
Test your knowledge with more question sets
Sign in to access a wider variety of questions and get notified when new practice sets are added to this module.
LPUSH tasks "task-c" → pushes to the LEFT (head)
LPUSH tasks "task-b"
LPUSH tasks "task-a"
State: [task-a, task-b, task-c] ← head is left
RPUSH tasks "task-d" → pushes to the RIGHT (tail)
State: [task-a, task-b, task-c, task-d]
LPUSH key val [val ...] → push one or more values to the head, return new length
RPUSH key val [val ...] → push one or more values to the tail, return new length
LPOP key [count] → pop from the head, return value(s)
RPOP key [count] → pop from the tail, return value(s)
LLEN key → number of elements
LRANGE key start stop → return elements from index start to stop (inclusive)
LINDEX key index → return element at index (0 = head, -1 = tail)
BLPOP key [key ...] timeout → block until an element is available at the head
BRPOP key [key ...] timeout → block until an element is available at the tail
# Worker process (blocking, waiting for work)
BLPOP tasks 0 → blocks here
# Producer (in another process/connection)
LPUSH tasks "job-99" → wakes the blocked worker
HSET user:1001 name "Jatin" email "j@example.com" role "engineer"
HGET user:1001 name → "Jatin"
HGET user:1001 email → "j@example.com"
HGETALL user:1001 → ["name", "Jatin", "email", "j@example.com", "role", "engineer"]
HSET key field value [field value ...] → set one or more fields, return count of new fields
HSETNX key field value → set field only if it does not exist
HGET key field → return the value of field
HMGET key field [field ...] → return values of multiple fields (nil for missing)
HGETALL key → return all field-value pairs as a flat list
HKEYS key → return all field names
HVALS key → return all values
HLEN key → return the number of fields
HEXISTS key field → 1 if field exists, 0 if not
HDEL key field [field ...] → delete one or more fields, return count deleted
HINCRBY key field increment → atomically increment integer field
HINCRBYFLOAT key field increment
SADD key member [member ...] → add members, return count of new members added
SREM key member [member ...] → remove members, return count removed
SPOP key [count] → remove and return random member(s)
SRANDMEMBER key [count] → return random member(s) without removing
SMEMBERS key → return all members (use with care on large sets)
SCARD key → return the number of members (cardinality)
SISMEMBER key member → 1 if member exists, 0 if not
SMISMEMBER key member [member ...] → membership test for multiple members (Redis 6.2+)
SINTER key [key ...] → return members in all sets (intersection)
SUNION key [key ...] → return members in any set (union)
SDIFF key [key ...] → return members in first set not in any subsequent set (difference)
SINTERSTORE dest key [key ...] → store intersection result in dest
SUNIONSTORE dest key [key ...] → store union result in dest
SDIFFSTORE dest key [key ...] → store difference result in dest
ZADD leaderboard GT 2500 "player:alice" → updates alice only if 2500 > current score
ZADD leaderboard NX 1000 "player:dave" → adds dave only if not already present
ZRANK key member → return rank (0 = lowest score); nil if not found
ZREVRANK key member → return rank from highest (0 = highest score)
ZRANGE key start stop [REV] [BYSCORE|BYLEX] [LIMIT offset count] [WITHSCORES]
# Get top 3 players (highest scores)
ZRANGE leaderboard 0 2 REV WITHSCORES
1) "player:bob"
2) "2300"
3) "player:charlie"
4) "1900"
5) "player:alice"
6) "1500"
# Get all players with score between 1000 and 2000
ZRANGE leaderboard 1000 2000 BYSCORE WITHSCORES
1) "player:alice"
2) "1500"
3) "player:charlie"
4) "1900"
ZSCORE key member → return the score of a member
ZCARD key → return total number of members
ZCOUNT key min max → count members with score between min and max
ZLEXCOUNT key min max → count members with lex range (same-score members)
ZREM key member [member ...] → remove members
ZREMRANGEBYRANK key start stop → remove by rank range
ZREMRANGEBYSCORE key min max → remove by score range
ZPOPMIN key [count] → remove and return lowest-scored member(s)
ZPOPMAX key [count] → remove and return highest-scored member(s)
# A player completes a level and earns 500 points
ZINCRBY leaderboard:season:1 500 "user:1001"
# Get user's current rank (1-based, descending)
# ZREVRANK returns 0-based rank; add 1 for display
rank = ZREVRANK leaderboard:season:1 "user:1001" → 4 (5th place)
# Get top 10
ZRANGE leaderboard:season:1 0 9 REV WITHSCORES
# Get users near a specific user (their neighborhood in the leaderboard)
ZREVRANK leaderboard:season:1 "user:1001" → 4
ZRANGE leaderboard:season:1 2 6 REV WITHSCORES → positions 3-7
ZADD events:user:1001 1717000000 "event:login:abc123"
ZADD events:user:1001 1717003600 "event:purchase:def456"
ZADD events:user:1001 1717007200 "event:logout:ghi789"
# Events from the last hour (timestamps are seconds since epoch)
now = current_unix_timestamp
ZRANGEBYSCORE events:user:1001 (now-3600) now WITHSCORES
# Prune events older than 30 days
ZREMRANGEBYSCORE events:user:1001 0 (now - 2592000)