Lists, Hashes, Sets, and Sorted Sets — Internal Encodings22 min read
Module F-3·22 min read
The full data structure surface and how each uses listpack vs ziplist vs skiplist vs hashtable encoding under the hood — what triggers encoding upgrades and how encoding determines memory usage.
Who this module is for: You know Redis has data structures beyond Strings — you have seen them in documentation — but you have never understood when to reach for each one, why they exist, or what is happening under the hood when you use them. This module teaches the full command surface and internal mechanics of Redis's four most important composite types. By the end, you will not just know the commands; you will know which structure to reach for before you start writing code.
In F-2, you learned that a Redis String is not a string — it is a binary-safe byte array with encoding-aware storage. The same design philosophy applies to every Redis type: the shape of your data and the operations you need should determine which type you use.
This is different from how you might think about SQL. In PostgreSQL, you store everything in tables and columns, then join and aggregate at query time. In Redis, you choose the structure that makes your operations atomic and your memory efficient. The wrong choice does not cause a crash — it causes unnecessary complexity, non-atomic operations, and wasted RAM.
The four structures in this module — Lists, Hashes, Sets, and Sorted Sets — cover the vast majority of Redis use cases. Master these four and you can implement caching, queues, counters, leaderboards, sessions, pub/sub fan-out, social graphs, and rate limiters without reaching for anything else.
A Redis List is an ordered sequence of strings, linked as a doubly-linked list (or, for small sizes, a listpack — more on this in the encoding section below). You can push elements to either end and pop elements from either end. This makes it both a stack (LIFO) and a queue (FIFO) depending on how you use it.
LPUSH tasks "task-c" → pushes to the LEFT (head) LPUSH tasks "task-b" LPUSH tasks "task-a"
LPUSH key val [val ...] → push one or more values to the head, return new length RPUSH key val [val ...] → push one or more values to the tail, return new length LPOP key [count] → pop from the head, return value(s) RPOP key [count] → pop from the tail, return value(s)
Multi-element pop (count argument) was added in Redis 6.2. Before that, you had to call LPOP in a loop.
Inspecting without consuming:
LLEN key → number of elements LRANGE key start stop → return elements from index start to stop (inclusive) LINDEX key index → return element at index (0 = head, -1 = tail)
LRANGE key 0 -1 returns all elements. Redis list indexes are zero-based, and negative indexes count from the tail: -1 is the last element, -2 is the second-to-last, and so on.
Blocking pop — the pattern that powers job queues:
BLPOP key [key ...] timeout → block until an element is available at the head BRPOP key [key ...] timeout → block until an element is available at the tail
BLPOP myqueue 0 blocks the client indefinitely until another client pushes to myqueue. The moment a push occurs, the blocked client is woken and the element is returned. This is how you build a job queue without polling:
# Worker process (blocking, waiting for work) BLPOP tasks 0 → blocks here
# Producer (in another process/connection) LPUSH tasks "job-99" → wakes the blocked worker
The timeout argument is in seconds. 0 means block forever (until an element arrives or the connection is closed). A non-zero timeout returns (nil) if nothing arrives within the timeout — useful for workers that need to periodically check for shutdown signals.
BLPOP accepts multiple keys and returns from the first one that has data — this lets a single worker consume from multiple queues with a single connection.
Moving between lists atomically:
LMOVE source destination LEFT|RIGHT LEFT|RIGHT
LMOVE source destination RIGHT LEFT pops from the right of source and pushes to the left of destination atomically. This is the correct way to implement a reliable queue: move a job from pending to processing, and if the worker crashes, the job is still in processing and can be recovered.
The non-atomic alternative — RPOP from one list then LPUSH to another — has a window where the job is in neither list if your application crashes between the two commands.
Trimming a list to a fixed size:
LTRIM key start stop
After LTRIM mylog 0 999, only the first 1,000 elements remain. Use this after every LPUSH to cap a capped log or recent-activity feed.
A Redis Hash is a map of field names to values — exactly what you might call a dictionary, object, or associative array in your programming language. Both fields and values are strings.
HSET user:1001 name "Jatin" email "j@example.com" role "engineer" HGET user:1001 name → "Jatin" HGET user:1001 email → "j@example.com" HGETALL user:1001 → ["name", "Jatin", "email", "j@example.com", "role", "engineer"]
HSET key field value [field value ...] → set one or more fields, return count of new fields HSETNX key field value → set field only if it does not exist
In Redis 4.0+, HSET accepts multiple field-value pairs. The old HMSET command (which did the same thing) is deprecated. Use HSET.
Getting fields:
HGET key field → return the value of field HMGET key field [field ...] → return values of multiple fields (nil for missing) HGETALL key → return all field-value pairs as a flat list HKEYS key → return all field names HVALS key → return all values HLEN key → return the number of fields
HGETALL returns a flat alternating list: field1, value1, field2, value2, ... Client libraries typically convert this to a map for you. For large hashes, be careful with HGETALL — it returns everything in one call and can be slow for hashes with thousands of fields.
Checking and deleting:
HEXISTS key field → 1 if field exists, 0 if not HDEL key field [field ...] → delete one or more fields, return count deleted
Incrementing numeric fields:
HINCRBY key field increment → atomically increment integer field HINCRBYFLOAT key field increment
This is the right way to track per-user counters inside a Hash object:
A common pattern is to store a user object as a JSON string:
SET user:1001 '{"name":"Jatin","email":"j@example.com","role":"engineer","posts":42}'
This works. But it has a critical flaw: every update requires reading the entire JSON, deserializing, modifying, reserializing, and writing it back. If two processes do this concurrently, one will overwrite the other's changes. You cannot atomically update a single field.
With a Hash:
HINCRBY user:1001 posts 1 → atomic, safe from concurrent updates HSET user:1001 role "senior-engineer" → update one field, other fields untouched
Use a Hash when your object has fields you update independently. Use a String JSON blob only when you always read and write the entire object atomically (rare) or when the object is too complex for the flat Hash model.
A Redis Set is an unordered collection of unique strings. It behaves like a mathematical set: no duplicates, no guaranteed order, and support for set-theoretic operations (union, intersection, difference).
SADD key member [member ...] → add members, return count of new members added SREM key member [member ...] → remove members, return count removed SPOP key [count] → remove and return random member(s) SRANDMEMBER key [count] → return random member(s) without removing
Querying:
SMEMBERS key → return all members (use with care on large sets) SCARD key → return the number of members (cardinality) SISMEMBER key member → 1 if member exists, 0 if not SMISMEMBER key member [member ...] → membership test for multiple members (Redis 6.2+)
Set operations:
SINTER key [key ...] → return members in all sets (intersection) SUNION key [key ...] → return members in any set (union) SDIFF key [key ...] → return members in first set not in any subsequent set (difference)
SINTERSTORE dest key [key ...] → store intersection result in dest SUNIONSTORE dest key [key ...] → store union result in dest SDIFFSTORE dest key [key ...] → store difference result in dest
The STORE variants write the result to a new key and return the count. Use these when you want to cache a set operation result rather than computing it repeatedly.
A Redis Sorted Set (also called a ZSet) is like a Set with an added requirement: every member has a score (a double-precision float), and members are ordered by score from lowest to highest. Within the same score, members are ordered lexicographically.
This single addition — the score — makes Sorted Sets one of the most versatile Redis types. You can implement leaderboards, rate limiters, priority queues, time-series indexes, and autocomplete with a Sorted Set.
ZADD key [NX|XX] [GT|LT] [CH] [INCR] score member [score member ...]
The options make ZADD flexible:
NX — add only; never update existing members
XX — update only; never add new members
GT — only update if new score > current score
LT — only update if new score < current score
CH — return count of members changed (added + updated); default returns only count added
INCR — instead of setting the score, increment it; equivalent to ZINCRBY
ZADD leaderboard GT 2500 "player:alice" → updates alice only if 2500 > current score ZADD leaderboard NX 1000 "player:dave" → adds dave only if not already present
Querying by rank (position):
ZRANK key member → return rank (0 = lowest score); nil if not found ZREVRANK key member → return rank from highest (0 = highest score) ZRANGE key start stop [REV] [BYSCORE|BYLEX] [LIMIT offset count] [WITHSCORES]
As of Redis 6.2, ZRANGE is the unified range query command. Before 6.2, you used separate commands:
ZRANGE key start stop — by rank, ascending
ZREVRANGE key start stop — by rank, descending
ZRANGEBYSCORE key min max — by score, ascending
ZREVRANGEBYSCORE key max min — by score, descending
All of these still work in Redis 7, but ZRANGE with REV and BYSCORE options is the modern approach.
# Get all players with score between 1000 and 2000 ZRANGE leaderboard 1000 2000 BYSCORE WITHSCORES 1) "player:alice" 2) "1500" 3) "player:charlie" 4) "1900"
Querying scores and cardinality:
ZSCORE key member → return the score of a member ZCARD key → return total number of members ZCOUNT key min max → count members with score between min and max ZLEXCOUNT key min max → count members with lex range (same-score members)
Removing:
ZREM key member [member ...] → remove members ZREMRANGEBYRANK key start stop → remove by rank range ZREMRANGEBYSCORE key min max → remove by score range ZPOPMIN key [count] → remove and return lowest-scored member(s) ZPOPMAX key [count] → remove and return highest-scored member(s)
Incrementing scores:
ZINCRBY key increment member → atomically add increment to member's score, return new score
This is the safe way to update a leaderboard score without a read-modify-write cycle:
ZINCRBY leaderboard 150 "player:alice" → alice's score is now 1650 (was 1500)
The leaderboard is the canonical Sorted Set use case. Here is how it works in practice:
# A player completes a level and earns 500 points ZINCRBY leaderboard:season:1 500 "user:1001"
# Get user's current rank (1-based, descending) # ZREVRANK returns 0-based rank; add 1 for display rank = ZREVRANK leaderboard:season:1 "user:1001" → 4 (5th place)
# Get top 10 ZRANGE leaderboard:season:1 0 9 REV WITHSCORES
# Get users near a specific user (their neighborhood in the leaderboard) ZREVRANK leaderboard:season:1 "user:1001" → 4 ZRANGE leaderboard:season:1 2 6 REV WITHSCORES → positions 3-7
All operations are O(log N). The Sorted Set's skip list data structure makes rank queries, range queries, and insertions all logarithmic — the same asymptotic complexity as a balanced binary tree, but with better real-world constant factors for this access pattern.
Before Redis 7.0, the small-list encoding was ziplist. Redis 7.0 replaced it with listpack (more compact, same concept). The thresholds are controlled by list-max-listpack-size and list-max-ziplist-size config.
A quicklist is a doubly-linked list where each node contains a listpack (a compact byte array). This gives you the cache efficiency of contiguous memory for small sublists and the O(1) push/pop of a linked list.
The listpack encoding stores field-value pairs sequentially in a byte array. For a small user profile object (< 128 fields), Redis uses essentially zero pointer overhead. The moment you add the 129th field — or a field with a value longer than 64 bytes — the entire Hash is converted to a hashtable with pointer-based buckets.
intset is a sorted array of integers stored in minimal bytes (2, 4, or 8 bytes per element depending on range). It is remarkably compact and fast to search (binary search). The moment you add a non-integer member, the entire set converts to listpack or hashtable.
The skiplist encoding uses a probabilistic skip list for the ordered-by-score structure and a hashtable for O(1) score lookup by member. Both structures point to the same member/score pairs in memory — the memory cost is one set of data with two indexes.
The listpack encoding is a sorted byte array: fast for small sets (no pointer overhead), but linear-time lookup. This is fine for ≤ 128 members.
The default thresholds are good starting points, but they are not sacred. If your use case has objects that always stay small, you can raise the listpack thresholds to keep more objects in the compact encoding:
# In redis.conf hash-max-listpack-entries 256 # was 128 hash-max-listpack-value 128 # was 64
The trade-off: larger listpack means more scanning per operation (linear time within the listpack). For objects you always HGETALL (read all fields), a larger listpack is fine. For objects where you HGET specific fields frequently, raising the threshold too high can make individual field lookups slower.
In practice: profile with OBJECT ENCODING and OBJECT IDLETIME. If your hashes are at hashtable encoding with only 10 fields, your encoding threshold is too low.
Lists — ordered sequences with head/tail access. Use for queues (RPUSH/BLPOP), stacks (LPUSH/LPOP), activity feeds (LPUSH + LTRIM). LMOVE for reliable queues.
Hashes — field-value maps. Use for entity objects (user profiles, sessions). HINCRBY for atomic per-field counters. Prefer over JSON strings when you update individual fields.
Sets — unordered unique collections. Use for uniqueness tracking, social graphs (SINTER for mutual follows), deduplication. Set operations (SINTER/SUNION/SDIFF) are atomic.
Sorted Sets — unique members ordered by float score. Use for leaderboards (ZINCRBY + ZREVRANK), time-series indexes (timestamp as score), priority queues (ZPOPMIN), rate limiters.
Encodings — all four types start in a compact listpack/intset encoding and upgrade to pointer-based structures when they exceed configured thresholds. Know your thresholds; inspect with OBJECT ENCODING.
Next: F-4 — Redis's Single-Threaded Event Loop — Speed, Limits, and Multithreading — how the event loop handles 100K+ RPS, what Redis 6+ multithreading parallelises, and the one failure mode that brings a Redis instance to its knees.
Knowledge Check
A team needs to build a reliable background job processing system. A worker process must claim a job, process it, and acknowledge completion. If the worker crashes mid-processing, the job must not be lost. Which combination of Redis List commands provides this reliability guarantee without race conditions?
You are tasked with building a real-time global leaderboard for a multiplayer game. The leaderboard must rank millions of players by their score (which updates constantly), allowing you to quickly fetch the top 10 players and query an individual player's current rank. Which Redis data structure is designed precisely for this use case?
An engineer creates a Redis Hash to store user profile data using HSET user:123 name "Alice" role "Admin". They then notice via OBJECT ENCODING user:123 that the Hash is encoded internally as a listpack. They add a new field: HSET user:123 bio "..." where the biography string is 5,000 bytes long. What happens to the internal encoding of this Hash?
Test your knowledge with more question sets
Sign in to access a wider variety of questions and get notified when new practice sets are added to this module.