Module F-1·20 min read

The problem Redis solves vs relational databases, the in-memory model, Redis vs Memcached, installing Redis, and your first key-value operations — zero prior knowledge assumed.

JJS

Written by Jatin Jain Saraf · Senior Software Engineer

F-1 — What Is Redis and Why Does It Exist?

Q: A backend application uses PostgreSQL to load a user's dashboard, taking about 15 milliseconds per request. As traffic grows, the database CPU maxes out. The engineering team proposes adding Redis as a caching layer. What is the fundamental hardware reason why fetching this data from Redis will be significantly faster and offload the database?

Redis stores its entire dataset in RAM (memory), where read speeds are measured in nanoseconds, whereas PostgreSQL primary storage is on disk, where read speeds are measured in microseconds or milliseconds. — The massive performance advantage of Redis fundamentally comes down to hardware physics. Disk I/O (even with fast NVMe SSDs) requires moving data from storage to memory, taking tens to hundreds of microseconds. RAM access takes roughly 100 nanoseconds—about 1,000 times faster. Because Redis serves all reads and writes directly from RAM, it skips the disk entirely on the hot path, achieving sub-millisecond latencies and enormous throughput that disk-based databases cannot match for simple lookups.

Q: When making an architectural choice between Redis and Memcached for a new project, which of the following best describes why modern engineering teams overwhelmingly choose Redis?

Memcached only supports string values and lacks persistence, whereas Redis supports rich data structures (Hashes, Lists, Sets), pub/sub messaging, and optional on-disk persistence. — While Memcached is an excellent, simple, and multi-threaded key-value store, its feature set stops at caching simple strings. Redis has become the industry standard because it acts as a "data structure server." By offering Lists, Sets, Sorted Sets, Hashes, and Streams, Redis allows developers to build complex atomic operations (like leaderboards or rate limiters) directly in the cache. Additionally, Redis offers persistence (saving to disk) and built-in clustering, making it far more versatile for use cases beyond pure, volatile caching.

Q: A junior developer needs to find all keys in a production Redis database related to "user sessions" to debug an issue. They open the `redis-cli` and plan to execute `KEYS session:user:*`. A senior engineer stops them and tells them to use `SCAN` instead. Why is `KEYS *` dangerous in production?

Redis is single-threaded; `KEYS *` blocks the entire server while iterating over the entire dataset, causing all other application requests to time out until the scan finishes. — This is one of the most critical operational rules of Redis. Because the core command execution in Redis is single-threaded, it can only process one command at a time. The `KEYS` command searches the entire keyspace in one unbroken operation. On a database with millions of keys, this can take seconds. During those seconds, every other `GET` or `SET` command from your application is queued and blocked, resulting in massive latency spikes or application outages. `SCAN`, conversely, uses a cursor to iterate through the keyspace in small batches, allowing other commands to execute in between batches.

Who this module is for: You have heard of Redis — maybe you have even used it to "add caching" to a project — but you have never understood what it actually is under the hood. You might be a backend engineer who copies the redis.set() and redis.get() pattern without understanding why it is fast, what guarantees it provides, or when it is the wrong tool. This module assumes nothing. By the end, you will have a working Redis installation, a precise mental model of what Redis is and is not, and your first real key-value operations running against a live server.

The Problem Redis Was Built to Solve

To understand Redis, you need to first understand the bottleneck it eliminates.

Imagine a typical web application. A user requests their profile page. Your application server receives the request and runs a query against PostgreSQL:

sql

SELECT users.*, preferences.*, subscription_tier
FROM users
JOIN preferences ON preferences.user_id = users.id
JOIN subscriptions ON subscriptions.user_id = users.id
WHERE users.id = $1;

PostgreSQL runs this query. It reads pages from disk (or from its buffer cache), executes the join, and returns the result. On a well-tuned database with proper indexes, this might take 2–5 milliseconds.

That sounds fast. But now multiply it:

10,000 users make requests per second
Each request runs this query (and usually several more)
Your database is now handling 50,000–100,000 queries per second
Each query consumes a connection, CPU, I/O, and memory

The database becomes the bottleneck. You scale it vertically (bigger machine), then horizontally (read replicas), then you hit the wall of what SQL databases are designed to do: store data reliably, enforce constraints, execute complex queries — not answer the same simple lookup 100,000 times per second.

The problem Redis solves: answering simple, repeated data lookups at a speed and scale that disk-based databases cannot match.

The core insight is obvious once stated: if you have already computed the answer to a question and the answer has not changed, do not compute it again. Store it somewhere faster than your database and return it instantly.

Redis is that somewhere faster.

What Makes Redis Different: The In-Memory Model

Every database you have likely worked with — PostgreSQL, MySQL, MongoDB, SQLite — stores its primary data on disk. Disk is durable (data survives a power outage) but slow relative to RAM. Even with SSDs, a disk read involves seeking to the right location and waiting for the storage controller. With spinning hard drives, this is measured in milliseconds. With NVMe SSDs, it is measured in microseconds.

RAM is different. A RAM read is measured in nanoseconds — roughly 100 nanoseconds vs 100 microseconds for SSD, which is 1,000x faster.

Redis stores its entire dataset in RAM. When you set a key, it goes into memory. When you get a key, it comes from memory. There is no disk I/O on the hot path for reads or writes.

This is what gives Redis its speed. A well-configured Redis instance on modest hardware can handle 1 million read operations per second with single-digit millisecond latency. That is not an exaggeration from a benchmark. That is what engineers encounter in production.

The trade-off you accept: RAM is volatile. If the Redis process crashes or the machine loses power, data stored only in memory is gone. Redis has mechanisms to address this (RDB snapshots, AOF logging — covered in P-1 and P-2), but you must understand the default: without persistence configured, Redis loses data on restart.

This is not a bug. It is a design decision. For pure caching — where your primary database is the source of truth — losing the cache on restart is acceptable. You simply warm the cache again. For other use cases (as a message broker, a session store, a rate limiter), you need persistence, and Redis supports it.

Redis vs Memcached: The Question Engineers Ask First

If you have researched caching before, you have encountered Memcached. Both are in-memory key-value stores. The question "which one?" comes up in every engineering team. Here is the honest comparison:

Memcached gets right:

Pure simplicity. It does one thing: store and retrieve values by key. No data structures, no persistence, no pub/sub, no scripting.
Multi-threaded by design, which can utilize multiple CPU cores more efficiently for pure throughput.
Slightly lower memory overhead per key due to simpler internals.

Where Memcached stops:

The value is always a string. There are no lists, sets, sorted sets, or hashes. You cannot atomically increment a counter, push to a list, or add to a set.
No persistence. Data is gone on restart, period.
No pub/sub. No streams. No scripting.
No built-in replication or clustering (community solutions exist but are not first-class).
No transactions or atomic multi-key operations.

What Redis adds:

Rich data structures (Strings, Hashes, Lists, Sets, Sorted Sets, HyperLogLogs, Bitmaps, Streams, Geospatial indexes).
Optional persistence (RDB snapshots, AOF).
Pub/Sub messaging.
Lua scripting and Redis Functions for atomic compound operations.
Transactions (MULTI/EXEC).
Built-in replication, Sentinel (automatic failover), and Cluster (horizontal sharding).

The practical answer in 2025: Choose Redis. The days of "Memcached for simple caching, Redis for everything else" are largely over. Redis has caught up to and surpassed Memcached's throughput in most real-world workloads, and its additional capabilities mean you get more done with one infrastructure component. Unless you have a specific, measured reason to prefer Memcached (a very old codebase already using it, or a specific multi-threaded throughput requirement), Redis is the default choice.

The Redis Data Model: A Key-Value Store That Is More Than Key-Value

The simplest description of Redis is a key-value store: you associate a key (a string) with a value, and you retrieve the value by key.

text

SET user:1001:name "Jatin Jain Saraf"
GET user:1001:name
→ "Jatin Jain Saraf"

But calling Redis a "key-value store" is like calling PostgreSQL a "file storage system." It is technically accurate and practically misleading.

The key is always a string (up to 512 MB, though you should use much shorter keys in practice). The value can be one of several types, and the type determines what operations you can perform:

Type	What It Stores	Example Use Case
String	Text, integers, binary data	Cache a JSON blob, atomic counter
Hash	Field → value map	User profile object
List	Ordered sequence of strings	Activity feed, task queue
Set	Unordered collection of unique strings	Unique visitors, tags
Sorted Set	Set with a float score per member	Leaderboard, rate limiter
HyperLogLog	Probabilistic cardinality estimator	Unique page views at scale
Bitmap	Bit array indexed by offset	Feature flags, presence tracking
Stream	Append-only log with consumer groups	Event sourcing, message queue
Geospatial	Coordinates with radius queries	Location search

Each type comes with its own set of commands. A String supports SET, GET, INCR, APPEND. A Sorted Set supports ZADD, ZRANGE, ZRANGEBYSCORE, ZRANK. You cannot run ZADD on a String key — Redis will return a type error.

This matters enormously in practice. When you reach for the right data structure, Redis gives you atomic operations that would be impossible to implement safely with simple key-value pairs. A leaderboard with ZADD and ZRANK is atomic. A leaderboard implemented with GET + application-side sort + SET is not.

The Client-Server Model

Redis operates as a server process, listening for connections on a port (default: 6379). Your application — or the redis-cli command line tool — is the client. It connects to the Redis server, sends commands, and receives responses.

The protocol Redis uses for this communication is called RESP (REdis Serialization Protocol). It is a simple, text-based protocol designed to be easy to implement and fast to parse. You will rarely need to think about RESP directly — client libraries (ioredis for Node.js, redis-py for Python, Jedis/Lettuce for Java) handle it for you — but understanding that it exists explains why Redis is network-bound and why latency matters.

When your application calls redis.get('key'), here is what happens:

The client library serializes the command into RESP format
The serialized bytes travel over the network (or a Unix socket if local) to the Redis server
Redis receives the bytes, deserializes the command, and looks up the key in memory
Redis serializes the result and sends it back
The client library deserializes the response and returns it to your code

The actual memory lookup in step 3 takes nanoseconds. The network round-trip in steps 2 and 4 takes microseconds to milliseconds depending on whether Redis is local or remote.

This is why network architecture matters for Redis. Running Redis on the same machine as your application (communicating over a Unix socket) gives you sub-millisecond latency. Running Redis over a network connection from a different region gives you latency measured in tens of milliseconds, which defeats much of the purpose.

Installing Redis

macOS (Homebrew):

bash

brew install redis
brew services start redis   # start as a background service
redis-cli ping              # should return PONG

Ubuntu / Debian:

bash

sudo apt update
sudo apt install redis-server
sudo systemctl start redis-server
sudo systemctl enable redis-server
redis-cli ping

Docker (recommended for local development — no installation required):

bash

docker run -d --name redis -p 6379:6379 redis:7-alpine
redis-cli -h 127.0.0.1 -p 6379 ping

The Docker approach is what most teams use for local development because it gives you a clean, disposable Redis instance on any OS, and you can specify the exact Redis version.

Verify the installation:

bash

redis-cli ping
# PONG
redis-cli --version
# redis-cli 7.x.x

Your First Session with redis-cli

redis-cli is the official Redis command-line interface. Open a terminal and connect to your local Redis instance:

bash

redis-cli

You will see a prompt:

127.0.0.1:6379>

This tells you the IP and port you are connected to. Now let us run through the fundamental operations.

Setting and getting a key:

text

127.0.0.1:6379> SET greeting "Hello, Redis"
OK
127.0.0.1:6379> GET greeting
"Hello, Redis"

SET returns OK on success. GET returns the value, or (nil) if the key does not exist.

Key expiry — setting a TTL:

text

127.0.0.1:6379> SET session:user:1001 "eyJhbGci..." EX 3600
OK
127.0.0.1:6379> TTL session:user:1001
(integer) 3599
127.0.0.1:6379> PTTL session:user:1001
(integer) 3598847

EX 3600 sets the key to expire after 3600 seconds (1 hour). TTL returns the remaining time to live in seconds. PTTL returns it in milliseconds. Once the key expires, GET returns (nil) and TTL returns -2.

Checking if a key exists:

text

127.0.0.1:6379> EXISTS greeting
(integer) 1
127.0.0.1:6379> EXISTS nonexistent
(integer) 0

EXISTS returns 1 if the key exists, 0 if it does not. It accepts multiple keys: EXISTS key1 key2 key3 returns the count of keys that exist.

Deleting a key:

text

127.0.0.1:6379> DEL greeting
(integer) 1
127.0.0.1:6379> GET greeting
(nil)

DEL returns the number of keys actually deleted (so 0 if the key did not exist).

Atomic increment:

text

127.0.0.1:6379> SET page:views:homepage 0
OK
127.0.0.1:6379> INCR page:views:homepage
(integer) 1
127.0.0.1:6379> INCR page:views:homepage
(integer) 2
127.0.0.1:6379> INCRBY page:views:homepage 10
(integer) 12

INCR atomically increments an integer value by 1 and returns the new value. INCRBY increments by a specified amount. "Atomically" here means no two concurrent INCR operations on the same key can produce the same value — Redis processes them one at a time. This is a safe counter without a lock.

Inspecting key types:

text

127.0.0.1:6379> SET name "Jatin"
OK
127.0.0.1:6379> TYPE name
string
127.0.0.1:6379> LPUSH tasks "buy milk" "pay rent"
(integer) 2
127.0.0.1:6379> TYPE tasks
list

TYPE tells you what data structure is stored at a key. Useful for debugging unexpected type errors.

Scanning all keys (safely):

text

127.0.0.1:6379> SCAN 0 MATCH * COUNT 10
1) "0"
2) 1) "name"
   2) "tasks"
   3) "page:views:homepage"

SCAN is the safe alternative to KEYS *. KEYS * blocks Redis while it iterates through every key — dangerous on production instances with millions of keys. SCAN is cursor-based: it returns a cursor and a batch of results. When the cursor returns to 0, the full scan is complete. You will use SCAN in production. You will not use KEYS *.

Key Naming Conventions

Redis keys are arbitrary strings, but most teams adopt a convention to avoid collisions and make keys self-documenting:

{entity}:{id}:{field}

Examples:

text

user:1001:profile          → cached user profile object
user:1001:session          → session token
course:nodejs:module-count → module count for a course
rate:api:user:1001         → rate limit counter for user 1001
lock:scanner:block:44928   → distributed lock for block processing

Guidelines:

Use : as the separator (it is the most common convention)
Keep keys short — every key name is stored in memory
Be consistent — inconsistent naming makes debugging painful
Include the entity type first — makes SCAN filtering easier with patterns like SCAN 0 MATCH user:*

What Redis Is Not

Understanding the boundaries is as important as understanding the capabilities.

Redis is not a replacement for a relational database. It does not enforce schemas, foreign keys, or constraints. It cannot run joins or complex queries across multiple keys. It does not support SQL. Your PostgreSQL database remains the source of truth; Redis sits in front of it as a fast cache, alongside it as a message broker, or adjacent to it as a session store.

Redis is not infinitely durable by default. With default settings and no persistence configured, a Redis restart loses all data. This is intentional for pure cache use cases. If your data must survive a restart, you must configure RDB or AOF persistence (covered in P-1 and P-2).

Redis is not horizontally scalable by default. A single Redis instance is a single server with a single dataset in memory. Horizontal scaling requires Redis Cluster (covered in A-12 through A-15) and adds significant operational complexity.

Redis is not free of failure modes. The single-threaded event loop means one slow command blocks all other clients. Memory limits and eviction policies can silently drop data. Pub/Sub loses messages when subscribers disconnect. Distributed locking with Redis is genuinely hard to do correctly. These are not reasons to avoid Redis — they are reasons to understand it deeply, which is exactly what this course is for.

The Mental Model Going Forward

Here is the mental model you should carry through every module in this course:

Redis is a data structure server. It stores named data structures in RAM and provides atomic operations on them. Its speed comes from memory, its richness comes from its data types, and its complexity comes from the distributed systems problems that arise when you run it at scale.

Every module in this course adds a layer to this model:

Foundation: the data structures, memory layout, caching mechanics
Practitioner: persistence, messaging, atomic operations
Architect: scripting, distributed locking, high availability

You will never look at redis.set() and redis.get() the same way again.

Summary

Redis is an in-memory data structure server — not just a cache, not a database replacement
Its speed comes from RAM: 100–1,000x faster than SSD reads for simple lookups
The trade-off: RAM is volatile; without persistence configured, data is lost on restart
Redis wins over Memcached because of its rich data types, persistence options, pub/sub, scripting, and built-in HA
The client-server model means every operation pays a network round-trip; architecture placement matters
Install Redis with Docker for local development: docker run -d -p 6379:6379 redis:7-alpine
redis-cli gives you direct access: SET, GET, INCR, TTL, TYPE, SCAN are your first vocabulary
Key naming convention: {entity}:{id}:{field} — consistent, readable, filterable
Redis is not a database replacement — your primary database remains the source of truth

Next: F-2 — Strings, Numbers, and Binary Safety — where we go deeper into the String type, understand why Redis Strings are not what you think they are, and explore the full command surface for the most-used data type in Redis.

Knowledge Check

A backend application uses PostgreSQL to load a user's dashboard, taking about 15 milliseconds per request. As traffic grows, the database CPU maxes out. The engineering team proposes adding Redis as a caching layer. What is the fundamental hardware reason why fetching this data from Redis will be significantly faster and offload the database?

When making an architectural choice between Redis and Memcached for a new project, which of the following best describes why modern engineering teams overwhelmingly choose Redis?

A junior developer needs to find all keys in a production Redis database related to "user sessions" to debug an issue. They open the redis-cli and plan to execute KEYS session:user:*. A senior engineer stops them and tells them to use SCAN instead. Why is KEYS * dangerous in production?

Test your knowledge with more question sets

Sign in to keep reading

The rest of this module is free — sign in with Google to unlock it and track your progress.

Next Module F-2: Strings, Numbers, and Binary Safety

Discussion

Join the discussion

Loading comments...