Why sequential redis.get() in a loop is the most common Redis performance mistake, what RESP looks like on the wire, how pipelining eliminates per-command RTT, and when to use MGET/MSET vs explicit pipelines.
Who this module is for: You use Redis in a loop — fetching a list of keys one by one, or updating a batch of records sequentially. Your Redis operations work, but you have not thought about what is happening at the network level. This module explains why that pattern is slow, what the RESP protocol looks like on the wire, and how pipelining eliminates the per-command network round-trip to make batch operations 10–100x faster.
The Network Round-Trip Problem
Every Redis command you issue pays a cost that has nothing to do with Redis itself: the time for bytes to travel from your application to the Redis server and back. This is the Round-Trip Time (RTT).
On a typical cloud setup — application and Redis in the same region, connected via a private network — RTT is 0.5–2ms. On a local machine (Redis and app on the same host via TCP loopback), it is < 0.1ms.
Now consider this pattern — common in codebases everywhere:
javascript
With 1,000 keys and 1ms RTT, this takes at minimum 1,000 × 1ms = 1 second. Redis itself processes each GET in microseconds. You are spending 99.9% of your time waiting for network round-trips.
This is the problem pipelining solves.
What the RESP Protocol Looks Like
Before diving into pipelining, it helps to understand the protocol Redis uses. RESP (REdis Serialization Protocol) is a text-based protocol designed to be simple to implement and fast to parse.
Every Redis command is serialized as a RESP Array. For SET mykey "hello":
text
The server's response for OK:
+OK\r\n ← Simple string
For a GET response returning "hello":
text
For (nil):
$-1\r\n ← Null bulk string (length -1)
For an integer (e.g., INCR returning 42):
:42\r\n ← Integer
For an error:
-ERR wrong number of arguments for 'get' command\r\n
Why this matters for pipelining: RESP is a streaming protocol. Redis reads commands from a socket buffer, processes them, and writes responses back. It does not require request-response pairing — you can send 100 commands before reading any response, and Redis will process them in order and write 100 responses in order.
What Pipelining Is
Pipelining is the technique of sending multiple commands to Redis in a single network write — without waiting for a response between each command. Redis processes them in order and sends all responses back together.
Without pipelining (naive):
text
With pipelining:
text
One network write, one network read, 1,000 results. RTT paid once instead of 1,000 times.
Pipelining in Node.js (ioredis)
javascript
The exec() call sends all queued commands to Redis in a single write and waits for all responses.
Benchmarking the difference:
javascript
Handling Errors in a Pipeline
Each command in a pipeline has its own result slot. An error in one command does not abort the pipeline — it is reported as an error in that slot's [error, result] pair:
javascript
Unlike transactions (covered in P-5), a pipeline error does not roll back other commands.
MGET and MSET: Built-in Batching
For the specific case of getting or setting multiple String keys, Redis provides MGET and MSET as atomic multi-key commands:
text
These send a single command (not a pipeline) and are slightly more efficient than a pipeline for pure String batches because there is only one command to parse. But they only work for String keys — for Hashes, Lists, or mixed types, you need pipelining.
javascript
Pipeline Size: How Many Commands to Batch
There is no fixed "correct" pipeline size. Larger pipelines reduce RTT cost but increase:
Memory for queued commands on the client side
Memory for pending responses on the Redis server side
Time before you can start processing results (you wait for the whole pipeline to finish)
Practical guidelines:
Small pipelines (10–100 commands): Fine for most operations — enrichment, batch reads, fan-out writes.
Medium pipelines (100–1,000 commands): The sweet spot for bulk data operations.
Large pipelines (1,000+ commands): Break into chunks. Sending 100,000 commands at once can overwhelm Redis's output buffer and cause latency spikes for other clients.
javascript
Pipelining vs Transactions vs Lua Scripts
These three are often confused because they all involve "sending multiple commands." Here is the distinction:
Feature
Pipelining
Transaction (MULTI/EXEC)
Lua Script
Atomic?
No
Yes (but no rollback)
Yes
Round-trips
1 (for the batch)
2 (MULTI + EXEC)
1
Can read-then-write atomically?
No
No (reads happen at queue time)
Yes
Error handling
Per-command
Per-command, rest continues
Script-level
Use case
Batch throughput
Preventing interleaving
Complex atomic logic
Pipelining is not atomic. Two pipelines from different clients can interleave. If you need the commands to execute without any other client's commands interleaving, use a transaction (MULTI/EXEC, covered in P-5) or a Lua script (covered in A-2).
Connection Pooling and Pipelining Together
A common misconception: "pipelining replaces connection pooling." It does not. They solve different problems.
Connection pooling — keeps multiple TCP connections open so multiple coroutines/threads can issue commands concurrently without waiting for a shared connection.
Pipelining — reduces RTT for sequential operations within a single connection.
In a Node.js application with ioredis:
javascript
ioredis's auto-pipelining batches commands that are issued in the same event loop tick into a single network write. You often get pipelining benefits without manually creating a pipeline — but explicit pipelines give you more control over batch size and error handling.
Monitoring Pipeline Performance
INFO commandstats
Shows per-command call counts and aggregate microseconds:
text
A high usec_per_call on individual commands with high call counts is the signal that you have sequential loops that should be pipelined.
SLOWLOG GET 10
Shows commands that took longer than slowlog-log-slower-than microseconds (default 10,000 = 10ms). Pipelines themselves are rarely the bottleneck — they appear as fast in the slow log because Redis processes each command quickly. What you want to avoid is many slow sequential single commands.
Summary
Every Redis command pays a network RTT — typically 0.5–2ms on cloud infrastructure
Sequential loops of await redis.get(key) pay N × RTT for N keys — often the biggest Redis performance mistake
Pipelining sends multiple commands in one network write, paying RTT once instead of N times
redis.pipeline() in ioredis: queue commands, then exec() to send all at once
Each pipeline command has its own result — errors in one command do not abort others
For pure String batches: MGET / MSET are atomic single-command alternatives
Next: F-8 — Pub/Sub and the Message Fanout Model — where we cover Redis's publish-subscribe system, how it differs from Streams, when to use it, and its most important limitation: messages are fire-and-forget.
Knowledge Check
An application needs to fetch the profiles of 500 users from Redis. The network round-trip time (RTT) between the application server and Redis is 2ms. If the developer uses a for loop with await redis.get(key) for each user, roughly how long will the operation take purely in network latency, and how long would it take if they used pipelining instead?
A developer creates a pipeline with three commands: a SET, an LPUSH targeting the same key (which will cause a type error), and another SET. They call await pipeline.exec(). How does Redis handle the execution and the return values?
While reviewing code to fetch 10,000 user profiles, a senior engineer suggests breaking the pipeline() batch into chunks of 500 commands rather than sending all 10,000 commands in a single pipeline execution. Why is chunking large pipelines important?
Test your knowledge with more question sets
Sign in to access a wider variety of questions and get notified when new practice sets are added to this module.
importRedisfrom'ioredis';const redis =newRedis();// Create a pipelineconst pipeline = redis.pipeline();// Queue commands — nothing is sent yetpipeline.get('user:1');pipeline.get('user:2');pipeline.set('counter',0);pipeline.incr('counter');// Execute all queued commands in one batchconst results =await pipeline.exec();// results: [[null, "value1"], [null, "value2"], [null, "OK"], [null, 1]]// Each element is [error, result]
// Naive sequential: ~1000ms for 1000 keys at 1ms RTTconst start1 =Date.now();for(let i =0; i <1000; i++){await redis.get(`user:${i}`);}console.log(`Sequential: ${Date.now()- start1}ms`);// Pipelined: ~2-5ms for 1000 keys (single round-trip)const start2 =Date.now();const pipe = redis.pipeline();for(let i =0; i <1000; i++){ pipe.get(`user:${i}`);}await pipe.exec();console.log(`Pipelined: ${Date.now()- start2}ms`);
const pipeline = redis.pipeline();pipeline.set('key1','value1');pipeline.lpush('key1','will-fail');// type error: key1 is a String, not a Listpipeline.set('key3','value3');const results =await pipeline.exec();// [// [null, 'OK'],// [ReplyError: WRONGTYPE Operation against a key..., null],// [null, 'OK']// ]
MGET key1 key2 key3 → returns array of values in order
MSET key1 v1 key2 v2 key3 v3 → sets all keys atomically
// MGET for bulk String readsconst values =await redis.mget('user:1','user:2','user:3');// ["value1", "value2", null] ← null for missing keys// MSET for bulk String writesawait redis.mset('key1','val1','key2','val2','key3','val3');
// Chunk large batchesasyncfunctionbatchGet(keys: string[], chunkSize =500){constresults:(string |null)[]=[];for(let i =0; i < keys.length; i += chunkSize){const chunk = keys.slice(i, i + chunkSize);const pipe = redis.pipeline(); chunk.forEach(key=> pipe.get(key));const chunkResults =await pipe.exec(); results.push(...chunkResults!.map(([, val])=> val as string |null));}return results;}
// ioredis automatically pipelines commands issued concurrently via Promise.all// This is called "auto-pipelining"const[user, session, config]=awaitPromise.all([ redis.get('user:1001'), redis.get('session:abc123'), redis.hgetall('config:global'),]);