Module F-8·18 min read

SUBSCRIBE, PUBLISH, PSUBSCRIBE for pattern channels, the no-persistence guarantee, keyspace notifications, horizontal scaling across app servers, and when to use Streams instead of Pub/Sub.

JJS

Written by Jatin Jain Saraf · Senior Software Engineer

F-8 — Pub/Sub and the Message Fanout Model

Q: A developer is building a real-time chat application using Redis Pub/Sub. When a user sends a message, it is published to the `chat:global` channel. A user complains that when they momentarily lose their internet connection and reconnect a minute later, they do not see the messages that were sent while they were offline. Why does this happen, and what is the proper architectural fix?

Redis Pub/Sub is strictly fire-and-forget; it does not persist messages. If a subscriber is offline when a `PUBLISH` occurs, the message is permanently lost to them. The fix is to use Redis Streams or append messages to a List/Sorted Set alongside the Pub/Sub notification. — Redis Pub/Sub offers zero delivery guarantees and no message persistence. It is a live broadcasting mechanism. When a message is published, Redis immediately pushes it to all currently connected subscribers and then discards it. If a client is disconnected even for a millisecond when the `PUBLISH` command runs, they will never receive that message. For chat applications where history and missed messages matter, Redis Streams (which persist messages) or a hybrid approach (storing in a List/ZSet and notifying via Pub/Sub) must be used.

Q: In a Node.js microservice, you need to listen for global configuration updates via Pub/Sub while occasionally writing new configurations back to Redis. Why do you need to instantiate *two* separate Redis client connections in your code?

Once a Redis connection issues a `SUBSCRIBE` or `PSUBSCRIBE` command, it enters a dedicated "subscriber mode." In this mode, the connection is blocked from issuing standard commands like `GET`, `SET`, or `PUBLISH`. — By design, the Redis protocol dictates that a client connection in subscriber mode can only be used to receive messages or manage subscriptions (using `SUBSCRIBE`, `UNSUBSCRIBE`, `PSUBSCRIBE`, `PUNSUBSCRIBE`, `PING`, or `QUIT`). It cannot be used to execute data manipulation commands. Therefore, any application that acts as both a publisher and a subscriber must maintain at least two distinct connection pools: one dedicated strictly to listening, and one for sending commands.

Q: An application relies on Redis Keyspace Notifications to trigger a background job whenever a caching key expires. The team enabled this by running `CONFIG SET notify-keyspace-events "Ex"`. Under heavy load, they notice some background jobs are never triggering, even though the keys are definitely expiring. Given the mechanics of Pub/Sub, what is the fundamental flaw in this architecture?

Keyspace notifications are dispatched via Pub/Sub, which is at-most-once delivery. If the worker service restarts, crashes, or suffers a brief network partition at the exact moment the key expires, the notification is lost forever and the job will never run. — Keyspace notifications are built directly on top of Redis Pub/Sub, meaning they inherit its "fire-and-forget" semantics. If your worker process goes offline (e.g., during a deployment) exactly when a key expires, the notification is published, no one receives it, and it disappears. Using key expiration as a trigger for critical, guaranteed background jobs is an anti-pattern. Critical jobs should be managed using durable queues (like Redis Streams, BullMQ, or delayed ZSets) where messages are persisted until acknowledged.

Who this module is for: You have heard of Redis Pub/Sub and may have used it for real-time notifications or chat. But you have not understood its most important limitation — messages are not persisted and any subscriber that is offline loses them permanently. This module covers the full Pub/Sub model, pattern-based subscriptions, its correct use cases, and when to use Streams instead.

What Pub/Sub Is

Redis Pub/Sub is a message fanout system. Publishers send messages to named channels. Subscribers listening on those channels receive every message in real time. Redis acts as the broker — it receives the message from the publisher and immediately delivers it to all connected subscribers.

text

Publisher → PUBLISH notifications:user:1001 "new message"
                  ↓
Redis broker
                  ↓
Subscriber A ← receives "new message"
Subscriber B ← receives "new message"
Subscriber C ← receives "new message"

The key characteristic: delivery is best-effort and immediate. If no subscriber is connected when a message is published, the message is gone. If a subscriber disconnects and reconnects, it misses all messages published during the disconnection window.

Commands

Subscribing

SUBSCRIBE channel [channel ...]

Once a client issues SUBSCRIBE, it enters a special "subscriber mode." In this mode, the only commands it can issue are SUBSCRIBE, UNSUBSCRIBE, PSUBSCRIBE, PUNSUBSCRIBE, PING, and RESET. The connection is dedicated to receiving messages.

text

127.0.0.1:6379> SUBSCRIBE notifications:global
Reading messages... (press Ctrl-C to quit)
1) "subscribe"
2) "notifications:global"
3) (integer) 1          ← number of channels now subscribed to

Publishing

PUBLISH channel message

Returns the number of subscribers that received the message (0 if nobody is subscribed).

text

127.0.0.1:6379> PUBLISH notifications:global "server maintenance at 02:00 UTC"
(integer) 3   ← 3 subscribers received it

Unsubscribing

UNSUBSCRIBE [channel ...]   → unsubscribe from channels; no args = unsubscribe from all

Pattern Subscriptions

PSUBSCRIBE subscribes to channels matching a glob pattern:

text

PSUBSCRIBE pattern [pattern ...]
PUNSUBSCRIBE [pattern ...]

text

# Subscribe to all user-specific notification channels
PSUBSCRIBE notifications:user:*

# This will receive messages published to:
# notifications:user:1001
# notifications:user:1002
# notifications:user:99999
# ... any channel matching the pattern

Pattern messages include the matched channel name in the received message:

text

1) "pmessage"
2) "notifications:user:*"   ← the pattern that matched
3) "notifications:user:1001"   ← the actual channel
4) "your order has shipped"    ← the message body

Inspecting Active Pub/Sub State

text

PUBSUB CHANNELS [pattern]      → list all active channels (with at least 1 subscriber)
PUBSUB NUMSUB [channel ...]    → subscriber count per channel
PUBSUB NUMPAT                  → number of active pattern subscriptions
PUBSUB SHARDCHANNELS [pattern] → list active shard channels (for Redis Cluster, 7.0+)
PUBSUB SHARDNUMSUB [channel]   → subscriber count for shard channels

text

127.0.0.1:6379> PUBSUB CHANNELS
1) "notifications:global"
2) "events:trade"

127.0.0.1:6379> PUBSUB NUMSUB notifications:global events:trade
1) "notifications:global"
2) (integer) 3
3) "events:trade"
4) (integer) 1

Message Delivery Guarantees (and the Lack Thereof)

This is the most important thing to understand about Redis Pub/Sub:

There are no delivery guarantees.

If a subscriber is offline when a message is published → message lost
If a subscriber's connection drops mid-delivery → message lost (TCP retransmission handles byte-level loss, but Redis does not retry at the application level)
If the Redis server goes down → all pending messages lost
Messages are not stored anywhere — there is no way to replay past messages
A slow subscriber receiving messages faster than it can process them will have messages buffered in Redis's output buffer. If the buffer exceeds client-output-buffer-limit, Redis disconnects the client and the remaining buffered messages are lost.

PUBLISH returns immediately regardless of whether subscribers received the message.

This is fundamentally different from a message queue (BullMQ, RabbitMQ, Kafka). A queue persists messages until a consumer acknowledges them. Pub/Sub does not.

Building a Real-Time Notification System

Here is a typical Pub/Sub pattern in a Node.js backend:

typescript

import Redis from 'ioredis';

const publisher = new Redis();
const subscriber = new Redis();

// Subscriber connection: dedicated to receiving messages
subscriber.subscribe('notifications:global', 'system:alerts');

subscriber.on('message', (channel: string, message: string) => {
  console.log(`[${channel}] ${message}`);
  // Forward to WebSocket clients connected to this server
  broadcastToWebSocketClients(channel, message);
});

// Publisher: normal Redis connection, just calls PUBLISH
async function notifyGlobal(message: string) {
  await publisher.publish('notifications:global', message);
}

async function notifyUser(userId: string, message: string) {
  await publisher.publish(`notifications:user:${userId}`, message);
}

Why two separate connections? A subscribed connection is in subscriber mode — you cannot issue normal commands on it. You need a separate connection for everything else (including PUBLISH).

Pattern Subscription in Node.js

typescript

subscriber.psubscribe('notifications:user:*');

subscriber.on('pmessage', (pattern: string, channel: string, message: string) => {
  const userId = channel.split(':')[2];
  const userSocket = connectedUsers.get(userId);
  if (userSocket) {
    userSocket.send(message);
  }
});

Horizontal Scaling: One Subscriber Per Server

When you run multiple application servers behind a load balancer, each server has its own WebSocket clients. A message published on server A's Redis connection needs to reach WebSocket clients on servers B and C.

The solution is to have every application server subscribe to the relevant channels. When a message is published (by any server), Redis delivers it to all subscribers — including the subscribers on every application server:

text

WebSocket Client → Server A → PUBLISH channel msg → Redis
                                                        ↓
                           Server A subscriber ← msg (forwards to A's WS clients)
                           Server B subscriber ← msg (forwards to B's WS clients)
                           Server C subscriber ← msg (forwards to C's WS clients)

This is the correct pattern for scaling real-time features across application servers. Each server maintains its own subscriber connection per channel family.

Pub/Sub vs Streams

The question comes up constantly: "should I use Pub/Sub or Streams for this?" Here is the honest answer:

Concern	Pub/Sub	Streams
Message persistence	No — fire and forget	Yes — messages stored until explicitly deleted
Replay missed messages	No	Yes — consumers can read from any position
Consumer groups (competing consumers)	No	Yes — multiple consumers sharing a stream
Message acknowledgement	No	Yes — XACK
Offline consumers	Messages lost	Messages waiting when consumer reconnects
Delivery guarantee	At-most-once	At-least-once (with ACK)
Latency	Lowest possible	Slightly higher (storage + ACK overhead)
Complexity	Simple	Moderate

Use Pub/Sub when:

You need real-time fanout to many subscribers simultaneously
Missing messages during disconnection is acceptable (or impossible, e.g., the subscriber is always connected)
You are broadcasting to WebSocket/SSE clients where reconnecting clients will reload state anyway
Cache invalidation signals (you do not need to replay past invalidations)
Live activity feeds where new users see only new events, not history

Use Streams when:

Messages must not be lost
Consumers may go offline and must catch up on missed messages
You need multiple independent consumers processing the same stream (consumer groups)
You need durable event sourcing or audit logs
Job queues where every job must be processed exactly once

Streams are covered in depth in F-9.

Client Output Buffer Limits

Redis has a protection mechanism for slow Pub/Sub subscribers. If a subscriber cannot read messages fast enough, they accumulate in Redis's output buffer for that client connection. If the buffer exceeds the configured limit, Redis forcibly disconnects the slow subscriber.

Default configuration:

client-output-buffer-limit pubsub 32mb 8mb 60

This means: disconnect a Pub/Sub subscriber if their output buffer exceeds 32MB at any point, or if it stays above 8MB for more than 60 consecutive seconds.

This is a safety valve — a slow subscriber should not cause Redis to run out of memory buffering messages for them. In practice, if your subscriber is consistently hitting this limit, it means your publisher is producing faster than the subscriber can consume, and you should consider Streams with consumer groups instead.

Keyspace Notifications: Pub/Sub for Redis Events

Redis can publish internal events to Pub/Sub channels when keys are modified — when a key expires, is deleted, or is written. This feature is called keyspace notifications.

Enable it in redis.conf or at runtime:

CONFIG SET notify-keyspace-events "KEA"

The value is a combination of flags:

K — keyspace events (channel: __keyspace@{db}__:{key})
E — keyevent events (channel: __keyevent@{db}__:{event})
g — generic commands (DEL, EXPIRE, RENAME)
$ — String commands
l — List commands
z — Sorted Set commands
x — expired events
e — evicted events
A — alias for g$lszxet (all events)

text

# Subscribe to all key expiry events on db 0
SUBSCRIBE __keyevent@0__:expired

# You'll receive:
# 1) "message"
# 2) "__keyevent@0__:expired"
# 3) "session:user:1001"   ← the key that expired

Keyspace notifications are useful for:

Cache warming triggers (when a key expires, recompute and re-cache it)
Session expiry hooks (notify your app when a session token expires)
Monitoring and alerting (react when specific keys are deleted or modified)

Caution: Keyspace notifications add overhead to every Redis write operation. Enable them only for the event types you actually need, and disable them if you are not using them.

Summary

Pub/Sub is fire-and-forget fanout: messages are delivered to all connected subscribers instantly and not persisted
SUBSCRIBE puts a connection in subscriber-only mode; you need a separate connection for PUBLISH and other commands
PSUBSCRIBE subscribes to channels matching a glob pattern
There are no delivery guarantees — offline subscribers miss messages permanently
For real-time WebSocket/SSE fanout across app servers, have every server subscribe — Redis delivers to all
Use Streams when you need durability, replay, or consumer groups
Client output buffer limits (client-output-buffer-limit pubsub) protect Redis from slow subscribers
Keyspace notifications let you subscribe to Redis's internal events (key expiry, deletion, etc.)

Next: F-9 — Streams: Append-Only Logs and Consumer Groups — the durable alternative to Pub/Sub that gives you persistent message storage, consumer groups, and at-least-once delivery guarantees.

Knowledge Check

A developer is building a real-time chat application using Redis Pub/Sub. When a user sends a message, it is published to the chat:global channel. A user complains that when they momentarily lose their internet connection and reconnect a minute later, they do not see the messages that were sent while they were offline. Why does this happen, and what is the proper architectural fix?

In a Node.js microservice, you need to listen for global configuration updates via Pub/Sub while occasionally writing new configurations back to Redis. Why do you need to instantiate *two* separate Redis client connections in your code?

An application relies on Redis Keyspace Notifications to trigger a background job whenever a caching key expires. The team enabled this by running CONFIG SET notify-keyspace-events "Ex". Under heavy load, they notice some background jobs are never triggering, even though the keys are definitely expiring. Given the mechanics of Pub/Sub, what is the fundamental flaw in this architecture?

Test your knowledge with more question sets

Sign in to keep reading

The rest of this module is free — sign in with Google to unlock it and track your progress.

PreviousModule F-7: Pipelining and the RESP Protocol Next Module F-9: Streams: Append-Only Logs and Consumer Groups

Discussion

Join the discussion

Loading comments...