Node.js In-Depth
Module P-7·20 min read

Redis with ioredis, cache-aside pattern, Nodemailer, file uploads to S3/R2, outbound HTTP with fetch, and idempotent webhook processing.

Written by Jatin Jain Saraf · Architect of SupraScan

Module P-7 — Connecting External Services and Caching

What this module covers: Production APIs don't live in isolation. They send email, upload files to object storage, call third-party APIs, and cache expensive queries to keep response times fast. This module covers Redis with ioredis and the cache-aside pattern, sending transactional email with Nodemailer, uploading files to S3-compatible object storage, making outbound HTTP requests with the native fetch API, and processing webhooks idempotently. Each section gives you production-ready patterns, not just the happy path.

Redis and Caching with ioredis

Redis is an in-memory data store. The two primary use cases in a Node.js API are caching (store the result of an expensive query, serve it for subsequent requests) and session/token storage (store refresh tokens so they can be revoked).

npm install ioredis

Redis client singleton

// src/db/redis.ts
import Redis from 'ioredis';
import { env } from '../config/env.js';
 
const redis = new Redis(env.REDIS_URL, {
  maxRetriesPerRequest: 3,
  retryStrategy: (times) => Math.min(times * 50, 2000), // exponential backoff, max 2s
  enableReadyCheck: true,
  lazyConnect: false,
});
 
redis.on('connect', () => console.log('[Redis] Connected'));
redis.on('error', (err) => console.error('[Redis] Error:', err.message));
 
export default redis;

The Cache-Aside Pattern

The most common caching strategy: check the cache first, hit the database only on a miss, then populate the cache for next time.

// src/services/posts.service.ts
import redis from '../db/redis.js';
import * as postsRepo from '../repositories/posts.repository.js';
import { NotFoundError } from '../errors/AppError.js';
 
const CACHE_TTL = 300; // 5 minutes in seconds
 
export async function getPostById(id: number) {
  const cacheKey = `post:${id}`;
 
  // 1. Check cache
  const cached = await redis.get(cacheKey);
  if (cached) {
    return JSON.parse(cached);
  }
 
  // 2. Cache miss — query the database
  const post = await postsRepo.findById(id);
  if (!post) throw new NotFoundError('Post');
 
  // 3. Populate cache with TTL
  await redis.setex(cacheKey, CACHE_TTL, JSON.stringify(post));
 
  return post;
}

A cold cache under a traffic spike is like one ATM with a thousand people reading the same "temporarily out of service" sign at once and independently deciding to walk to the same bank branch — every one of those thousand requests treats the miss as their own personal errand.

Cache Stampede: When a Miss Multiplies Itself

The getPostById function above has a race: if post:42 expires (or was never cached) and a thousand requests for it land within the same few milliseconds, every single one sees a cache miss and every single one queries the database — for the same row, at the same time. A popular post going viral is exactly when this hurts most: the cache is supposed to protect the database from load, and the moment it's most needed is the moment it stops helping.

Three mitigations, cheapest first:

Jittered TTLs — don't let a batch of keys expire at the exact same instant (common when they were all populated by the same batch job or warm-up script):

const jitter = Math.floor(Math.random() * 30); // 0-30s
await redis.setex(cacheKey, CACHE_TTL + jitter, JSON.stringify(post));

Locking — the first request to see a miss acquires a short-lived lock and populates the cache; everyone else waits briefly and retries the cache read instead of all hitting the database at once:

export async function getPostById(id: number) {
  const cacheKey = `post:${id}`;
  const cached = await redis.get(cacheKey);
  if (cached) return JSON.parse(cached);
 
  const lockKey = `lock:${cacheKey}`;
  const gotLock = await redis.set(lockKey, '1', 'EX', 10, 'NX');
 
  if (!gotLock) {
    // Someone else is already populating this key — wait briefly and retry once
    await new Promise((r) => setTimeout(r, 100));
    const retried = await redis.get(cacheKey);
    if (retried) return JSON.parse(retried);
    // Still nothing — fall through and query anyway rather than fail the request
  }
 
  const post = await postsRepo.findById(id);
  if (!post) throw new NotFoundError('Post');
 
  await redis.setex(cacheKey, CACHE_TTL, JSON.stringify(post));
  if (gotLock) await redis.del(lockKey);
 
  return post;
}

Probabilistic early refresh — recompute the value slightly before it actually expires, with a probability that rises as the remaining TTL shrinks, so a background refresh usually wins the race instead of the whole crowd hitting an actual expiry at once. Best suited to expensive-to-recompute values (a report, an aggregation) rather than a single-row lookup.

Pick locking for hot single keys under a thundering herd, jittered TTLs as a cheap default for anything cached in batches, and probabilistic early refresh for expensive-to-recompute values where a few extra seconds of staleness beats a cold recompute.

Cache invalidation

The cache must be invalidated when data changes — stale cache is worse than no cache:

export async function updatePost(id: number, data: UpdatePostInput, userId: number) {
  const post = await postsRepo.findById(id);
  if (!post) throw new NotFoundError('Post');
  if (post.authorId !== userId) throw new ForbiddenError();
 
  const updated = await postsRepo.update(id, data);
 
  // Invalidate after successful write
  await redis.del(`post:${id}`);
 
  return updated;
}
 
export async function deletePost(id: number, userId: number) {
  const post = await postsRepo.findById(id);
  if (!post) throw new NotFoundError('Post');
  if (post.authorId !== userId) throw new ForbiddenError();
 
  await postsRepo.delete(id);
  await redis.del(`post:${id}`);
}

Caching list queries

Lists are harder — when a post is created, which lists are stale? The simplest approach: use short TTLs for lists and longer TTLs for individual records.

export async function listPublishedPosts(page: number, limit: number) {
  const cacheKey = `posts:published:${page}:${limit}`;
  const cached = await redis.get(cacheKey);
  if (cached) return JSON.parse(cached);
 
  const posts = await postsRepo.findPublished({ page, limit });
 
  // Short TTL for lists — 60 seconds is fine for most feeds
  await redis.setex(cacheKey, 60, JSON.stringify(posts));
 
  return posts;
}

Redis for refresh token storage

Using Redis instead of a Postgres table for refresh tokens — faster lookups, automatic expiry:

// src/repositories/tokens.repository.ts
import redis from '../db/redis.js';
 
const TOKEN_TTL = 7 * 24 * 60 * 60; // 7 days in seconds
 
export async function create(userId: number, token: string): Promise<void> {
  // Store with user ID for revoke-all-sessions capability
  await redis.setex(`refresh:${token}`, TOKEN_TTL, String(userId));
}
 
export async function findByToken(token: string): Promise<number | null> {
  const userId = await redis.get(`refresh:${token}`);
  return userId ? parseInt(userId) : null;
}
 
export async function deleteByToken(token: string): Promise<void> {
  await redis.del(`refresh:${token}`);
}
 
// Revoke all sessions for a user (requires a different data structure)
// See the Set-based approach in the Architect phase for multi-device logout

Sending Email with Nodemailer

Nodemailer is the standard Node.js library for sending email. In production you connect it to a transactional email provider (SendGrid, Resend, Postmark, SES) — never your own SMTP server.

npm install nodemailer
npm install -D @types/nodemailer

Email transport singleton

// src/services/email.service.ts
import nodemailer from 'nodemailer';
import { env } from '../config/env.js';
 
// Transactional email via SMTP (works with SendGrid, Resend, Mailgun, SES)
const transporter = nodemailer.createTransport({
  host: env.SMTP_HOST,
  port: env.SMTP_PORT,
  secure: env.SMTP_PORT === 465, // true for port 465, false for 587
  auth: {
    user: env.SMTP_USER,
    pass: env.SMTP_PASS,
  },
});
 
// For development — log emails to console instead of sending
const devTransport = nodemailer.createTransport({
  jsonTransport: true, // logs to console
});
 
const transport = env.NODE_ENV === 'production' ? transporter : devTransport;

Sign in to keep reading

The rest of this module is free — sign in with Google to unlock it and track your progress.

Sign in & Register
PreviousModule P-6: Configuration, Security Hardening, and Rate LimitingNext Module P-8: Logging, Observability Basics, and Deployment

Discussion

0

Join the discussion

Loading comments...
PortfolioBlog

© 2026 Jatin Jain Saraf (JJS). All rights reserved.