Module A-20·11 min read

Native fetch socket pool mechanics, Web Crypto vs libuv thread pool, and profiling ReadableStream memory leaks under RPC load.

JJS

Written by Jatin Jain Saraf · Senior Software Engineer

Module 19 — The Modern Web Standard Shift: Undici, fetch, and Web Crypto

Q: When using Node.js's native `fetch` (built on Undici), what is a critical consequence of failing to consume the response body (e.g., throwing an error before reading `response.json()` or `response.text()`)?

The unconsumed `ReadableStream` holds the underlying TCP connection open, permanently leaking it and preventing it from returning to the connection pool. — Every `fetch` response contains a `ReadableStream`. If this stream is not fully consumed (or explicitly cancelled via `response.body.cancel()`), Undici cannot safely return the underlying socket to the connection pool. This leads to a connection leak, eventually exhausting the pool and bringing down the service. Always consume the body, even on error paths.

Q: For maximum possible HTTP throughput when making internal service-to-service calls, which approach provides the best performance in modern Node.js?

Utilizing `undici.Pool.request` directly, which exposes the underlying engine's full performance without the overhead of the `fetch` API's cross-browser compatibility layer. — While native `fetch` is built on Undici, the `fetch` standard itself requires extra specification compliance checks (like CORS, redirect handling, and stream wrapping) which adds overhead. By using `undici.Pool.request` directly, you bypass this compatibility layer, achieving significantly higher requests-per-second for raw server-to-server communication.

Q: When performing cryptographic operations at very high throughput (e.g., 50,000 tx/sec), what is the main advantage of using `crypto.createHash('sha256')` over asynchronous APIs like `crypto.pbkdf2`?

It is a synchronous operation that runs extremely fast in V8 without consuming slots in the limited libuv thread pool and avoids the overhead of Promise scheduling (`await`). — Operations like `crypto.pbkdf2` and `crypto.scrypt` are computationally expensive and therefore offloaded to the libuv thread pool (which defaults to 4 threads). If you have high throughput, you will exhaust this pool quickly. `crypto.createHash` is extremely fast and executes synchronously on the main thread, avoiding thread pool contention and the micro-latency of Promise `await` ticks.

What this module covers: Node.js 18 shipped native fetch — built on Undici, a from-scratch HTTP/1.1 and HTTP/2 client that replaced the legacy http.request internals. The connection pool model changed. Garbage collection behavior changed. Performance characteristics changed. Engineers who understand http.request semantics may be surprised by subtle differences in how fetch handles keep-alive, connection reuse, and stream consumption. This module covers Undici's architecture, how to avoid ReadableStream memory leaks when consuming large RPC responses, and the Web Crypto API's standard, Promise-native surface for cryptographic operations — which still runs on the same libuv thread pool as legacy crypto, and where that matters for hot-path performance.

Undici: The New HTTP Client Foundation

undici is the HTTP client library underlying Node.js's native fetch. It was written from scratch (unlike the legacy http module which dates to 2009) with modern HTTP semantics, connection pooling, and proper backpressure.

Connection Pool Architecture

Unlike http.request which uses a per-hostname keep-alive agent, Undici uses a Pool or Client with configurable connection counts. Undici's pool is a fleet of taxis outside the RPC node's building: fetch grabs whatever cab is idling, while Pool.request lets you pre-book exactly how many cabs wait there and how long each engine idles between fares.

Client and Pool solve different problems. A Client manages exactly one connection to a single origin — use it when you're deliberately talking to one specific backend instance and want to reason about a single socket's lifecycle (e.g. a sticky connection to one RPC node replica). A Pool manages multiple connections to a single origin and load-balances requests across them — this is what you want for the general case of calling one RPC service at high concurrency, which is why the rest of this module uses Pool.

javascript

import { Pool, Client, fetch } from 'undici';

// Client: a single connection to one specific host — useful when you need to
// pin requests to one backend instance rather than round-robin across a pool
const client = new Client('https://blockchain-rpc-replica-3.internal', {
  keepAliveTimeout: 60_000,
  connect: { timeout: 5_000 },
});

async function getFromPinnedReplica(path) {
  const { statusCode, body } = await client.request({ path, method: 'GET' });
  if (statusCode !== 200) throw new Error(`RPC error: ${statusCode}`);
  return body.json();
}

// Explicit pool: control connection count to a specific host
const pool = new Pool('https://blockchain-rpc.internal', {
  connections: 20,             // max simultaneous connections
  pipelining: 10,              // pipeline up to 10 requests per connection
  keepAliveTimeout: 60_000,    // keep connections alive 60s
  connect: {
    timeout: 5_000,            // connection timeout
    keepAlive: true,
  },
});

// High-throughput RPC calls over the pool
async function getRpcBlock(height) {
  const { statusCode, body } = await pool.request({
    path: `/api/v1/blocks/${height}`,
    method: 'GET',
    headers: { 'Authorization': `Bearer ${RPC_TOKEN}` },
  });

  if (statusCode !== 200) {
    throw new Error(`RPC error: ${statusCode}`);
  }

  // IMPORTANT: always consume the body
  return body.json();
}

A caveat on pipelining: 10: HTTP/1.1 pipelining sends multiple requests on a connection before earlier responses arrive, but the responses must still come back in order. Pipelining assumes every queued request will succeed and that the connection never needs a mid-stream fallback. If one pipelined request fails or the connection drops partway through, Undici cannot safely retry or reorder the requests still queued behind it on that connection — they fail too, even though nothing was wrong with them individually. This is why pipelining is not a free 10x: it trades resilience under partial failure for lower per-request overhead, and it's a poor fit for RPC endpoints with any tail latency variance or occasional 5xx responses. Most production Undici pools leave pipelining at the default of 1 (no pipelining) and get their concurrency from connections instead — reach for pipelining only when the upstream is known to be reliable and strictly FIFO-safe.

`ReadableStream` Memory Leaks: The Critical Mistake

Every fetch response (and Undici response) has a body that is a ReadableStream. If you don't consume the body, the connection is not released back to the pool:

Production story: during a UPI transaction spike, a service's RPC calls started timing out intermittently, and the on-call engineer's first theory was that the upstream RPC provider was down. It wasn't. A health-check probe running every few seconds called fetch(rpcHealthUrl), and on any non-200 response threw immediately without draining the body — exactly the LEAK pattern below. Each failed health check silently held one connection open. The pool was configured for 20 connections; within a few minutes of the RPC node returning intermittent 503s under load, every connection in the pool was pinned to an unconsumed body, and legitimate transaction-verification calls queued behind health checks that would never release their socket. The fix was the same body-draining discipline shown below — but the diagnosis cost 40 minutes because the symptom (RPC timeouts) pointed everyone at the RPC provider instead of the caller's own connection pool.

javascript

// LEAK: body not consumed — connection held indefinitely
async function checkHealth(url) {
  const response = await fetch(url);
  if (!response.ok) {
    // BUG: we throw here without reading the body
    // The response body stream is never consumed → connection never released
    throw new Error(`Health check failed: ${response.status}`);
  }
  // We only read body on success — failure path leaks the connection
  return response.json();
}

// FIX: always consume the body
async function checkHealth(url) {
  const response = await fetch(url);
  if (!response.ok) {
    // Consume the error body even if we don't need it
    await response.text();  // or response.body?.cancel()
    throw new Error(`Health check failed: ${response.status}`);
  }
  return response.json();
}

javascript

// Better: use a try-finally pattern to always clean up
async function safeFetch(url, options) {
  const response = await fetch(url, options);

  try {
    if (!response.ok) {
      throw new Error(`HTTP ${response.status}: ${await response.text()}`);
    }
    return await response.json();
  } catch (err) {
    // Ensure body is consumed even on error
    if (response.body && !response.bodyUsed) {
      await response.body.cancel();
    }
    throw err;
  }
}

Large Response Streaming

For blockchain RPC responses that return multi-MB block data, streaming avoids loading the entire response into memory:

javascript

// Stream a large block response instead of buffering it
async function streamLargeBlock(height) {
  const response = await fetch(`https://rpc.internal/block/${height}`);

  if (!response.ok) {
    await response.text();
    throw new Error(`RPC error: ${response.status}`);
  }

  // Process the response as a stream — never loads the full block into memory
  const reader = response.body.getReader();
  const parser = createStreamingParser();

  try {
    while (true) {
      const { done, value } = await reader.read();
      if (done) break;
      parser.push(Buffer.from(value));
    }
    return parser.finish();
  } finally {
    reader.releaseLock();
  }
}

Tuning `fetch`'s Own Connection Pool

The examples so far treat fetch and pool.request as if they were unrelated APIs — fetch "just works" while Pool is where the tuning knobs live. That's not quite right: fetch is itself backed by an Undici Agent (a collection of per-origin pools), and that agent is configurable via setGlobalDispatcher. Every fetch call in the process uses whatever dispatcher is currently set globally, unless overridden per-call.

javascript

import { Agent, setGlobalDispatcher } from 'undici';

// Configure the pool that ALL global fetch() calls use, process-wide
setGlobalDispatcher(new Agent({
  connections: 20,            // max connections per origin, same knob as Pool
  keepAliveTimeout: 60_000,   // matches the pool example above
  connect: { timeout: 5_000 },
}));

// This fetch call now runs through the tuned Agent — no code change needed
// at every call site, just at process startup
const response = await fetch('https://blockchain-rpc.internal/api/v1/blocks/12345');

javascript

// Per-call override: use a dedicated dispatcher for one specific request
// instead of (or in addition to) the global one
import { Agent } from 'undici';

const rpcAgent = new Agent({ connections: 20, pipelining: 1 });

const response = await fetch('https://blockchain-rpc.internal/api/v1/blocks/12345', {
  dispatcher: rpcAgent,
});

In short: fetch is not pool-less and Pool.request is not the only tunable path — they're two entry points into the same Undici dispatcher machinery. Reach for setGlobalDispatcher when you want every fetch call in the service to share consistent connection limits without threading a Pool instance through your codebase; reach for Pool/Client directly when you want fine-grained control (like getFromPinnedReplica above) over specific origins.

Request Timeouts with `AbortSignal.timeout()`

An RPC call to a degraded blockchain node can hang far longer than is useful — without an explicit timeout, fetch will wait on the underlying socket indefinitely (bounded only by OS-level TCP timeouts, which can be minutes). AbortSignal.timeout() gives you a one-line, GC-friendly way to bound any fetch call:

javascript

async function getRpcBlockWithTimeout(height) {
  try {
    const response = await fetch(`https://blockchain-rpc.internal/api/v1/blocks/${height}`, {
      // Aborts the request (and releases the connection) if no response
      // arrives within 3 seconds — critical for RPC nodes that hang under load
      signal: AbortSignal.timeout(3_000),
    });
    if (!response.ok) {
      await response.text();
      throw new Error(`RPC error: ${response.status}`);
    }
    return response.json();
  } catch (err) {
    if (err.name === 'TimeoutError') {
      // Distinguish "RPC node too slow" from other network failures
      throw new Error(`RPC node timed out fetching block ${height}`);
    }
    throw err;
  }
}

Pool.request and Client.request accept the same signal option, so the pattern is identical whether you're calling through global fetch, a custom Agent, or a Pool directly. Always set an explicit timeout on outbound RPC calls in a payment pipeline — an unbounded hang on one RPC node is indistinguishable from a full pool exhaustion until you've waited long enough to notice.

Keep-Alive Timeout Mismatches: The Classic ECONNRESET War Story

A connection kept alive on the client side can be silently closed by the server (or an intermediate load balancer) while the client still believes it's usable. This happens when the client's keepAliveTimeout is longer than the server's: the server closes the idle socket after its own timeout, but the client, unaware, reuses the same socket for the next request a moment later — resulting in an ECONNRESET on a request that never had anything to do with the actual RPC call it was carrying.

javascript

// Client keeps sockets alive for 60s...
const pool = new Pool('https://blockchain-rpc.internal', {
  keepAliveTimeout: 60_000,
});

// ...but if the RPC node's own server (or an nginx/ALB in front of it) closes
// idle sockets after, say, 30s, there's a 30-second window where the client
// thinks the socket is alive and the server has already torn it down.
// The next request sent on that socket gets ECONNRESET — not because
// anything is wrong with the request, but because of the timeout mismatch.

The fix is not to catch and retry ECONNRESET (though a retry-once-on-idempotent-GET policy is reasonable defense in depth) — it's to make the client's keepAliveTimeout shorter than the server's, so the client always closes and reopens idle sockets before the server has a chance to. If you don't control the server, set your client's keepAliveTimeout conservatively low (a few seconds under whatever the server/load balancer's documented idle timeout is) rather than assuming a large number is "safer."

Performance Profiling: `fetch` vs `http.request`

javascript

// Benchmark: native fetch vs undici.Pool.request vs http.request
// Test: 10,000 requests to a local server, keep-alive enabled

// Results (ops/sec on Node.js 22):
// http.request (legacy): 18,400 req/sec
// fetch (undici): 21,200 req/sec (+15%)
// undici Pool.request: 31,500 req/sec (+71%)
// undici Client (single connection, pipelined): 48,200 req/sec (+162%)

For maximum throughput between internal services: use undici.Pool directly instead of fetch. The fetch API adds overhead for cross-browser compatibility. Pool.request exposes Undici's full performance.

Web Crypto API: Same Thread Pool, Different Consumer Model

Node.js's crypto module routes expensive operations (PBKDF2, scrypt, key derivation) through the libuv thread pool via callback- or Promise-based APIs. The Web Crypto API (globalThis.crypto.subtle) is not a thread-pool-free alternative — under the hood, subtle.digest/sign/encrypt/deriveBits and similar operations are also dispatched to the libuv thread pool, through an internal CryptoJob mechanism, precisely so the expensive computation doesn't block the event loop. The difference from legacy crypto is the API surface (Promise-based Web Standards API, portable to browsers) and some internal scheduling details — not whether the thread pool is involved.

Web Crypto Still Uses the Thread Pool

javascript

// Legacy crypto (uses libuv thread pool for key derivation):
const key = await new Promise((resolve, reject) => {
  crypto.pbkdf2('password', 'salt', 100000, 32, 'sha256', (err, dk) => {
    if (err) reject(err); else resolve(dk);
  });
});
// → occupies one thread pool slot for ~100ms

// Web Crypto (also dispatches to the libuv thread pool via a CryptoJob —
// same underlying resource contention, different API surface):
const keyMaterial = await crypto.subtle.importKey(
  'raw',
  new TextEncoder().encode('password'),
  { name: 'PBKDF2' },
  false,
  ['deriveBits']
);
const derivedBits = await crypto.subtle.deriveBits(
  { name: 'PBKDF2', salt: new TextEncoder().encode('salt'), iterations: 100000, hash: 'SHA-256' },
  keyMaterial,
  256
);
// → Occupies a thread pool slot just like crypto.pbkdf2 — switching to Web Crypto
// does not reduce thread pool pressure, it just gives you the standard API surface

Web Crypto Operations (Thread Pool, Standard API Surface)

javascript

// SHA-256 hash via Web Crypto — dispatched to the libuv thread pool like other
// subtle.* operations; the benefit here is the standard, Promise-native API surface,
// not reduced thread pool usage
async function hashTransaction(txData) {
  const bytes = new TextEncoder().encode(JSON.stringify(txData));
  const hashBuffer = await crypto.subtle.digest('SHA-256', bytes);
  return Buffer.from(hashBuffer).toString('hex');
}

// HMAC signing — used for webhook signatures
async function signWebhookPayload(payload, secret) {
  const key = await crypto.subtle.importKey(
    'raw',
    new TextEncoder().encode(secret),
    { name: 'HMAC', hash: 'SHA-256' },
    false,
    ['sign']
  );
  const signature = await crypto.subtle.sign(
    'HMAC',
    key,
    new TextEncoder().encode(payload)
  );
  return Buffer.from(signature).toString('hex');
}

// AES-GCM encryption for PII in transit
async function encryptPII(data, keyBytes) {
  const key = await crypto.subtle.importKey(
    'raw', keyBytes, { name: 'AES-GCM' }, false, ['encrypt']
  );
  const iv = crypto.getRandomValues(new Uint8Array(12));
  const encrypted = await crypto.subtle.encrypt(
    { name: 'AES-GCM', iv },
    key,
    new TextEncoder().encode(data)
  );
  return { iv: Buffer.from(iv).toString('hex'), data: Buffer.from(encrypted).toString('hex') };
}

Impact on libuv Thread Pool

javascript

// Monitoring thread pool pressure when mixing legacy crypto and Web Crypto

// Legacy crypto operations that consume thread pool slots:
crypto.pbkdf2(...)          // always thread pool
crypto.scrypt(...)          // always thread pool
crypto.randomBytes(...)     // thread pool (use crypto.randomFillSync for small amounts)

// Web Crypto operations ALSO consume thread pool slots, via an internal CryptoJob:
crypto.subtle.digest(...)   // thread pool, dispatched as a CryptoJob
crypto.subtle.sign(...)     // thread pool
crypto.subtle.encrypt(...)  // thread pool
crypto.subtle.deriveBits(...) // thread pool

// Operations that do NOT use the thread pool (synchronous, run inline in V8):
crypto.createHash('sha256').update(data).digest()  // synchronous, fast
crypto.createHmac('sha256', key).update(data).digest()  // synchronous
crypto.randomUUID()         // synchronous, Web Crypto
crypto.getRandomValues(...) // synchronous for typical buffer sizes

// At 50K tx/sec requiring SHA-256: use synchronous crypto.createHash.
// Not async crypto.subtle.digest — it queues onto the same limited thread pool
// (default size 4) as pbkdf2/scrypt, and adds Promise/await scheduling overhead
// on top. crypto.subtle is the right choice when you need the Web Standards API
// surface (e.g. portable code, AES-GCM, HMAC key objects) — not when you need
// the fastest possible hash on a hot path.
const hash = crypto.createHash('sha256').update(txData).digest('hex');

Summary

Concept	Key Takeaway
Undici	Underlying `fetch` in Node.js 18+. Modern connection pool. Better throughput than legacy `http`.
`Pool.request`	Direct Undici API, 71% faster than `fetch`. Use for inter-service HTTP at high throughput.
Body consumption	Always consume the response body. Unconsumed bodies hold connections in the pool forever.
`response.body.cancel()`	Explicitly release the connection without reading the body. Use in error paths.
Web Crypto vs `crypto`	Web Crypto (`subtle.*`) offloads to the same libuv thread pool as legacy `crypto` via internal CryptoJobs — it's a Promise-native Web Standards API, not a thread-pool-free path.
Sync `crypto.createHash`	For hot paths: synchronous SHA-256 via `crypto.createHash` is fastest. No await overhead.
`crypto.randomUUID()`	Synchronous, Web Standards, no thread pool. Best for request ID generation.

Next: Automated Post-Mortem Diagnostics: process.report →

Knowledge Check

When using Node.js's native fetch (built on Undici), what is a critical consequence of failing to consume the response body (e.g., throwing an error before reading response.json() or response.text())?

For maximum possible HTTP throughput when making internal service-to-service calls, which approach provides the best performance in modern Node.js?

When performing cryptographic operations at very high throughput (e.g., 50,000 tx/sec), what is the main advantage of using crypto.createHash('sha256') over asynchronous APIs like crypto.pbkdf2?

Test your knowledge with more question sets

Sign in to keep reading

The rest of this module is free — sign in with Google to unlock it and track your progress.

PreviousModule A-19: Crossing the Boundary: Rust, N-API, and WASI for Cryptographic Throughput Next Module A-21: Automated Post-Mortem Diagnostics: process.report

Discussion

Join the discussion

Loading comments...

Module 19 — The Modern Web Standard Shift: Undici, fetch, and Web Crypto

Undici: The New HTTP Client Foundation

Connection Pool Architecture

ReadableStream Memory Leaks: The Critical Mistake

Sign in to keep reading

Discussion

`ReadableStream` Memory Leaks: The Critical Mistake