Module A-8·22 min read

Why Express middleware chains collapse under extreme throughput and how Fastify's Radix tree router with compiled JSON Schema achieves 3x gains.

JJS

Written by Jatin Jain Saraf · Senior Software Engineer

Module 7 — Routing Engines at Scale: Vanilla HTTP vs Radix Tree Frameworks

Q: Why does Fastify's Radix tree routing provide significantly better performance than Express's routing at scale?

Radix tree routing matches based on URL prefixes in a way that depends on the path's length rather than the number of registered routes, avoiding the linear O(N) regular expression matching Express uses for every request. — Express evaluates routes linearly (O(N)) using regular expressions (`path-to-regexp`), meaning the routing cost grows linearly with the number of registered routes. Fastify uses a Radix tree (via `find-my-way`), making route lookup a function of the path's own length (its segment count) rather than the number of registered routes — effectively constant with respect to route count, and barely impacting CPU even with hundreds of routes.

Q: How does Fastify achieve 2-3x faster JSON serialization compared to standard `JSON.stringify()`?

By compiling response schemas at startup using `fast-json-stringify`, enabling it to generate optimized code that accesses object properties directly without inspecting their types at runtime. — When a response schema is provided, Fastify uses `fast-json-stringify` to compile an optimized serialization function at startup. This compiled function avoids the runtime type inspection required by `JSON.stringify`, leading to massively increased throughput.

Q: During a load test, you use `clinic flame` and notice a massive amount of CPU time is spent inside `path-to-regexp`. What is the most likely cause?

The server is running Express with many registered routes, and the route matching overhead is saturating the CPU. — `path-to-regexp` is the underlying library Express uses for route matching. A flamegraph showing extensive time spent here confirms that Express's linear, regex-based routing is the primary bottleneck, a typical issue in Express apps with many routes under high load.

What this module covers: Your ingestion endpoint receives 50,000 requests per second. Before your code runs, the framework has already spent CPU time parsing the URL, finding the matching route, running middleware, and deserializing the body. At high throughput, this overhead is measurable — sometimes it is the difference between handling your load and dropping requests. This module covers why Express's linear middleware scan fails under extreme concurrency, how Fastify's Radix tree router achieves route matching whose cost depends on path length, not on the number of registered routes, and how compiled JSON Schema validation eliminates per-request interpretation overhead.

The Overhead Before Your Code Runs

For a payment gateway receiving a POST to /api/v2/payments/process, the framework must:

Parse the URL (string split, decode percent-encoding)
Find the matching route handler (scan routes or traverse a tree)
Execute middleware chain (authentication, rate limiting, body parsing)
Deserialize the request body (JSON.parse)
Validate the payload (schema check)
Hand control to your handler

At 100 req/sec, steps 1–5 cost microseconds and are invisible. At 50,000 req/sec, they cost milliseconds that compound into measurable throughput limits. The framework is not neutral — it has a throughput ceiling determined by its internal architecture.

Express: Linear Scan Middleware Chain

Express's routing model is a linked list of middleware functions. Every incoming request walks this list sequentially until a matching handler is found.

javascript

// What Express does internally for each request:
app.use(cors());                     // check #1
app.use(helmet());                   // check #2
app.use(express.json());             // check #3
app.use(rateLimiter);                // check #4
app.post('/api/v1/auth', handler);   // check #5 — match!
app.post('/api/v1/users', handler);  // never reached for /auth
app.post('/api/v2/payments', handler);  // never reached
// ... 200 more routes

For a request to /api/v1/auth, Express checks: is this cors? Yes, run it. Is this helmet? Yes, run it. Is this json? Yes, parse the body. Is this rateLimiter? Yes, run it. Is the method POST and path /api/v1/auth? Yes — match.

The path-matching cost: Express uses path-to-regexp for route matching. For each route, it compiles the path pattern to a RegExp and tests the incoming URL against it. The test is O(N) in the number of routes.

With 200 routes and the matching route at position 180: every request triggers 180 RegExp tests. At 50,000 req/sec, that's 9 million RegExp executions per second — a measurable CPU load before any application logic runs.

javascript

// Measure Express routing overhead directly
const start = process.hrtime.bigint();
app.handle(mockRequest, mockResponse, () => {});
const routingNs = Number(process.hrtime.bigint() - start);
console.log(`Routing overhead: ${routingNs / 1000}μs`);

Typical Express routing overhead on a 100-route app: 15–60μs per request. At 50K req/sec: 750ms–3s of CPU per second just in routing. That's 75–300% of a single CPU core dedicated to route matching.

Radix Tree Routing: O(m) Route Matching, Independent of Route Count

The hotel-concierge analogy: Express checks every name on the guest list at the door, one by one; Fastify's radix tree is a hotel concierge who reads your reservation code once and walks straight to your floor, your hallway, your room.

Fastify uses find-my-way — a Radix tree (compressed trie) router. Instead of scanning routes sequentially, it traverses a tree where common path prefixes are compressed into single nodes.

text

Routes registered:
  POST /api/v1/auth
  POST /api/v1/users
  POST /api/v1/users/:id
  GET  /api/v2/payments
  POST /api/v2/payments/process
  GET  /api/v2/payments/:id

Radix tree structure:
  /api/
    v1/
      auth         → POST handler
      users        → POST handler
        /:id       → POST handler
    v2/
      payments     → GET handler
        /process   → POST handler
        /:id       → GET handler

Matching /api/v2/payments/process:

Does the URL start with /api/? Yes → descend
Does the next segment start with v1 or v2? v2 → descend
Does the next segment start with payments? Yes → descend
Is the remainder /process? Yes → exact match → return handler

4 string prefix comparisons, regardless of the total number of routes. Adding 100 more routes to a different branch (/admin/...) does not change the cost of matching /api/v2/payments/process. The cost of matching is a function of the path's own length (segment count) — roughly O(m) where m is the number of path segments — and is effectively constant with respect to K, the number of registered routes. Route count does not appear in the cost at all, so it is not "logarithmic in K" — it simply doesn't depend on K.

Fastify: Architecture for Throughput

Fastify's design reflects a single principle: minimize overhead on the hot path.

JSON Schema Compilation via `ajv`

Every time Express parses and validates a request body at runtime, it interprets the validation logic dynamically. Fastify pre-compiles JSON Schema into optimized validator functions at startup using ajv:

javascript

// Fastify with compiled schema validation
const fastify = Fastify({ logger: false });

// Schema is compiled ONCE at startup — not on every request
const paymentSchema = {
  type: 'object',
  required: ['amount', 'senderId', 'recipientId'],
  properties: {
    amount: { type: 'integer', minimum: 1, maximum: 1000000000 },
    senderId: { type: 'string', pattern: '^[A-Z0-9]{32}$' },
    recipientId: { type: 'string', pattern: '^[A-Z0-9]{32}$' },
    memo: { type: 'string', maxLength: 256 },
  },
  additionalProperties: false,
};

fastify.post('/api/v2/payments', {
  schema: {
    body: paymentSchema,
    response: {
      200: {
        type: 'object',
        properties: {
          transactionId: { type: 'string' },
          status: { type: 'string' },
        }
      }
    }
  }
}, async (request, reply) => {
  // By the time this runs:
  // - Route matched via Radix tree (O(m) on path length, independent of route count K)
  // - Body validated via compiled ajv function (no interpretation)
  // - request.body is type-safe and validated
  const payment = request.body;
  const result = await processPayment(payment);
  return result;  // serialized via fast-json-stringify (compiled)
});

What ajv compilation produces: instead of interpreting the schema on every request, ajv generates a JavaScript function like this:

javascript

// What ajv generates at startup (conceptually):
function validatePayment(data) {
  if (typeof data.amount !== 'number') return false;
  if (data.amount < 1 || data.amount > 1000000000) return false;
  if (typeof data.senderId !== 'string') return false;
  if (!/^[A-Z0-9]{32}$/.test(data.senderId)) return false;
  // ... etc
  return true;
}
// This runs 5-10x faster than interpreting the schema on every call

`fast-json-stringify`: Compiled Response Serialization

Standard JSON.stringify is generic — it inspects every key and value at runtime to determine how to serialize them. fast-json-stringify pre-compiles a response schema into a serialization function:

javascript

import fastJsonStringify from 'fast-json-stringify';

// Compiled ONCE at startup
const serializePaymentResponse = fastJsonStringify({
  type: 'object',
  properties: {
    transactionId: { type: 'string' },
    status: { type: 'string' },
    amount: { type: 'integer' },
    timestamp: { type: 'integer' },
  }
});

// Per-request: 2-3x faster than JSON.stringify
const responseBody = serializePaymentResponse({
  transactionId: 'TX123',
  status: 'accepted',
  amount: 5000,
  timestamp: Date.now(),
});

Fastify wires this automatically when you provide a response schema.

Benchmarking: The Actual Numbers

Using autocannon for load testing with 100 concurrent connections:

bash

# Install autocannon
npm install -g autocannon

# Test Express
autocannon -c 100 -d 10 -m POST \
  -H "Content-Type: application/json" \
  -b '{"amount":5000,"senderId":"ABCD1234ABCD1234ABCD1234ABCD1234","recipientId":"EFGH5678EFGH5678EFGH5678EFGH5678"}' \
  http://localhost:3000/api/v2/payments

# Test Fastify
autocannon -c 100 -d 10 -m POST \
  -H "Content-Type: application/json" \
  -b '{"amount":5000,"senderId":"ABCD1234ABCD1234ABCD1234ABCD1234","recipientId":"EFGH5678EFGH5678EFGH5678EFGH5678"}' \
  http://localhost:3001/api/v2/payments

Representative results on an 8-core server (handler does no I/O — pure routing/validation overhead):

Framework	Req/sec	Avg latency	P99 latency
Express (default)	18,400	5.4ms	14ms
Express (no middleware)	32,100	3.1ms	8ms
Fastify (no schema)	48,200	2.1ms	5ms
Fastify (compiled schema)	67,800	1.5ms	3ms
Vanilla `http` (no framework)	74,200	1.3ms	2.5ms

Fastify with compiled schemas is 3.7x faster than Express with typical middleware. For a 50K req/sec target: Express cannot reach it on 8 cores; Fastify can.

Beyond Fastify: HTTP/2 and `uWebSockets.js`

The table above tops out at vanilla http on HTTP/1.1. Two further options exist for teams that have exhausted Fastify's headroom and still need more:

HTTP/2 via @fastify/http2 — multiplexes many logical requests over a single TCP connection (no per-request handshake, header compression via HPACK), which mainly pays off for many small, concurrent requests over a persistent connection — closer to how payment terminals or full-node RPC clients behave than to a typical browser workload. It layers onto the same Fastify routing and schema-compilation model shown throughout this module, so the migration cost is mostly protocol/TLS configuration, not a rewrite.
uWebSockets.js — a from-scratch C++ HTTP/WebSocket implementation with its own (non-Fastify-compatible) API, routinely benchmarked at several times vanilla http's throughput. The tradeoff is real: no Fastify plugin ecosystem, no find-my-way radix router, no ajv/fast-json-stringify integration — you re-implement whatever of those you need yourself. It's the right call when routing and serialization overhead, not application logic, is confirmed (via clinic flame, below) to be the actual ceiling — not a default starting point.

For most ingestion services in this course's UPI/blockchain-indexer narrative, Fastify with compiled schemas is the right stopping point: the jump to HTTP/2 or uWebSockets.js trades meaningful ecosystem and maintainability cost for throughput most teams don't need until they've already saturated Fastify in production.

Fastify's Plugin Architecture: Encapsulation at Scale

For large applications with hundreds of routes, Fastify's plugin system provides scope isolation:

javascript

const fastify = Fastify();

// Each plugin is encapsulated — middleware registered inside
// only applies to routes inside that plugin
await fastify.register(async (ingestionPlugin) => {
  // Rate limiter only for ingestion routes
  ingestionPlugin.addHook('preHandler', rateLimiter);

  ingestionPlugin.post('/api/v2/payments', paymentSchema, paymentHandler);
  ingestionPlugin.post('/api/v2/transfers', transferSchema, transferHandler);

}, { prefix: '/ingestion' });

await fastify.register(async (adminPlugin) => {
  // Auth only for admin routes
  adminPlugin.addHook('preHandler', adminAuthenticator);

  adminPlugin.get('/admin/stats', statsHandler);
  adminPlugin.post('/admin/config', configHandler);

}, { prefix: '/admin' });

// Public routes: no middleware
fastify.get('/health', healthHandler);
fastify.get('/metrics', metricsHandler);

Each registered plugin creates a child scope. Hooks and decorators registered inside a plugin are invisible to routes in sibling plugins. This eliminates the "every request checks every middleware" problem of Express — middleware only runs for the routes that need it.

The Full Hook Lifecycle

The preHandler hook shown above is only one point in a longer, ordered lifecycle. Fastify runs hooks at fixed stages for every request, and understanding the full sequence is what makes the encapsulation model above actually useful — you choose where in the lifecycle a plugin's logic runs, not just whether it runs:

javascript

fastify.addHook('onRequest', async (request, reply) => {
  // Earliest hook — runs before the body is even read off the socket.
  // Good for: request-ID tagging, coarse IP allow/deny lists.
  request.receivedAt = process.hrtime.bigint();
});

fastify.addHook('preParsing', async (request, reply, payload) => {
  // Runs after onRequest, before the raw body stream is parsed.
  // Good for: decompression, decryption of an encrypted request body —
  // anything that needs to transform bytes before JSON parsing happens.
  return payload;
});

fastify.addHook('preValidation', async (request, reply) => {
  // Runs after body parsing, before schema validation.
  // Good for: normalizing fields before ajv validates them (e.g.
  // trimming whitespace from a senderId before pattern-matching it).
});

fastify.addHook('preHandler', async (request, reply) => {
  // Runs after validation, immediately before the route handler.
  // Good for: authentication, rate limiting — the hook shown earlier
  // in this module (rateLimiter, adminAuthenticator).
});

fastify.addHook('onSend', async (request, reply, payload) => {
  // Runs after the handler returns, before bytes go on the wire.
  // Good for: response header injection, compressing the payload,
  // stripping sensitive fields the handler forgot to omit.
  return payload;
});

Ordering per request: onRequest → preParsing → (route matching + body parsing) → preValidation → (schema validation) → preHandler → (your handler runs) → onSend → response sent. Because plugin encapsulation applies to every hook, not just preHandler, an ingestion plugin can attach preParsing decompression only to the routes that receive compressed payloads, while the admin plugin attaches preHandler auth only to /admin/* — each hook type only runs where it's registered, at the specific lifecycle stage it's registered for.

Production Incident: 400 Routes and a Decade of API Versions

Context: A UPI payment gateway built on Express accumulated over 400 registered routes across three years of API evolution — /api/v1/*, /api/v2/*, and /api/v3/* all running simultaneously, because bank-side integration partners upgraded on their own schedules and nothing could be safely removed without breaking a partner still calling the old version.

The symptom: Linear route matching alone — before any middleware, before any handler code — was adding double-digit milliseconds of latency per request at peak load. The routes nearest the end of the registration list (frequently the newest, least-used v3 endpoints, registered last) paid the highest cost, which meant the newest API version had the worst routing latency purely as an artifact of registration order.

What this did not fix by itself: swapping Express for a radix-tree router like Fastify would flatten the routing-cost curve regardless of route count — a legitimate and necessary fix — but it doesn't address the underlying reason the gateway had 400 routes to match in the first place. Three concurrent, fully-maintained API versions is an organizational cost (three sets of handlers, three sets of validation schemas, three code paths to keep secure and correct) that a faster router masks without resolving.

The actual fix: alongside the framework migration, the team introduced a route-deprecation and sunset policy — every new API version shipped with a published sunset date for the version it replaced, partner integrations were tracked against that date, and routes past their sunset window were actively removed rather than left running indefinitely "just in case." Route count dropped from 400+ to under 180 within two quarters. The lesson: a faster router buys you headroom, but only a deprecation policy stops the route count — and the maintenance burden behind it — from growing without bound.

HTTP Keep-Alive and Connection Reuse

For persistent connections from payment terminals or blockchain full nodes, HTTP keep-alive eliminates per-request TCP handshake overhead.

javascript

// Configure keep-alive on Fastify
const fastify = Fastify({
  // Keep connections alive for 72 seconds
  // (longer than typical 60s load balancer timeout — this MUST stay higher than the
  // LB's idle timeout. If the server's keep-alive is shorter than the LB's, the server
  // can close a connection the LB still considers open, causing intermittent 502s. Always
  // set keepAliveTimeout a few seconds above the LB's timeout, never lower.)
  keepAliveTimeout: 72_000,

  // Time allowed for client to send headers after connection is established
  connectionTimeout: 5_000,

  // Max requests per connection before closing (prevents memory accumulation)
  maxRequestsPerSocket: 1000,
});

javascript

// Configure keep-alive on outbound connections (e.g., to external APIs)
import { Agent } from 'node:http';

const keepAliveAgent = new Agent({
  keepAlive: true,
  maxSockets: 100,          // max connections to same host
  keepAliveMsecs: 30_000,   // send keep-alive probes every 30s
  maxFreeSockets: 20,       // keep 20 idle connections ready
});

// Use with fetch or http.request
fetch(url, { agent: keepAliveAgent });

For a blockchain indexer making thousands of outbound RPC calls to full nodes: without keep-alive, each call does a TCP handshake (~3ms). With keep-alive at 10,000 RPC calls/sec: 30 seconds of TCP handshake time saved per second of operation.

`undici`: The Current Best Practice for Outbound Calls

The http.Agent approach above is the legacy API — it still works, but undici is Node's modern HTTP client (Node's own fetch() is actually built on it), and at this throughput tier its connection-pool model measurably outperforms http.Agent. Instead of an Agent attached per-request, undici uses a Pool (or Agent in its own, distinct sense — a collection of pools keyed by origin) that manages connections, pipelining, and queuing internally:

javascript

import { Pool } from 'undici';

// One pool per upstream origin — e.g., a specific full node's RPC endpoint
const rpcPool = new Pool('https://fullnode.example.com', {
  connections: 100,        // equivalent to maxSockets
  pipelining: 1,           // requests pipelined per connection (1 = disabled)
  keepAliveTimeout: 30_000,
  keepAliveMaxTimeout: 60_000,
});

async function callRpc(method, params) {
  const { statusCode, body } = await rpcPool.request({
    path: '/',
    method: 'POST',
    headers: { 'content-type': 'application/json' },
    body: JSON.stringify({ jsonrpc: '2.0', method, params, id: 1 }),
  });

  if (statusCode !== 200) throw new Error(`RPC call failed: ${statusCode}`);
  return body.json();
}

undici's pool avoids per-request Agent lookup overhead, supports HTTP pipelining explicitly (rather than Node core's more conservative defaults), and is actively where Node's own HTTP client investment is going — fetch(), Response, and Request in modern Node are undici under the hood. For new outbound-call code at this module's throughput tier, reach for undici's Pool/Client over constructing an http.Agent directly; keep the Agent pattern above only when working in or maintaining code that predates the undici adoption.

`autocannon` + `clinic.js`: The Three-Tool Profiling Stack

Throughput measurement: autocannon

bash

autocannon -c 100 -d 30 \
  --renderStatusCodes \
  --json > results.json \
  http://localhost:3000/api/v2/payments

# Key metrics from results.json:
# requests.average: mean req/sec
# latency.p99: 99th percentile latency
# errors: connection errors (indicates server saturation)

CPU profiling: clinic flame

bash

clinic flame -- node server.js &
SERVER_PID=$!

# Run load
autocannon -c 100 -d 20 http://localhost:3000/api/v2/payments

kill $SERVER_PID
# Opens flamegraph in browser — identify wide plateaus in hot paths

Event loop diagnosis: clinic doctor

bash

clinic doctor -- node server.js &
SERVER_PID=$!

autocannon -c 100 -d 20 http://localhost:3000/api/v2/payments

kill $SERVER_PID
# Reports: ELU, GC frequency, I/O wait, event loop lag
# Identifies whether bottleneck is CPU, I/O, or event loop saturation

The Production Incident: Express Middleware Saturating a Payment Gateway

Context: A UPI payment gateway using Express with 8 middleware functions and 150 registered routes. Normal throughput: 8,000 req/sec. During a bank-wide reconciliation period, traffic peaked at 32,000 req/sec.

What happened: CPU across 16 workers hit 98% utilization. Response latency climbed from 12ms to 340ms. New connections began timing out. The database was at 15% capacity — it was not the bottleneck.

Diagnosis with clinic flame:

The flamegraph showed 28% of CPU time inside path-to-regexp — Express's route matching library. For each of 32,000 req/sec, Express was running 150 RegExp tests (the matching route was near the end of the list). Total: 4.8 million RegExp tests/second, consuming 28% of all CPU across 16 cores.

The migration:

javascript

// Before: Express with 150 routes
const app = express();
app.use(cors(), helmet(), express.json(), rateLimiter, ...);
app.post('/api/v1/...', handler);
// ... 149 more routes

// After: Fastify with compiled schemas
const fastify = Fastify({ logger: false });

await fastify.register(fastifyRateLimit, { max: 1000, timeWindow: '1 minute' });

// Routes with compiled schemas — zero RegExp, compiled validation
fastify.post('/api/v1/payments', { schema: paymentSchema }, paymentHandler);
// ... etc

Result after migration: At 32,000 req/sec, CPU dropped to 42% across 16 workers (from 98%). Latency: 8ms average (from 340ms). The route matching overhead that had consumed 28% of CPU dropped to ~2%.

Summary

Concept	Key Takeaway
Express routing	Linear scan: O(N) RegExp tests per request. 15–60μs overhead for 100 routes.
Radix tree	O(m) prefix traversal, where m is path length. Route count (K) barely affects matching cost.
`ajv` compilation	Schema compiled once at startup. 5–10x faster validation vs runtime interpretation.
`fast-json-stringify`	Response schema compiled once. 2–3x faster than `JSON.stringify`.
Fastify vs Express	3.7x throughput advantage at high req/sec when validation is included.
Fastify plugins	Scoped encapsulation — middleware runs only for relevant routes.
Keep-alive	Eliminates 3ms TCP handshake per request for persistent connections.
`autocannon`	Throughput and latency measurement. The baseline profiling tool.
`clinic flame`	CPU flamegraph. Identifies time spent in framework internals vs application code.
`clinic doctor`	Event loop health. ELU, GC frequency, I/O wait — the diagnostic layer above flamegraphs.

The routing layer gets requests to your code. Module 8 covers what to do once they're there — how to structure large ingestion systems as a Modulith to eliminate internal network overhead while maintaining clean architectural boundaries.

Next: Module 8 — The Modern Hybrid Monolith: High-Throughput Modulith Architecture →

Knowledge Check

Why does Fastify's Radix tree routing provide significantly better performance than Express's routing at scale?

How does Fastify achieve 2-3x faster JSON serialization compared to standard JSON.stringify()?

During a load test, you use clinic flame and notice a massive amount of CPU time is spent inside path-to-regexp. What is the most likely cause?

Test your knowledge with more question sets

Sign in to keep reading

The rest of this module is free — sign in with Google to unlock it and track your progress.

PreviousModule A-7: Core Scaling: Multi-Process Clustering & IPC Latency Next Module A-9: The Modern Hybrid Monolith: High-Throughput Modulith Architecture

Discussion

Join the discussion

Loading comments...

Module 7 — Routing Engines at Scale: Vanilla HTTP vs Radix Tree Frameworks

The Overhead Before Your Code Runs

Express: Linear Scan Middleware Chain

Radix Tree Routing: O(m) Route Matching, Independent of Route Count

Fastify: Architecture for Throughput

JSON Schema Compilation via ajv

fast-json-stringify: Compiled Response Serialization

Sign in to keep reading

Discussion

JSON Schema Compilation via `ajv`

`fast-json-stringify`: Compiled Response Serialization