Module 2 — Event Loop Saturation & Thread Pool Offloading
What this module covers: The event loop from Module 0 was presented as a single loop. That was a simplification. The event loop has six distinct phases, each with its own queue, processed in strict order. When you have thousands of incoming transactions per second, microtask queues competing with I/O callbacks, timers firing at sub-millisecond intervals, and cryptographic operations queuing for a thread pool that defaults to 4 threads — the phase structure determines everything about your latency profile. This module covers the event loop precisely, shows you how to measure saturation, and gives you the tools to offload work correctly.
The Six Phases of the Event Loop
The Node.js event loop is not a simple loop over a single queue. It is a phased loop — in each iteration, it processes up to six distinct phases, and the order is non-negotiable.
text
Between every phase and between every callback within a phase, Node.js drains two special queues:
These micro-queues drain completely before the next phase or callback runs. This ordering has critical implications for high-throughput ingestion.
Phase 1: Timers
Executes callbacks scheduled by setTimeout and setInterval whose thresholds have elapsed. The "threshold" is a minimum — the callback won't run before the specified time, but it may run later if other phases are busy.
Production implication: If your ingestion pipeline uses setTimeout(fn, 100) as a flush trigger for a batch write, that timer will not fire at exactly 100ms if the poll phase is busy processing incoming transaction data. At 50K events/sec, the poll phase can be continuously occupied, delaying timers by hundreds of milliseconds.
Phase 2: Pending Callbacks
Executes I/O callbacks that were deferred to the next loop iteration by the OS (typically some TCP errors). Rarely populated in normal operation.
Phase 3: Idle, Prepare
Internal to libuv. Not accessible from JavaScript.
Phase 4: Poll
The most important phase for I/O-intensive applications.
The poll phase does two things:
Calculates how long to block waiting for new I/O events (0ms if there are pending timers or setImmediate callbacks, otherwise up to some calculated maximum)
Processes I/O callbacks in the poll queue
For a blockchain indexer receiving a continuous stream of transactions: incoming socket data triggers epoll notifications → libuv adds callbacks to the poll queue → the poll phase drains the poll queue. As long as data keeps arriving, the poll phase stays busy.
The blocking calculation is critical: if the poll queue keeps filling faster than it drains, the event loop never moves past the poll phase. Timers don't fire. setImmediate callbacks don't run. This is event loop starvation.
Phase 5: Check
Executes setImmediate callbacks. This phase runs after the poll phase, not before. If you want code to run "soon" but after current I/O has been processed, setImmediate is correct. If you want code to run "immediately" (before any I/O callbacks), process.nextTick is correct.
javascript
Phase 6: Close Callbacks
Executes close event callbacks (socket.on('close', ...)). Cleanup only.
Microtask Queues: The Invisible Priority System
Before every phase transition and between every callback, Node.js drains microtask queues in priority order:
Both queues drain completely before the event loop moves forward. This has a dangerous implication: if you continuously add items to these queues, the event loop never advances.
process.nextTick and Promise microtasks are like an airport's expedite lane that must be completely empty before the main line can move at all — if passengers keep cutting into it, the main line never advances no matter how short it looks from outside. A phase transition is the main line; the microtask queues are the expedite lane. Add faster than it drains, and everything behind it — I/O callbacks, timers, setImmediate — simply waits.
The Microtask Starvation Pattern
javascript
javascript
Promise Chain Depth and Starvation
javascript
Event Loop Utilization (ELU): The Critical Production Metric
Event Loop Utilization (ELU) measures the ratio of time the event loop spends actively executing JavaScript vs idling in the poll phase waiting for I/O.
text
javascript
javascript
Alert thresholds for a UPI payment gateway:
ELU
Status
Action
< 0.70
Healthy
No action
0.70–0.85
Watch
Profile for CPU hotspots
0.85–0.95
Warning
Offload CPU work, scale
> 0.95
Critical
Immediate intervention, add instances
Event Loop Lag: Measuring Actual Delay
ELU tells you how busy the loop is. Event loop lag tells you how delayed callbacks are relative to when they were scheduled.
javascript
A setImmediate callback should run in < 0.1ms under no load. Under saturation:
At ELU 0.80: lag typically 2–10ms
At ELU 0.90: lag typically 10–50ms
At ELU 0.95: lag typically 50–200ms
For a blockchain indexer, 50ms event loop lag means WebSocket subscribers receive transaction confirmations 50ms late — and that 50ms compounds across every downstream consumer.
perf_hooks.monitorEventLoopDelay(): The Production-Grade Alternative
The hand-rolled setImmediate probe above gives you one point sample per interval — a single number, once a second. A probe that happens to fire during a quiet moment reports a healthy number even if lag spiked to 300ms a few milliseconds earlier and has since recovered. Point samples cannot tell you anything about the distribution of lag between samples.
Node's built-in histogram API is the standard tool for production: it continuously samples event loop delay in the background and reports percentiles.
javascript
Wire p50/p99/max into your dashboards instead of (or alongside) the setImmediate probe. The histogram captures every sample in the window, not just whichever moment your probe happened to fire.
Production story: A UPI payment gateway's dashboards showed ELU sitting at a comfortable 0.6 average, yet support tickets kept describing intermittent multi-hundred-millisecond stalls on a subset of requests. Averages couldn't see it — ELU smooths an entire sampling window into one ratio, and a handful of brief spikes barely move it. After wiring in monitorEventLoopDelay(), the p99 told the real story: 300ms lag spikes clustered tightly around GC pauses, each lasting only a few milliseconds but landing exactly when a batch of transaction objects went out of scope. The average absorbed those spikes as noise; the percentile histogram made them impossible to miss.
The libuv Thread Pool
libuv maintains a thread pool for operations that cannot be made truly non-blocking at the OS level. This pool is separate from the event loop thread and runs in parallel.
What Uses the Thread Pool
javascript
Critical for blockchain applications:crypto.pbkdf2, crypto.scrypt, and crypto.createSign all use the thread pool. Every transaction signature verification is a thread pool job.
The Default 4-Thread Limit
The thread pool defaults to 4 threads. If you submit more than 4 concurrent blocking operations, they queue. This queue has no limit.
javascript
A job, once submitted, cannot be cancelled. There is no cancel() or abort() for a thread pool job — once crypto.pbkdf2 (or any other thread-pool-backed call) is queued, it runs to completion even if the request that triggered it times out, the client disconnects, or the result is no longer needed. This matters directly for the incident later in this module: rejecting or timing out requests at the HTTP layer does nothing to drain the underlying thread pool queue, because the already-queued jobs have no cancellation hook to call. The only way to relieve pressure is to stop submitting new jobs or wait for the backlog to drain on its own.
Sizing UV_THREADPOOL_SIZE for Cryptographic Workloads
There is no single canonical formula here — despite how often one gets quoted. A reasonable starting heuristic, not a rule:
UV_THREADPOOL_SIZE ≈ number_of_cpu_cores × 2 (a starting point to benchmark from, not gospel)
For a blockchain indexer on an 8-core server performing signature verification on every transaction:
bash
A note on the "128" you'll see cited everywhere: libuv's actual hard ceiling on UV_THREADPOOL_SIZE is 1024 threads, not 128. The number 128 that circulates in blog posts and Stack Overflow answers is not a technical limit — it's a commonly repeated convention, likely inherited from older defaults in unrelated systems. Don't treat it as a wall you can't cross; treat it (and any × 2 / × 4 multiplier) as a starting heuristic to benchmark against your actual workload, not a fixed technical constraint.
Why not just set it as high as possible? More threads = more memory (each thread has its own stack), more context switching, and eventually diminishing returns as CPU cores are shared. The heuristic above keeps thread count roughly proportional to available parallelism, but the only way to know the right number for your workload is to measure thread pool queue depth under realistic load and tune from there.
javascript
worker_threads vs Thread Pool: Choosing the Right Tool
libuv's thread pool handles C-level blocking operations. worker_threads is for JavaScript-level CPU-bound work.
libuv Thread Pool
worker_threads
Language
C / native code
JavaScript
Control
Indirect (through APIs)
Direct (you write the worker)
Communication
Callback when done
Message passing / SharedArrayBuffer
Use case
crypto, fs, dns
Heavy JS computation
Default limit
4 (configurable)
No system limit (you manage the pool)
When to Use worker_threads
Use worker_threads when your hot path has CPU-bound work in JavaScript — complex data transformation, in-process schema validation at scale, or Merkle tree construction in JS.
javascript
SharedArrayBuffer: Zero-Copy Communication
For high-frequency worker communication, message passing involves serialization (structuredClone or JSON). SharedArrayBuffer eliminates this:
javascript
Preventing Event Loop Blocking During JSON Parsing
Large JSON payloads are a common blocking source. JSON.parse is synchronous and runs entirely on the main thread.
The Problem: Multi-MB Blockchain Payloads
A full Ethereum block can be 1–10MB of JSON. JSON.parse on a 5MB payload takes 20–80ms on modern hardware. On the event loop thread, this means 20–80ms where nothing else runs.
javascript
Solution 1: Offload to a Worker Thread
javascript
Solution 2: Streaming JSON Parser
For very large payloads, a streaming parser processes JSON incrementally, yielding to the event loop between chunks:
javascript
The Phase Interaction: setImmediate vs process.nextTick vs Promise
Understanding the exact execution order is critical for scheduling work correctly in a high-throughput pipeline.
javascript
For a blockchain indexer, this means:
javascript
The Production Incident: Thread Pool Exhaustion During UPI Festival Spike
Context: A UPI payment gateway processing ~1,500 transactions/second normally. During a major festival sale, traffic spikes to 12,000 transactions/second.
The architecture:
javascript
This code has a second, quieter bug, independent of thread pool sizing. Setting the PBKDF2 iteration count to 1 does not make crypto.pbkdf2 produce the same output as crypto.createHmac('sha512', key).update(data).digest(). PBKDF2 appends a 4-byte block-index suffix to the input before each HMAC round, even when there's only one round — so derivedKey is never equal to a true HMAC-SHA512(key, data). This verifyHmac was never actually verifying against a real HMAC signature; it was comparing against something else entirely. Under this incident that correctness bug was masked by the performance bug — every request timed out before verification even completed, so nobody could tell it was also computing the wrong value.
What happened at 12,000 req/sec:
Default UV_THREADPOOL_SIZE = 4. Each crypto.pbkdf2 call takes ~8ms. At 12,000 req/sec, the thread pool needed to complete 12,000 operations/sec. Maximum throughput: 4 threads × (1000ms / 8ms) = 500 operations/sec.
The queue of pending thread pool operations grew to over 20,000 within 2 seconds. Each incoming payment request waited in the thread pool queue. The event loop remained responsive (ELU: 0.12 — barely busy), but every request timed out because verifyHmac took 30–120 seconds to resolve — not because of CPU, but because of thread pool starvation.
The fix:
javascript
After the fix: At 12,000 req/sec, createHmac verifications ran at 0.1ms each on the main thread. ELU increased to 0.42 (acceptable). No thread pool queue buildup. Response time: 4–12ms end-to-end.
Where I/O callbacks run. Continuous data = loop stays in poll phase.
Microtask queues
process.nextTick then Promises, before every phase. Infinite recursion starves the loop.
ELU
Ratio of active to idle event loop time. Alert at > 0.85.
Event loop lag
Actual delay between scheduling and execution. Measurable with setImmediate probe.
Thread pool
Default 4 threads for crypto, fs, dns. Size with UV_THREADPOOL_SIZE = cores × 2.
Thread pool exhaustion
Requests don't fail — they just wait forever. ELU stays low while requests queue.
worker_threads
JavaScript CPU work off the main thread. Use for heavy JS computation, not I/O.
SharedArrayBuffer
Zero-copy communication between threads. Use for high-frequency data transfer.
Streaming JSON
Parse multi-MB payloads with clarinet or in a worker thread. Never JSON.parse large payloads on the main thread.
setImmediate vs nextTick
setImmediate yields to the event loop (check phase). nextTick is immediate (before next phase).
The event loop determines when your code runs. The next layer down determines what happens when data actually arrives at the kernel. Module 3 goes into the OS-level I/O multiplexing that the event loop is built on — epoll, kqueue, and the precise journey a transaction takes from the NIC to your JavaScript callback.
Which mechanism should be used to schedule code to run in the check phase of the current event loop iteration?
What is the most likely cause of a Node.js application experiencing severe latency when processing thousands of concurrent requests that require cryptographic signature verification using crypto.pbkdf2, despite having low CPU utilization and an Event Loop Utilization (ELU) of 0.15?
What does an Event Loop Utilization (ELU) of 0.95 indicate about the Node.js process?
Test your knowledge with more question sets
Sign in to access a wider variety of questions and get notified when new practice sets are added to this module.
// Common pattern for blockchain indexers:// After writing to DB, immediately schedule validation (before timers, after I/O)db.write(tx).then(()=>{setImmediate(()=>validateAndIndex(tx));});
// DANGEROUS: infinite nextTick recursionfunctionprocessTransactions(queue){if(queue.length===0)return;const tx = queue.shift();handleTransaction(tx); process.nextTick(()=>processTransactions(queue));// ← recurses forever}// At 50K queued transactions:// - processTransactions runs, schedules nextTick// - nextTick queue drains: runs processTransactions again// - Schedules another nextTick// - nextTick queue drains again... 50,000 times// - The event loop NEVER moves past the current phase// - All I/O, timers, and setImmediate are starved
// CORRECT: yield to the event loop periodicallyfunctionprocessTransactions(queue){constBATCH_SIZE=100;let processed =0;while(queue.length>0&& processed <BATCH_SIZE){const tx = queue.shift();handleTransaction(tx); processed++;}if(queue.length>0){// Use setImmediate to yield to the event loop after each batch// This allows I/O and timers to run between batchessetImmediate(()=>processTransactions(queue));}}
// This Promise chain processes 10,000 transactions without yieldingasyncfunctionprocessAll(transactions){for(const tx of transactions){awaitparseTransaction(tx);// adds to microtask queueawaitvalidateSignature(tx);// adds to microtask queueawaitwriteToDatabase(tx);// adds to microtask queue}}// At 10,000 transactions with 3 awaits each: 30,000 microtasks// Before the event loop advances past the current phase,// all 30,000 resolve synchronously if they're CPU-bound// → I/O callbacks starved for the duration of this batch
ELU = active_time / (active_time + idle_time)
ELU = 0.0 → event loop idle, no work to do
ELU = 0.5 → event loop 50% busy, 50% waiting for I/O
ELU = 0.9 → event loop 90% busy — danger zone
ELU = 1.0 → event loop fully saturated, no capacity for new work
// Measuring ELU in productionimport{ eventLoopUtilization }from'node:perf_hooks';// Capture baseline at startupconst startELU =eventLoopUtilization();// Measure delta every 5 secondssetInterval(()=>{const currentELU =eventLoopUtilization(startELU);console.log(`ELU: ${(currentELU.utilization*100).toFixed(1)}%`);if(currentELU.utilization>0.85){console.warn('EVENT LOOP SATURATED — offload CPU work or scale horizontally');}},5000);
// Export to Prometheus for production alertingimport{ register,Gauge}from'prom-client';const eluGauge =newGauge({name:'nodejs_event_loop_utilization',help:'Event loop utilization (0–1)',});let lastELU =eventLoopUtilization();setInterval(()=>{const elu =eventLoopUtilization(lastELU); eluGauge.set(elu.utilization); lastELU =eventLoopUtilization();},1000);
// Measure event loop lag directlyfunctionmeasureLag(){const start = process.hrtime.bigint();setImmediate(()=>{const lag =Number(process.hrtime.bigint()- start)/1_000_000;// msconsole.log(`Event loop lag: ${lag.toFixed(2)}ms`);});}setInterval(measureLag,1000);
import{ monitorEventLoopDelay }from'node:perf_hooks';const histogram =monitorEventLoopDelay({resolution:20});// sample every 20mshistogram.enable();setInterval(()=>{console.log({min: histogram.min/1e6,// nanoseconds → msmax: histogram.max/1e6,mean: histogram.mean/1e6,p50: histogram.percentile(50)/1e6,p99: histogram.percentile(99)/1e6,}); histogram.reset();// clear the window for the next reporting interval},5000);
// These operations use the libuv thread pool:import{ readFile }from'node:fs';import{ lookup }from'node:dns';import{ pbkdf2, scrypt, randomBytes }from'node:crypto';import{ createGzip }from'node:zlib';// Network sockets do NOT use the thread pool — they use epoll directly// TCP, UDP, pipes: always async, never thread pool
// Demonstrating thread pool exhaustion// Run with 4 simultaneous crypto operations — fills the poolconst promises =Array.from({length:20},()=>newPromise((resolve, reject)=>{ crypto.pbkdf2('password','salt',100000,64,'sha512',(err, key)=>{if(err)reject(err);elseresolve(key);});}));// Operations 5–20 will queue behind operations 1–4// Total time ≈ ceil(20/4) × per-operation-time// NOT 20 × per-operation-time (parallel), NOT 1 × per-operation-time (4 parallel)
# Set before starting Node.js — cannot change at runtimeUV_THREADPOOL_SIZE=16node indexer.js
# Or in your startup script:exportUV_THREADPOOL_SIZE=16node indexer.js
// Measuring thread pool saturation// Monitor the delay between submitting a thread pool job and its completionfunctionmeasureThreadPoolLag(){const start =Date.now(); crypto.randomBytes(32,()=>{// lightweight thread pool jobconst lag =Date.now()- start;if(lag >10){console.warn(`Thread pool lag: ${lag}ms — pool may be saturated`);}});}setInterval(measureThreadPoolLag,500);
// main.js — worker thread pool for transaction validationimport{Worker, isMainThread, parentPort, workerData }from'node:worker_threads';import{ cpus }from'node:os';// Declared here (not inside the block below) since a function declared inside// an if-block is only visible within that block — this needs to be visible at// module scope so it can be exported once, after the if/else.let validateTransaction;if(isMainThread){// Pool of validation workersconstPOOL_SIZE=cpus().length;const workers =[];const queue =[];const pending =newMap();let requestId =0;// Spawn poolfor(let i =0; i <POOL_SIZE; i++){const worker =newWorker(newURL(import.meta.url)); worker.on('message',({ id, result, error })=>{const{ resolve, reject }= pending.get(id); pending.delete(id); error ?reject(newError(error)):resolve(result);// Process next queued workif(queue.length>0){const{id: nextId, payload }= queue.shift(); worker.postMessage({id: nextId, payload });}}); workers.push({ worker,busy:false});}// Submit work to pool — `export` only works at module top level, so this// assigns the module-scoped binding declared above and gets exported once,// after the if/else blockvalidateTransaction=function(payload){returnnewPromise((resolve, reject)=>{const id = requestId++; pending.set(id,{ resolve, reject });const idleWorker = workers.find(w=>!w.busy);if(idleWorker){ idleWorker.worker.postMessage({ id, payload });}else{ queue.push({ id, payload });// queue if all workers busy}});};}else{// Worker: CPU-bound validation logic parentPort.on('message',({ id, payload })=>{try{const result =heavyValidation(payload);// runs in worker thread parentPort.postMessage({ id, result });}catch(err){ parentPort.postMessage({ id,error: err.message});}});functionheavyValidation(payload){// Schema validation, signature verification in JS, Merkle proof check// This runs on its own thread — main thread event loop unaffectedreturntrue;}}export{ validateTransaction };
// Shared ring buffer between main thread and workers// Zero serialization — both threads read/write the same memoryconstBUFFER_SIZE=1024*1024;// 1MB ring bufferconst sharedBuffer =newSharedArrayBuffer(BUFFER_SIZE);const view =newUint8Array(sharedBuffer);const control =newInt32Array(newSharedArrayBuffer(8));// control[0] = write position, control[1] = read position// Main thread writes transaction datafunctionwriteTransaction(txBytes){const writePos =Atomics.load(control,0); view.set(txBytes, writePos);Atomics.store(control,0,(writePos + txBytes.length)%BUFFER_SIZE);Atomics.notify(control,0,1);// wake one waiting worker}// Worker readsAtomics.wait(control,0,Atomics.load(control,0));// sleep until dataconst readPos =Atomics.load(control,1);// ... read and process
// BAD: synchronous parse of large payload on main threadapp.post('/ingest-block',(req, res)=>{let body =''; req.on('data',chunk=> body += chunk); req.on('end',()=>{const block =JSON.parse(body);// ← blocks event loop for 20–80ms if body is 5MBprocessBlock(block); res.sendStatus(200);});});
import{Worker}from'node:worker_threads';// parser-worker.jsparentPort.on('message',({ id, jsonString })=>{try{const parsed =JSON.parse(jsonString);// runs in worker, main thread free parentPort.postMessage({ id,result: parsed });}catch(err){ parentPort.postMessage({ id,error: err.message});}});// main thread — JSON parse in workerasyncfunctionparseBlockPayload(jsonString){returnnewPromise((resolve, reject)=>{const id = nextId++; parserWorker.postMessage({ id, jsonString }); parserWorker.once('message',({id: msgId, result, error })=>{if(msgId === id) error ?reject(error):resolve(result);});});}
// Using 'clarinet' for streaming JSON parsingimportclarinetfrom'clarinet';functionparseBlockStream(readable){returnnewPromise((resolve, reject)=>{const parser = clarinet.createStream();const transactions =[];let inTransactionsArray =false;let depth =0; parser.on('openarray',()=>{ depth++;if(depth ===2) inTransactionsArray =true;}); parser.on('closearray',()=>{ depth--;if(depth <2) inTransactionsArray =false;}); parser.on('value',(value)=>{if(inTransactionsArray){ transactions.push(value);}}); parser.on('end',()=>resolve(transactions)); parser.on('error', reject); readable.pipe(parser);// non-blocking: parser processes chunks as they arrive});}
// Transaction arrives → parse → write to DB → notify subscribers// BAD: using setTimeout(fn, 0) for subscriber notification// This defers to the NEXT iteration of the event loop// → adds minimum one full event loop iteration of latency// CORRECT: using setImmediate// Runs in the check phase of the CURRENT iteration// → subscriber notified in the same loop iteration as the DB write completesdb.write(transaction).then(()=>{setImmediate(()=>notifySubscribers(transaction));// same loop iteration});
// Fix 1: Increase thread pool size// UV_THREADPOOL_SIZE=32 node payment-gateway.js// Fix 2: Replace pbkdf2 (slow, thread-pool-bound, AND not a valid HMAC in this form)// with createHmac — the actually-correct, actually-fast way to compute an HMACfunctionverifyHmac(signature, data, key){const hmac = crypto.createHmac('sha512', key); hmac.update(data);const computed = hmac.digest('hex');return crypto.timingSafeEqual(Buffer.from(computed),Buffer.from(signature));// crypto.createHmac computes a true HMAC-SHA512 directly — no PBKDF2 detour,// no block-index suffix, no thread pool. Runs synchronously on the main thread.// It's also fast: ~0.1ms per verification, so the main thread handles// 10,000/sec before saturation.}// Fix 3: For slow operations (genuine pbkdf2 for key derivation),// use worker_threads with a pool sized to CPU count