Auto-instrumentation traces HTTP and database calls transparently, but Kafka breaks the chain by default — a message sitting in a topic carries no notion of "which trace is this part of." To get one trace spanning producer → topic → consumer, the trace context has to travel inside the message itself, via the W3C traceparent header.

With this in place, a single traceparent travels HTTP ingest span → Kafka publish span → (message header, across the wire) → Kafka consume span → database write span. Jaeger renders it as one continuous waterfall across two separate Node.js processes, instead of two disconnected traces that happen to touch the same topic.

Correlating Traces with Logs

The logging and tracing sections above are covered as separate tools, but in production you want to go from "this trace was slow" to "here are the exact log lines emitted during that span" without manually lining up timestamps. The bridge: read the active span's IDs and attach them to every log line.

traceId and spanId now show up as queryable fields alongside every other structured field from the Pino section above. Click a slow span in Jaeger, copy its traceId, and pull every log line emitted during that request with a single query — no timestamp guesswork.

---

Event Loop Utilization: The First Metric to Check

ELU was introduced in Module 2. In production, it is the first metric to check during a latency incident.

---

Generating V8 CPU Profiles

A CPU profile samples the call stack every N microseconds. The result: a list of functions and how many samples they appear in. Functions that appear in many samples are consuming CPU time.

Method 1: --cpu-prof Flag

Method 2: V8 Profiler API

---

Reading Flame Graphs

A flame graph is an X-ray of your event loop — the widest bones are exactly where all the weight is sitting.

A flame graph visualizes CPU time as a stack of colored bars:

• X axis: total CPU time (wider = more CPU)
• Y axis: call stack depth (higher = deeper in the call chain)
• Color: random, for visual distinction
• Width of a bar: proportion of CPU time spent in that function and its callees

Reading rules:

1. Wide plateaus at the top are hot spots — functions that consume a large fraction of CPU
2. Narrow spikes are expected call depths — not performance problems
3. Look for width at the TOP of the stack — if JSON.parse is 35% wide at the top level, it's consuming 35% of total CPU

Actionable patterns:

---

The clinic.js Diagnostic Toolchain

clinic.js is the most comprehensive Node.js diagnostic suite. Three tools, each answering a different question.

clinic doctor: What Is Wrong?

clinic doctor analyzes four signals:

1. Event loop delay — is the event loop lagging? (indicates blocking code)
2. CPU usage — is the process CPU-bound?
3. Memory — is memory growing continuously? (indicates leak)
4. Handles/requests — are there open handles preventing process exit? (indicates resource leak)

What the report tells you:

clinic flame: Where Is the CPU Going?

clinic flame generates a proper flamegraph from --perf profiling data, with:

• Merged V8 and native frames (you see both JS and C++ in one view)
• Click-to-zoom for deep inspection
• Filtering by function name

Interpreting clinic flame output for a transaction parser:

clinic bubbleprof: Where Is the Async Time Going?

bubbleprof visualizes async operations as bubbles — the size of each bubble represents how long async operations took. It shows where time is spent waiting rather than executing.

Useful for diagnosing:

• Database queries that are slower than expected
• HTTP client calls with unexpected latency
• Async chains that add unnecessary await depth

---

Always-On Production Profiling: Datadog Continuous Profiler & Pyroscope

--cpu-prof and clinic flame are one-off sessions: you start them deliberately, during a known incident or load test, then stop. They answer "what is this process doing right now?" They cannot answer "what was this process doing at 3am last Tuesday, five minutes before the alert fired?" — by the time you're looking, the moment is gone.

Always-on profilers close that gap by sampling CPU (and often heap) continuously in production, at low enough overhead to leave running permanently, and retaining the samples the same way you retain metrics — queryable by service, by tag, by time range.

Grafana Pyroscope follows the same model with an open-source agent and storage backend, if you'd rather not depend on a vendor.

In practice these are complementary, not competing: continuous profiling tells you when and roughly where a regression started ("CPU time in parseTransaction doubled the moment deploy X went out"), and a targeted clinic flame session gets you the rest of the way to the exact line.

---

Heap Snapshots: Finding Memory Leaks

When clinic doctor shows continuously growing memory, take a heap snapshot:

Common memory leak patterns in Node.js:

---

auto_explain for Node.js: async_hooks + Performance Timing

The equivalent of PostgreSQL's auto_explain for Node.js: automatically log slow async operations above a threshold.

Caveat: async_hooks is not free to run. Every async resource creation and teardown fires an init/destroy callback, and at 50K events/second with several async resources per transaction (socket reads, timers, DB queries), that adds up to real per-operation overhead — async_hooks has a well-documented history of measurably regressing the exact event loop it's meant to be monitoring, particularly under heavy async churn. Treat this as a tool you enable for a bounded window during a targeted debugging session — behind a flag, or toggled via a signal handler — rather than as always-on production instrumentation.

---

The Production Incident: Undetected Memory Leak via Event Listener

Context: A blockchain indexer WebSocket subscription service. Engineers received an alert: process memory at 94% of limit after 6 hours of operation. Restart fixed it temporarily.

Diagnosis with heap snapshot comparison:

Snapshot 1 (at startup): 180MB heap Snapshot 2 (6 hours later): 1.8GB heap

Chrome DevTools comparison view showed:

183,000 EventEmitter instances — one per WebSocket subscriber (12,000 subscribers × ~15 listeners each = 180,000 accumulated listener objects that were never cleaned up).

The broken code:

The fix:

After the fix: memory stabilized at 220MB regardless of subscriber count or runtime duration.

---

Summary

---

You can now diagnose performance problems with precision. Module 13 covers the connection layer — the pool configurations and process management patterns that determine how many requests your service can actually handle concurrently.

Next: Module 13 — Advanced Connection Pooling & Process Management →

---