Module F-5·18 min read

DNS → TCP handshake → TLS → HTTP, hop by hop, with the milliseconds attributed — so you know which layer to blame before you start guessing.

JJS

Written by Jatin Jain Saraf · Senior Software Engineer

Module F-5 — The Anatomy of a Request

Q: A service adds an outbound HTTPS call to a third-party API 120ms away. Latency rises by roughly 400ms per request, though the vendor's own p99 is 80ms. The most likely cause?

Every request opens a new connection, paying DNS, a TCP handshake (1 RTT), and a TLS handshake (1–2 RTT) before any data moves — so an agent with keep-alive and connection pooling should recover most of it. — At 120ms RTT, a cold connection costs roughly 120ms for the TCP handshake plus 120–240ms for TLS, plus DNS and the request itself — which lands close to the observed 400ms while the vendor's own processing stays at 80ms. The module calls out Node.js specifically here: outbound HTTP calls need an agent with `keepAlive: true`, and without pooling every request pays the full handshake. Option A misreads the evidence; the vendor's number describes their processing, not your connection setup, and raising a timeout hides latency rather than removing it. Option B is implausible at this magnitude — serialisation is a single-digit-to-low-tens-of-milliseconds cost, not hundreds. Option C is the fatalistic version of the right diagnosis: distance sets the RTT, but the *number* of round trips is under your control, which is exactly what connection reuse changes.

Q: A team reduces an API response from 60 KB to 12 KB and sees a much larger improvement for first-time visitors than the 5× size reduction would suggest. Why?

TCP slow start limits a new connection's first burst to roughly 14 KB, so 60 KB needed about three round trips to transfer while 12 KB fits in one — the saving is in round trips, not bytes. — The initial congestion window is about 10 packets — roughly 14 KB of payload — and doubles each round trip, so the module's table puts ≤14 KB at one round trip and ~60 KB at three. At a high RTT, removing two round trips is worth far more than the byte count implies, and it's why "get the critical response under 14 KB" is specific advice rather than a general "smaller is better." The effect is strongest for first-time visitors precisely because an established connection has already ramped its window up. Option A inverts the magnitudes; parse time is small relative to round trips at distance. Option C is not generally true and wouldn't explain a super-linear improvement concentrated in new connections. Option D describes a mechanism that doesn't exist — payload size doesn't change the routing path.

Q: Which change does the module identify as the largest available latency win for a request pattern currently paying ~800ms end to end with a 40ms handler?

Reusing connections — keep-alive between every pair of hops, pooled outbound connections, and TLS terminated near the user — since DNS, TCP, TLS, and slow start account for essentially the entire gap between the 800ms cold path and the 220ms warm one. — The cold-vs-warm budget table is the module's central result: the same request with the same handler costs ~800ms on a new connection and ~220ms on a reused one, and the only structural difference is whether a connection already existed. Hence the stated ordering — fix connection reuse first, optimise the handler after. Option A targets 40ms out of 800ms. Option D misidentifies the constraint: slow start limits the *window*, not the link speed, so more bandwidth does not remove those round trips. Option B is the trap: 0-RTT does eliminate handshake latency, but the module flags that it permits replay of the first application data, making it unsafe for state-changing requests — so it is a targeted optimisation for idempotent traffic, not a blanket setting.

What this module covers: A request took 800ms and the server-side trace says the handler ran for 40ms. Where did the other 760ms go? This module accounts for every hop: DNS resolution and the caching layers that usually hide it, the TCP three-way handshake, TCP slow start and why the first 14 KB is special, TLS 1.2 vs 1.3 handshake costs and session resumption, then the server-side path and the response. It ends with the cold-connection vs warm-connection budget side by side — because connection reuse is the single largest and cheapest latency win available, and understanding why is what makes Modules F-6, F-10, and P-4 make sense.

The 760 Milliseconds Nobody Owns

Here's an incident pattern that repeats in every organisation. A user in Bengaluru reports that an API call takes about a second. The APM dashboard shows the handler executing in 40ms, p99 at 60ms. Both observations are correct, and the gap between them is entirely made of work that happens before your code runs and after it returns.

Most of that gap is round trips. And a round trip is not a thing you can optimise away with better code — it's a fixed toll set by distance and by how many times the protocols insist on going back and forth before any useful bytes move.

Analogy: a request is a courier delivery, and your application code is only the moment the package is handed over. Before that: looking up the address (DNS), the courier driving to the building (TCP handshake), showing ID at reception and getting a badge (TLS), and being escorted to the right floor (routing). Optimising the handover from 40 seconds to 20 seconds is irrelevant if the drive takes an hour. Almost all real latency work is about the drive.

Why this matters in production: if you can't attribute latency to a hop, you'll optimise the wrong one. Teams routinely spend a sprint shaving 15ms off a query while every request pays 300ms for a TLS handshake it shouldn't be doing at all.

Step 1: DNS — Turning a Name Into an Address

Before a single byte reaches your server, the client needs an IP address. Full recursive resolution walks a hierarchy:

text

Browser cache        → hit? done, ~0 ms
OS resolver cache    → hit? done, ~0–1 ms
Configured resolver  → hit? ~1–20 ms (ISP or 1.1.1.1/8.8.8.8)
  └─ Root nameserver          → "ask .com"
     └─ TLD nameserver        → "ask ns1.yourdomain.com"
        └─ Authoritative NS   → "here's the A record"

A completely cold recursive lookup can cost 20–120ms, sometimes more with a distant or overloaded resolver. A warm one is effectively free. In practice the great majority are warm, which is exactly why DNS is invisible until the day it isn't.

Three things about DNS that show up in real designs:

TTL is a promise nobody fully keeps. You set a 60-second TTL expecting fast failover; some resolvers and some client runtimes cache longer, and long-lived processes may resolve once at startup and never again. So DNS-based failover has a real tail — a fraction of clients keep hitting the dead IP for minutes. This is why serious failover uses a load balancer with a stable address, or anycast, rather than swapping A records and hoping (Modules F-11, A-9).

Every distinct hostname is a separate resolution. A page pulling from your API, a CDN, an analytics vendor, a font host, and a payment iframe pays DNS separately for each. This is why dns-prefetch and preconnect hints exist, and why consolidating third-party origins is a measurable front-end win.

Server-side runtimes often don't cache DNS the way you assume. Node.js delegates to the OS resolver with no in-process caching by default, so a service making outbound calls to a hostname can resolve on every connection. At high request rates this becomes visible — both as latency and as load on the resolver. The fix is an explicit caching resolver or connection reuse that makes the question moot.

Step 2: TCP — One Round Trip Before You Say Anything

TCP requires a three-way handshake to establish a connection:

text

Client ──SYN────────▶ Server
Client ◀────SYN-ACK── Server
Client ──ACK────────▶ Server     (data can ride along with this ACK)

That's one full round trip (1 RTT) before any application data flows. On a same-city connection that's ~5–20ms. Mumbai to Virginia, it's ~180ms — gone, before your server has heard what the client wants.

Then comes the part fewer people know about, and it matters more than the handshake for small responses: TCP slow start.

A new connection doesn't know how much bandwidth is available, so it starts conservatively and doubles each round trip. The initial congestion window on modern Linux is 10 packets — roughly 14 KB of payload. The consequences are sharp:

Response size	Round trips to deliver (new connection)
≤ 14 KB	1
≤ ~42 KB	2
≤ ~100 KB	3
≤ ~220 KB	4

So a 90 KB JSON response over a fresh connection to a server 180ms away costs roughly 3 RTTs of transfer — ~540ms — not because bandwidth is short, but because the connection hasn't earned permission to send faster yet. Bandwidth is nearly irrelevant here; round trips are everything.

Two design consequences worth remembering:

Getting the critical response under ~14 KB is a real, measurable win on first requests. It's why "inline the critical CSS" advice exists, and why a chatty API returning small payloads over a warm connection can outperform one returning a single large payload over a cold one.
An established connection has already ramped up. Its congestion window is wide open, so the second request on the same connection transfers far faster than the first. This is the mechanical reason keep-alive matters so much — and the reason a system that opens a new connection per request never gets out of slow start.

Step 3: TLS — Proving Identity Costs Round Trips

TLS negotiates a cipher suite, validates the certificate chain, and derives session keys. The cost depends on version:

	Round trips	Cost at 180ms RTT
TLS 1.2 full handshake	2 RTT	~360 ms
TLS 1.3 full handshake	1 RTT	~180 ms
TLS 1.3 session resumption	1 RTT	~180 ms
TLS 1.3 with 0-RTT	0 RTT	~0 ms (with caveats)

TLS 1.3 halving the handshake is one of the largest single latency improvements available, and it's a configuration change rather than an engineering project. If a service still negotiates TLS 1.2, that's an extra round trip on every new connection, forever.

Details that bite in practice:

Certificate chain size is transfer cost during the handshake. A chain with unnecessary intermediates makes the handshake bigger — and it's happening inside slow start, where every extra packet may cost a round trip. ECDSA certificates are substantially smaller than RSA ones.
0-RTT resumption isn't free of consequence. It lets a client send application data with its first packet, which means that data can be replayed by an attacker. It's safe for idempotent requests and unsafe for anything that mutates state — a rare case where a latency optimisation has a correctness precondition (Module P-16 covers idempotency properly).
TLS termination location decides how much of this you pay. Terminating at a CDN edge 20ms away means the expensive handshake happens over a 20ms RTT, and the edge holds a warm, pooled connection back to your origin. That's the single biggest reason a CDN improves API latency even for uncacheable responses — Module F-15.

Step 4: The Server Side, and the Response

Now the request is finally on the wire. Server-side, before your handler:

text

Load balancer accept + routing decision      ~1 ms      (Module F-11)
TLS termination if not already done          see above
Reverse proxy / gateway: auth, rate limit    ~1–10 ms   (Modules F-15, P-18)
Application: framework, middleware, parsing  ~1–5 ms
├─ Connection acquired from the pool         0–???  ms  ← the hidden queue (P-4)
├─ Database query                            ~1–50 ms
└─ Serialisation                             ~1–10 ms
Response: compression, then transfer         see slow start

Two of those lines deserve attention because they're where surprises live.

Connection pool acquisition has no upper bound. If every pooled connection is busy, the request waits — and it waits with no slow query, no slow handler, and nothing in a naive trace to point at. Module F-4 listed this as a top source of tail latency; Module P-4 explains how to size the pool so it doesn't happen.

Compression is a CPU-for-bytes trade. Gzip or Brotli on a 200 KB JSON response might cost 5ms of CPU and save 3 round trips of transfer — an obvious win at distance. On a tiny response over a fast local link it's pure overhead. And compressing already-compressed content (JPEG, video, anything gzipped upstream) burns CPU for nothing.

The Budget: Cold Connection vs Warm

Here's the whole thing, end to end, for a client 180ms away from the origin. Same request, same handler, two connection states.

Hop	Cold connection	Warm (keep-alive)
DNS resolution	40 ms	0 ms (cached)
TCP handshake	180 ms	0 ms (reused)
TLS handshake (1.3)	180 ms	0 ms (reused)
Request transfer	90 ms (½ RTT)	90 ms
Server processing	40 ms	40 ms
Response transfer (60 KB)	270 ms (slow start, 3 RTT)	90 ms (window open)
Total	~800 ms	~220 ms

There's the missing 760ms from the opening. Nothing in it was your code, and the only structural difference between the two columns is whether a connection already existed.

This is the payoff of the module, so state it plainly: connection reuse is worth more than almost any application-level optimisation you can make. Which produces a checklist:

Keep-alive on, everywhere. Between client and edge, edge and origin, and service to service. In Node.js, outbound fetch/http calls need an agent with keepAlive: true — historically it defaulted off, and an unpooled outbound HTTPS call pays the full handshake on every request.
Pool outbound connections to every dependency — databases, caches, internal services, third-party APIs.
Terminate TLS close to the user and keep warm connections from the edge to the origin.
Prefer TLS 1.3 and small certificate chains.
Then, once the connection cost is gone, optimise the handler.

And the constraint that makes this a genuine architectural issue rather than a config note: some runtimes can't hold connections at all. A serverless invocation may start with no warm connection and no pool, so it pays the cold column — per invocation. That is one of the central costs of the model, covered in Module F-10 and paid for at the database in Module P-4.

Where This Shows Up in Your Stack

PostgreSQL: a new Postgres connection costs a TCP handshake, optional TLS, authentication, and a forked backend process with its own memory. It's far more expensive than an HTTP connection, which is why pg.Pool is not optional and why pgBouncer exists (Module P-4).
Next.js and third-party APIs: a route handler calling a payment or email provider over HTTPS without a keep-alive agent pays DNS + TCP + TLS on every request. Two dependencies at 100ms RTT each can add ~600ms of pure handshake to a request that does no meaningful work.
Redis: the whole reason Redis feels instant is that the connection is already open and the payload is tiny — one round trip, no handshake, no slow start. Break the pooling and Redis stops looking fast, which is a connection story rather than a Redis story.
Cross-region calls: every synchronous cross-region hop in a request path costs a full RTT, and slow start applies again if the connection is new. Module A-9's preference for asynchronous replication over synchronous cross-region calls is largely this table.
Mobile clients: radio wake-up on a cellular network can add 100ms+ before the first packet even leaves the device, and the RTT is both higher and more variable. The cold column is closer to the norm on mobile than on desktop.

Summary

Application code is usually a small share of user-perceived latency. The rest is round trips, and round trips are set by distance and protocol, not by code quality.
DNS: cold recursive resolution is 20–120ms, warm is free. TTLs aren't reliably honoured, so DNS is a weak failover mechanism; every extra hostname is another lookup.
TCP: one RTT for the handshake, then slow start caps the first burst at ~14 KB. A 90 KB response on a fresh connection can take 3 RTTs to transfer regardless of bandwidth.
TLS: 1.2 costs 2 RTT, 1.3 costs 1 RTT, resumption and 0-RTT cut further — with 0-RTT only safe for idempotent requests.
Server-side, watch connection pool acquisition (unbounded wait, invisible in naive traces) and treat compression as a CPU-for-round-trips trade.
Cold vs warm is ~800ms vs ~220ms for the same request. Connection reuse — keep-alive, pooling, edge termination — outperforms nearly any handler optimisation.
Runtimes that can't hold connections pay the cold column every time. That's a structural cost of serverless and edge compute, not a tuning oversight.

The next module, HTTP/1.1, HTTP/2, and HTTP/3, follows this thread up a layer: given that round trips dominate, how do the protocol versions differ in how many they need and how well they share a connection — including the head-of-line blocking problem that HTTP/2 solved at one layer and left standing at another.

Knowledge Check

A service adds an outbound HTTPS call to a third-party API 120ms away. Latency rises by roughly 400ms per request, though the vendor's own p99 is 80ms. The most likely cause?

A team reduces an API response from 60 KB to 12 KB and sees a much larger improvement for first-time visitors than the 5× size reduction would suggest. Why?

Which change does the module identify as the largest available latency win for a request pattern currently paying ~800ms end to end with a 40ms handler?

Test your knowledge with more question sets

Sign in to keep reading

The rest of this module is free — sign in with Google to unlock it and track your progress.

PreviousModule F-4: Latency, Throughput, and Tail Latency Next Module F-6: HTTP/1.1, HTTP/2, and HTTP/3

Discussion

Join the discussion

Loading comments...