Module F-6·18 min read

Head-of-line blocking at both the application and transport layer, multiplexing, keep-alive, and why connection reuse is the cheapest performance win you have.

JJS

Written by Jatin Jain Saraf · Senior Software Engineer

Module F-6 — HTTP/1.1, HTTP/2, and HTTP/3

Q: A team migrates from HTTP/1.1 to HTTP/2 and sees a solid improvement in office testing, but users on mobile networks report little change or occasional regressions. What explains this?

HTTP/2 multiplexes all streams over one TCP connection, and TCP's in-order delivery guarantee means a single lost packet stalls every stream — whereas HTTP/1.1's six separate connections confine a loss to one queue. On lossy links the multiplexing advantage narrows and can invert. — This is transport-layer head-of-line blocking, the problem HTTP/2 could not solve and HTTP/3 exists to address. TCP delivers a single ordered byte stream and has no knowledge of the independent HTTP/2 streams inside it, so it buffers everything after a lost packet until the retransmission lands — stalling streams whose data already arrived. The module notes the uncomfortable corollary: HTTP/1.1 with six connections is *more* loss-resilient, since a loss affects only one queue. It also warns that benchmarking on a clean office network overstates the gain for exactly the population that suffers most loss. Option A is wrong on the facts — HTTP/2 in browsers requires TLS but not specifically 1.3, and this isn't a fallback story. Option B describes a limit that doesn't exist. Option C is negligible; HPACK saves far more in bytes than it costs in CPU.

Q: After moving internal services to gRPC over HTTP/2 behind an L4 load balancer, a team finds that autoscaling adds instances which receive almost no traffic while the original instances stay saturated. What's happening?

Each client holds one long-lived multiplexed connection, so balancing by connection no longer tracks request volume — and existing connections never move to new backends. The fixes are L7 request-level balancing and a maximum connection age that forces periodic reconnection. — The module calls this the consequence nobody expects. Under HTTP/1.1, connection count roughly tracks request count, so connection-level distribution works; under HTTP/2 one connection carries thousands of requests, and an L4 balancer pins all of a client's traffic to whichever backend it first connected to. Scaling out therefore adds capacity that existing connections never reach. The stated remedies are an L7 balancer that distributes individual requests and a deliberate connection-lifetime limit that forces redistribution. Option C stays at L4, where the unit of distribution is the problem — least-connections still counts connections. Option B misattributes it to client caching rather than to connection longevity. Option D invents a failure mode; the new instances are healthy, they're simply never addressed.

Q: A service's p99 is 2 seconds, dominated by a database query missing an index. A proposal argues that enabling HTTP/3 will materially improve it. What does the module say?

It won't materially help. The gains concentrate on many-small-resources workloads, high-RTT links, and lossy networks; a request dominated by a slow query is unaffected by the protocol version. — The module names this case explicitly, in the bluntest available terms: if p99 is 2 seconds because of a missing index, HTTP/3 will make it 1.98 seconds. Protocol upgrades save round trips and handshake costs, and this request's time is server-side work — nothing in the transport touches it. Option A generalises real benefits to a case where they don't apply, which is exactly the over-attribution being warned against. Option B is wrong on both counts: supporting both is required in practice because UDP is sometimes blocked, and dual support doesn't cancel HTTP/3's benefits for clients that use it. Option D overstates the CPU cost — it's per-connection transport overhead, and it does not slow down an unrelated database query.

What this module covers: Given Module F-5's conclusion that round trips dominate, the natural next question is how well each protocol version uses a connection. This module covers HTTP/1.1's one-request-at-a-time limitation and the six-connections-per-origin workaround it forced, HTTP/2's binary framing and multiplexing, the TCP-level head-of-line blocking HTTP/2 could not fix, and how HTTP/3 sidesteps it by abandoning TCP for QUIC over UDP. It also covers the parts that surprise people in production: why multiplexing changes how load balancers distribute traffic, why HTTP/2 is what makes gRPC possible, and the cases where none of this matters at all.

The Problem: One Connection, One Request at a Time

HTTP/1.1's defining limitation is simple. A connection carries one request/response exchange at a time. Send a request, wait for the full response, then send the next. The specification does define pipelining — sending several requests without waiting — but responses must come back in order, so one slow response blocks everything queued behind it. In practice it was so poorly supported by intermediaries that browsers disabled it, and it's effectively dead.

That's head-of-line blocking at the application layer: one slow item delays every item behind it in the same queue.

Analogy: HTTP/1.1 is a single-lane road with no overtaking. If the truck in front is slow, everyone behind it is slow, regardless of how fast their cars are or how empty the road ahead looks.

Browsers worked around it by opening six parallel connections per origin — six lanes instead of one. That helped, and it introduced its own costs. Each connection pays a fresh TCP handshake, a fresh TLS handshake, and starts in slow start (Module F-5), and six connections competing for the same bandwidth means six congestion controllers each with an incomplete view. A page with 80 resources still queues them through six lanes.

Which produced the era of front-end workarounds you may remember, all of them there to dodge the connection limit rather than to make anything genuinely better:

Domain sharding — serving assets from img1.example.com, img2.example.com to get 6 connections per hostname, at the cost of extra DNS lookups and extra handshakes.
Sprite sheets — one image containing dozens, cut up with CSS, so one request replaced thirty.
Bundling and inlining — concatenating all JS into one file, base64-inlining images, so that a change to one line invalidates the cache for everything.

Why this matters in production: every one of those techniques becomes harmful under HTTP/2, and plenty of build pipelines still do them. Under a multiplexed protocol, one giant bundle means a one-line change re-downloads the whole thing, and domain sharding forces extra connections that multiplexing was designed to eliminate. Protocol upgrades don't just make things faster; they invert previous best practices, which is precisely why "best practice" without a stated reason ages badly.

HTTP/2: Multiplexing Over One Connection

HTTP/2 keeps HTTP's semantics — methods, status codes, headers all mean the same thing — and replaces how they're framed on the wire.

Binary framing. Instead of newline-delimited text, HTTP/2 sends binary frames, each tagged with a stream ID. That single change enables everything else, because frames from different streams can be interleaved on one connection.

Multiplexing. Many concurrent requests share one TCP connection, and responses can come back in any order and interleaved. Head-of-line blocking at the application layer is genuinely gone — a slow response no longer blocks the others.

HPACK header compression. HTTP headers are repetitive and, with cookies, often larger than the body of a small API response. HPACK compresses them and maintains a dynamic table of previously-sent headers, so repeated headers cost a few bytes rather than a few hundred. On a page making 100 API calls with a 1 KB cookie, this alone saves ~100 KB of upstream traffic.

Stream prioritisation and flow control. Clients can signal that the stylesheet matters more than a below-the-fold image. Support has been inconsistent across servers and CDNs, so treat it as a hint rather than a mechanism you rely on.

Server push let servers send resources before they were requested. It's now effectively deprecated — Chrome removed support — because servers routinely pushed things clients already had cached, wasting bandwidth. 103 Early Hints covers the useful case (telling a client what to start fetching) without the guessing.

The result: one connection, one handshake, one congestion window that ramps up once and is then shared by everything. On a page with many small resources, this is a large win and the workarounds above become counterproductive.

text

HTTP/1.1, 6 connections, 30 resources:
  6 × (TCP + TLS handshake) = 6 × 2 RTT of setup
  30 resources queued through 6 lanes, in order per lane

HTTP/2, 1 connection, 30 resources:
  1 × (TCP + TLS handshake) = 2 RTT of setup
  30 concurrent streams, interleaved, responses in any order
  Headers compressed across all 30

What HTTP/2 Couldn't Fix: TCP-Level Head-of-Line Blocking

Here's the subtlety that justifies HTTP/3's existence, and it's a good example of a problem being pushed down a layer rather than removed.

TCP guarantees in-order, reliable delivery of a single byte stream. It has no idea that the stream contains interleaved HTTP/2 frames belonging to independent requests. So when one packet is lost, TCP holds back every byte that arrived after it until the retransmission arrives — because delivering them out of order would violate TCP's contract.

The consequence: one lost packet stalls every multiplexed stream on that connection, even the ones whose data arrived perfectly.

text

Streams A, B, C multiplexed on one TCP connection.
Packet carrying a fragment of stream A is lost.

TCP's view:  bytes 1000–1500 missing → buffer everything after it, wait for retransmit
Result:      B and C are stalled too, despite their data having arrived
Cost:        one full RTT (or more) for the retransmission

And here's the uncomfortable part: HTTP/1.1 with six connections is more resilient to packet loss than HTTP/2 with one, because a loss on one connection only stalls that connection's queue. On a clean network HTTP/2 wins comfortably. On a lossy network — mobile, congested Wi-Fi, poor last-mile links — the advantage narrows and can reverse.

Why this matters in production: this is why measuring a protocol change on an office network and concluding "HTTP/2 is 30% faster" can mislead. The population that gains the most from multiplexing overlaps heavily with the population most exposed to packet loss.

HTTP/3 and QUIC: Fixing It by Leaving TCP

You cannot fix TCP head-of-line blocking inside TCP — the ordering guarantee is TCP. So QUIC reimplements the transport on top of UDP, which provides almost nothing and therefore gets out of the way.

Independent streams. QUIC understands streams natively. A lost packet stalls only the stream it belonged to; the others continue. This is the headline feature and the reason HTTP/3 exists.

Integrated handshake. TLS 1.3 is built into QUIC rather than layered on top, so connection setup is 1 RTT for a new connection and 0 RTT for resumption — versus TCP's handshake plus TLS's handshake as separate costs (Module F-5's table).

Connection migration. A QUIC connection is identified by a connection ID rather than the source IP/port tuple. A phone moving from Wi-Fi to cellular keeps the same connection instead of re-establishing everything. For mobile apps this is a genuine, user-visible improvement.

Better loss recovery. Unambiguous packet numbering removes retransmission ambiguities that TCP has carried for decades, giving more accurate RTT estimates and faster recovery.

The trade-offs, named honestly:

More CPU on the server. QUIC runs in user space rather than the kernel's highly-optimised TCP stack, and per-packet encryption costs more. Expect measurably higher CPU per connection.
UDP is sometimes blocked or deprioritised by corporate firewalls and some middleboxes, so clients need to fall back to HTTP/2 gracefully — meaning you support both, not one.
Less mature tooling. Packet capture, debugging, and load-balancer support are all better-trodden for TCP.

The Comparison, and When It Actually Matters

	HTTP/1.1	HTTP/2	HTTP/3
Transport	TCP	TCP	QUIC (UDP)
Framing	Text	Binary	Binary
Concurrency per connection	1	Many (multiplexed)	Many (independent)
Connections per origin (browser)	~6	1	1
App-layer head-of-line blocking	Yes	No	No
Transport-layer head-of-line blocking	Per connection	Yes — one loss stalls all streams	No
Header compression	None	HPACK	QPACK
New-connection setup	TCP + TLS (2–3 RTT)	TCP + TLS (2–3 RTT)	1 RTT (0 with resumption)
Connection survives network change	No	No	Yes

Where the upgrade pays: many small resources on one origin (a typical web page, a chatty API), high-latency links where handshake savings compound, and lossy or mobile networks (HTTP/3 specifically).

Where it barely matters: a single large file download — one stream, nothing to multiplex, bandwidth-bound rather than round-trip-bound. Server-to-server calls inside one datacentre where RTT is already sub-millisecond and loss is negligible. And any request whose time is dominated by a slow database query, which no protocol version will help.

That last one is worth saying out loud, because protocol upgrades are easy to over-attribute. If your p99 is 2 seconds because of a missing index, HTTP/3 will make it 1.98 seconds.

The Consequence Nobody Expects: Load Balancing

Multiplexing changes traffic distribution in a way that catches teams out, and it's the most operationally relevant item in this module.

Under HTTP/1.1, a client opens several connections and a load balancer distributing connections across backends spreads load reasonably well. Under HTTP/2, one client holds one long-lived connection carrying thousands of requests. Distribute by connection and you've pinned all of that client's traffic to one backend.

Consequences:

An L4 (TCP-level) load balancer distributing HTTP/2 connections distributes load badly, because connection count no longer tracks request count. You need an L7 load balancer that balances individual requests across backends (Module F-11).
Scaling out doesn't rebalance existing traffic. New backends receive no traffic from existing long-lived connections, so autoscaling can add capacity that sits idle while the original instances stay hot. Mitigations are deliberate connection-lifetime limits (max_connection_age in gRPC/Envoy terms) that force periodic reconnection and hence redistribution.
Draining a backend during deploys needs protocol awareness. With long-lived multiplexed connections, "stop sending new connections" no longer means "stop sending requests." Graceful shutdown must send GOAWAY and let in-flight streams finish (Modules F-11, O-4).

This is also the reason gRPC requires HTTP/2 (Module F-8): gRPC needs multiple concurrent, independently-flowing streams over one connection, including long-lived bidirectional streams, which HTTP/1.1 simply cannot express. And gRPC services inherit exactly the load-balancing problem above — which is a significant part of why service meshes exist (Module A-5).

Where This Shows Up in Your Stack

Next.js and CDNs: any modern CDN serves HTTP/2 and usually HTTP/3 to browsers by default. But the connection from the CDN to your origin is often HTTP/1.1 — worth verifying, because that's the hop carrying every cache miss.
Build pipelines: aggressive bundling was an HTTP/1.1 workaround. Under HTTP/2, moderate code-splitting is generally better: many small cacheable chunks multiplex fine, and a one-line change invalidates one chunk instead of a 2 MB bundle. Neither extreme is right — dozens of tiny files still cost per-request overhead and compress worse individually.
Node.js: the http2 module exists, but most Node services sit behind a proxy that terminates HTTP/2 and speaks HTTP/1.1 to the app. That's usually fine — the RTT is negligible over a loopback or in-datacentre link — but it does mean the app never sees HTTP/2 semantics, which matters if you intended to serve gRPC directly.
gRPC service-to-service: you get multiplexing benefits and inherit the connection-pinning problem. Client-side load balancing or a mesh proxy, plus connection age limits, is the standard answer (Modules A-5, F-11).
Observability: protocol version belongs in your request metadata. "Slow only for HTTP/1.1 clients" and "slow only on lossy networks" are both real patterns, and neither is visible if you don't record the dimension (Module O-1).

Summary

HTTP/1.1 carries one exchange per connection, pipelining is dead in practice, and browsers used ~6 connections per origin to compensate — which is why sharding, sprites, and mega-bundles existed.
HTTP/2 multiplexes many streams over one connection with binary framing and HPACK header compression, eliminating application-layer head-of-line blocking and making those workarounds counterproductive.
HTTP/2 inherits TCP's head-of-line blocking: one lost packet stalls every stream on the connection. On lossy networks its advantage narrows and can invert.
HTTP/3 runs over QUIC on UDP for genuinely independent streams, a 1-RTT (or 0-RTT) integrated TLS 1.3 handshake, and connection migration across network changes — at the cost of higher server CPU, occasional UDP blocking, and less mature tooling.
The gains concentrate on many-small-resources, high-RTT, and lossy links. A single large download or a slow query sees almost nothing.
Multiplexing breaks connection-based load balancing. One client, one long-lived connection means L7 request-level balancing, connection age limits, and GOAWAY-aware draining.
gRPC requires HTTP/2 for concurrent independent streams — and inherits the same load-balancing problem.

The next module, Client–Server, N-Tier, and the Monolith, moves from the wire to the shape of the application behind it — starting with the argument that the monolith is almost always the correct first answer, and what a tier actually buys you.

Knowledge Check

A team migrates from HTTP/1.1 to HTTP/2 and sees a solid improvement in office testing, but users on mobile networks report little change or occasional regressions. What explains this?

After moving internal services to gRPC over HTTP/2 behind an L4 load balancer, a team finds that autoscaling adds instances which receive almost no traffic while the original instances stay saturated. What's happening?

A service's p99 is 2 seconds, dominated by a database query missing an index. A proposal argues that enabling HTTP/3 will materially improve it. What does the module say?

Test your knowledge with more question sets

Sign in to keep reading

The rest of this module is free — sign in with Google to unlock it and track your progress.

PreviousModule F-5: The Anatomy of a Request Next Module F-7: Client–Server, N-Tier, and the Monolith

Discussion

Join the discussion

Loading comments...