Module A-5·30 min read

Registries, client- vs server-side discovery, the sidecar model, and a frank accounting of what a mesh costs in latency and operational surface.

JJS
Written by Jatin Jain Saraf · Senior Software Engineer

Module A-5 — Service Discovery and Service Mesh

What this module covers: Why a list of addresses in a config file is stale before it deploys, and the five answers in the order they were invented — static config, DNS, client-side discovery, server-side discovery, and the sidecar. The DNS trap that catches everyone: a connection pool holding keep-alive sockets never re-resolves, so a TTL you tuned carefully changes nothing. Self-registration versus platform registration, and the stale entry left by a process that was killed. Health checks in depth, including the readiness probe that turns a database blip into a total outage, and connection draining without which every deploy returns 502s. Then the service mesh: what the data plane genuinely gives you — mTLS with rotating identities, uniform retries and breakers, traffic shifting, and per-hop telemetry — and a frank accounting of what it costs in latency, memory, operational surface and specialist skill, because that bill is routinely understated. Finally, the ladder of cheaper options and the conditions under which each rung is the right one.


Addresses Are Not Stable, So a List of Them Is Always Wrong

In fixed infrastructure, payments.internal resolved to 10.0.4.12 and stayed there for years. A config file was a reasonable answer.

Then instances became ephemeral. Autoscaling adds and removes them by the minute; a rolling deploy replaces every one of them; a spot instance is reclaimed with two minutes' notice (Module O-8); a container is rescheduled onto a different node with a different IP. So the question a caller needs answered is no longer "what is the address of payments" but:

Which instances of payments exist right now, which of them are healthy, and which should I send this request to?

That is service discovery, and it has three parts that are worth keeping separate because different mechanisms solve them to different standards: a registry of what exists, health information about what is usable, and a selection policy for choosing among the usable ones (Module F-11's load-balancing algorithms).

Analogy: a hospital switchboard. A printed directory of extensions is fine while consultants have permanent offices. Once they rotate between wards each shift, the directory is wrong by Tuesday — so you need a live board showing who is on duty and not currently in surgery, plus a rule for which of three available registrars to page. Notice that "on duty" and "free to take a call" are different facts, and conflating them is how you page a surgeon mid-operation or, worse, mark the entire department unavailable because the ward telephone is broken.


Five Mechanisms, in the Order They Were Invented

1. Static configuration

Addresses in a config file or environment variable. Correct for fixed infrastructure and for a small number of stable endpoints. It fails the moment instances are created and destroyed automatically, and it fails silently — the config still parses, requests just go nowhere.

2. DNS

Universal, requires no library, and works from any language. Also full of traps, and they are worth enumerating because DNS-based discovery is where most teams start.

  • TTL is a hint, not a contract. Many clients cache more aggressively than the TTL suggests. The JVM historically cached resolutions for the process lifetime by default; various libraries and OS resolvers cache independently. A 5-second TTL you set carefully may be observed by nobody.
  • No health awareness beyond removal. DNS tells you an address exists; whether the process behind it is answering is a separate question, and removing a record propagates only as fast as the caches allow.
  • Round-robin only. No least-connections, no zone awareness, no outlier ejection. You get whatever order the resolver returns.
  • No ports. SRV records carry them and support is patchy enough that most systems do not rely on it.

And the trap that catches nearly everyone:

A connection pool holding keep-alive sockets does not re-resolve DNS.

Your HTTP client resolves payments.internal once, opens ten connections, and reuses them for hours because keep-alive is doing exactly what it is supposed to (Module F-6). The instance behind those sockets is terminated, or the load balancer's IPs rotate, and your client keeps trying the address it resolved at start-up — while dig from the same host returns the correct answer, which is what makes this so confusing to diagnose. The mitigations are a maximum connection lifetime (recycle sockets every 60 seconds regardless of health), a maximum requests-per-connection, or a client that re-resolves on a schedule. This is also why AWS documents that clients of an ALB must respect DNS TTLs: the ALB's own addresses change, and a long-lived pool will eventually be pointed at nothing.

3. Client-side discovery

The client queries a registry (Consul, etcd, Eureka, ZooKeeper), receives the full list of healthy instances, and load-balances itself.

Gains: no extra network hop, and the client can be genuinely smart — least-connections, zone-aware routing to avoid cross-AZ charges (Module O-8), subsetting so each client talks to a subset of a large fleet rather than holding a thousand connections, and immediate reaction to a registry change.

Costs: a discovery library per language, which must be kept in step across every service — the coupling Module A-4 warned about — and every client is now responsible for getting load balancing and health interpretation right. In a polyglot estate this is the thing that does not scale, because "upgrade the discovery library everywhere" becomes a quarterly project.

4. Server-side discovery

The client talks to one stable address — a load balancer, a Kubernetes Service, a proxy — which knows the instances.

Gains: language-agnostic, centralised policy, nothing to upgrade in the clients.

Costs: an extra hop on every call, and the balancer is a critical-path component you must make highly available. It also cannot do client-aware things like zone-local preference as well as a client can, though modern balancers approximate it.

5. Sidecar (the mesh data plane)

A proxy process runs beside every service instance. The application connects to localhost, and the sidecar handles discovery, load balancing, retries, encryption and telemetry.

Gains: language-agnostic and client-side-smart, with no shared balancer to bottleneck, and policy managed centrally without touching application code.

Costs: the subject of the second half of this module.

Extra hopPer-language workSmart LBCentral policy
Static confignononenono
DNSnononenono
Client-sidenoa library per languageyeshard
Server-sideyesnonelimitedyes
Sidecaryes (localhost)noneyesyes

Registration: Who Tells the Registry, and What Happens When Nobody Does

Self-registration. The service registers itself on start-up and deregisters on shutdown. Simple, and it has a specific failure: a process that is SIGKILLed, OOM-killed, or on a node that lost power never deregisters. The registry now advertises an instance that does not exist, and every client sending traffic there gets connection refused — or worse, a timeout, if the whole node vanished and the packets go nowhere.

The mitigation is a TTL with a heartbeat: the registration expires unless renewed, so a dead instance ages out. Which is the same lease pattern as Module P-19's job claim, Module A-2's lock and Module P-16's idempotency record, for the same reason — any ownership must expire on its own — and it comes with the same tuning trade: a short TTL detects death quickly and generates constant heartbeat write traffic against a consensus system (Module A-1), while a long TTL leaves a black hole in the pool for its duration.

Third-party registration. The platform registers on the service's behalf, because the platform is the thing that knows whether the process exists. Kubernetes does this: a pod passing its readiness probe is added to the EndpointSlice, and one that dies is removed by the control plane whether or not it managed to run any shutdown code.

Prefer platform registration. It removes an entire class of stale-entry bug, because the component doing the registering is not the component that died.


Health Checks Are Where Discovery Goes Wrong

Discovery decides where; health decides whether. Getting the second wrong causes outages, and there is one mistake that causes most of them.

Liveness and readiness are different questions

  • Liveness: "is this process broken beyond recovery?" A failing liveness check means restart me.
  • Readiness: "should I receive traffic right now?" A failing readiness check means take me out of the pool, but leave me running.

Conflating them produces a restart loop: a slow dependency makes the check fail, the platform restarts a perfectly healthy process, the restart makes things worse, and the loop continues until someone intervenes.

The readiness probe that causes a total outage

Sign in to keep reading

The rest of this module is free — sign in with Google to unlock it and track your progress.

Sign in & Register

Discussion

0

Join the discussion

Loading comments...

© 2026 Jatin Jain Saraf (JJS). All rights reserved.