---

The Write Path Is the Whole Problem

Reads replicate and cache. Writes are where the design lives, and there are only two shapes:

Route writes to an owning region. Correct, conflict-free, and the write pays the round trip. Mitigations that make this acceptable in practice: optimistic UI (show the result locally, reconcile on response), making writes asynchronous where the user does not need confirmation, and accepting the write at the edge into a durable local queue then forwarding — which is fine when the operation is idempotent and eventual, and wrong when the user needs a definite answer ("did my payment succeed?").

Accept the write locally and replicate. Fast, available under partition, and you now own conflicts.

Last-write-wins is silent data loss

Module A-1 established the mechanism; it needs restating here because it is the default in more systems than teams realise, including DynamoDB Global Tables:

Last-write-wins does not resolve a conflict. It picks one write and destroys the other, using evidence that may be wrong. It is acceptable where losing one of two concurrent writes is genuinely fine — a presence flag, a theme preference, a cache entry. It is not acceptable for anything additive or authored: a document, a cart, a set of collaborators, a counter, a ledger. In those cases the discarded write is information nobody can reconstruct, and the user experience is "it didn't save" with nothing in any log.

Detect concurrency before deciding anything

You cannot make a good resolution decision without knowing that a conflict exists, and a timestamp comparison cannot tell you. Version vectors (Module A-1) can: they distinguish "B happened after A" from "A and B are concurrent." Systems that expose concurrency return siblings — both versions — which is honest and pushes the decision to the application. That is more work and strictly better than silently choosing.

CRDTs, and the constraints they cannot express

When no human can arbitrate — offline-first clients, collaborative editing, active-active writes — the data type itself must converge deterministically. That is a CRDT, and the useful catalogue is short:

And the limitation that matters most, which is a theorem rather than an implementation gap:

A CRDT cannot enforce any invariant that requires knowing about other replicas' writes.

Concretely, none of these can be guaranteed under active-active with local writes:

• Uniqueness — "this username is taken." Two regions can both accept it.
• Non-negativity — "inventory must not go below zero." Two regions each sell the last unit.
• A global limit — "at most 5 seats on this plan," "at most 100 attendees."

Those require coordination, which means either a single writer for that key (topology 4) or a consensus write (topology 6). The honest statement to make to a product owner is:

"Never oversell" and "always writable in every region" are incompatible. Pick one, per invariant.

Escrow: partition the budget so local decisions are safe

The technique that resolves many of these cases without coordination, and which is genuinely useful:

You have traded a small amount of stranded inventory for the ability to decide locally. That trade is almost always worth it for rate limits, quotas, seat counts, and stock at reasonable volumes — and it is the same idea as Module P-18's per-tenant limits, applied per region instead of per tenant.

---

Data Residency Reshapes the Topology, Including the Parts You Forgot

When law requires that EU personal data stay in the EU (Module O-9 develops the legal frame), the consequences are structural rather than a configuration flag:

• There is no single global table. The partition key must be jurisdiction, which forces topology 4 whether you wanted it or not.
• A global secondary index may itself be a violation if it materialises regulated fields outside the region.
• Backups and DR targets are in scope. A disaster-recovery region in another jurisdiction is a violation of exactly the rule you are complying with — which means residency and DR must be designed together, with the DR site inside the same jurisdiction.
• Cross-region analytics is in scope. A warehouse that ingests everything (Module P-22) is a residency leak by design unless it is partitioned or the data is tokenised before it leaves (Module P-23's argument for ETL over ELT in this specific case).
• And the one that catches everyone: logs, traces and metrics. Observability pipelines routinely ship everything to one central region, and request logs contain IP addresses, user IDs, emails in URLs, and request bodies. Your compliance story can be perfect at the database layer and violated by your APM agent. Regionalise the telemetry pipeline, or scrub before it crosses a boundary (Module O-1).

---

Failover: Why Two Regions Cannot Safely Fail Over Automatically

The failure of failover is a genre. The recurring causes:

• DNS TTLs and pooled connections keep clients pointed at the dead region (Modules A-5, A-9's routing section) — the reason a global load balancer or anycast is worth having.
• Stateful components need promotion, not just routing. A replica must be promoted (Module P-5), and until it is, the new region has no writable database — so routing traffic there produces errors rather than service.
• The untested runbook. Failover involves promotion order, connection-string changes, cache warm-up and a data-loss decision, and a procedure written once and never rehearsed will not execute correctly under pressure (Modules O-6, O-7).
• Cold caches, so the surviving region's per-request cost is higher exactly when it is taking double traffic — which is Module A-8's warning about scaling into an overload.

And the structural one, which is Module A-1's two-node problem at region scale:

With exactly two regions, neither can safely decide that the other has failed.

Region A cannot distinguish "B is down" from "I am partitioned from B" — and if B is alive and reaches the same conclusion about A, both promote, both accept writes, and you have split brain with divergent data and no protocol to reconcile it. There is no majority available in a set of two.

The fix is a witness (or arbiter) in a third location — a small, cheap quorum member that holds no data but votes. Two regions plus a witness is three votes, so a majority exists and exactly one side can win. This is why serious multi-region designs have an odd number of failure domains even when they have an even number of serving regions, and it is the single most common omission in a two-region plan.

Absent a witness, the honest options are manual failover (a human decides, accepting the RTO cost) or accepting that automatic failover carries a split-brain risk. Choose deliberately; do not let a health-check script make the decision by default.

---

The Cost, Which Is Larger Than the Design Documents Say

Cross-region architecture has a specific bill, and cross-region data transfer is the most under-estimated line in this course:

• Every write replicated to N regions costs egress N−1 times. At meaningful write volumes this becomes one of the largest items on the invoice — replication traffic is charged per gigabyte, in both directions in some configurations, and it grows linearly with both write volume and region count (Module O-8).
• N regions means N sets of baseline capacity, and in active-passive one of them earns nothing.
• Chatty cross-region calls are billed as well as slow. A service in region A calling a database in region B pays 75 ms and per-gigabyte egress on every query — which is why the "just point it at the other region's database" shortcut is expensive twice.
• Cross-AZ traffic is also charged in most clouds, which is why topology-aware routing (Module A-5) is a cost control as much as a latency one.

And the operational cost: per-region dashboards are mandatory. A global average hides a dead region completely — 33% of your users failing shows up as a modest dip in an aggregate. Every SLI needs a region dimension, plus synthetic checks originating in each region (Module O-2).

Why this matters in production: the multi-region failures that happen are rarely "the region went down." They are a stale DNS entry sending 40% of traffic to a region with no writable database; a last-write-wins policy nobody chose, silently discarding user edits for eight months; a residency violation discovered in an audit because the tracing pipeline ships everything to one region; a failover that was never tested and takes four hours; and an egress bill that doubled without a corresponding change in traffic. Every one of those is a consequence of the topology being adopted for one reason and inheriting the failure modes of another.

---

Where This Shows Up in Your Stack

• PostgreSQL: cross-region streaming replication is realistically asynchronous — synchronous would put 75 ms on every commit — so your RPO is your replication lag, and you must know it (Module O-7). Logical replication enables selective and bidirectional setups, and bidirectional means conflicts, which core Postgres does not resolve for you. Run Patroni with its DCS (etcd/Consul) spanning three failure domains including a witness, never two, or automatic failover is a split-brain generator (Module A-1).
• Redis: do not replicate a cache across regions. Give each region its own cache, warmed from its own reads — it is cheaper, simpler, avoids egress, and a cache miss is not a correctness problem (Module F-12). Active-active Redis (CRDB) exists in the commercial product with CRDT semantics, and the same caveat applies as to all CRDTs: it converges, and it cannot enforce a global invariant.
• Kafka: MirrorMaker 2 or Cluster Linking replicate topics across regions, and the detail that bites is that offsets do not translate directly between clusters, so a consumer failing over must translate its position or accept reprocessing (which is fine if it is idempotent — Module P-16). Decide per topic which region is authoritative; bidirectional replication of the same topic invites loops and duplicate processing.
• DynamoDB Global Tables: the canonical concrete example of the trap — multi-region active-active with last-write-wins by default. It works exactly as documented and will silently discard concurrent writes, so it fits topology 4 (partition so no two regions write the same item) far better than topology 5.
• Spanner / CockroachDB: the real implementations of topology 6, and both let you pin data to a locality, which converts a global-consensus cost into a local one for records that belong to one place. That capability is the reason to choose them over hand-rolling topology 4.
• CDN and edge: the topology-1 tools, and the ones to exhaust first. An edge function that serves a cached, personalised-by-cookie fragment removes far more perceived latency than a second database region, at a fraction of the cost and none of the consistency risk (Modules F-10, F-15).

---

Summary

• Three different requirements hide in "we need multi-region": user latency (wants data near readers), regional failure survival (wants a complete tested stack elsewhere), and data residency (wants partitioning by jurisdiction and forbids global replication). They lead to different architectures, and conflating them is the primary design error.
• "High availability" alone rarely justifies it. Multi-AZ gives independent facilities at single-digit-millisecond latency, so synchronous replication works and there are no conflicts. The honest drivers for crossing a region boundary are latency and law.
• The physics is fixed: ~75 ms London–Virginia, ~250 ms London–Sydney, and a consensus write costs a round trip to a quorum. Good multi-region design is about removing cross-region round trips from the critical path, not speeding them up.
• Start at topology 1 — CDN plus one region — and measure how much of your latency is actually the dynamic call before building anything more. Most perceived slowness for distant users is cacheable.
• Topology 4, active-active over partitioned data, is where most systems should stop. One region is the single writer for each record, so conflicts cannot occur, both regions are warm (making failover real), and writes are local. Its requirements: a partition key aligned with users, a saga for cross-partition operations, a migration path for users who move, and a fence so a recovered region stops writing another's partition.
• Route by geography for entry, and by home region for writes. The home region is a property of the user carried in the session, not of their current location — otherwise a travelling user's writes are geo-routed into a region that does not own their data.
• DNS-based failover disappoints because of caching and pooled connections; anycast or a health-checked global load balancer reconverges in seconds without depending on client caches.
• Last-write-wins does not resolve conflicts — it destroys one write using evidence that may be wrong, since ordinary clock skew inverts the order of writes made 50 ms apart. Acceptable for a presence flag or theme preference; unacceptable for anything authored or additive, where the lost write is unreconstructable and the user experience is "it didn't save" with nothing logged. It is the default in DynamoDB Global Tables.
• Detect concurrency before resolving it. Version vectors distinguish "later" from "concurrent"; returning siblings is honest and better than silently choosing.
• CRDTs converge automatically and provably cannot enforce invariants that depend on other replicas' writes — uniqueness, non-negativity, or a global limit. So "never oversell" and "always writable everywhere" are incompatible, per invariant, and that sentence belongs in a product conversation. LWW-Registers are a CRDT wrapper around the loss, and add-wins sets accumulate tombstones that need compaction.
• Escrow partitions a global budget into per-region allocations, enabling purely local decisions with no coordination and no overselling, at the cost of occasionally stranded inventory. It is the practical answer for seats, quotas, rate limits and stock.
• Residency reshapes the topology and reaches further than the database: no global table, secondary indexes that may themselves violate, DR targets that must stay in-jurisdiction, analytics pipelines that leak by design, and — most commonly missed — logs, traces and metrics containing PII shipped to one central region. Your database compliance can be perfect and your APM agent non-compliant.
• With exactly two regions, neither can safely declare the other failed — "you are down" and "I am partitioned" are indistinguishable, and both promoting means split brain with no reconciliation protocol. Add a witness in a third location so a majority exists, or accept manual failover; do not let a health-check script make the call by default.
• Failover fails for mundane reasons: stale DNS and pooled connections, a replica that was never promoted so the region has no writable database, an unrehearsed runbook, and cold caches raising per-request cost exactly when traffic doubles.
• Cross-region egress is the most under-estimated cost in this course. Every write replicated to N regions is charged N−1 times, chatty cross-region calls pay latency and per-gigabyte transfer, and cross-AZ traffic is billed too — which makes topology-aware routing a cost control.
• Every SLI needs a region dimension, plus synthetics originating per region, because a global average hides a completely dead region as a modest dip.

The next module, Real-Time Systems, takes on the connection model this one assumed away. Everything so far has been request/response; A-10 covers long polling, server-sent events and WebSockets, what it takes to hold a million concurrent connections, presence that is not a lie, and the reconnect storm that arrives the moment a region — or a single deploy — drops every socket at once.

---