What this module covers: CAP is the most cited and least accurately quoted result in distributed systems. This module states it precisely — during a network partition you choose between linearizability and availability — then dismantles the three misreadings that make it useless in design reviews: that you "pick two of three", that CA is an option, and that it says anything about performance. It then covers why a GC pause is indistinguishable from a cut cable, and why PACELC — which adds the else case, where you trade latency against consistency on every single request — is the framework you actually make decisions with. Finishes by classifying real systems and naming what each one pays.
The Theorem Only Speaks During a Partition
CAP gets invoked in architecture reviews as if it were a personality quiz: we're an AP shop, we picked availability. That framing is wrong in a way that matters, because it makes the theorem sound like a philosophy when it is a narrow impossibility result about one specific situation.
Here is the whole statement, and it is worth being able to recite:
When the network partitions — when some nodes cannot reach other nodes — a distributed system must choose between remaining consistent and remaining available. It cannot be both.
That is it. Not "pick two of three". Not a claim about throughput. A conditional statement whose antecedent is there is a partition right now. When there is no partition, CAP is silent — it permits a system to be both perfectly consistent and perfectly available, and plenty of systems are.
The proof is almost insultingly simple, which is why the theorem is true and why arguing with it is a waste of a meeting. Take two nodes, A and B, holding a replica each of the same value x = 0. The link between them dies. A client writes x = 1 to node A. A second client reads x from node B.
Node B has exactly two options:
Answer with 0. It has responded, so it is available. It has returned a value that a linearizable register would not have returned, because the write to A completed before the read to B began. Consistency lost.
Refuse to answer — block, or return an error — until it can reach A again. Consistency preserved. It did not respond, so it is not available.
There is no third option. No amount of engineering creates one, because the information about the write is on the other side of a link that does not carry information. This is not a limitation of current databases; it is a limitation of causality.
Analogy: two ticket counters at opposite ends of a stadium, sharing one seating chart, connected by a single phone line. The line goes dead. A customer walks up to counter B and asks for seat 14A. Counter B can sell it — and risk that counter A sold it thirty seconds ago, producing two people with a claim to one seat. Or it can say "I can't sell anything until I can reach the other counter," and the queue goes home. It cannot both sell the seat and guarantee the seat is free. The phone line is the only thing that ever made that guarantee possible.
What the Three Letters Actually Mean
Most of the confusion around CAP is vocabulary, not logic. Each letter means something narrower and stranger than its everyday sense.
C is linearizability. Every read observes the most recent completed write, as though there were exactly one copy of the data and all operations happened one at a time in some order consistent with real time. Note what this is not: it is not ACID's C, which is about constraints and invariants inside a transaction. Module P-1 covered that collision — two unrelated properties sharing a letter — and it remains the single most common source of nonsense in CAP discussions. When someone says "we need CAP consistency because we have foreign keys," they have merged two different guarantees.
A is availability in an unusually strict sense: every request that reaches a non-failing node receives a non-error response, eventually, with no time bound. This definition is strict in one direction and absurdly generous in another. Strict, because a system that returns 503 on 0.01% of requests is not available in CAP's sense at all. Generous, because a node that answers after four minutes is available in CAP's sense and useless in yours. Your SLO's availability (Module O-2) is a completely different measurement — a ratio over a window, with a latency threshold baked in. A system can be CAP-available and blow every SLO it has.
P is partition tolerance, and this is the letter people get most wrong: it is not a feature you build. It is an assumption about the environment — that the network is allowed to drop and delay messages arbitrarily. You do not get to decide whether your network does that. You only get to decide what your system does when it happens.
Letter
Everyday reading
What CAP means
C
"The data is correct / constraints hold"
Linearizability — one logical copy, real-time ordering
A
"Our uptime is good"
Every request to a live node gets a non-error answer, eventually
P
"We handle network faults gracefully"
The network may lose messages — a property of the world, not the system
Once P is understood as an assumption rather than a capability, "pick two of three" collapses on contact.
Three Things CAP Does Not Say
1. It is not a design style, and you do not "pick two." The three-way Venn diagram — the one on every slide deck — implies you sit down and select two properties as a strategy. But you do not select P; the network selects it for you. The actual decision surface is one binary choice, taken at the moment a partition is detected, and taken separately for each operation: refuse, or serve possibly-stale data. Everything else in the diagram is scenery.
The practical consequence: a statement like "we're an AP system" is nearly content-free. The useful version names the operation and the behaviour. When the primary is unreachable, the cart-add endpoint accepts writes into the local replica and reconciles later, accepting that two devices can add the same item twice; the checkout endpoint returns 503 rather than charge against inventory it cannot verify. Same system, both branches, per operation. That is a design; "we're AP" is a bumper sticker.
2. CA is not an achievable option for a distributed system. This is the most consequential misreading, because it lets a team believe they have opted out. A "CA system" would be one that is both linearizable and available even during a partition — which the two-node proof above rules out. What "we chose CA" means in practice is: we did not decide what happens during a partition. And an undecided system does not get to be both. It gets whatever its defaults produce, which is usually the worst of both: a node that hangs on a lock it cannot release (unavailable) while a second node accepts conflicting writes (inconsistent). You lose C and A simultaneously, and discover it during the incident rather than during the design review.
There is one honest case that looks like CA, and it is worth naming precisely because it is so common: a single-node database is not a distributed system, so CAP has nothing to say about it. One Postgres instance with no replicas has no internal network to partition. It is trivially linearizable and its availability is bounded by exactly one machine's uptime plus however long a restore takes. That is a real and often correct architecture. But the moment you add a streaming replica and route any read to it, you have built a distributed system and inherited the choice — asynchronous replication means that replica can serve a value the primary has already superseded, and a failover across a partition can lose acknowledged writes outright (Module P-5's split-brain).
3. CAP says nothing whatsoever about performance when the network is healthy. This is the misreading that does the most day-to-day damage, because it gets used to explain latency. Our p99 is 400ms because we're a CP system, that's CAP. No. CAP permits a CP system to answer in one microsecond when there is no partition. If your reads are slow on a normal Tuesday, the theorem is not involved — something else is, and that something else is what the second half of this module is about.
Why this matters in production: partitions in a single well-run datacentre are rare — occasional minutes per year. If CAP were the whole story, the consistency-versus-availability trade would be an edge case you handle in a runbook and otherwise ignore. Teams reason as though that were true, and then cannot explain why their strongly-consistent reads cost 12ms instead of 0.4ms when nothing is broken. CAP is the rare case. The trade-off you are actually paying for, on every request, all year, is the one CAP does not mention.
A Partition Is Whatever Your Peers Cannot Distinguish From One
Before getting to that, correct one more mental model: the picture of a partition as a backhoe through a fibre bundle. Cable cuts happen, but they are a minority of the partitions your system will actually experience, and the picture makes people underestimate the frequency badly.
A partition, operationally, is node X cannot get a timely response from node Y. Everything in this list produces exactly that observation, indistinguishable from the outside:
A stop-the-world GC pause. A JVM or a .NET runtime under memory pressure can stop for seconds. During that window the process does not read its socket, does not send heartbeats, does not do anything. Its peers see silence. There is no way for them to tell "paused" from "unreachable" from "dead", and this is not a tooling gap — it is fundamental. The pause is a partition of one node from everyone.
An overloaded node. A machine at 99% CPU, or with a saturated disk queue, has request latency climbing hyperbolically (Module F-4's utilisation curve). It is answering, just past everyone's timeout. Functionally partitioned.
An asymmetric or one-way route. A misconfigured firewall rule, an ACL applied in one direction, a routing change: A reaches B, B cannot reach A. Half the cluster thinks the other half is dead while the other half thinks everything is fine. This produces the ugliest split-brain scenarios, because the two sides disagree about who is partitioned.
A saturated NIC or a noisy-neighbour hypervisor. Packet loss high enough that retransmits blow your timeout budget is a partition with a probability attached.
A SIGSTOP, a live migration, or a laptop-style suspend on a VM. The process resumes believing no time has passed. It may still hold a lease it no longer owns — which is why leases need fencing tokens (Module A-2).
Two consequences follow, and they are both load-bearing:
Partition detection is timeout-based, and therefore a guess. There is no partition-detection primitive; there is only "I waited 3 seconds and heard nothing." Set the timeout tight and you declare partitions that were merely GC pauses, triggering failovers you did not need. Set it loose and you spend 30 seconds unavailable before reacting to a real failure. That is a real trade with no correct answer, and Module A-6 is where the numbers get chosen.
Therefore partitions are far more frequent than "network incidents" in your dashboards. Every long GC pause and every overload event is a partition from the cluster's point of view. Which means the CAP branch fires more often than the naive model predicts — usually as a brief spike of write rejections or a burst of conflicting versions that nobody connects back to a garbage collection log.
PACELC: The Trade You Make on Every Request, Not Once a Year
PACELC (Abadi) keeps CAP and adds the clause CAP omits. Read it as two conditionals:
If there is a Partition, choose Availability or Consistency. Else — no partition, everything healthy — choose Latency or Consistency.
That else is the whole contribution, and it is the half you live in. With no partition at all, a system that wants a read to be linearizable must confirm with other nodes before answering. Confirmation means at least one network round trip to a quorum, and the round trip has a price set by geography, not by your code:
Coordination scope
Round trip
Floor on a strongly-consistent operation
Same host
—
~0 (single-node, no coordination)
Same datacentre / AZ-local quorum
~0.5 ms
~1 ms, plus fsync (Module F-2)
Cross-AZ within a region
~1–2 ms
a few ms
Mumbai ↔ Virginia
~180 ms
~180 ms, unavoidably
That last row is the one that ends architecture debates. A globally linearizable write with a participant in Virginia and a participant in Mumbai cannot commit faster than the speed of light between them, ~180ms RTT. No amount of tuning moves it, because it is not a software number. If your latency budget (Module F-4) is 200ms end to end, one cross-continent strong write consumes essentially all of it.
So the else branch is where your latency budget is actually spent, and it is a dial rather than a switch. Postgres exposes it directly:
conf
Two lines of configuration, and they are the EL versus EC decision for the entire system. Notice the cost is stated in both directions, which is the rule from Module F-1: on costs latency on every single write; local costs durability of acknowledged writes at exactly the worst moment. There is no line you can write that costs nothing.
The same dial exists per-request in stores designed for it:
ts
The interesting line is the reason in the second comment. ConsistentRead: true is not the safe default to sprinkle everywhere — it costs latency and it costs money on every call, so applying it globally is how a team pays for linearizability on their product catalogue, where nobody would have noticed staleness measured in milliseconds. Consistency is a per-operation purchase.
Why this matters in production: the else branch is the one that shows up in your p99 chart, your egress bill, and your capacity plan, because it fires on 100% of requests instead of during a handful of minutes a year. When someone reaches for CAP to explain a latency regression, the honest answer is that they are looking at PACELC's EL/EC dial and calling it by the wrong name — and the fix is a per-operation consistency decision, not a debate about the theorem.
Where Real Systems Land
PACELC's four-way classification is more useful than CP/AP because it separates what a system does when broken from what it does when healthy. A caveat first: most modern stores are tunable, so a single label describes the default or typical configuration rather than a law. Read the table as "what you get if you do not think about it."
System
PACELC
Notes on the mechanism, and what it costs
Single-node Postgres
n/a
Not a distributed system. No internal network, so no partition and no CAP choice. Linearizable and limited to one machine's fate. Adding a read replica is what enrols you.
Postgres + async replica
PA/EL
Replica serves stale reads; a failover across a partition can lose acknowledged writes. synchronous_commit = on moves it toward PC/EC at the cost of an RTT per write.
Cassandra
PA/EL
Leaderless, quorum-tunable, designed to keep accepting writes on both sides of a partition and reconcile with last-write-wins and hinted handoff. Default consistency levels favour latency. Cost: conflicting versions are your application's problem, and clock-based conflict resolution silently discards a write. QUORUM/QUORUM pushes it toward PC/EC and gives back the latency.
DynamoDB
tunable
Eventually-consistent reads by default (EL); ConsistentRead: true and transactions available per call (EC). Costs latency and roughly double the read capacity. The per-request granularity is the point.
Redis (replicated)
PA/EL
Asynchronous replication; a failover can lose recently acknowledged writes, and Redis Cluster does not offer linearizability. WAIT lets you block for replica acknowledgement — which is the EC dial, priced in latency. Module P-12 goes further.
etcd / ZooKeeper
PC/EC
Raft / ZAB majority quorum. The minority side of a partition refuses writes. Deliberately chosen because these hold configuration and locks, where a stale answer is worse than no answer. Cost: write throughput bounded by consensus, and the minority is down.
MongoDB
PC/EC (default-ish)
Single primary per replica set; majority write and read concerns give strong guarantees. Weaker concerns trade back toward EL.
Spanner
PC/EC
External consistency globally. Refuses on the minority side; pays real latency when healthy. See below.
Kafka
PC/EC per partition
acks=all with min.insync.replicas=2 refuses produces when the ISR shrinks below the threshold — availability traded for not losing writes. acks=1 is the other choice. Module P-15.
Two things to take from this table beyond the labels.
PC does not mean "the system goes down during a partition." A three-node quorum system split 2–1 keeps serving on the majority side, which is where most of your traffic probably is. Only the minority partition refuses. "CP means unavailable" overstates it considerably — the correct statement is that some clients, the ones stranded with the minority, see errors.
PA does not mean "the data is wrong." It means conflict resolution moved into your application, where you now own it. That is sometimes exactly right: a shopping cart merged by union, a set of "seen" flags, a like counter as a CRDT-style counter. It is disastrous for anything with a uniqueness or non-negativity invariant — seat reservations, account balances, inventory. The question is not "how much consistency do we like" but "does this data type have a merge function that is correct?"
What Spanner Pays to Be PC/EC
Spanner is worth a section because it is regularly cited as having "beaten CAP", and it did not. It bought its way to the far end of both dials and posted the price openly.
It is PC: on a partition, the minority side of each Paxos group stops serving writes. Nothing exotic — it refuses, like etcd refuses.
It is EC, and this is where the money goes. Spanner provides external consistency (linearizability plus a real-time ordering guarantee across the whole database) using TrueTime: a clock API backed by GPS receivers and atomic clocks in every datacentre, which returns not a timestamp but an interval — earliest and latest possible now, with a bounded uncertainty. Because the uncertainty is bounded rather than unknown, a transaction can be assigned a commit timestamp and then made to wait out the uncertainty window before acknowledging, guaranteeing that any transaction starting afterwards sees a strictly later timestamp.
That commit-wait is latency added deliberately, on the happy path, to purchase ordering. Plus Paxos round trips to the replicas — which, if your replicas span continents, is the ~180ms floor from the table above.
So the costs, named plainly: specialised hardware in every datacentre (which is why this arrived as a managed service rather than something you install), write latency deliberately inflated by the clock-uncertainty wait, cross-region writes bounded by the speed of light, and unavailability for the minority side of any partition. What you get in exchange is that application developers stop reasoning about staleness — a genuinely enormous saving that most teams undervalue until they have spent a quarter debugging read-your-own-writes bugs.
The general shape here is the one worth internalising: you cannot remove the trade-off, only relocate it — into hardware, into latency, into money, or into your application's conflict-handling code. Module A-1 takes TrueTime apart properly alongside Lamport clocks, vector clocks, and hybrid logical clocks, which are what you use when you do not have atomic clocks in a rack.
Choose Per Operation, Not Per System
The practical output of this module is not a label for your architecture. It is a small table you can actually fill in, one row per operation that matters:
Operation
On partition
When healthy
Why, and what it costs
Read product page
Serve stale
EL — nearest replica
A 30-second-old description harms nobody; costs occasional visible staleness after an edit
Add to cart
Accept, merge later
EL
Union merge is correct for a cart; costs a re-appearing removed item in a rare conflict
Reserve last seat
Refuse (503)
EC — quorum
Double-booking is unrecoverable; costs ~5–10ms per reservation and outright refusal for stranded clients
Debit balance
Refuse
EC
Negative balances are a financial invariant, not a UX preference; same costs, accepted
Record page view
Accept, lossy
EL
An approximate count is fine; costs a small undercount during incidents
Read own profile after edit
Serve from primary, or read-your-writes token
EC for that user only
Users disbelieve a UI that forgets their own edit; costs primary load — Module P-11's session guarantees
Three habits make this table honest:
Write the refusal as a product behaviour, not an exception. "Refuse" that surfaces as a stack trace and a spinner is a worse outage than stale data. Refuse with a specific, retryable error shape (Module F-8), a queued-for-later path, or a degraded mode that says what is unavailable (Module A-7).
Notice when "consistency" was never the requirement. Frequently the actual requirement is no double charge, and that is idempotency (Module P-16), not linearizability. Or it is this counter must not go negative, which a conditional write or a reservation pattern handles without global ordering. Reaching for strong consistency because the invariant felt important is how teams buy 180ms they did not need.
State the branch you take when detection is wrong. Since partition detection is a timeout and therefore sometimes a false positive, every "refuse on partition" decision is also a decision to occasionally refuse during a GC pause. Whether that is acceptable is a separate question from whether the invariant matters.
Where This Shows Up in Your Stack
PostgreSQL:synchronous_commit and synchronous_standby_names are the EL/EC dial for writes; on costs an RTT plus a remote flush per commit, local costs acknowledged writes on failover. Reading from a replica is an EL choice whether or not anyone decided it — check pg_stat_replication's lag before assuming it is small, and remember a long-running query on the replica can stall apply entirely.
Node.js: the client-side half of the P branch. A partition reaches your code as a timeout, so statement_timeout, a client-side deadline, and a circuit breaker (Module A-7) are what turn "the database is unreachable" into a fast typed error instead of a pool full of hung connections behind a 30-second TCP timeout. Retrying a write into a partition without an idempotency key is how one refusal becomes two charges.
Redis: PA/EL by construction — asynchronous replication means a failover can drop writes the client already saw acknowledged, so Redis is the wrong home for a uniqueness invariant even though SETNX looks exactly like one (Module A-2 covers why the naive lock is unsafe). WAIT gives partial EC at the cost of blocking on replica acknowledgement.
DynamoDB / Cassandra: the per-request consistency parameter is the most under-used design lever in either system. Audit where ConsistentRead: true or QUORUM is set: usually a handful of call sites genuinely need it, and either everything has it (paying latency and capacity everywhere) or nothing does (including the seat reservation).
Summary
CAP is a conditional, not a design style — during a partition, a system chooses between linearizability and availability. Outside a partition it says nothing at all, and permits both.
The proof is two nodes and a dead link: the far node either answers with a value that may be stale, or refuses to answer. There is no third branch, and no engineering produces one.
The letters are narrower than they sound. C is linearizability, not ACID's C (Module P-1). A means every request to a live node gets a non-error answer with no time bound — unrelated to your SLO. P is an assumption about the network, not a feature you build.
"Pick two of three" is wrong because P is not yours to pick. The real decision is one binary branch, taken per operation, at the moment a partition is detected.
CA is not an option for a distributed system. A single-node database is genuinely CA-shaped because it is not distributed — but "we chose CA" for a replicated system means "we never decided", and an undecided system loses consistency and availability together during the incident.
CAP says nothing about performance on a healthy network. Using it to explain a p99 regression is the misquote that does the most daily damage.
A partition is whatever peers cannot distinguish from one: GC pauses, overloaded nodes, asymmetric routes, saturated NICs, suspended VMs. Detection is timeout-based and therefore a guess, which makes partitions far more frequent than your network dashboard suggests.
PACELC adds the clause that matters:else — with no partition — you trade latency against consistency on every request. That branch fires 100% of the time, priced by geography: ~0.5ms same-datacentre, ~180ms Mumbai↔Virginia, with no software fix.
Classify by default behaviour, then tune per operation. Cassandra PA/EL, DynamoDB tunable per call, etcd and Kafka PC/EC by design, Spanner PC/EC bought with atomic clocks and a deliberate commit-wait (Module A-1). PC does not mean down — only the minority side refuses. PA does not mean wrong — it means conflict resolution moved into your application, where it needs a correct merge function.
The takeaway: CAP eliminates one fantasy — that a partition can leave you both consistent and available. PACELC describes the trade you are actually making all day, on every request, and it is the one worth writing down per operation with its cost named.
The next module, Consistency Models, replaces this module's single binary with the spectrum it hides: linearizable, sequential, causal, and eventual, plus the session guarantees — read-your-writes, monotonic reads — that determine whether a user believes your interface. CAP tells you a choice exists; consistency models tell you what the options in between actually promise, and which of them you can get for free.
Knowledge Check
In a design review, a staff engineer argues that because the service runs in a single cloud region with redundant switches, the team can treat the cluster as CA and skip deciding what happens when nodes lose contact with each other. What does the module say about this position?
A team's cross-region reads sit at a p99 of roughly 200ms. There have been no network incidents for months, and monitoring confirms no partitions. An engineer explains the latency as "the CAP trade-off — we chose consistency, so we pay for it." How should this be corrected?
During a deploy, one node in a five-node quorum cluster suffers a multi-second stop-the-world GC pause. Peers stop hearing from it, writes it was coordinating fail, and a burst of conflicting versions appears. The on-call engineer says this cannot be a CAP situation because the network was healthy throughout. What is the correct reading?
Test your knowledge with more question sets
Sign in to access a wider variety of questions and get notified when new practice sets are added to this module.
# EC — every commit waits for one replica to flush WAL before returning.
# Cost: +1 network RTT and +1 fsync on the commit path, on every write, forever.
synchronous_standby_names = 'ANY 1 (replica_a, replica_b)'
synchronous_commit = on
# EL — commit returns after the local flush. Faster by an RTT.
# Cost: a failover can lose transactions the client was already told succeeded.
# synchronous_commit = local
// Same table, two operations, two different answers to the E-else question.// EL: served from whichever replica is nearest. Single-digit ms.// Cost: may not reflect a write that completed moments ago.const product =await ddb.get({ TableName:'products', Key:{ sku }, ConsistentRead:false,// the default, and the cheaper read});// EC: quorum read. Higher latency, and roughly double the read capacity billed.// Cost is deliberate: overselling stock is worse than 8ms.const stock =await ddb.get({ TableName:'inventory', Key:{ sku }, ConsistentRead:true,});