The cost is WAL volume, potentially large on wide tables, and therefore replication bandwidth and storage. Set it per table where the before-image is genuinely needed, not globally.

Unchanged TOASTed values are not in the WAL record. Postgres stores large values (long text, JSON, vectors from Module P-21) out of line, and if an UPDATE does not modify such a column, its value is not included in the logical change event — it appears as a placeholder such as __debezium_unavailable_value. A consumer that writes the event straight into a warehouse or search index therefore overwrites a real document body with a placeholder string. There are exactly three responses: set REPLICA IDENTITY FULL (which does include them), have the consumer merge the event onto its existing state rather than replacing it, or have the consumer re-read the row from the source. This bug is famous for reaching production because it only affects rows with large columns updated without touching them, which is a shape that test data rarely has.

DDL is not in the logical stream. Postgres logical decoding emits row changes, not schema changes, so a consumer discovers a renamed column by receiving unfamiliar fields. Debezium mitigates this with a schema-history topic; a hand-rolled consumer needs an explicit change-management process. Which is a good moment to say it plainly: writing your own logical-decoding consumer is more work than it appears — snapshots, schema tracking, restart positions, heartbeats and slot management are the actual product — so use Debezium or your platform's managed CDC unless you have a specific reason not to.

---

The Initial Snapshot Is the Hard Part, Not the Stream

Streaming changes from now on is straightforward. Getting the existing 400 million rows into the destination consistently with that stream is where designs go wrong.

The naive sequence — lock the table, copy it, start streaming — is correct and unusable, because locking a production table for the hours a copy takes is an outage (Module O-4).

The standard mechanism, and the reason it works:

1. Create the replication slot first. From this moment every change is retained, whatever happens next.
2. Read the table in a repeatable-read snapshot (Module P-1), exported from the slot's creation point, so the copy sees a single consistent instant.
3. Stream from the slot afterwards. Changes that occurred during the copy are replayed from the log.
4. Apply everything as idempotent upserts keyed on the primary key, so a row appearing in both the snapshot and the stream converges rather than duplicating (Module P-16).

Point 4 is what makes the overlap harmless and is the reason CDC pipelines can be at-least-once without being wrong.

For very large tables, modern tooling uses incremental snapshotting (the DBLog approach in Debezium): the table is copied in chunks while streaming runs concurrently, using watermark events inserted into the log to determine, for each chunk, whether a streamed change supersedes a snapshotted row. No long transaction, no long-held snapshot — which matters because a repeatable-read transaction open for six hours blocks vacuum for six hours (Module F-13) and, on a replica, guarantees the recovery conflicts of Module P-5. The practical upshot: chunked, resumable snapshots are not a refinement, they are what makes CDC viable on a table you cannot afford to hold a snapshot over.

You also need to *re-*snapshot occasionally: after a slot is invalidated, after a bug in a consumer, after a schema change that requires reinterpretation. Treat "re-snapshot this table" as a routine operation with a runbook, not an emergency.

---

Delivery, Ordering, and the Referential Integrity You Lose

Delivery is at-least-once. A consumer that crashes after applying a change but before confirming its LSN will receive that change again. So every CDC consumer must be idempotent (Module P-16) — and here idempotency is cheap, because a row change keyed by primary key is naturally an upsert. Carry the LSN or a version and gate the write, which is Module P-17's version predicate again:

Ordering is per-key, if you arrange it. Partition your Kafka topic by primary key (Module P-15) and all changes to one row are ordered, which is the guarantee that matters — a stale update can never overwrite a fresh one for the same row. What you do not get is ordering across keys or across tables, because different keys are in different partitions and consumed in parallel.

That loss has a concrete consequence: referential integrity does not survive the pipeline. An order and its order_items are written in one source transaction, and downstream they may arrive on different partitions, minutes apart. A consumer that enforces foreign keys on the destination will reject the child row that arrives before its parent. The options:

• Do not enforce foreign keys downstream. Standard for warehouses and search indexes; eventual consistency is accepted and reconciled by the periodic reload.
• Buffer and wait for the parent, with a timeout — real complexity for a real requirement.
• Group by transaction ID. Debezium exposes transaction metadata, so a consumer can apply a whole source transaction atomically. This costs throughput and buffering, and is worth it only when the destination genuinely needs transactional consistency.
• Use a single partition to preserve global order, and accept that your throughput ceiling is one consumer.

Two more properties worth knowing. Heartbeats matter: on a database where the CDC'd tables are quiet but other tables are busy, the slot's confirmed position never advances, so WAL accumulates from the tables you are not capturing — a heartbeat mechanism periodically advances the slot and prevents exactly the disk-fill scenario above. And lag is your SLI: measure the difference between source commit time and destination apply time, per table, because every downstream consumer's freshness is bounded by it.

---

ETL or ELT: Where the Transformation Belongs

ETL transforms before loading: extract, reshape and clean in a pipeline, then write the finished model to the destination. ELT loads raw data first and transforms inside the destination with SQL (dbt being the common tooling).

ELT has become the default for good reasons, all of which follow from Module P-22:

• Warehouse compute is elastic and cheap, so transformation there costs less than a bespoke pipeline's servers.
• Raw data is preserved, so a transformation bug is fixed by re-running SQL rather than by re-extracting from a source that may no longer hold the history. This is the biggest practical advantage: replayability.
• Transformations become version-controlled, tested SQL that analysts can own, rather than pipeline code only engineers can change.
• Adding a new derived model does not require touching ingestion.

ETL remains correct in specific cases, and they are not edge cases:

• PII must not land in the destination at all. If the warehouse is not permitted to hold card numbers or health data, masking or tokenising before load is a hard requirement (Module O-9) — you cannot delete your way out of having loaded it.
• Volume must be reduced first, because loading raw high-cardinality event data would cost more than it is worth.
• Data residency forbids the raw data crossing a boundary that the destination sits on the other side of.
• The destination cannot express the transformation — parsing a proprietary binary format, calling an external enrichment API.

The honest framing is not ETL versus ELT but "transform as late as possible, subject to what you are not allowed to store."

---

CDC or the Outbox? They Answer Different Questions

This is the architectural decision, and conflating the two produces the most expensive coupling in an event-driven system.

CDC captures physical row changes. The event's shape is your table's shape. So every consumer becomes coupled to your schema: rename a column and downstream breaks; split a table and downstream breaks; and consumers must reconstruct business meaning from field diffs — "status went from pending to shipped" rather than "the order shipped." You cannot refactor your database without a cross-team migration, which is the opposite of the autonomy microservices were meant to buy (Module A-4).

The outbox publishes intentional domain events. In the same transaction as the business write, insert a row describing what happened in a contract you designed:

The event is explicit, versioned, and stable across schema refactors — a published interface rather than a leaked implementation.

And the pattern most mature systems land on, outbox plus CDC: write domain events to an outbox table in the business transaction, and let log-based CDC stream that one table to Kafka. You get the outbox's explicit contract with none of the polling, no relay process to operate, and no DELETE-driven bloat on the outbox table — plus, because the outbox row is only ever inserted, the CDC stream for it is trivially simple. It is the combination that gives you a stable published interface and a low-latency, low-load transport.

Use raw table CDC where the destination genuinely wants your data rather than your events: the warehouse (Module P-22), the search index (Module P-20), a read replica in another engine, and cache invalidation — where it finally solves the problem Module P-13 left open, since a CDC-driven invalidation catches writes from migrations and manual fixes that an application-level cache purge never sees.

---

Schema Evolution Downstream, and Making Deletion Reach Everywhere

Once changes stream to N consumers, your schema is an API with unknown callers. The discipline that keeps this working:

• Additive changes only, by default. Adding a nullable column is safe; renaming or retyping one is a coordinated migration across every consumer. This is Module O-4's expand/contract, applied to a data contract rather than a deployment.
• A schema registry with compatibility enforcement (Avro/Protobuf with backward compatibility checks) turns "we broke the warehouse" from a discovery into a rejected build.
• Tolerant readers. A consumer should ignore unknown fields rather than failing, so a producer adding a field does not require a lockstep deploy.
• Version the payload in outbox events, and support two versions during a transition.

The deletion story is the one with legal weight. CDC is what makes deletion propagable: a DELETE in the source emits a delete event, and each consumer can remove its copy. Three complications, all of which Module O-9 develops:

• Soft deletes break propagation. If your application never issues a real DELETE, the stream carries an UPDATE setting deleted_at, and each consumer must be written to interpret that as a removal. Most are not, so the data quietly persists downstream.
• Kafka's log is immutable. Even after every consumer deletes its copy, the personal data remains in the topic's retention window — and in a compacted topic, a tombstone eventually removes the key, but only on compaction. For a genuinely immutable log the answer is crypto-shredding: store the personal fields encrypted with a per-subject key and delete the key, which renders every copy of every record unreadable at once without rewriting any of them.
• You must know your consumers. A deletion requirement you cannot enumerate the destinations of is a deletion requirement you cannot satisfy. Maintain the list.

Why this matters in production: CDC's failures are mostly invisible from the source. The primary is healthy, the application is fine, and meanwhile the connector has been down for four hours (so the warehouse is stale, the search index is stale, and the WAL is growing), or a TOAST placeholder has been overwriting document bodies for a week, or a consumer that ignores deleted_at has been retaining data you have promised to erase. The three alarms that catch nearly all of it: retained WAL bytes per slot, per-table end-to-end lag, and a periodic row-count-plus-checksum reconciliation between source and destination.

---

Where This Shows Up in Your Stack

• PostgreSQL: wal_level = logical (a restart), a PUBLICATION naming the tables, and a logical replication slot. Alert on pg_replication_slots.active = false and on retained WAL bytes, and set max_slot_wal_keep_size so a dead consumer breaks itself rather than the database. Set REPLICA IDENTITY FULL only on tables whose before-images you need, and know the TOAST placeholder behaviour before it corrupts a destination. For Postgres-to-Postgres, built-in logical replication needs no Debezium at all. pg_recvlogical is the debugging tool that shows you what is actually in the stream.
• Debezium / Kafka Connect: the default answer, because snapshots, schema history, restart offsets, heartbeats and incremental snapshotting are the real work. Partition on the primary key for per-key ordering (Module P-15), enable heartbeats when the captured tables are quieter than the database, and use the transaction-metadata topic if a destination needs source transactions applied atomically.
• Node.js: you can consume logical decoding directly, and for a single-purpose stream it is reasonable — but budget for snapshotting and schema tracking, not just the stream. On the consumer side, the whole apply path is an upsert gated on LSN, which makes idempotency almost free (Module P-16).
• Redis: CDC-driven cache invalidation is the proper fix for Module P-13's hardest case, because it invalidates on the database's view of a change and therefore catches migrations, cascades and manual fixes that an application-level purge misses. The trade is that invalidation is now asynchronous, so the staleness window is your CDC lag.
• Search and vectors: the sound sync mechanism from Module P-20 and Module P-21, with the same benefit that a reindex becomes "reset the offset and replay."
• Warehouse sinks: land raw, transform with SQL (ELT), and reconcile periodically with row counts and checksums per table — because the pipeline being up is not evidence that it is complete.

---

Summary

• Polling updated_at is not a change feed. It cannot see deletes, cannot see writes that bypass the application, costs the OLTP database continuous work, and loses rows to a race that is unfixable in principle: now() is transaction-start time while visibility is at commit, so a long transaction commits a row behind your watermark and it is never read. Overlap windows only shrink the probability, which is why the bug fires during load spikes.
• Trigger-based capture is correct and lives on your write path. It fixes deletes, out-of-band writes and the watermark race, at the cost of doubling writes, adding latency to user transactions, creating a high-churn table that bloats, requiring per-table maintenance, and making bulk migrations dramatically more expensive.
• Log-based CDC reads the log the database already writes, so you get every committed change including deletes, in commit order, with before/after images and transaction boundaries, with no application code and no polling load, resumable by LSN.
• A replication slot will fill your disk if its consumer stops — retention is unconditional, and a Postgres primary with a full disk stops accepting writes. An inactive logical slot also blocks vacuum. Alert on retained WAL bytes and active = false, set max_slot_wal_keep_size so the stream breaks instead of the database, and drop slots you stop using.
• Two Postgres details silently ruin streams: REPLICA IDENTITY defaults to logging only the primary key, so before-images are unavailable until you set FULL (paying WAL volume); and unchanged TOASTed columns are absent from the change event, so a consumer that blindly writes the event replaces a real document body with a placeholder — merge onto existing state, re-read the row, or use REPLICA IDENTITY FULL.
• The snapshot is the hard part. Create the slot first so nothing is lost, copy in a consistent exported snapshot, then stream — applying everything as idempotent upserts keyed on the primary key so the overlap converges. Prefer chunked incremental snapshotting, because a six-hour repeatable-read transaction blocks vacuum for six hours and guarantees recovery conflicts on a replica. Treat re-snapshotting as a routine runbook operation.
• Delivery is at-least-once and ordering is per-key. Partition on the primary key and gate the destination write on LSN so a replay is a no-op. Accept that referential integrity does not survive — a child can arrive before its parent — so either do not enforce foreign keys downstream, buffer with a timeout, or apply whole source transactions atomically at a throughput cost. Enable heartbeats when captured tables are quiet, and treat per-table end-to-end lag as your SLI.
• Prefer ELT — load raw, transform in the destination with SQL — because warehouse compute is elastic and, decisively, raw data preserved means a transformation bug is fixed by re-running SQL rather than re-extracting history you may no longer have. Choose ETL when PII must not land at all, when volume must be reduced first, when residency forbids the raw copy, or when the destination cannot express the transformation.
• CDC captures rows; the outbox publishes events. CDC couples every consumer to your schema and makes them infer meaning from field diffs; the outbox gives an explicit, versioned contract that survives refactoring, but publishes nothing for out-of-band writes. Outbox plus CDC is usually the right answer — write events in the business transaction, stream that one insert-only table with log-based CDC, and you get the stable contract with no relay to operate and no polling. Use raw table CDC for replication targets: warehouse, search index, vectors, and cache invalidation, where it catches the writes an application-level purge never sees (Module P-13).
• Once changes stream to N consumers your schema is an API with unknown callers: additive changes by default, a schema registry that rejects incompatible builds, tolerant readers that ignore unknown fields, and versioned event payloads. Deletion must reach every copy — soft deletes silently break propagation, an immutable Kafka log needs crypto-shredding (delete the per-subject key rather than the records), and you must be able to enumerate your consumers to satisfy an erasure request at all (Module O-9).
• CDC fails invisibly from the source, so the three alarms that matter are retained WAL per slot, per-table end-to-end lag, and a periodic row-count-and-checksum reconciliation — because a pipeline being up is not evidence that it is complete.

The next module is the Practitioner capstone: a payment system. It is the design that requires nearly everything in this phase at once — idempotency end to end because money cannot be charged twice, a double-entry ledger because balances must be derivable rather than mutable, explicit state machines because a payment has many terminal states, an outbox because the charge and the notification cannot be one transaction, and reconciliation because the processor is a system you do not control.

---