You already know the shape of the problem. A downstream system needs to reflect what’s in a source database, and the source keeps changing. The nightly batch that used to reconcile them runs longer every quarter and now overlaps business hours. So someone reaches for change data capture, and the freshness problem goes away. The operational problem it swaps in shows up later.
So this isn’t a from-scratch definition. You know what change data capture is: detecting inserts, updates, and deletes at a source and propagating them as they happen, instead of re-reading the whole dataset on a schedule. The part worth your time is the part the tooling glosses over. There are three ways to actually capture change, they fail in completely different ways, and most teams never chose which one they’re running.
The three ways to capture a change
The method is the decision. Everything downstream inherits its properties, and the wrong one for your source is a slow-motion incident.
Log-based capture reads the database’s own transaction log: the write-ahead log in PostgreSQL, the binlog in MySQL, the redo log in Oracle. It’s the strongest option. It captures every change in commit order, imposes almost no query load on the source, and it sees deletes, which the other two often miss. The cost is that you’re now coupled to the database’s internals: replication-slot management, log-retention settings, and a failover story that gets interesting exactly when you can least afford it. Worse, a log-based consumer that stalls holds a replication slot open, the source can’t reclaim the log, and disk fills on the database you were only supposed to be watching. Your read path becomes a write-availability risk for the source.
Trigger-based capture installs triggers on the source tables that write each change into a shadow table you then read. It works where you can’t reach the transaction log, and the change record is clean and queryable. The cost is write amplification: every insert on the source now does a second insert, on the hot path of the transaction. And the triggers live inside the source schema, so every future migration has to account for them, and a DBA who doesn’t know they’re load-bearing can drop one without noticing.
Query-based capture, or polling, asks the source “what changed since I last looked,” usually by watching an updated_at column or a monotonically increasing id. It needs no special access and it’s the easiest to reason about, which is why it’s the most common thing running in production. It also has the most ways to be quietly wrong. It can’t see hard deletes. It misses a value that changes and reverts between two polls. It scans the source repeatedly, so aggressive polling adds load and lax polling adds lag. And it breaks silently the first time an application updates a row without touching the column you’re watching.
| Sees deletes | Load on source | Ordering | Main risk | |
|---|---|---|---|---|
| Log-based | Yes | Very low | Commit order | Coupled to DB internals |
| Trigger-based | Yes | Write amplification | Per-table | Triggers rot in the schema |
| Query-based | No | Repeated scans | Approximate | Silently misses changes |
None of these is the “right” one in the abstract. Log-based earns its coupling when freshness and completeness are non-negotiable. Trigger-based is a fair fallback when the log is off-limits. Polling is genuinely fine for low-volume, lag-tolerant sources with no hard deletes, and a surprising amount of “we need CDC” is exactly that on inspection.
The failure you’ll debug at 2 a.m. is the one your method ships with, so choosing on purpose is most of the work.
Capturing the change was the easy part
Getting the change out of the source is maybe 20% of the job. The rest is making sure it arrives correctly, once, in order, and stays debuggable when it doesn’t. This is the part that turns “we added CDC” into “we operate a pipeline.”
- Exactly-once is something you engineer, not something you get. Real pipelines are at-least-once. A broker redelivers after a failure, a consumer restarts mid-batch, and the same change lands twice. If the downstream write isn’t idempotent — keyed on a stable change id, an upsert rather than an insert — you’ve just created a duplicate, and duplicates in a synced dataset are invisible until they aren’t.
- Ordering stops being free the moment you scale out. Log-based capture is ordered at the source, but fan events across broker partitions for throughput and global order is gone. A delete can arrive ahead of the insert it was meant to follow. Rare, and corrupting when it happens.
- Schema drift is the quiet one. The source is owned by another team on another release cadence. They add, rename, or drop a column, and depending on your pipeline the change flows through as garbage, halts the stream, or gets dropped on the floor while every dashboard stays green. That last outcome is the expensive one.
- Replay is a day-one requirement that feels like day-two. The first question after any failure is “can we replay from before it broke?” If the honest answer is “re-snapshot the whole source table and reconcile against what already landed,” you’re in for a long night. A CDC setup with no first-class replay-from-checkpoint story is one that simply hasn’t failed yet.
Notice what these have in common. Every one is an operational property, not a feature you can see in a demo. They’re invisible until production, which is precisely when they cost the most to be missing.
Change data capture is easy to start and expensive to operate, and the gap between those two is wider here than for almost any other integration pattern.
So what actually decides whether CDC works?
Not the capture method on its own. The method decides how you read change; whether the whole thing holds up is decided downstream, by whether idempotency, ordering, drift handling, and replay are things you built deliberately or things you’ll discover during an incident.
This is why “does it work in the demo” tells you almost nothing. A CDC pipeline demos beautifully the day you build it, because none of the failure modes above have fired yet. The schema hasn’t drifted. Nothing has redelivered. You haven’t needed to replay. The real question is what the pipeline does on the day one of those happens, and whether you can see it when it does. If tracing a bad row back to the change that caused it means grepping connector logs on a box you’d half forgotten was running, you don’t have observability, you have hope.
That observability question is where the capture method stops mattering and the platform around it starts to. A change event you can follow as a trace — indexed, queryable, retained — is a five-minute investigation. The same event with no trace is an afternoon. The difference isn’t which log you tailed. It’s whether the pipeline was built to be operated.
A CDC pipeline is only as good as your ability to see what it did.
Where this goes next
Change data capture is the correct approach for keeping systems in sync. It’s also a genuine distributed-systems problem wearing a friendly name, and the cost lands in month three, not on launch day. The capture method you pick decides how you read change; whether the whole thing survives production is decided downstream, by whether idempotency, ordering, drift handling, replay, and visibility were designed in on purpose or discovered during an incident.
So the useful question to sit with isn’t “which capture method.” It’s whether the pipeline around the capture was built to be operated: can you follow a change end to end, replay from a checkpoint, and see a schema drift before a customer does. Get those right and CDC is one of the most reliable tools you have. Skip them and it’s a quiet source of wrong data that looks healthy right up until it isn’t.