How Databricks Handles CDC Replay Safety
May 2026
Databricks replay safety comes from Delta transaction rules plus an explicit ordering contract for each business key.
Short Answer
Databricks handles CDC replay safety by applying changes through Unity Catalog MERGE on Delta tables and only updating a matched row when the incoming _skippr_order_token is newer than the stored token. That keeps an older replay from overwriting more recent state.
Delete replay safety follows the same logic. Skippr records delete intent in a tombstone table and uses that newer-or-older comparison to prevent a stale insert from bringing back a row that was deleted later in source order.
Why Teams Struggle with This
This matters in lakehouse pipelines because retries often happen far from the SQL layer. A file upload can be retried, a compute job can restart, or a batch can be reprocessed. Delta ACID transactions help, but they still need a row-ordering rule to decide which mutation should win.
- A successful transaction is not the same as the correct final row winning.
- Replayed updates are harmless only when the merge condition compares row freshness.
- Delete safety depends on preserving delete intent, not only on removing a row once.
- Teams need business keys because reconciliation is defined per logical entity.
How Skippr Handles It
Skippr treats Databricks replay safety as part of the destination contract, not as a downstream repair task. CDC-managed Delta tables get the _skippr_order_token column and the companion tombstone tables automatically, so the warehouse rules stay inspectable.
A concrete example: customer_id = 7 is updated from plan = basic to plan = pro, then an earlier replay of plan = starter arrives from a retry. The MERGE guard compares order tokens and keeps plan = pro, which is the state an analyst expects to see in the final-state table.
- Databricks CDC uses Unity Catalog MERGE on Delta tables.
- Delta Lake ACID transactions support exactly-once final-state application in Skippr.
- Tombstone tables stop older inserts from resurrecting newer deletes.
- The same warehouse contract can feed directly into generated dbt models.
What the First Useful Version Looks Like
Databricks replay safety works best when you think about rows, not files. The file transport can retry as much as it needs to as long as the row-ordering rule is enforced at apply time.
That is the useful reason to care about _skippr_order_token rather than treating it as extra metadata.