Skippr Docs

Appearance

MySQL CDC to Databricks Final State Guide ​

June 2026

MySQL to Databricks is an integration where binlog row events need a clear Delta MERGE contract before the tables are worth modeling.

Short Answer ​

MySQL CDC to Databricks works by reading WRITE_ROWS, UPDATE_ROWS, and DELETE_ROWS events from the MySQL binlog and applying those mutations with Unity Catalog MERGE on Delta tables. Skippr creates _skippr_order_token STRING columns and tombstone tables automatically, so a replayed older row does not overwrite a newer Delta row.

For example, if invoices.id 330 gets a new paid_at timestamp, MySQL emits an UPDATE_ROWS event with the full after image and Skippr stages that row for Databricks. The MERGE updates the target only when the new order token is greater than the token already stored for id 330. If the invoice row is deleted later, the tombstone ensures a stale insert does not recreate it.

Why Teams Struggle with This ​

This integration combines source-side replication requirements with warehouse-side catalog rules. MySQL must provide row-based binlog events with full row images, and the Databricks token must be allowed to write to the selected catalog and schema before MERGE can keep the table current.

  • MySQL CDC needs binlog_format = ROW and binlog_row_image = FULL.
  • The replication user must have REPLICATION SLAVE and REPLICATION CLIENT privileges.
  • Databricks final-state behavior depends on Unity Catalog MERGE with order-token comparison rather than naive last-write-wins loading.
  • Delete handling still relies on tombstones so older replays cannot revive removed keys.

How Skippr Handles It ​

Skippr keeps the pipeline deterministic from both ends. It stores the MySQL binlog filename and position after each committed batch, then applies those same row changes to Databricks with the documented MERGE pattern on Delta tables. The result is a warehouse table that reflects the latest accepted state instead of every mutation ever seen.

That is especially useful when dbt models start depending on the table. The upstream contract is already explicit: newer tokens win, older replays lose, and deletes are preserved through tombstones.

  • MySQL row-event CDC with durable binlog-position resumes.
  • Databricks MERGE on Delta tables with _skippr_order_token guards.
  • Automatic tombstone tables for anti-resurrection delete handling.
  • A clean path from operational MySQL rows into current-state analytics tables.

What the First Useful Version Looks Like ​

Choose this integration when the team wants current-state Delta tables fed directly from MySQL mutations.

The most useful test is to replay an older UPDATE_ROWS event mentally and make sure the Databricks table would still keep the newer value.