Skippr Docs

Appearance

MySQL CDC to BigQuery Final State Guide ​

June 2026

MySQL to BigQuery is strongest when binlog row events and BigQuery MERGE semantics are designed together.

Short Answer ​

MySQL CDC to BigQuery works by reading MySQL binlog row events and applying them with BigQuery MERGE DML. On the source side, Skippr reads WRITE_ROWS, UPDATE_ROWS, and DELETE_ROWS events when binlog_format is ROW and binlog_row_image is FULL. On the destination side, BigQuery uses MERGE with _skippr_order_token STRING columns and tombstone tables to preserve final-state correctness.

For example, if orders.id 1201 changes from status "paid" to "shipped", MySQL emits an UPDATE_ROWS event with the full row image and Skippr carries that change into BigQuery. The MERGE only updates the current row for id 1201 when the incoming order token is greater than the token already stored. If the order is deleted later, the tombstone blocks an older replayed insert from putting it back.

Why Teams Struggle with This ​

This integration only behaves well when both sides are configured for row-level fidelity. MySQL has to emit complete row events, and BigQuery has to treat those events as inputs to an atomic MERGE contract rather than as append-only warehouse facts.

  • MySQL CDC requires binlog_format = ROW and binlog_row_image = FULL so the destination receives the right before and after context.
  • The replication user needs REPLICATION SLAVE and REPLICATION CLIENT privileges before Skippr can read the binlog reliably.
  • BigQuery final-state correctness depends on MERGE plus order-token comparison, not on the arrival order of batched writes.
  • Delete safety depends on the companion tombstone table, especially during retries and restarts.

How Skippr Handles It ​

Skippr preserves the MySQL resume point by storing the binlog filename and position after each committed batch, so the stream can restart from a durable place instead of re-reading blindly. BigQuery then applies the same mutations with the documented MERGE DML pattern and automatic tombstone creation.

That combination makes the integration easy to reason about with concrete examples. You can tell whether a specific row came from WRITE_ROWS, UPDATE_ROWS, or DELETE_ROWS, and you can explain why BigQuery did or did not accept a later replay.

  • MySQL binlog CDC with WRITE_ROWS, UPDATE_ROWS, and DELETE_ROWS support.
  • Durable resume through stored binlog filename and position.
  • BigQuery atomic MERGE updates guarded by _skippr_order_token.
  • Automatic tombstone tables that stop deleted keys from returning.

What the First Useful Version Looks Like ​

Pick this integration when MySQL is the operational database but analysts need current-state tables in BigQuery instead of an event archive.

The pipeline gets much easier once you treat the binlog position and the BigQuery order token as the same ordering story told at two different layers.