Skippr Docs

Appearance

Postgres CDC to BigQuery Final State Guide ​

June 2026

Postgres to BigQuery is an integration where WAL ordering and BigQuery MERGE semantics need to agree on what the current row should be.

Short Answer ​

Postgres CDC to BigQuery works by streaming row-level changes from PostgreSQL WAL logical replication with the pgoutput plugin and then applying those changes with BigQuery MERGE DML. PostgreSQL provides inserts, updates with full row after images, and deletes, while BigQuery adds the _skippr_order_token STRING column and tombstone table that keep older LSN-derived mutations from overwriting newer state.

A concrete example is public.orders id 700 moving from status "pending" to "paid". PostgreSQL emits the update through logical replication, Skippr stores the committed LSN as the resume point, and BigQuery updates the row only when the incoming order token is greater than the token already attached to id 700. A later delete for that key records a tombstone and removes the row.

Why Teams Struggle with This ​

This integration depends on getting the PostgreSQL replication setup right before the warehouse side can do its job. The source needs wal_level = logical, a replication user with REPLICATION rights or superuser access, and at least one replication slot. BigQuery then depends on MERGE to turn those WAL changes into current-state rows.

  • PostgreSQL CDC requires WAL logical replication through pgoutput.
  • Replication slots matter because PostgreSQL retains WAL only until Skippr has confirmed it.
  • Updates arrive as full row after images, which is what lets BigQuery build a final-state row instead of a partial patch record.
  • Deletes still need tombstones to stop older inserts from reviving a removed key.

How Skippr Handles It ​

Skippr keeps the integration durable end to end. It stores the committed LSN after each committed batch, reuses the replication slot across restarts, and applies the same ordered mutations to BigQuery through atomic MERGE statements. That means source ordering and destination reconciliation tell the same story.

The result is easier to debug than a pile of landed change files. When a row looks wrong, you can ask which LSN won, whether BigQuery rejected an older replay, and whether a tombstone already recorded a newer delete.

  • PostgreSQL logical replication with inserts, updates, and deletes.
  • Durable resume behavior from stored committed LSN values.
  • BigQuery MERGE with _skippr_order_token guards on updates.
  • Automatic tombstone tables for delete protection.

What the First Useful Version Looks Like ​

Choose this integration when PostgreSQL is the operational source and BigQuery is where analysts expect one current row per business key.

The practical review point is whether the PostgreSQL publication and the BigQuery merge key reflect the same tables and identities.