Skippr Docs

Appearance

How BigQuery Shapes the Warehouse Contract ​

July 2026

BigQuery is a clear warehouse contract when the team wants current-state tables in one dataset, atomic CDC reconciliation, and a simple query model for downstream SQL.

Short Answer ​

BigQuery shapes the warehouse contract around one project, one dataset, and a set of destination tables that represent current state rather than a raw mutation log. The destination docs and CDC docs are aligned on that point. Skippr writes tables into the chosen dataset, adds _skippr_order_token STRING to CDC-managed tables, and creates companion tombstone tables so replayed older writes and deletes are resolved inside BigQuery.

That gives teams a straightforward query model. If project: analytics-prod and dataset: raw_data contain customers, analysts query raw_data.customers as the current-state table. They do not need to reconstruct row freshness from a pile of append-only change events. BigQuery MERGE is atomic and consistent, so the warehouse contract stays centered on the destination table and its tombstone companion, not on a separate cleanup job.

Why Teams Struggle with This ​

BigQuery projects go off course when teams understand the connector as "load files into BigQuery" but never define what the table is supposed to mean afterward. The useful contract is not just that rows land. It is that the dataset holds final-state tables whose freshness rules are visible and consistent.

  • A valid service account is not enough if the team still expects the main table to behave like an append-only mutation archive.
  • Ignoring bigquery.jobs.create breaks the warehouse contract in practice because MERGE and other warehouse-side operations cannot run.
  • Choosing the wrong dataset or location can create a clean-looking connector config that still points at the wrong analytical landing zone.
  • If reviewers never mention _skippr_order_token or tombstones, they are skipping the part that actually defines CDC correctness.

How Skippr Handles It ​

Skippr keeps BigQuery easy to reason about because the connector surface is small: project, dataset, and optional location, with authentication handled by GOOGLE_APPLICATION_CREDENTIALS. That keeps the warehouse boundary visible while the service account key stays outside config.

The CDC behavior is equally explicit. BigQuery MERGE applies newer rows only when the incoming order token wins, and tombstone tables prevent stale inserts from reviving deleted business keys. That is the warehouse contract readers should learn, not just the credential setup.

  • Targets one BigQuery dataset as the primary landing boundary.
  • Uses atomic MERGE statements for CDC final-state reconciliation.
  • Adds _skippr_order_token columns and _skippr_tombstones_{table} companions automatically.
  • Depends on concrete service-account roles such as BigQuery Data Editor and BigQuery Job User.

What the First Useful Version Looks Like ​

The first useful version is one dataset such as raw_data and one CDC-managed table with a business key the team already understands. That is enough to teach the BigQuery contract clearly before you scale to more sources.

If downstream users actually need full mutation history, say that separately and store it separately. The main BigQuery table in this pattern is strongest when it stays the current-state interface.