Skippr Docs

Appearance

How to Review BigQuery Warehouse Config Before Production ​

July 2026

BigQuery is ready for production when the project and dataset boundary is clear, the service account can create jobs and write tables, and the team understands the destination as final-state CDC tables.

Short Answer ​

Review a BigQuery warehouse config by checking project, dataset, and optional location, then verifying that GOOGLE_APPLICATION_CREDENTIALS points to a service-account key with the permissions the docs call out: dataset creation, table writes, and job creation. For production CDC, also confirm that the team expects final-state tables maintained through BigQuery MERGE, not an append-only mutation archive.

A concrete review might say: write to project: analytics-prod, dataset: raw_data, location: US, and run with a service account that has BigQuery Data Editor plus BigQuery Job User. That is the setup that lets Skippr create or update tables, run jobs, and maintain CDC-managed rows with _skippr_order_token columns and companion tombstone tables that block stale replays and delete resurrection.

Why Teams Struggle with This ​

BigQuery warehouse reviews usually fail on permissions or expectations. Teams may create the dataset but forget jobs.create, or they may understand the destination as current-state tables only after analysts start asking why replayed older mutations did not overwrite newer rows.

  • The project and dataset values should reflect the real raw landing boundary, not a temporary evaluation project.
  • The service account needs the equivalent of bigquery.datasets.create, bigquery.tables.create, bigquery.tables.updateData, and bigquery.jobs.create.
  • The optional location should match the intended dataset region before the first production tables are created.
  • CDC review should include BigQuery MERGE semantics plus the automatic order-token and tombstone design from the CDC destination docs.

How Skippr Handles It ​

Skippr keeps BigQuery review focused because the public surface is small: project, dataset, and location, with authentication handled through a standard GCP service-account key path. That makes the warehouse boundary easy to inspect before the first load.

The CDC behavior is equally concrete. Skippr applies changes with BigQuery MERGE, adds _skippr_order_token columns to CDC-managed tables, and creates tombstone tables so a replayed older insert does not bring back a key that was deleted later.

  • Uses a simple project, dataset, and optional location warehouse definition.
  • Authenticates through GOOGLE_APPLICATION_CREDENTIALS and standard BigQuery roles.
  • Applies CDC with atomic final-state MERGE behavior in BigQuery.
  • Automatically creates order-token columns and tombstone tables for replay and delete safety.

What the First Useful Version Looks Like ​

The first useful version is one raw dataset, one service account with the right roles, and one source landing into a CDC-managed table that the team can inspect directly.

If the review has not yet settled the dataset region or the service-account scope, fix those early. They are more expensive to correct after several tables are already in use.