Skippr Docs

Appearance

Common BigQuery Warehouse Setup Mistakes ​

May 2026

BigQuery warehouse setup usually breaks on credentials, dataset permissions, or fuzzy expectations about how final-state CDC is applied.

Short Answer ​

The most common BigQuery warehouse mistakes are using a service account that can run neither jobs nor table writes, pointing Skippr at the wrong dataset or location, and assuming CDC will behave like append-only loading. Skippr expects a project, dataset, and optional location, then uses BigQuery MERGE with a _skippr_order_token column and tombstone tables when CDC is enabled.

That means setup is not finished when authentication works once. If the service account can query but lacks BigQuery Data Editor or BigQuery Job User capabilities, the first silver or gold dataset creation can fail later. If the dataset lives in EU but the project setup assumes US, jobs fail for a reason that looks unrelated to the model you were trying to build.

Why Teams Struggle with This ​

BigQuery feels simple because the connector surface is short, but teams often hide important warehouse decisions behind one credential file. The docs make the real contract clearer: the service account needs dataset creation, table write, and job execution permissions, and CDC depends on atomic MERGE behavior rather than raw inserts.

  • Teams often set GOOGLE_APPLICATION_CREDENTIALS correctly but grant only partial access, so dataset creation or table updates fail later.
  • Project ID, dataset name, and dataset location are easy to mix up when multiple environments share one GCP account.
  • Some teams expect CDC rows to accumulate as history, even though Skippr applies final-state MERGE rules in BigQuery.
  • Missing bigquery.datasets.create shows up only when Skippr tries to create downstream datasets, not during the first credential check.

How Skippr Handles It ​

Skippr keeps the warehouse setup explicit instead of burying it in an opaque bootstrap step. The BigQuery connector asks for the exact project, dataset, and location you want, and the docs call out the role set that covers table creation, data updates, and query execution.

When CDC is part of the plan, Skippr also makes the reconciliation contract visible. BigQuery gets _skippr_order_token columns, tombstone tables, and atomic MERGE statements, so stale replays lose cleanly and deleted keys do not come back by accident.

  • Direct configuration for project, dataset, and location instead of hidden warehouse defaults.
  • A service-account model grounded in GOOGLE_APPLICATION_CREDENTIALS and the documented BigQuery roles.
  • Automatic _skippr_order_token columns and tombstone tables for CDC-managed tables.
  • Atomic BigQuery MERGE application so final-state tables stay deterministic after retries.

What the First Useful Version Looks Like ​

A useful first version is a dedicated service account, one clearly named raw dataset, and a confirmed location that matches how the rest of the GCP project is run.

If CDC is in scope, review one target table as a final-state table from day one and make sure the team expects MERGE semantics rather than append-only facts.