Skippr Docs

Appearance

What a Good First BigQuery Warehouse Setup Looks Like ​

July 2026

The smallest production BigQuery setup is one dataset and one service account with the right roles for loads and MERGE-based current-state tables.

Short Answer ​

A good first BigQuery warehouse setup is one GCP project, one dataset, one location, and one service account that can create datasets, write tables, and run jobs. The docs are direct about the surface: project, dataset, and optional location in config, with GOOGLE_APPLICATION_CREDENTIALS pointing at the service account JSON key file. That is enough for a real production landing zone.

The smallest good production version also assumes final-state warehouse behavior, not just raw file loading. BigQuery CDC support uses MERGE, automatic _skippr_order_token columns, and tombstone tables, so the first destination should be a dataset where that service account can both land raw tables and run the SQL required to keep current state correct.

Why Teams Struggle with This ​

BigQuery looks easy to start, but the first production boundary is only good if the dataset, location, and service-account permissions all line up. One missing role is enough to turn a simple setup into a half-working warehouse.

  • The service account needs dataset creation, table write, and job creation permissions before the first load is truly production-ready.
  • Choosing the wrong dataset location creates friction later, even when auth is correct.
  • A checked-in JSON key file is a credential leak, not a warehouse setup strategy.
  • If the team expects append-only raw history as the only interface, it may miss that the documented destination is optimized for current-state reconciliation.

How Skippr Handles It ​

Skippr keeps BigQuery narrow enough to reason about. One project, one dataset, and one location describe the landing zone, while one service account key file authenticates the runner through the standard Google env var.

That setup also lines up with warehouse semantics that matter in production. Skippr can create tables, run jobs, and use BigQuery MERGE to reject stale updates and honor tombstones without a second reconciliation layer.

  • Uses project, dataset, and location as the visible warehouse contract.
  • Authenticates with GOOGLE_APPLICATION_CREDENTIALS and a service account key file.
  • Relies on BigQuery Data Editor and BigQuery Job User style permissions for the first useful setup.
  • Supports CDC final state through MERGE, _skippr_order_token, and tombstone tables.

What the First Useful Version Looks Like ​

The first useful version is one dataset such as raw_data, one location such as US, and one service account with the documented BigQuery roles. That is enough to prove dataset creation, table loads, and the job path that CDC also depends on.

Only after that should you worry about wider warehouse topology. The first production destination should be a dataset that operators can understand and permissions that auditors can explain.