Skippr Docs

Appearance

How to Configure BigQuery as a Warehouse Destination ​

May 2026

BigQuery setup is straightforward once the dataset, location, and service-account roles are settled before the first load.

Short Answer ​

Configure BigQuery in Skippr by setting warehouse.kind to bigquery, then filling in project, dataset, and location. Authentication comes from GOOGLE_APPLICATION_CREDENTIALS, which should point to a service-account JSON key file rather than embedding credentials in skippr.yaml.

A clean first setup usually means creating a dedicated service account with the BigQuery Data Editor and BigQuery Job User roles, exporting the key-file path, and choosing the exact dataset region up front. If your raw dataset is raw_data in project analytics-prod and location US, Skippr can create datasets and load tables there as long as that service account can create jobs and update table data.

Why Teams Struggle with This ​

BigQuery setups fail less from syntax than from IAM and location mismatches. Teams often confirm the project and dataset names, then discover the key file points to the wrong service account or the account can run queries but cannot create datasets or update tables.

  • The connector needs a valid GCP project ID, dataset name, and matching dataset location such as US or EU.
  • Authentication depends on GOOGLE_APPLICATION_CREDENTIALS pointing to a real JSON key file on the runner.
  • A first-time load can require bigquery.datasets.create, bigquery.tables.create, bigquery.tables.updateData, and bigquery.jobs.create.
  • If the service account lacks dataset-creation rights, BigQuery may reject a setup that otherwise looks correct.

How Skippr Handles It ​

Skippr keeps the BigQuery surface area small. You configure the warehouse with project, dataset, and location, and the runtime picks up the service-account credentials from the standard Google environment variable. That makes it easy to run locally, in CI, or on an orchestrator without rewriting the connector config.

Once the permissions are in place, the first successful run is easy to inspect. You should see Skippr create or target the chosen dataset, launch the necessary BigQuery jobs, and start loading raw tables into a consistent warehouse namespace.

  • Minimal BigQuery config with project, dataset, and optional location.
  • Service-account auth through GOOGLE_APPLICATION_CREDENTIALS instead of inline secrets.
  • Compatibility with BigQuery Data Editor and BigQuery Job User role patterns from the docs.
  • A clean starting point for later CDC final-state pipelines into BigQuery.

What the First Useful Version Looks Like ​

Before the first production load, confirm the service account can create jobs in the target project and write into the chosen dataset.

If the team works across regions, lock the dataset location first so storage, governance, and downstream query placement do not drift later.