Skippr Docs

Appearance

What Access Skippr Needs to Write to BigQuery ​

June 2026

BigQuery write access is mostly a service-account and IAM problem, with CDC relying on MERGE once the dataset and job permissions are in place.

Short Answer ​

Skippr needs a GCP service account key file exposed through GOOGLE_APPLICATION_CREDENTIALS, plus permissions to create datasets, create tables, update table data, and create jobs. The destination docs name the concrete permissions as bigquery.datasets.create, bigquery.tables.create, bigquery.tables.updateData, and bigquery.jobs.create, and they point to BigQuery Data Editor plus BigQuery Job User as the role pattern that covers them.

That is enough for a clean write path into a dataset such as raw_data in project analytics-prod. When CDC is enabled, BigQuery then applies MERGE-based final-state reconciliation using _skippr_order_token columns and tombstone tables, so the service account needs to run the same job and table-write operations that make those MERGE statements possible.

Why Teams Struggle with This ​

BigQuery write issues usually look like generic auth failures even when the key file is fine. The service account may authenticate successfully and still be missing the exact project or dataset permissions needed for creation, loading, or MERGE execution.

  • The connector needs a valid service-account JSON key file referenced by GOOGLE_APPLICATION_CREDENTIALS.
  • The service account needs dataset creation, table creation, table data update, and job creation permissions.
  • Dataset location still matters because jobs and datasets have to line up regionally.
  • CDC depends on MERGE semantics, so a service account that can only upload raw files but not run BigQuery jobs is not sufficient.

How Skippr Handles It ​

Skippr keeps the BigQuery write boundary simple by asking only for project, dataset, and optional location, while the service account stays on the standard Google credential path. That makes access review much easier than a custom warehouse bootstrap that hides which IAM capabilities the runtime will actually use.

The CDC destination docs then make the final-state behavior explicit. BigQuery uses MERGE DML plus automatic order-token columns and tombstone tables, so the result is a current-state warehouse table, not an append-only mutation archive that needs separate cleanup SQL.

  • Uses GOOGLE_APPLICATION_CREDENTIALS for service-account authentication.
  • Aligns with the documented BigQuery Data Editor and BigQuery Job User role pattern.
  • Relies on BigQuery MERGE for exactly-once final-state CDC reconciliation.
  • Automatically manages _skippr_order_token columns and tombstone tables on CDC-managed tables.

What the First Useful Version Looks Like ​

The first useful version is one project, one raw dataset, and one service account whose roles are narrow but complete enough to create and update the tables Skippr manages.

Before widening the warehouse footprint, confirm the service account can both write data and run the BigQuery jobs that CDC reconciliation depends on.