Skippr Docs

Appearance

How BigQuery Destination Authentication Works in Skippr ​

June 2026

BigQuery destination authentication uses a GCP service account key file, and the useful question is whether that identity can create datasets, write tables, and run the jobs Skippr relies on.

Short Answer ​

BigQuery destination authentication in Skippr uses a GCP service account JSON key file referenced by GOOGLE_APPLICATION_CREDENTIALS. The warehouse config itself stays focused on project, dataset, and optional location, while the service account supplies the identity that BigQuery will trust for dataset creation, table writes, and job execution.

That means a correct auth model is role-based, not just credential-file based. If you point Skippr at project analytics-prod, dataset raw_data, and location US, the service account still needs the permissions the docs call out: dataset creation, table creation or updates, and job creation. The recommended BigQuery Data Editor and BigQuery Job User roles exist because a warehouse load needs all three behaviors.

Why Teams Struggle with This ​

Teams often think BigQuery auth is finished once the key file is present. The docs are more practical than that. Authentication and authorization are inseparable here because Skippr does real warehouse work after login, including CDC MERGE operations when CDC is enabled.

  • The identity comes from GOOGLE_APPLICATION_CREDENTIALS, which must point to a valid service-account JSON key file.
  • The service account needs bigquery.datasets.create, bigquery.tables.create, bigquery.tables.updateData, and bigquery.jobs.create for a normal first load.
  • The recommended role pattern is BigQuery Data Editor plus BigQuery Job User.
  • CDC destinations in BigQuery use atomic MERGE, so job execution permission is part of the destination auth model, not an afterthought.

How Skippr Handles It ​

Skippr keeps BigQuery destination auth narrow and legible. You do not bury warehouse credentials in the config file. You declare the warehouse location and let the standard Google credential environment variable identify the service account that will do the work.

That makes the first production setup easier to explain to platform teams. One service account owns the landing zone. One project and dataset define where data lands. And when CDC is enabled, the same identity can run the MERGE statements that keep final-state tables reconciled.

  • Authentication through a GCP service account key file referenced by GOOGLE_APPLICATION_CREDENTIALS.
  • Minimal warehouse config with project, dataset, and optional location.
  • A documented role pattern that covers dataset creation, table writes, and job execution.
  • CDC support backed by BigQuery MERGE, _skippr_order_token, and tombstone-table semantics.

What the First Useful Version Looks Like ​

The first useful version is one service account, one clearly named raw dataset, and one successful load that proves object creation and job execution under the same identity.

If BigQuery is part of a larger GCP estate, keep the warehouse service account dedicated to the pipeline. Auth models stay easier to reason about when they are not shared with unrelated workloads.