Skippr Docs

Appearance

What Network or Staging Access BigQuery Needs ​

July 2026

BigQuery does not need a separate staging bucket in the documented flow, but it does need the right service-account roles, dataset boundary, and job-creation access.

Short Answer ​

BigQuery needs a service account that can create datasets, create and update tables, and create jobs in the target project. The destination docs are explicit about project, dataset, and optional location, plus the role pattern of BigQuery Data Editor and BigQuery Job User. In other words, the useful load boundary is not a bucket staging path. It is one project-and-dataset boundary backed by a service account that can run the necessary BigQuery jobs.

That is why BigQuery feels different from Redshift or Snowflake staging discussions. If the raw landing area is raw_data in US, Skippr needs the JSON key file behind GOOGLE_APPLICATION_CREDENTIALS, permission to create or update tables there, and permission to submit the jobs that actually perform the work. When CDC is enabled, the same boundary also has to support MERGE-based final-state reconciliation with _skippr_order_token columns and tombstone tables.

Why Teams Struggle with This ​

BigQuery warehouse work is usually blocked by IAM and dataset boundaries, not by a missing object-store hop. Teams often prove the key file exists, then discover the service account cannot create jobs, cannot create the dataset, or is pointed at the wrong location.

  • The service account needs bigquery.datasets.create, bigquery.tables.create, bigquery.tables.updateData, and bigquery.jobs.create.
  • The target dataset and its location have to be chosen intentionally before the warehouse grows around them.
  • A working credential file is not enough if the project-level roles are incomplete.
  • CDC uses BigQuery MERGE and tombstone protection, so the table path has to support that final-state model.

How Skippr Handles It ​

Skippr keeps the BigQuery load path narrow by centering it on project, dataset, location, and one service account key file. That makes it easier to explain to a GCP admin because you are asking for one clear BigQuery boundary, not for broad cloud storage access that the documented connector does not require.

It also helps with CDC planning. The same BigQuery boundary that lands raw tables is the one that runs MERGE-based reconciliation for current-state tables, so you can evaluate permissions and final-state behavior together instead of treating CDC as a separate warehouse product.

  • Uses GOOGLE_APPLICATION_CREDENTIALS for service-account authentication.
  • Targets one BigQuery project, dataset, and optional location explicitly.
  • Relies on BigQuery jobs rather than a separate external staging bucket in the documented setup.
  • Applies CDC with MERGE plus automatic order-token and tombstone support.

What the First Useful Version Looks Like ​

The first useful version is one service account, one raw dataset, one location, and one small landed table that proves the account can both write data and create jobs.

If the account still cannot create the dataset or run the job, stop there. BigQuery becomes straightforward once those permissions are boringly correct.