What Network or Staging Access Databricks Needs

July 2026

Databricks destination access means the runner can reach the workspace, upload through the Files API, and, when configured, use the chosen SQL warehouse for COPY INTO.

Short Answer

Databricks needs a reachable workspace_url, a valid personal access token, and write access to the selected Unity Catalog catalog and schema. The docs also make an important staging boundary explicit: Skippr uploads data as Parquet through the Files API, and if you set warehouse_id, that SQL warehouse must be reachable and authorized for COPY INTO or query execution. The load path is therefore workspace plus Files API first, with an optional SQL warehouse added when you want that execution path.

That matters in real deployments because one part can work while another part still fails. A token might authenticate to https://my-workspace.cloud.databricks.com and upload files, yet still be blocked from writing to main.default. Or the catalog permissions may be correct while the chosen warehouse_id is unavailable for the COPY INTO path you expected to use.

Why Teams Struggle with This

Databricks destination setup is usually blocked by a boundary mismatch, not by lack of features. Teams often confirm the workspace exists, then discover the runner cannot use the exact write path they planned: Files API, Unity Catalog writes, or the optional SQL warehouse.

• The runner must reach the Databricks workspace URL from the environment where Skippr runs.
• The token must have write access to the selected Unity Catalog catalog and schema.
• If warehouse_id is configured, that SQL warehouse must be usable for COPY INTO or query execution.
• A workspace login by itself does not prove the landing schema and load path are actually ready.

How Skippr Handles It

Skippr keeps the Databricks boundary clear by exposing the actual moving parts instead of hiding them: workspace_url, token, optional warehouse_id, catalog, and schema. That helps both platform teams and data teams because they can talk about a concrete Files API path and an optional SQL execution path instead of a vague Databricks integration.

It also lines up cleanly with the CDC docs. Once the landing path works, the same destination can use Unity Catalog MERGE with _skippr_order_token columns and tombstones, so final-state behavior is attached to a warehouse path the team already understands.

• Uploads Parquet files through the Databricks Files API.
• Optionally uses a configured SQL warehouse for COPY INTO workflows.
• Targets one Unity Catalog catalog and schema explicitly.
• Supports CDC final-state reconciliation through Unity Catalog MERGE.

What the First Useful Version Looks Like

The first useful version is one workspace, one token, one catalog, one schema, and one small table load that proves both file upload and schema writes are working.

Only add the optional SQL warehouse after that base path is solid. It is much easier to debug COPY behavior once the Files API and catalog permissions are already known good.