Skippr Docs

Appearance

How to Configure Databricks as a Warehouse Destination ​

May 2026

Databricks warehouse setup goes smoothly when the workspace token, catalog permissions, and optional SQL warehouse are decided up front.

Short Answer ​

Configure Databricks by setting warehouse.kind to databricks, then supplying workspace_url, token, and the Unity Catalog target with catalog and schema. If you want COPY INTO support or warehouse-driven execution, also include warehouse_id for the SQL warehouse Skippr should use.

A clean first setup often looks like catalog: main, schema: default, a personal access token in DATABRICKS_TOKEN, and a workspace URL such as https://dbc-xxxxxxxx.cloud.databricks.com. That gives Skippr enough information to upload Parquet through the Files API and, when configured, use the named SQL warehouse for load and reconciliation work.

Why Teams Struggle with This ​

Databricks setups usually break at the boundary between storage access and SQL execution rights. A token can be valid for the workspace but still lack permission to write to the selected catalog and schema, or the token can write files while the chosen SQL warehouse is unavailable for COPY INTO.

  • The workspace_url must point at the correct Databricks workspace, not just the organization landing page.
  • The token should come from DATABRICKS_TOKEN rather than sitting inline in config.
  • The selected catalog and schema need write access for the token.
  • If warehouse_id is set, that SQL warehouse must be reachable and authorized for Skippr operations.

How Skippr Handles It ​

Skippr maps directly onto the Databricks concepts the docs call out: workspace URL, token, optional SQL warehouse, and Unity Catalog target. That keeps the setup compact while still leaving room for the practical production choice between simple file-backed loads and warehouse-backed COPY INTO execution.

Once it is working, the first run gives a good signal that the foundation is right. Files can land through the Databricks APIs, target tables can appear in the expected catalog and schema, and later CDC tables can reuse that same warehouse configuration for final-state MERGE behavior.

  • Direct config for workspace_url, token, catalog, schema, and optional warehouse_id.
  • Token handling via environment interpolation with ${DATABRICKS_TOKEN}.
  • Alignment with Unity Catalog permissions and Databricks SQL warehouse access.
  • A clear upgrade path from initial loads into CDC-managed Delta tables.

What the First Useful Version Looks Like ​

Before promoting the setup, verify the token can both write to the chosen schema and use the SQL warehouse you name in config.

If the team has multiple catalogs, decide the raw landing catalog up front so downstream model ownership does not become a naming cleanup project later.