Skippr Docs

Appearance

How to Manage Databricks Destination Credentials in Skippr ​

July 2026

Keep Databricks destination credentials safe by leaving workspace and Unity Catalog settings visible, injecting DATABRICKS_TOKEN at runtime, and granting that token only the catalog, schema, and SQL warehouse access the docs call for.

Short Answer ​

Manage Databricks destination credentials in Skippr by keeping workspace_url, catalog, schema, and optional warehouse_id in config while loading token from ${DATABRICKS_TOKEN}. The docs are clear that the token is the secret. The workspace address and Unity Catalog target are not.

That matters even more when CDC is part of the plan. A token that can write to the target catalog and schema can land Parquet files, drive optional COPY INTO, and support the documented Unity Catalog MERGE flow for final-state tables with _skippr_order_token and tombstone protection. The token should therefore be scoped to that exact landing area, not to the entire workspace by default.

Why Teams Struggle with This ​

Databricks credential mistakes usually come from treating the token as the whole deployment story. In practice, the token only works if the catalog, schema, and optional SQL warehouse permissions line up with the landing design.

  • Putting a personal access token directly in skippr.yaml creates an avoidable secret leak.
  • A token can authenticate successfully and still fail if it cannot write to the chosen catalog or schema.
  • If warehouse_id is set, the same token also needs permission to use that SQL warehouse for COPY INTO or query execution.
  • Keeping the workspace URL hidden makes config harder to review without improving security.

How Skippr Handles It ​

Skippr keeps Databricks configuration readable because only the token is secret-bearing. The destination file can still show the workspace, catalog, schema, and optional SQL warehouse so operators know exactly where bronze data lands.

The same clarity helps with CDC support. Skippr uses Databricks MERGE semantics in Unity Catalog, so a narrowly scoped token plus a clearly named target schema is enough to support both initial loads and final-state change application.

  • Uses token: ${DATABRICKS_TOKEN} for the secret-bearing field.
  • Keeps workspace_url, catalog, schema, and optional warehouse_id visible in config.
  • Supports Databricks Files API uploads and optional COPY INTO with the same token.
  • Handles CDC through Unity Catalog MERGE, order tokens, and tombstone tables.

What the First Useful Version Looks Like ​

The first useful version is one workspace, one catalog such as main, one schema, and one token that can write there. That proves both authentication and permissions before you land more datasets.

If the team later changes SQL warehouses, update the visible warehouse_id without changing the secret-handling model. The token still belongs in the environment.