Skippr Docs

Appearance

How Databricks Shapes the Warehouse Contract ​

July 2026

Databricks is a clear warehouse contract when the team wants the main interface to be Delta tables in one catalog and schema, not a loose collection of staged files.

Short Answer ​

Databricks shapes the warehouse contract around Unity Catalog tables in one catalog and schema. The destination docs say Skippr uploads data as Parquet through the Files API and can use COPY INTO when warehouse_id is configured, but those mechanics are not the primary interface downstream users query. The real contract is the Delta tables created in the selected catalog and schema.

CDC makes that even clearer. Skippr uses Unity Catalog MERGE, adds _skippr_order_token STRING, and creates tombstone tables so the main table reflects current state by business key. A setup such as catalog: main and schema: raw means consumers query main.raw.orders as the contract. The uploaded Parquet files and optional SQL warehouse are part of how data lands, not the thing downstream teams should treat as authoritative.

Why Teams Struggle with This ​

Databricks projects often blur the difference between landing mechanics and warehouse contract. Teams may focus on tokens, files, or warehouse_id and forget that analysts and dbt models will rely on the Delta table in Unity Catalog, not on the temporary upload path.

  • A token with workspace access is not enough if it cannot write to the chosen catalog and schema where the contract is supposed to live.
  • If warehouse_id is present, reviewers should know whether it is required for COPY INTO or left over from a different setup.
  • Treating uploaded Parquet as the published interface weakens the cleaner table contract Databricks can offer.
  • Skipping order-token and tombstone behavior turns CDC into a black box instead of a warehouse rule the team can explain.

How Skippr Handles It ​

Skippr makes the Databricks contract explicit because the config names the workspace, optional SQL warehouse, catalog, and schema directly. That keeps the durable landing area visible and lets the token stay an environment-backed secret instead of another hidden dependency.

The CDC path also maps well to how Databricks users already think. Delta tables remain the thing you query, while MERGE applies newer changes by key and tombstones keep deletes from being undone by stale replays.

  • Targets explicit Unity Catalog catalog and schema values.
  • Uploads Parquet through the Files API and can use COPY INTO when a SQL warehouse is configured.
  • Uses Delta MERGE for CDC final-state reconciliation.
  • Adds order-token and tombstone structures so the main Delta table stays the contract.

What the First Useful Version Looks Like ​

The first useful version is one workspace, one catalog, one schema, and one table whose business key the team already understands. That gives you a Databricks contract grounded in tables, not in staging details.

If people still talk about the upload files more than the Delta table they are producing, tighten the documentation before scaling. The best Databricks setups keep the landing mechanics secondary to the table interface.