Skippr Docs

Appearance

How to Query Recent CDC Tables in Databricks ​

May 2026

In Databricks, recent CDC queries should start from the reconciled Delta table, not from staging files or order-token guesses.

Short Answer ​

To query recent CDC tables in Databricks, use the reconciled Delta target table for current rows and filter with a real timestamp column that represents business recency. The MERGE result in the destination is the thing analysts should read, because that is where conflicting mutations have already been resolved by _skippr_order_token.

When you need recent deletes, query the companion tombstone table in addition to the target table. Do not treat _skippr_order_token as a substitute for updated_at or event time. It tells Databricks which mutation is newer, not which hour bucket a user wants to chart.

Why Teams Struggle with This ​

Teams sometimes look too far upstream when they want recent data. They inspect landing files, staging logic, or retry history. That is useful for debugging, but not for normal analytics. The queryable product is the Delta table after MERGE.

  • The target table is the right read path for current-state analytics.
  • A real timestamp column is still needed for recent-window filtering.
  • Tombstone tables are the right place to see recent deletes.
  • The order token solves replay ordering, not user-facing time semantics.

How Skippr Handles It ​

Skippr keeps the Databricks path clean because the destination contract already includes the final-state table and the tombstone companion. Readers do not need to inspect upload folders or infer row freshness from retry metadata.

A concrete example: to count recently activated subscriptions, query the subscriptions target table where status = active and activated_at >= current_timestamp() - INTERVAL 1 DAY. To count recent deletions for the same entity, read the tombstone companion rather than trying to detect missing rows after the fact.

  • Delta target tables are the normal source for recent current-state queries.
  • Tombstone tables surface recent delete activity without mixing it into the target read path.
  • Order tokens stay focused on mutation ordering and replay safety.
  • Unity Catalog MERGE means the destination table is already the reconciled truth.

What the First Useful Version Looks Like ​

The fastest way to make Databricks CDC queries confusing is to ask staging artifacts to answer business questions.

The better pattern is target table for current truth, tombstone table for deletes, and a business timestamp for recency.