Skippr Docs

Appearance

Kafka CDC to Synapse Final-State Guide ​

June 2026

Kafka to Synapse CDC succeeds when Debezium row identity is carried all the way into a guarded warehouse MERGE.

Short Answer ​

Kafka CDC to Synapse starts with Debezium-formatted messages and ends with a SQL MERGE. Skippr consumes the topic, parses create, update, and delete envelopes, and applies those mutations to Synapse only when the incoming _skippr_order_token is newer than the row already present.

The destination-specific requirement is that Synapse needs a precise MERGE predicate. If the Kafka topic key identifies an order row by order_id, then Synapse must use that exact business key in the target table. The integration is only as correct as the row identity preserved across that boundary.

Why Teams Struggle with This ​

This integration often appears simpler than it is because the source is a topic and the destination is SQL. The difficult part is making sure both sides agree on what one row means.

  • Kafka offsets track consumption progress, not warehouse truth.
  • Debezium key fields can get lost if the pipeline focuses only on the payload.
  • Delete handling needs tombstones before the warehouse row is removed.
  • Older messages can still arrive later, so MERGE needs an order-token guard.

How Skippr Handles It ​

Skippr keeps the Kafka side documented and durable by using a stable project-derived group_id and Kafka offset commits for resume.

On Synapse, Skippr adds _skippr_order_token as NVARCHAR, creates tombstone tables automatically, and runs MERGE through the documented TDS path. That makes the row contract explicit from Debezium key to Synapse target.

  • Debezium envelope parsing extracts mutation kind, key fields, and payload.
  • Kafka consumer groups preserve restart position cleanly.
  • Synapse MERGE reconciles only newer mutations into the target row.
  • Delete semantics remain visible through companion tombstone tables.

What the First Useful Version Looks Like ​

Choose this integration when Kafka is the shared CDC transport but Azure Synapse is where the business wants current-state warehouse tables.

The integration stays healthy when the team treats row identity and ordering as first-class parts of the design, not as assumptions hidden in a topic.