Appearance

Kafka CDC to Postgres Final-State Guide ​

June 2026

Kafka to Postgres CDC is less about consuming the topic and more about defining the row contract behind the upsert.

Short Answer ​

Kafka CDC reaches PostgreSQL final state by reading Debezium envelopes from a topic, loading the batch into staging, and then applying INSERT ... ON CONFLICT with order-token guards. The source gives Skippr op, before, and after payloads plus key fields; PostgreSQL turns those into the current row for each business key.

The destination-specific rule is that PostgreSQL only protects correctness if the conflict key matches the Debezium row identity. If the topic key says one customer row is identified by customer_id, then the warehouse upsert must use that same key or the final-state table will drift even though consumption is healthy.

Why Teams Struggle with This ​

This setup often fails because teams over-trust the topic and under-design the conflict target. PostgreSQL will happily upsert the wrong thing if you tell it to.

  • Kafka resume via consumer offsets does not guarantee the target table has the right row identity.
  • Debezium updates need a clear business key in PostgreSQL, not just a payload shape.
  • Delete messages need tombstones before target rows are removed.
  • Staging is part of the correctness path, not just a loading convenience.

How Skippr Handles It ​

Skippr consumes Kafka with a stable group_id, so offsets are committed and reused across restarts. That gives the source side a durable resume model exactly as the docs describe.

On PostgreSQL, Skippr creates staging tables, _skippr_order_token, and companion tombstones automatically, then reconciles with ON CONFLICT only when the incoming mutation is newer. That keeps the setup teachable: topic in, guarded upsert out.

  • Debezium envelopes are parsed directly from Kafka topics.
  • Stable consumer groups preserve restart position.
  • PostgreSQL staging plus ON CONFLICT handles bulk reconciliation cleanly.
  • Order-token and tombstone rules protect final-state correctness.

What the First Useful Version Looks Like ​

This integration is useful when PostgreSQL is the warehouse endpoint and the team wants to preserve a clear SQL-native reconciliation path.

The main discipline is to keep the Debezium key definition and the PostgreSQL conflict key exactly aligned.