Skippr Docs

Appearance

DynamoDB CDC to ClickHouse Final-State Guide ​

May 2026

DynamoDB to ClickHouse CDC is mainly about understanding that ClickHouse converges to final state through merges, not immediate row replacement.

Short Answer ​

DynamoDB CDC reaches final state in ClickHouse by combining DynamoDB Streams records with ClickHouse ReplacingMergeTree semantics. The source emits INSERT, MODIFY, and REMOVE events for items, and Skippr writes rows with order tokens so ClickHouse can keep the newest version of a logical row as merges run.

What changes for ClickHouse is timing. A MODIFY to a product item might be visible alongside an older version for a short period because ReplacingMergeTree deduplicates during background merges. If readers need point-in-time correctness right after fresh mutations, they should query with FINAL rather than assuming the table has already compacted.

Why Teams Struggle with This ​

This integration goes wrong when teams assume ClickHouse behaves like a transactional row store at read time. It does not. The warehouse contract needs to match ClickHouse merge behavior.

  • DynamoDB item identity can be composite, so ClickHouse needs the full business key, not a partial key.
  • Recent duplicates can be visible until a background merge occurs.
  • REMOVE events still need tombstone tracking or older inserts can come back after replay.
  • Readers who skip FINAL on fresh data may think CDC is wrong when the real issue is query semantics.

How Skippr Handles It ​

Skippr gives the source side a stable resume path by storing the last processed DynamoDB sequence number per shard. That matters because restart safety is part of CDC correctness, not just an operational convenience.

For ClickHouse, Skippr adds _skippr_order_token, uses ReplacingMergeTree for upserts, and creates a tombstone table with MergeTree semantics for deletes. That keeps the final-state contract explicit: newer item versions win, deletes are recorded, and query-time consistency expectations are visible.

  • DynamoDB Streams records become ordered warehouse mutations.
  • ClickHouse keeps the row with the highest version or order token during merges.
  • Delete intent is preserved in tombstone tables before target rows are removed.
  • FINAL is available for reads that need immediate post-merge correctness.

What the First Useful Version Looks Like ​

Choose this setup when ClickHouse query performance matters and the team understands the difference between convergence and instant replacement.

The destination is strong here, but only if readers know that fresh CDC tables may need FINAL before background merges finish.