Skippr Docs

Appearance

How ClickHouse Handles CDC Replay Safety ​

May 2026

ClickHouse replay safety depends on versioned rows and query discipline, not on pretending background merges are instant.

Short Answer ​

ClickHouse handles CDC replay safety by inserting versioned rows into ReplacingMergeTree tables and letting the highest _skippr_order_token win as merges converge. An older replayed mutation can still be written physically, but it should lose logically once ClickHouse keeps the row with the newest version.

Deletes are handled separately through tombstone tracking plus conditional deletion. That matters because replay safety in ClickHouse is not only about which row wins during deduplication. It is also about making sure an old insert cannot undo a newer delete for the same business key.

Why Teams Struggle with This ​

The important caveat is that ClickHouse merges are eventual. Right after fresh CDC writes, recent duplicates can still be visible until a background merge runs. That is why replay safety in ClickHouse includes query-time behavior as well as write-time behavior.

  • ReplacingMergeTree does not give immediate point-in-time deduplication on every query.
  • Readers can see multiple versions of the same key if they query too soon without FINAL.
  • A weak version or business-key definition makes replay handling look random.
  • Delete protection still needs tombstones because deduplication alone does not explain resurrection safety.

How Skippr Handles It ​

Skippr sets up the pieces ClickHouse needs: a String _skippr_order_token column on CDC-managed tables, tombstone tables for deletes, and a design expectation that recent correctness-sensitive reads may need FINAL.

A concrete example is an orders table keyed by order_id. If two versions of order_id = 42 exist briefly after a retry, ClickHouse can still converge on the row with the highest order token. When the dashboard needs the latest state immediately, querying with FINAL gives a point-in-time answer instead of waiting for background merges.

  • CDC final-state behavior uses ReplacingMergeTree engine semantics.
  • Delete handling includes tombstone tracking and ALTER TABLE ... DELETE behavior.
  • The docs explicitly warn that recent duplicates may be visible until background merges complete.
  • FINAL is the practical tool for fresh, consistency-sensitive reads.

What the First Useful Version Looks Like ​

ClickHouse can be replay-safe, but the shape of that safety is different from MERGE-first warehouses.

If you understand that writes converge and hot queries may need FINAL, the behavior becomes predictable instead of surprising.