Skippr Docs

Appearance

MongoDB CDC to Redshift Final State Guide ​

June 2026

MongoDB to Redshift is not just a transport path; it is a document CDC stream feeding a staged MERGE contract.

Short Answer ​

MongoDB CDC to Redshift works by reading insert, update, and delete events from MongoDB change streams and then COPY-loading those changes into a Redshift staging table before MERGE applies the final state. Skippr also creates a _skippr_order_token column and a tombstone table so Redshift only accepts newer mutations for the same key.

For example, if a catalog document with _id 500 changes from status "draft" to "published", the change-stream update carries the new after image and Skippr stages it for Redshift. The MERGE step updates the target row only when that incoming order token is newer than the one Redshift already has. A later delete for _id 500 becomes both a tombstone record and a row removal.

Why Teams Struggle with This ​

The operational boundary is sharper in this pipeline than it is on a direct SQL destination. MongoDB still needs replica-set-backed change streams and resume tokens, but Redshift also needs its own staging bucket, prefix, and IAM role before MERGE can safely turn those mutations into final-state rows.

  • A standalone MongoDB deployment cannot provide the change streams this integration needs.
  • Redshift CDC uses a staging table plus MERGE, so the S3 staging bucket and iam_role_arn are part of correctness, not just convenience.
  • Document identity still has to be stable at the destination, usually by carrying the MongoDB key forward.
  • Tombstones matter because a replayed older insert should not recreate a document that a newer delete already removed.

How Skippr Handles It ​

Skippr ties the source and destination semantics together instead of leaving the handoff implicit. It stores MongoDB resume tokens for restart safety, stages Redshift loads through the documented COPY workflow, and uses MERGE with order-token guards when the same key appears again.

That gives the pipeline a useful audit trail. When someone asks why a Redshift row disappeared or changed, you can explain the source operation type, the order token that won, and whether a tombstone blocked an older replay.

  • MongoDB change-stream capture with insert, update, and delete support.
  • Durable resume position through stored resume tokens.
  • Redshift staging plus MERGE reconciliation with newer-token checks.
  • Delete handling through companion tombstone tables.

What the First Useful Version Looks Like ​

Choose this integration when Redshift is where the team wants current-state analytics, not just landed JSON-like change events.

The main thing to verify early is that the MongoDB document key you care about can stay stable all the way through the Redshift MERGE step.