CDC: PostgreSQL to Athena | Real-Time Replication Guide

Real-time Change Data Capture from PostgreSQL to Athena with exactly-once delivery, order-token guards, and tombstone anti-resurrect protection.

Source: PostgreSQL. Destination: Athena.

CDC apply: Iceberg MERGE.

Sample config

skippr:
  workspace: cdc_postgres_to_athena
pipelines:
  cdc_postgres_to_athena:
    data_source: data_sources.source
    data_sink: data_sinks.warehouse
    cdc:
      default:
        business_key_columns:
          - id
data_sources:
  source:
    Postgres:
      host: localhost
      port: 5432
      user: replicator
      password: "${POSTGRES_PASSWORD}"
      database: mydb
      cdc_mode: snapshot_then_cdc
data_sinks:
  warehouse:
    Athena:
      database: my_glue_db
      s3_staging_dir: "s3://my-bucket/athena-results/"

Environment

export AWS_ACCESS_KEY_ID="your-key"
export AWS_SECRET_ACCESS_KEY="your-secret"

Source connector: postgres. Destination connector: athena.

See CDC overview.