Appearance

Book a call

Flink CDC alternative — Apache Flink CDC vs Skippr ​

Apache Flink CDC is a YAML pipeline layer on Apache Flink. You declare a source and a sink. Flink submits the job. YAML pipeline sources in the 3.7 docs are MySQL, PostgreSQL, Oracle, and SQL Server. Some of those engines are Debezium under the hood. MongoDB and Db2 stay Flink SQL sources, not YAML. The Iceberg pipeline sink arrived in 3.4.0 (16 May 2025). It creates tables, syncs added columns, and talks Hadoop, Hive, Glue, or a custom catalog. Iceberg versions listed for the sink run through 1.10. The source table must have a primary key. Writes are at-least-once; idempotence is the PK table, not exactly-once. Batch and streaming are both supported. Sinks in the 3.6 line also include Paimon, Hudi, Kafka, Doris, StarRocks, Elasticsearch, OceanBase, MaxCompute, and Fluss. The licence is Apache-2.0. You pay the Flink cluster (Standalone, YARN, or Kubernetes). This is not a Streamhouse. There is no Iceberg ∪ WAL.

Skippr is an open Streamhouse you run as one Rust binary. Write skippr.yml (or the same thing in Python), discover, sync. You query live state over Iceberg ∪ WAL, and the history is ordinary Iceberg in your bucket. No JobManager. ELT into a warehouse is one path, not the product.

FeatureSkipprApache Flink CDC
DeploySingle Rust binary, your VPC or on-premFlink cluster plus CDC pipeline YAML
SchemaIn-stream evolution, sibling columnsPipeline schema framework. Postgres schema-change.enabled defaults off
StorageOpen Iceberg on Parquet you ownIceberg, Paimon, Hudi, Kafka, and the other pipeline sinks
CDCLog-based; fails closed if the dest cannot honourLog CDC. PK required for Iceberg. At-least-once apply
Queryskipprd query on Iceberg ∪ WAL; history in any Iceberg engineAfter Flink commits. Flink SQL is a different job
PricingFree self-hosted engineApache-2.0. Pay TaskManagers, checkpoints, and disks.

A MySQL or Postgres log that already needs to land in Iceberg, Paimon, or StarRocks on a Flink estate you run. That is the product. YAML instead of a Connect cluster is why some teams skip Debezium-plus-sink. If the job is "this log through this Flink job," that is a real product.

Where the job is wider ​

Flink CDC is a pipeline generator. It is not a Streamhouse. There is no Iceberg ∪ WAL. There is no skipprd query on in-flight rows. Iceberg apply is at-least-once onto a PK table. Compaction and catalog ops stay yours.

The cluster is the ops. Checkpoint storage, TaskManager memory, and a YAML job that still needs Flink HA. Skippr will not give you Paimon or StarRocks sinks. Skippr also will not replace a Flink CDC job that already writes Iceberg in that cluster. Bare Debezium is a separate page. Managed Flink is Decodable.

If the job is "Flink CDC we already staff," Flink CDC is the narrower tool. If the job is live SQL on Iceberg you run, Flink CDC was never that.

Getting started ​

Install skipprd from the install guide, or pip install skippr. The three tabs write the same skippr.yml. Then discover, sync, and query.

yaml
skippr:
  workspace: analytics
  skipprd_el_storage_mode: local

pipelines:
  orders:
    data_source: data_sources.app_db
    data_sink: data_sinks.lake

data_sources:
  app_db:
    Postgres:
      host: db.internal
      user: skippr_reader
      password: ${APP_DB_PASSWORD}
      database: app
      tables: [orders, customers]
      cdc_mode: snapshot_then_cdc

data_sinks:
  lake:
    AthenaIceberg:
      warehouse: s3://my-iceberg-warehouse/
      glue_database_name: analytics
      athena_workgroup_name: primary
      athena_results_s3_bucket: my-athena-results
      region: us-east-1
    schema_sink: schema_sinks.lake_schema

schema_sinks:
  lake_schema:
    AthenaIceberg:
      warehouse: s3://my-iceberg-warehouse/
      glue_database_name: analytics
      athena_workgroup_name: primary
      athena_results_s3_bucket: my-athena-results
      region: us-east-1
bash
skipprd --workspace analytics --storage-mode local connect data-source postgres \
  --pipeline orders \
  --name app_db \
  --host db.internal \
  --user skippr_reader \
  --password '${APP_DB_PASSWORD}' \
  --database app \
  --tables orders \
  --tables customers \
  --cdc-mode snapshot_then_cdc

skipprd connect data-sink athenaiceberg \
  --pipeline orders \
  --name lake \
  --warehouse s3://my-iceberg-warehouse/ \
  --glue-database-name analytics \
  --athena-workgroup-name primary \
  --athena-results-s3-bucket my-athena-results \
  --region us-east-1

skipprd connect schema-sink athenaiceberg \
  --pipeline orders \
  --name lake_schema
python
from skippr import (
    Config,
    DataSinkAthenaIceberg,
    DataSourcePostgres,
    DataSourcePostgresSourceCdcMode,
    EnvRef,
    LocalStorage,
    Pipeline,
)

cfg = Config().workspace("analytics").storage(LocalStorage())

app_db = cfg.data_source(
    "app_db",
    DataSourcePostgres(
        host="db.internal",
        user="skippr_reader",
        password=EnvRef("APP_DB_PASSWORD"),
        database="app",
        tables=["orders", "customers"],
        cdc_mode=DataSourcePostgresSourceCdcMode.SNAPSHOTTHENCDC,
    ),
)

lake = cfg.data_sink(
    "lake",
    DataSinkAthenaIceberg(
        warehouse="s3://my-iceberg-warehouse/",
        glue_database_name="analytics",
        athena_workgroup_name="primary",
        athena_results_s3_bucket="my-athena-results",
        region="us-east-1",
    ),
    schema_sink="lake_schema",
)

cfg.pipeline("orders", Pipeline(data_source=app_db, data_sink=lake))
cfg.save("skippr.yml")
bash
export APP_DB_PASSWORD='your-postgres-password'
skipprd discover --pipeline orders
skipprd sync --pipeline orders --once
skipprd query "SELECT count(*) FROM analytics.orders"

Which to choose ​

Choose Apache Flink CDC if you want YAML CDC on a Flink cluster you already run, and the sink after the job is enough.

Choose Skippr if you want one binary, live SQL, and Iceberg files that stay yours. Skippr will not submit a Flink CDC job.

Install from Install. Connector and query reference live in Developer Docs. For optional onboarding in your VPC, Enterprise Jumpstart or Contact.