Skip to content

Streambed alternative — Streambed vs Skippr ​

Streambed is an Apache-2.0 Postgres-to-Iceberg CDC engine (Go, viggy28/streambed). It subscribes with logical replication, decodes pgoutput, buffers rows per table, flushes Parquet to S3, and commits Iceberg snapshots. Updates and deletes copy-on-write against existing Parquet. A query server embeds DuckDB and speaks the Postgres wire protocol (streambed sync --query-addr=:5433), so you can psql the lake. SQLite holds LSN and table registry for fast startup; the Iceberg snapshot summary field last_flush_lsn is the authority. The pipeline is one goroutine: decode, buffer, flush, ack. No Kafka. Postgres only. Docker image ghcr.io/viggy28/streambed (their README says use a version tag in production; latest tracks the newest stable). Flags take a STREAMBED_ prefix. There is no licence fee. You pay the process, S3, and people. This is not Iceberg ∪ WAL. Serving SQL is DuckDB on committed Iceberg, not a WAL union.

Skippr is an open Streamhouse you run as one Rust binary. Write skippr.yml (or the same thing in Python), discover, sync. You query live state over Iceberg ∪ WAL, and the history is ordinary Iceberg in your bucket. No goroutine pipeline to staff. ELT into a warehouse is one path, not the product.

FeatureSkipprStreambed
DeploySingle Rust binary, your VPC or on-premGo binary or Docker. Postgres + S3
SchemaIn-stream evolution, sibling columnsPostgres OIDs mapped to Iceberg types
StorageOpen Iceberg on Parquet you ownIceberg on your S3. CoW merge on update/delete
CDCLog-based; fails closed if the dest cannot honourPostgres logical replication only
Queryskipprd query on Iceberg ∪ WAL; history in any Iceberg engineDuckDB over Postgres wire on committed Iceberg
PricingFree self-hosted engineApache-2.0. Pay S3 and the process.

What Streambed is good at ​

Postgres WAL into Iceberg on a bucket you own, then psql against DuckDB, without a warehouse contract. That is the product. One-process CoW plus an Iceberg catalog you can also open with Spark or Trino is why some teams try it before buying a CDC vendor. If the job is "this Postgres into this bucket, queried with psql," that is a real product.

Where the job is wider ​

Streambed is a Postgres CDC writer plus a DuckDB query server. It is not a Streamhouse. There is no Iceberg ∪ WAL. In-flight rows wait in the buffer until flush. Copy-on-write rewrites Parquet on updates. MySQL, SQL Server, and Oracle are out of scope. There is no warehouse dest.

The single goroutine is the ops. One pipeline, in-memory buffers, SQLite plus the snapshot LSN. Skippr will not speak Streambed's STREAMBED_ flags. Skippr also will not replace a Streambed Iceberg prefix that already answers psql. RisingWave is the Postgres-wire streaming-database sibling (separate page).

If the job is "Streambed Postgres-to-Iceberg we already run," Streambed is the narrower tool. If the job is live SQL on Iceberg ∪ WAL you run, that is Skippr's job, not DuckDB on the last flush.

Getting started ​

Install skipprd from the install guide, or pip install skippr. The three tabs write the same skippr.yml. Then discover, sync, and query.

yaml
skippr:
  workspace: analytics
  skipprd_el_storage_mode: local

pipelines:
  orders:
    data_source: data_sources.app_db
    data_sink: data_sinks.lake

data_sources:
  app_db:
    Postgres:
      host: db.internal
      user: skippr_reader
      password: ${APP_DB_PASSWORD}
      database: app
      tables: [orders, customers]
      cdc_mode: snapshot_then_cdc

data_sinks:
  lake:
    AthenaIceberg:
      warehouse: s3://my-iceberg-warehouse/
      glue_database_name: analytics
      athena_workgroup_name: primary
      athena_results_s3_bucket: my-athena-results
      region: us-east-1
    schema_sink: schema_sinks.lake_schema

schema_sinks:
  lake_schema:
    AthenaIceberg:
      warehouse: s3://my-iceberg-warehouse/
      glue_database_name: analytics
      athena_workgroup_name: primary
      athena_results_s3_bucket: my-athena-results
      region: us-east-1
bash
skipprd --workspace analytics --storage-mode local connect data-source postgres \
  --pipeline orders \
  --name app_db \
  --host db.internal \
  --user skippr_reader \
  --password '${APP_DB_PASSWORD}' \
  --database app \
  --tables orders \
  --tables customers \
  --cdc-mode snapshot_then_cdc

skipprd connect data-sink athenaiceberg \
  --pipeline orders \
  --name lake \
  --warehouse s3://my-iceberg-warehouse/ \
  --glue-database-name analytics \
  --athena-workgroup-name primary \
  --athena-results-s3-bucket my-athena-results \
  --region us-east-1

skipprd connect schema-sink athenaiceberg \
  --pipeline orders \
  --name lake_schema
python
from skippr import (
    Config,
    DataSinkAthenaIceberg,
    DataSourcePostgres,
    DataSourcePostgresSourceCdcMode,
    EnvRef,
    LocalStorage,
    Pipeline,
)

cfg = Config().workspace("analytics").storage(LocalStorage())

app_db = cfg.data_source(
    "app_db",
    DataSourcePostgres(
        host="db.internal",
        user="skippr_reader",
        password=EnvRef("APP_DB_PASSWORD"),
        database="app",
        tables=["orders", "customers"],
        cdc_mode=DataSourcePostgresSourceCdcMode.SNAPSHOTTHENCDC,
    ),
)

lake = cfg.data_sink(
    "lake",
    DataSinkAthenaIceberg(
        warehouse="s3://my-iceberg-warehouse/",
        glue_database_name="analytics",
        athena_workgroup_name="primary",
        athena_results_s3_bucket="my-athena-results",
        region="us-east-1",
    ),
    schema_sink="lake_schema",
)

cfg.pipeline("orders", Pipeline(data_source=app_db, data_sink=lake))
cfg.save("skippr.yml")
bash
export APP_DB_PASSWORD='your-postgres-password'
skipprd discover --pipeline orders
skipprd sync --pipeline orders --once
skipprd query "SELECT count(*) FROM analytics.orders"

Which to choose ​

Choose Streambed if you want Apache-2.0 Postgres WAL to Iceberg on S3, and psql against DuckDB on those tables is the job.

Choose Skippr if you want one binary, live SQL on Iceberg ∪ WAL, and files that stay yours. Skippr will not run Streambed.

Install from Install. Connector and query reference live in Developer Docs. For optional onboarding in your VPC, Enterprise Jumpstart or Contact.