DataStage alternative — DataStage vs Skippr

IBM DataStage is a parallel ETL designer. You draw jobs. The PX engine fans stages across vCPUs. That is the product, on-prem or on Cloud Pak for Data. DataStage as a Service meters Capacity Unit-Hours: one vCPU for one hour is one CUH. Lite is 15 CUH a month. Standard is pay as you go. Bundles pre-buy 5,000, 10,000, or 25,000 CUH. IBM Cloud catalog has listed Standard around 1.94 USD a CUH (1 Aug 2026 list). Anywhere packs add a virtual processor core line (769 USD) plus an instance fee. watsonx.data integration is a different RU meter. A Presto connector can land into watsonx.data Iceberg; IBM's own notes say you can create tables and not alter existing ones, and an earlier rollback bug left inconsistent lake writes (fixed in 2.1.1). There is no public "DataStage Streamhouse." Query is the warehouse or lake after the job.

Skippr is an open Streamhouse you run as one Rust binary. Write skippr.yml (or the same thing in Python), discover, sync. You query live state over Iceberg ∪ WAL, and the history is ordinary Iceberg in your bucket. No CUH. ELT into a warehouse is one path, not the product.

Feature	Skippr	DataStage
Deploy	Single Rust binary, your VPC or on-prem	Designer plus PX runtime. Cloud Pak, as-a-Service, or Anywhere
Schema	In-stream evolution, sibling columns	Per-stage mapping on the canvas
Storage	Open Iceberg on Parquet you own	Target you configured. watsonx.data Iceberg is a connector, not the engine
CDC	Log-based; fails closed if the dest cannot honour	Separate IBM Data Replication / IIDR SKUs. Not the PX job itself
Query	skipprd query on Iceberg ∪ WAL; history in any Iceberg engine	Destination after the job. Knowledge Catalog is another IBM product
Pricing	Free self-hosted engine	CUH plus packs. See IBM Cloud catalog.

Talk through skipprd now

Paul Bot knows the docs. Google Meet as soon as you join, any time of day.

Chat now →

Any time · 20 minutes · Google Meet now

What DataStage is good at

PX is still how banks fan a fat transform across cores without rewriting it as Spark. The canvas is a language a generation of integration teams already speak: sequential file, transformer, lookup, dataset. Cloud Pak keeps that job next to Watson Studio and a catalog. If you have thousands of compiled jobs and a weekend batch window, that is a real estate.

Where the job is wider

DataStage is a job runtime. It is not a Streamhouse. There is no Iceberg ∪ WAL. There is no skipprd query on in-flight rows. A two-vCPU run for ten minutes is 0.33 CUH. Leave a fat environment up and the meter is the environment, not the row. Lite's 30-second minimum is a small trap; Standard has no minimum duration.

CDC is another IBM family (Data Replication, IIDR, StreamSets). Do not expect skipprd-style fail-closed log CDC from a PX canvas. watsonx.data write is a connector with table-create limits.

Skippr will not import a DataStage job or replace PX partitioning. Skippr also will not sit in Cloud Pak. If the job is "keep the IBM parallel graph," DataStage is the narrower tool. If the job is live SQL on Iceberg you run, DataStage was never that.

Getting started

Install skipprd from the install guide, or pip install skippr. The three tabs write the same skippr.yml. Then discover, sync, and query.

YAML

skippr:
  workspace: analytics
  skipprd_el_storage_mode: local

pipelines:
  orders:
    data_source: data_sources.app_db
    data_sink: data_sinks.lake

data_sources:
  app_db:
    Postgres:
      host: db.internal
      user: skippr_reader
      password: ${APP_DB_PASSWORD}
      database: app
      tables: [orders, customers]
      cdc_mode: snapshot_then_cdc

data_sinks:
  lake:
    AthenaIceberg:
      warehouse: s3://my-iceberg-warehouse/
      glue_database_name: analytics
      athena_workgroup_name: primary
      athena_results_s3_bucket: my-athena-results
      region: us-east-1
    schema_sink: schema_sinks.lake_schema

schema_sinks:
  lake_schema:
    AthenaIceberg:
      warehouse: s3://my-iceberg-warehouse/
      glue_database_name: analytics
      athena_workgroup_name: primary
      athena_results_s3_bucket: my-athena-results
      region: us-east-1

CLI

skipprd --workspace analytics --storage-mode local connect data-source postgres \
  --pipeline orders \
  --name app_db \
  --host db.internal \
  --user skippr_reader \
  --password '${APP_DB_PASSWORD}' \
  --database app \
  --tables orders \
  --tables customers \
  --cdc-mode snapshot_then_cdc

skipprd connect data-sink athenaiceberg \
  --pipeline orders \
  --name lake \
  --warehouse s3://my-iceberg-warehouse/ \
  --glue-database-name analytics \
  --athena-workgroup-name primary \
  --athena-results-s3-bucket my-athena-results \
  --region us-east-1

skipprd connect schema-sink athenaiceberg \
  --pipeline orders \
  --name lake_schema

Python

from skippr import (
    Config,
    DataSinkAthenaIceberg,
    DataSourcePostgres,
    DataSourcePostgresSourceCdcMode,
    EnvRef,
    LocalStorage,
    Pipeline,
)

cfg = Config().workspace("analytics").storage(LocalStorage())

app_db = cfg.data_source(
    "app_db",
    DataSourcePostgres(
        host="db.internal",
        user="skippr_reader",
        password=EnvRef("APP_DB_PASSWORD"),
        database="app",
        tables=["orders", "customers"],
        cdc_mode=DataSourcePostgresSourceCdcMode.SNAPSHOTTHENCDC,
    ),
)

lake = cfg.data_sink(
    "lake",
    DataSinkAthenaIceberg(
        warehouse="s3://my-iceberg-warehouse/",
        glue_database_name="analytics",
        athena_workgroup_name="primary",
        athena_results_s3_bucket="my-athena-results",
        region="us-east-1",
    ),
    schema_sink="lake_schema",
)

cfg.pipeline("orders", Pipeline(data_source=app_db, data_sink=lake))
cfg.save("skippr.yml")

export APP_DB_PASSWORD='your-postgres-password'
skipprd discover --pipeline orders
skipprd sync --pipeline orders --once
skipprd query "SELECT count(*) FROM analytics.orders"

Which to choose

Choose DataStage if you already run PX jobs on Cloud Pak and a warehouse after the run is enough.

Choose Skippr if you want one binary, no CUH, live SQL, and Iceberg files that stay yours. Skippr will not open a DataStage job.

Install from Install. Connector and query reference live in Developer Docs. For optional onboarding in your VPC, Enterprise Jumpstart or Contact.