Datastream alternative — Google Cloud Datastream vs Skippr
Google Cloud Datastream is serverless log CDC. It reads MySQL, PostgreSQL, AlloyDB, SQL Server, and Oracle on the source and writes inserts, updates, and deletes to BigQuery, Cloud Storage, or Apache Iceberg tables in BigQuery. Cloud SQL and Spanner are also listed destinations. It queries the logs on the database; it does not copy the log files to Google Cloud. A stream can hold 10,000 tables. Delivery is at-least-once. Extra event metadata is how you dedupe. The BigQuery dest has Merge (default, needs a primary key) and Append-only. Max row into BigQuery is 20 MB. You pay for GiB processed. Google says that internal byte count is often two to five times the payload. CDC is 2 USD per GiB for the first 2,500 GiB a month, then 1.50, 1.20, and 0.80 on the next bands. AlloyDB and Spanner CDC get the first 100 GiB free. Backfill is free for the first 500 GB a month, then 0.40 USD per GB. An idle stream does not bill. BigQuery CDC apply, Cloud Storage, and Dataflow are separate lines. This is not a Streamhouse. There is no Iceberg ∪ WAL.
Skippr is an open Streamhouse you run as one Rust binary. Write skippr.yml (or the same thing in Python), discover, sync. You query live state over Iceberg ∪ WAL, and the history is ordinary Iceberg in your bucket. No GiB processed tax. ELT into a warehouse is one path, not the product.
| Feature | Skippr | Google Cloud Datastream |
|---|---|---|
| Deploy | Single Rust binary, your VPC or on-prem | Serverless stream in a GCP project |
| Schema | In-stream evolution, sibling columns | Source keys into BQ. Merge needs a primary key |
| Storage | Open Iceberg on Parquet you own | BigQuery, GCS, or Iceberg tables in BigQuery |
| CDC | Log-based; fails closed if the dest cannot honour | Log CDC. At-least-once. Idle stream is free |
| Query | skipprd query on Iceberg ∪ WAL; history in any Iceberg engine | BigQuery after apply. Iceberg dest is BQ-hosted |
| Pricing | Free self-hosted engine | GiB processed (2 USD then tiers). Backfill after 500 GB is 0.40 USD/GB. |
What Google Cloud Datastream is good at
A Cloud SQL or Oracle log that already needs to land in BigQuery without a VM. That is the product. Agentless connectivity and 10,000 tables on one stream are why some estates skip Data Fusion Replication. Iceberg in BigQuery is a real dest if you already query Lakehouse tables there. If the job is "this database into this GCP dest," that is a real product.
Where the job is wider
Datastream is a CDC pipe. It is not a Streamhouse. There is no Iceberg ∪ WAL. There is no skipprd query on in-flight rows. Iceberg here is a BigQuery table type, not files you open with Trino after you leave GCP.
GiB math is the ops. Two to five times the payload is the processed-byte warning on Google's own page. Merge apply is a second BigQuery bill. Skippr will not capture Oracle or SQL Server the way Datastream does. Skippr also will not replace a Datastream that already merges into BigQuery.
If the job is "GCP log CDC we already staff," Datastream is the narrower tool. If the job is live SQL on Iceberg you run, Datastream was never that.
Getting started
Install skipprd from the install guide, or pip install skippr. The three tabs write the same skippr.yml. Then discover, sync, and query.
skippr:
workspace: analytics
skipprd_el_storage_mode: local
pipelines:
orders:
data_source: data_sources.app_db
data_sink: data_sinks.lake
data_sources:
app_db:
Postgres:
host: db.internal
user: skippr_reader
password: ${APP_DB_PASSWORD}
database: app
tables: [orders, customers]
cdc_mode: snapshot_then_cdc
data_sinks:
lake:
AthenaIceberg:
warehouse: s3://my-iceberg-warehouse/
glue_database_name: analytics
athena_workgroup_name: primary
athena_results_s3_bucket: my-athena-results
region: us-east-1
schema_sink: schema_sinks.lake_schema
schema_sinks:
lake_schema:
AthenaIceberg:
warehouse: s3://my-iceberg-warehouse/
glue_database_name: analytics
athena_workgroup_name: primary
athena_results_s3_bucket: my-athena-results
region: us-east-1skipprd --workspace analytics --storage-mode local connect data-source postgres \
--pipeline orders \
--name app_db \
--host db.internal \
--user skippr_reader \
--password '${APP_DB_PASSWORD}' \
--database app \
--tables orders \
--tables customers \
--cdc-mode snapshot_then_cdc
skipprd connect data-sink athenaiceberg \
--pipeline orders \
--name lake \
--warehouse s3://my-iceberg-warehouse/ \
--glue-database-name analytics \
--athena-workgroup-name primary \
--athena-results-s3-bucket my-athena-results \
--region us-east-1
skipprd connect schema-sink athenaiceberg \
--pipeline orders \
--name lake_schemafrom skippr import (
Config,
DataSinkAthenaIceberg,
DataSourcePostgres,
DataSourcePostgresSourceCdcMode,
EnvRef,
LocalStorage,
Pipeline,
)
cfg = Config().workspace("analytics").storage(LocalStorage())
app_db = cfg.data_source(
"app_db",
DataSourcePostgres(
host="db.internal",
user="skippr_reader",
password=EnvRef("APP_DB_PASSWORD"),
database="app",
tables=["orders", "customers"],
cdc_mode=DataSourcePostgresSourceCdcMode.SNAPSHOTTHENCDC,
),
)
lake = cfg.data_sink(
"lake",
DataSinkAthenaIceberg(
warehouse="s3://my-iceberg-warehouse/",
glue_database_name="analytics",
athena_workgroup_name="primary",
athena_results_s3_bucket="my-athena-results",
region="us-east-1",
),
schema_sink="lake_schema",
)
cfg.pipeline("orders", Pipeline(data_source=app_db, data_sink=lake))
cfg.save("skippr.yml")export APP_DB_PASSWORD='your-postgres-password'
skipprd discover --pipeline orders
skipprd sync --pipeline orders --once
skipprd query "SELECT count(*) FROM analytics.orders"Which to choose
Choose Google Cloud Datastream if you need serverless log CDC into BigQuery or GCS, and the warehouse after apply is enough.
Choose Skippr if you want one binary, live SQL, and Iceberg files that stay yours. Skippr will not run a Datastream.
Install from Install. Connector and query reference live in Developer Docs. For optional onboarding in your VPC, Enterprise Jumpstart or Contact.