Apache InLong alternative — Apache InLong vs Skippr

Apache InLong is a multi-service integration stack. It started as TubeMQ, Tencent's message queue, donated in 2019, and graduated as an Apache TLP in June 2022. Current docs are 1.12.0. You do not install one binary. You run Agent (files and databases), DataProxy (Flume-ng fan-in), a queue (TubeMQ, Kafka, or Pulsar), Manager, Dashboard, Audit, and Sort. Sort Standalone (Flume) is the low-latency log path. Sort on Flink is the exactly-once path into Hive, ClickHouse, Iceberg, Hudi, Doris, and the rest. Extract nodes include MySQL, Postgres, Oracle, SQL Server, MongoDB, Kafka, Pulsar, Redis, MQTT, and files. Their own node table still lists Iceberg as 0.12.x. The pitch is trillion-row ingest inside Tencent-shaped ops. The project is Apache-2.0. You pay the cluster.

Skippr is an open Streamhouse you run as one Rust binary. Write skippr.yml (or the same thing in Python), discover, sync. You query live state over Iceberg ∪ WAL, and the history is ordinary Iceberg in your bucket. No Agent / DataProxy / TubeMQ / Sort split. ELT into a warehouse is one path, not the product.

Feature	Skippr	Apache InLong
Deploy	Single Rust binary, your VPC or on-prem	Agent, DataProxy, MQ, Manager, Dashboard, Sort, Audit. Flink if you want lake exactly-once
Schema	In-stream evolution, sibling columns	Manager metadata plus Sort. Connector-dependent
Storage	Open Iceberg on Parquet you own	Hive, ClickHouse, Iceberg, Hudi, Kafka, HBase, Doris, JDBC, …
CDC	Log-based; fails closed if the dest cannot honour	DB extract nodes (MySQL, Postgres, Oracle, SQL Server, MongoDB). Sort writes the dest
Query	skipprd query on Iceberg ∪ WAL; history in any Iceberg engine	Destination after Sort. Audit is traffic reconcile, not SQL
Pricing	Free self-hosted engine	Apache-2.0, pay every JVM. See their docs.

Talk through skipprd now

Paul Bot knows the docs. Google Meet as soon as you join, any time of day.

Chat now →

Any time · 20 minutes · Google Meet now

What Apache InLong is good at

This is a bus for massive log and CDC fan-in. Topic publish and subscribe, a Dashboard, and an audit trail that counts bytes in and out of Agent, DataProxy, and Sort. Sort on Flink can commit Iceberg snapshots on checkpoint, with equality and position deletes for upsert. Sort Standalone keeps the seconds-latency path for logs that can tolerate a little inconsistency. If you already operate Flink and a queue at Tencent scale, that is a real platform.

Where the job is wider

InLong is five or six processes before the first row lands. TubeMQ, DataProxy, and Manager are not optional if you want the product they document. There is no Iceberg ∪ WAL. There is no skipprd query on in-flight rows. Query waits for Sort.

Their extract/load matrix still names Iceberg 0.12.x. Community notes on the Iceberg path call out missing upstream order (upsert consistency) and small files when you have many partitions. That is a lake writer on Flink, not a Streamhouse.

Skippr will not give you TubeMQ, the Dashboard, or InLong Audit. Skippr also will not ingest MQTT or Redis the way their extract nodes do. If the job is "Tencent-shaped mass ingest onto a queue, then Sort," InLong is the narrower tool. If the job is live SQL on Iceberg you run, InLong was never that.

Getting started

Install skipprd from the install guide, or pip install skippr. The three tabs write the same skippr.yml. Then discover, sync, and query.

YAML

skippr:
  workspace: analytics
  skipprd_el_storage_mode: local

pipelines:
  orders:
    data_source: data_sources.app_db
    data_sink: data_sinks.lake

data_sources:
  app_db:
    Postgres:
      host: db.internal
      user: skippr_reader
      password: ${APP_DB_PASSWORD}
      database: app
      tables: [orders, customers]
      cdc_mode: snapshot_then_cdc

data_sinks:
  lake:
    AthenaIceberg:
      warehouse: s3://my-iceberg-warehouse/
      glue_database_name: analytics
      athena_workgroup_name: primary
      athena_results_s3_bucket: my-athena-results
      region: us-east-1
    schema_sink: schema_sinks.lake_schema

schema_sinks:
  lake_schema:
    AthenaIceberg:
      warehouse: s3://my-iceberg-warehouse/
      glue_database_name: analytics
      athena_workgroup_name: primary
      athena_results_s3_bucket: my-athena-results
      region: us-east-1

CLI

skipprd --workspace analytics --storage-mode local connect data-source postgres \
  --pipeline orders \
  --name app_db \
  --host db.internal \
  --user skippr_reader \
  --password '${APP_DB_PASSWORD}' \
  --database app \
  --tables orders \
  --tables customers \
  --cdc-mode snapshot_then_cdc

skipprd connect data-sink athenaiceberg \
  --pipeline orders \
  --name lake \
  --warehouse s3://my-iceberg-warehouse/ \
  --glue-database-name analytics \
  --athena-workgroup-name primary \
  --athena-results-s3-bucket my-athena-results \
  --region us-east-1

skipprd connect schema-sink athenaiceberg \
  --pipeline orders \
  --name lake_schema

Python

from skippr import (
    Config,
    DataSinkAthenaIceberg,
    DataSourcePostgres,
    DataSourcePostgresSourceCdcMode,
    EnvRef,
    LocalStorage,
    Pipeline,
)

cfg = Config().workspace("analytics").storage(LocalStorage())

app_db = cfg.data_source(
    "app_db",
    DataSourcePostgres(
        host="db.internal",
        user="skippr_reader",
        password=EnvRef("APP_DB_PASSWORD"),
        database="app",
        tables=["orders", "customers"],
        cdc_mode=DataSourcePostgresSourceCdcMode.SNAPSHOTTHENCDC,
    ),
)

lake = cfg.data_sink(
    "lake",
    DataSinkAthenaIceberg(
        warehouse="s3://my-iceberg-warehouse/",
        glue_database_name="analytics",
        athena_workgroup_name="primary",
        athena_results_s3_bucket="my-athena-results",
        region="us-east-1",
    ),
    schema_sink="lake_schema",
)

cfg.pipeline("orders", Pipeline(data_source=app_db, data_sink=lake))
cfg.save("skippr.yml")

export APP_DB_PASSWORD='your-postgres-password'
skipprd discover --pipeline orders
skipprd sync --pipeline orders --once
skipprd query "SELECT count(*) FROM analytics.orders"

Which to choose

Choose Apache InLong if you want a queue-plus-Sort platform for mass ingest, and a warehouse or lake after Flink is enough.

Choose Skippr if you want one binary, live SQL, and Iceberg files that stay yours. Skippr will not replace InLong's Agent / TubeMQ / Sort stack.

Install from Install. Connector and query reference live in Developer Docs. For optional onboarding in your VPC, Enterprise Jumpstart or Contact.