Skippr Docs

Appearance

How to Define ClickHouse Ingestion Scope Clearly ​

July 2026

A clear ClickHouse boundary starts with one database and one extraction shape: a short table list when raw tables are the contract, or one query when the source should already be narrowed.

Short Answer ​

Define ClickHouse ingestion scope as the combination of one database and one extraction shape: either an explicit tables list or a single query. The connector docs make the priority rule clear. If query is present, it overrides tables, so that SQL text becomes the real source contract rather than a helper detail.

That matters because the connector reads over the ClickHouse HTTP API and namespaces output as clickhouse.{database}.{table}. A config that names database: analytics and tables: [events, metrics] exposes two concrete tables. A config that names query: SELECT user_id, count() FROM events WHERE event_date >= today() - 7 GROUP BY user_id exposes a shaped result set instead, and teams should review it that way.

Why Teams Struggle with This ​

ClickHouse scope gets muddy when teams say they are ingesting one database but quietly depend on a custom SQL statement that joins, filters, and aggregates before data ever leaves the source. The docs give you a simpler boundary than that, but only if you choose it explicitly.

  • Treating tables and query as interchangeable hides whether the source contract is table-level or SQL-defined.
  • Using a broad database name with no table decision usually turns review into guesswork about what the connector will actually read.
  • A query that bakes in date filters or joins can be valid, but it should be treated as source logic because query overrides tables.
  • Checking only credentials misses the real boundary question, which is whether the dataset is supposed to be raw tables or a pre-shaped result.

How Skippr Handles It ​

Skippr keeps the ClickHouse source surface compact: url, database, user, password, tables, and query. That is useful because the boundary does not disappear behind a larger abstraction. You can read the config and tell whether the pipeline is exposing whole tables or a custom SQL slice.

For most first releases, explicit tables are the cleaner contract. A short list such as events and metrics keeps the source recognizable. A query is better when the upstream team already agrees the handoff is a single derived result, not the tables beneath it.

  • Reads from ClickHouse over the documented HTTP interface.
  • Uses database plus either tables or one overriding query to define the extract.
  • Keeps the chosen boundary visible instead of burying it in connector internals.
  • Produces a namespace that still points back to the selected ClickHouse database and table names.

What the First Useful Version Looks Like ​

The first useful version is one reachable ClickHouse HTTP endpoint, one database, and one extraction decision that everyone can explain in a sentence. For example, either "read analytics.events and analytics.metrics" or "run this one query that already defines the feed."

If the team cannot say why the SQL belongs in the connector rather than downstream modeling, start with tables. That keeps the first ClickHouse boundary easier to review, test, and change.