How Kafka Defines the Raw Data Contract

July 2026

For Kafka, raw data is whatever one topic publishes, whether that is business events or Debezium envelopes, and the initial history depends on earliest versus latest.

Short Answer

The raw data contract for Kafka is the payload published to one topic. Skippr consumes that topic directly, so the raw layer is the message shape that producers already emit, namespaced as kafka.{topic}, with consumer-group and offset settings deciding how the first run enters the stream.

That makes the topic itself the contract. If the topic carries business events such as user_signed_up, raw data is those event payloads. If the topic carries Debezium envelopes, raw data is that envelope structure until Skippr applies CDC handling downstream. Choosing earliest versus latest changes how much topic history is included in the initial read, which is why offset policy belongs in the contract discussion.

Why Teams Struggle with This

Kafka gets hand-wavy fast when teams talk about the broker instead of the topic. The useful boundary is not Kafka in general. It is a specific topic, a specific payload shape, and a specific decision about where the consumer should begin.

• The topic name is the raw-data boundary, not the broker cluster as a whole.
• A switch from earliest to latest changes what the first load contains even when the payload schema stays the same.
• SASL, ACLs, and broker reachability secure the source, but they do not define the record shape analysts will see later.
• If multiple producers change the topic payload without coordination, the raw contract is unstable before Skippr ever runs.

How Skippr Handles It

Skippr keeps the topic contract visible through brokers, topic, group_id, auto_offset_reset, and mode. That lets engineers review message-boundary choices directly instead of discovering them from runtime behavior later.

It also gives Kafka a clean path into CDC when appropriate. A Debezium topic can stay a Debezium topic at the source, while a supported warehouse destination handles the current-state reconciliation that analysts actually need.

• Consumes from one topic with a configurable consumer group.
• Supports earliest and latest offset policies that affect initial history.
• Works in stream or batch mode depending on the feed you are modeling.
• Supports SASL settings when the cluster is secured, while keeping topic semantics visible in config.

What the First Useful Version Looks Like

The first useful version is one topic, one message family, and one offset policy that the team can explain in one sentence. That usually tells you whether the topic is a durable raw contract or just a noisy internal feed.

If the topic will remain the long-term source, document a sample payload and the expected starting offset behavior. Kafka contracts stay healthy when the payload and the consumer entry point are both explicit.