Skippr Docs

Appearance

How to Manage Kafka Source Credentials in Skippr ​

July 2026

Keep Kafka source credentials safe by leaving broker and topic details in config, using env vars for SASL secrets when needed, and matching the connector to either a local unauthenticated cluster or a secured broker setup.

Short Answer ​

Manage Kafka source credentials in Skippr by separating connection details from actual secrets. The docs keep brokers, topic, group_id, auto_offset_reset, and mode in config, while sasl_username and sasl_password should use environment variable interpolation only when the cluster runs with SASL.

That is important because Kafka does not always require a secret at all. A local broker may be intentionally unauthenticated, while a production cluster may require SASL_SSL, a named mechanism, and broker ACLs for topic reads and group commits. The safe setup is the one that matches the broker you actually run, not the one that assumes every field is confidential.

Why Teams Struggle with This ​

Kafka auth problems often come from hiding the wrong fields and exposing the wrong ones. Broker addresses and topic names are operational details. SASL credentials are secrets. When teams blur that line, they usually end up with unreadable config and a leaked password anyway.

  • Putting sasl_username and sasl_password directly into skippr.yaml creates a secret leak without simplifying operations.
  • Using one shared superuser across many topics makes it harder to scope topic and consumer-group ACLs.
  • Changing earliest versus latest is a data contract decision, not a credential decision, and should stay visible in config.
  • If the runner cannot reach the bootstrap brokers or commit offsets for the group, no amount of password rotation will make the source healthy.

How Skippr Handles It ​

Skippr keeps the Kafka source honest about what it needs: broker addresses, topic, consumer group, auth settings, and mode. That makes it easy to read the connector config and see how the consumer behaves without exposing the SASL values that secure it.

It also means the same pattern works across both local evaluation and production. You can leave auth unset for a local broker, then move to ${KAFKA_SASL_USERNAME} and ${KAFKA_SASL_PASSWORD} when the secured cluster is ready.

  • Supports local unauthenticated Kafka as well as secured SASL clusters.
  • Keeps brokers, topic, group_id, and mode as visible config.
  • Uses ${KAFKA_SASL_USERNAME} and ${KAFKA_SASL_PASSWORD} for secret values when SASL is enabled.
  • Matches the documented ACL and consumer-group model instead of hiding it behind a generic token story.

What the First Useful Version Looks Like ​

The first useful version is one topic, one consumer group, one offset policy, and one broker security mode. Validate the consumer can join the group and read messages before you add more topics or credentials.

If the topic carries Debezium CDC later, keep the same secret-handling pattern and let the warehouse destination handle final-state reconciliation on its side.