How to Manage Delta Lake Source Credentials in Skippr

July 2026

Keep Delta Lake source credentials safe by separating the visible table URI from the secret storage keys and interpolating backend credentials into storage_options only where the object store requires them.

Short Answer

Manage Delta Lake source credentials in Skippr by keeping table_uri visible in config and treating only the object store secrets as secret material. The docs show storage_options for backend-specific access, and the safe pattern is to interpolate values such as AWS_ACCESS_KEY_ID or AWS_SECRET_ACCESS_KEY from environment variables instead of writing them directly into skippr.yaml.

That lines up with how the connector actually works. Skippr reads a Delta table from s3://, ADLS, or a local path, so the credential model comes from the storage backend behind the table rather than from a separate Delta login. A path like s3://analytics-lake/orders is not the secret. The secret is the storage credential that can read the Delta log and data files.

Why Teams Struggle with This

Delta Lake access becomes messy when teams blur the table location and the storage credential into one hidden blob. The docs are clearer than that: table_uri, optional version, and optional filter describe what to read, while storage_options carries only the backend access details that need protection.

• Committing S3 or ADLS keys inside storage_options defeats the point of using environment interpolation.
• Treating the full table URI as a secret makes the config harder to review without reducing risk.
• Reading the wrong version can look like an auth issue when the real problem is table selection.
• If the runner cannot reach the object store or Delta log path, the credential can be correct and the read will still fail.

How Skippr Handles It

Skippr keeps the Delta Lake contract direct: one table_uri, optional version, optional filter, and backend storage_options when the object store needs them. That gives you a clean place to keep the read target visible while protecting only the values that unlock storage access.

A common production pattern is to keep AWS_REGION in config because it is operational context, then inject DELTA_AWS_ACCESS_KEY_ID and DELTA_AWS_SECRET_ACCESS_KEY through environment variables. The article can teach that split clearly because it is the exact split the docs use.

• Keeps table_uri, version, and filter as normal connector config.
• Supports ${ENV_VAR} interpolation inside storage_options for backend secrets.
• Works across S3, ADLS, and local Delta tables without inventing a new auth surface.
• Lets teams review the read target without exposing the credential that opens it.

What the First Useful Version Looks Like

The first useful version is one Delta table URI, one backend credential path, and one controlled read against a known version. That proves both storage access and the table contract before you add more tables.

If the table lives on S3, keep the bucket path and region visible, then load only the access key pair or role-backed credential through the environment the runner already uses.