How Delta Lake Source Authentication Works in Skippr

June 2026

Delta Lake source authentication is really storage authentication, because Skippr reads a Delta table URI and whatever credentials the backing object store requires.

Short Answer

Delta Lake source authentication in Skippr is storage-backed, not server-backed. Skippr reads a table_uri such as s3://my-bucket/path/to/table or an ADLS path, and authentication comes from the credentials supplied through storage_options for that backend, usually via environment-variable interpolation rather than inline secrets.

In practice, that means the auth model changes with the table location. An S3-backed Delta table may use AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY values passed through storage_options, while a local table may need no separate cloud credential at all. The constant part is that the runner must be authorized to read both the Delta transaction log and the data files behind the URI.

Why Teams Struggle with This

This connector gets misunderstood when teams talk about "Delta auth" as if Delta Lake were a standalone server. The docs are clearer than that: the authentication boundary is the object store or filesystem behind table_uri, and mistakes usually come from passing the wrong storage credential or pointing at the wrong table root.

• Authentication depends on the backend referenced by table_uri, so there is no one universal username-and-password pattern.
• Sensitive storage values belong in environment-backed storage_options, not directly in skippr.yaml.
• The runner needs read access to the Delta log and the referenced data files, which is stricter than merely listing a bucket or directory.
• If version is pinned, auth can be correct while the data still looks stale because the connector is intentionally reading an older table snapshot.

How Skippr Handles It

Skippr keeps that model honest by exposing the storage dependency instead of hiding it. You authenticate to the backend the table actually lives on, and the source config stays focused on the few fields that matter: table_uri, storage_options, and optional snapshot controls.

That makes review easier. If a table on S3 fails to open, you do not need to guess at a hidden Delta session layer. You check the URI, the interpolated storage credentials, and whether the same runner can read the Delta metadata and files from that location.

• Authentication that follows the storage backend behind table_uri rather than inventing a separate Delta credential scheme.
• Environment-friendly storage_options for backend-specific secrets and region settings.
• A documented requirement to read both the Delta transaction log and data files.
• Optional version and filter controls that stay separate from the auth mechanism itself.

What the First Useful Version Looks Like

The first useful version is one table URI and the smallest set of backend credentials needed to read it successfully. That gives you a stable source before you add historical versioning or filters.

If you are debugging access, start at the storage layer. Delta Lake authentication problems are almost always object-store or path-boundary problems in disguise.