Athena (S3 + Glue)

Install

See the Install guide for the full setup, including Windows PowerShell.

curl -fsSL https://install.skippr.io/install.sh | shClick to copy

Installing Skippr means accepting the Skippr EULA.

Uses Athena as the destination. Ingest and query share the same Athena: sink in data_sinks. source and warehouse in examples are logical names, not reserved words.

Configuration

data_sinks:
  warehouse:
    Athena:
      s3_bucket: my-bucket
      s3_prefix: bronze
      glue_database_name: my_database
      athena_workgroup_name: primary
      athena_results_s3_bucket: my-bucket
      region: us-east-1
      catalog: AwsDataCatalog

Field	Default	Description
s3_bucket	(required)	S3 bucket for Parquet landing
s3_prefix		Key prefix for landed objects
glue_database_name		Glue database for catalog and unqualified queries
athena_workgroup_name		Athena workgroup name
athena_results_s3_bucket		Query-results bucket name (not an s3:// URI)
region		AWS region (query/model; ingest ignores this)
catalog		Athena catalog name (query/model; ingest ignores this)
max_concurrency		Optional query concurrency (query/model; ingest ignores this)
discovery_cache_ttl_secs		Optional catalog cache TTL (query/model; ingest ignores this)

CLI

skippr connect warehouse athena \
  --s3-bucket my-bucket \
  --s3-prefix bronze \
  --athena-workgroup-name primary \
  --glue-database-name my_database \
  --athena-results-s3-bucket my-bucket \
  --region us-east-1 \
  --catalog AwsDataCatalog

Flag	Description
--s3-bucket	S3 bucket for Parquet landing
--s3-prefix	Key prefix for landed objects
--athena-workgroup-name	Athena workgroup name
--glue-database-name	Glue database name
--athena-results-s3-bucket	Query-results bucket name
--region	AWS region
--catalog	Athena catalog name
--max-concurrency	Optional query concurrency
--discovery-cache-ttl-secs	Optional catalog cache TTL

Config output

Running connect warehouse athena writes the Athena sink settings directly into the full skippr.yml engine config.

data_sinks:
  warehouse:
    Athena:
      s3_bucket: my-bucket
      s3_prefix: bronze
      glue_database_name: my_database
      athena_workgroup_name: primary
      athena_results_s3_bucket: my-bucket
      region: us-east-1
      catalog: AwsDataCatalog

Authentication

Authentication uses the AWS default credential chain.

• AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
• IAM roles, instance profiles, or task roles
• AWS SSO or shared config profiles

Permissions or Network Requirements

The AWS principal needs access to the target S3 bucket, Glue database, and Athena workgroup or result location you configure. The runner must also be able to reach the AWS APIs for the configured region.

Partitioned API sources (GA4 and similar)

Sources such as Google Analytics (GA4) declare replace_partition on a date column. Athena deletes the matching S3 prefix (for example …/google_analytics.events_daily/date=2024-01-15/) before writing new Parquet and registers Glue partitions on date. This differs from generic time-bucketing layouts that only use p_year / p_month / p_day.

See Source landing semantics.

Troubleshooting

Symptom	Fix
query or Glue metadata errors	Verify the Glue database name, Athena workgroup settings, and the configured result S3 location.
AccessDenied	Check the AWS credential chain and confirm access to S3, Glue, and Athena resources.
stale GA4 metrics for past dates	Use a source with replace_partition and sufficient lookback_days; re-run skippr sync

Next steps

• Pair this destination with a source from Source Connectors.
• See Quick Start for the recommended warehouse evaluation paths.