How to Migrate from Amazon S3 to Amazon Redshift

Skippr is the easiest way to migrate from S3 to Redshift and get fully cleansed and modeled data in one run command.

Source: S3. Destination: Redshift.

Sample config

skippr:
  workspace: s3_to_redshift
pipelines:
  s3_to_redshift:
    data_source: data_sources.source
    data_sink: data_sinks.warehouse
data_sources:
  source:
    S3:
      s3_bucket: my-data-bucket
      s3_prefix: "raw/"
data_sinks:
  warehouse:
    Redshift:
      database: analytics
      cluster_identifier: my-cluster
      db_user: admin
      schema: public
      region: us-east-1

Environment

export AWS_ACCESS_KEY_ID="your-key"
export AWS_SECRET_ACCESS_KEY="your-secret"
export REDSHIFT_CLUSTER_IDENTIFIER="my-cluster"
export REDSHIFT_DB_USER="admin"
export REDSHIFT_DATABASE="analytics"
export AWS_ACCESS_KEY_ID="your-key"
export AWS_SECRET_ACCESS_KEY="your-secret"

Source connector: s3. Destination connector: redshift-warehouse.

See Quick start.