Skippr Docs

Appearance

ELT ​

Managed ELT is a pipeline registry in Skippr Cloud. You declare the pipeline, start a run, and inspect status. Preview records the run; managed skipprd execution is coming soon.

If you want to run extract, load, and dbt on your machine or VPC, use the Production ELT runner on the same skippr binary (skippr discover, skippr sync, skippr model). This page is the managed Cloud control plane.

Status: Preview. You can create a pipeline, start a run, and inspect status. A started run stays RUNNING until the engine reports a result. See the API action reference for the full command list.

Before you start ​

  • Sign in with skippr login. See Cloud User Directory.
  • Ensure your policy allows the elt actions you need.

Concepts ​

  • A pipeline is a named source-to-destination registry entry, including intended loads into the Datalake.
  • A run is one recorded execution with lifecycle status. Preview records the run; the engine has not started yet.
  • Compacted files live in objects. Table metadata lives in tables.

Create a pipeline and start a run ​

Creating the pipeline is CLI, Terraform, CDKTF TypeScript, or CDKTF Python. Starting a run is a data-plane call. For provider credentials and the shared cloud provider block, see Terraform and CDKTF.

bash
skippr elt create-pipeline --name orders-bronze --source postgres://orders --sink iceberg://lake/orders
hcl
resource "cloud_pipeline" "orders_bronze" {
  name   = "orders-bronze"
  source = "postgres://orders"
  sink   = "iceberg://lake/orders"
}
ts
import { CloudProvider, CloudPipeline } from "@skippr/provider-cloud";

const cloud = new CloudProvider(this, "cloud", { region: "eu-central-1" });

new CloudPipeline(this, "orders-bronze", {
  name: "orders-bronze",
  source: "postgres://orders",
  sink: "iceberg://lake/orders",
  provider: cloud,
});
python
from skippr_cdktf import CloudProvider, CloudPipeline

cloud = CloudProvider(self, "cloud", region="eu-central-1")

CloudPipeline(
    self,
    "orders-bronze",
    name="orders-bronze",
    source="postgres://orders",
    sink="iceberg://lake/orders",
    provider=cloud,
)

Then start a run:

bash
skippr elt start-run --pipeline-name orders-bronze

Use DescribeRun with the returned runId. Preview runs stay RUNNING until the engine reports. StopRun requests a halt. UpdatePipeline changes source/sink in place.

Access ​

  • CLI: skippr elt <operation>
  • Hostname: elt.{region}.cloud.skippr.io
  • Auth: Bearer JWT or SigV4 service elt