What Makes SFTP Hard to Ingest Reliably?

April 2026

SFTP ingestion is mostly a reliability problem about file handoff discipline, not about whether Skippr can open an SSH connection.

Short Answer

SFTP is hard to ingest reliably because the source is usually a human-managed file drop, not a strongly typed system boundary. Skippr needs a host, username, either a password or private key, and a remote_path or glob. Reliability depends on those values being stable, but it also depends on the upstream team treating that path like a production handoff and not a miscellaneous folder.

The problem shows up quickly with patterns such as /data/*.json. That can be the right production shape if the directory only holds complete exports with a known cadence. It becomes unreliable when old files, partial drops, or multiple export types share the same location and the ingest has no strong way to distinguish them.

Why Teams Struggle with This

SFTP ingestion tends to inherit the operational habits of the humans or legacy systems placing files on the server. The connector itself is simple. The hard part is making sure the remote path, auth method, and file lifecycle are specific enough that a rerun tomorrow means the same thing as a run today.

• The source contract depends on host, username, auth choice, and remote_path, so each one should be reviewed as part of the production handoff.
• Both password and private-key auth are supported, but mixing them across environments makes incidents harder to untangle.
• A globbed remote_path is convenient and also easy to make too broad.
• The docs only promise that Skippr can read the chosen remote path, which means directory hygiene is the upstream system's responsibility.

How Skippr Handles It

Skippr keeps SFTP narrow on purpose. You specify the server, the auth method, and the exact path or glob to read. That gives the team a concrete artifact to review with the upstream owner: which files land here, how are they named, and which credential does the runner use to fetch them?

A useful production setup is usually one dedicated SSH user, private-key auth, and a directory reserved for a single export family. That is a much healthier starting point than pointing at a shared exports folder and hoping filenames stay consistent.

• Supports password or private-key authentication, with private keys being the better long-lived production default.
• Reads a specific remote path or glob so the file boundary is explicit in config.
• Keeps host, port, and username visible, which helps during network and access reviews.
• Works well when upstream teams treat the SFTP directory as a stable delivery contract.

What the First Useful Version Looks Like

The first useful production shape is one SFTP host, one dedicated reader account, private-key auth, and one directory for one export family. That creates a real file handoff instead of a best-effort scrape.

Before adding more patterns, confirm that the existing directory has a naming rule and a cleanup policy. Reliable SFTP ingestion depends as much on upstream file discipline as on the connector.