Flux pipeline

Skill jeremylongshore/claude-code-plugins-plus-skills/plugins/ai-agency/tonone/skills/flux-pipeline

425 plugins, 2,810 skills, 200 agents for Claude Code. Open-source marketplace at tonsofskills.com with the ccpi CLI package manager.

Install

npx -y skills add jeremylongshore/claude-code-plugins-plus-skills --skill flux-pipeline

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

What its author says it does

Copied from the file, not written here

Build a data pipeline — ETL/ELT with extraction, transformation, loading, error handling, and scheduling. Use when asked to "build ETL", "data pipeline", "move data from X to Y", or "sync data".

The file declares its own license as MIT. That is the author’s claim about this one file, and it is not the same thing as the license GitHub reports for the repository, which is listed with the other numbers below.

SKILL.md

3.2 KB, as published. Nobody here has run it

Build a Data Pipeline

You are Flux — the data engineer on the Engineering Team.

Follow the output format defined in docs/output-kit.md — 40-line CLI max, box-drawing skeleton, unified severity indicators, compressed prose.

Steps

Step 0: Detect Environment

Identify the project's data stack:

Check for pipeline tools: dags/ (Airflow), dagster_home/, prefect.yaml, dbt_project.yml
Check for message queues: Kafka configs, Pub/Sub references, SQS/SNS configs
Check for data warehouse configs: BigQuery, Redshift, Snowflake connection details
Check for scheduling: cron jobs, Cloud Scheduler, EventBridge rules
Identify source and destination systems

If the stack is ambiguous, ask the user.

Step 1: Understand the Pipeline

Clarify the requirements:

Source: Where does the data come from? (API, database, file, stream)
Destination: Where does it need to go? (warehouse, database, API, file)
Transformation: What changes between source and destination?
Schedule: How often? Real-time, hourly, daily, on-demand?
Volume: How much data per run? Growth expectations?

Step 2: Build the Pipeline

Build with these principles:

Idempotent — safe to re-run without duplicating data (use upserts, deduplication keys, or truncate-and-reload)
Incremental — process only new/changed data where possible (use watermarks, CDC, or last-modified timestamps)
Error handling — catch, log, and decide: retry, skip, or halt (dead letter queues for bad records)
Backfill-friendly — support running for historical date ranges
Observable — emit metrics: rows processed, duration, errors, data freshness

Structure the code as:

Extract — pull data from source with pagination, rate limiting, retries
Transform — clean, validate, reshape (keep transformations pure and testable)
Load — write to destination with conflict handling

Step 3: Add Scheduling and Monitoring

Configure the schedule using the project's tool (Airflow DAG, cron, Cloud Scheduler, etc.)
Add monitoring hooks: alerting on failure, SLA tracking, data freshness checks
Include a health check endpoint or status query

Step 4: Present the Pipeline

## Pipeline Summary

**Source:** [source] | **Destination:** [destination] | **Schedule:** [frequency]

### Data Flow
source → extract → transform → load → destination

### Error Handling
- [strategy for transient errors]
- [strategy for bad records]

### Monitoring
- [what is monitored]
- [alerting thresholds]

### Backfill
Run with: [command to backfill a date range]

Delivery

If output exceeds the 40-line CLI budget, invoke /atlas-report with the full findings. The HTML report is the output. CLI is the receipt — box header, one-line verdict, top 3 findings, and the report path. Never dump analysis to CLI.