agentsclimarketplace
Sign in

Airflow

Skill G1Joshi/Agent-Skills/skills/ai-ml/airflow

Apache Airflow workflow orchestration. Use for data pipelines.From its SKILL.md

Install
npx -y skills add G1Joshi/Agent-Skills --skill airflow

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

  • 12 stars12 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

SKILL.md

1.0 KB, 230 tokens by cl100k_base, as published. Nobody here has run it

Airflow

Apache Airflow is the standard for data engineering pipelines. v3.0 (2025) introduces Event-driven Triggers and a modern React UI.

When to Use

  • ETL/ELT: Scheduling nightly data warehouse loads.
  • ML Ops: Retraining models when new data arrives.
  • Dependency Management: "Run Task B only if Task A succeeds".

Core Concepts

DAGs (Directed Acyclic Graphs)

Defined in Python.

Task SDK

New in v3.0. Allows writing tasks in any language, not just Python.

Edge Executor

Run tasks on remote edge devices.

Best Practices (2025)

Do:

  • Use the TaskFlow API: @task decorators are cleaner than PythonOperator.
  • Use Datasets: Define data-aware scheduling (schedule=[Dataset("s3://bucket/file")]).

Don't:

  • Don't put top-level code in DAG files: It runs every scheduler heartbeat.

References

What ships with it

Read from the repository

Just SKILL.md. No reference files, no scripts.

Keep looking

Skills are one crate of 325,949. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.