agentsclimarketplace
Sign in

Airflow dag patterns

Skill ComeOnOliver/skillshub/skills/aiskillstore/marketplace/sickn33/airflow-dag-patterns

Build production Apache Airflow DAGs with best practices for operators, sensors, testing, and deployment. Use when creating data pipelines, orchestrating workflows, or scheduling batch jobs.From its SKILL.md

Install
npx -y skills add ComeOnOliver/skillshub --skill airflow-dag-patterns

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

SKILL.md

1.4 KB, 243 tokens by cl100k_base, as published. Nobody here has run it

Apache Airflow DAG Patterns

Production-ready patterns for Apache Airflow including DAG design, operators, sensors, testing, and deployment strategies.

Use this skill when

  • Creating data pipeline orchestration with Airflow
  • Designing DAG structures and dependencies
  • Implementing custom operators and sensors
  • Testing Airflow DAGs locally
  • Setting up Airflow in production
  • Debugging failed DAG runs

Do not use this skill when

  • You only need a simple cron job or shell script
  • Airflow is not part of the tooling stack
  • The task is unrelated to workflow orchestration

Instructions

  1. Identify data sources, schedules, and dependencies.
  2. Design idempotent tasks with clear ownership and retries.
  3. Implement DAGs with observability and alerting hooks.
  4. Validate in staging and document operational runbooks.

Refer to resources/implementation-playbook.md for detailed patterns, checklists, and templates.

Safety

  • Avoid changing production DAG schedules without approval.
  • Test backfills and retries carefully to prevent data duplication.

Resources

  • resources/implementation-playbook.md for detailed patterns, checklists, and templates.

What ships with it

22.9 KB alongside SKILL.md

GitHub clipped this repository’s file list, so this is at least 2 files and may be more.

resources/

Gives 0 of the 12 instructions most data pipelines skills give in 243 tokens

Counted across 149 of the 156 authors here whose files we hold, read 2026-09-06

  • Run a safe catch-up or sample benchmarkin 13 of 149, across 4 files
  • Rerun final accounting after the codified path executesin 13 of 149, across 4 files
  • Move compute to where the data already isin 13 of 149, across 4 files
  • Batch small files, requests, and writesin 13 of 149, across 4 files
  • Use manifests or checkpoints to skip completed filesin 13 of 149, across 4 files
  • Measure backlog across files, rows, and timestampsin 13 of 149, across 4 files
  • Codify the path as a CLI or scheduled jobin 12 of 149, across 3 files
  • Promote only the fastest correctness-preserving pathin 10 of 149, across 3 files
  • Separate the bottleneck categories before optimizingin 10 of 149, across 3 files
  • Prefer warehouse-native scans, joins, and appendsin 9 of 149, across 2 files
  • Make writes idempotent through keys, manifests, or replaceable stagingin 9 of 149, across 2 files
  • Retry failures with exponential backoffin 9 of 149, across 7 files

Grouped from the skills themselves: near-identical wordings counted once, and counted by distinct author, so one author publishing three of these counts once. Length counted with cl100k_base; the agent that loads this file may tokenize it differently.

Keep looking

Skills are one crate of 325,949. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.