Pipelines & ETL
567 rows, by stacks then stars
lensesio/agentic-engineering-for-apache-kafka/skills/kafka-shadowtraffic-java Skill
55★ repoAI agent skills for building, operating and troubleshooting Apache Kafka applications. Topic audit, consumer lag, schema review, security, connectors and DLQ
lensesio/agentic-engineering-for-apache-kafka/skills/kafka-topic-audit Skill
55★ repoAI agent skills for building, operating and troubleshooting Apache Kafka applications. Topic audit, consumer lag, schema review, security, connectors and DLQ
Motherduck build data pipeline
motherduckdb/agent-skills/skills/motherduck-build-data-pipeline Skill
no license53★ repoOpinionated AI agent skills for building applications with MotherDuck
Motherduck build data pipeline
motherduckdb/agent-skills/plugins/motherduck-skills-claude/skills/motherduck-build-data-pipeline Skill
no license53★ repoOpinionated AI agent skills for building applications with MotherDuck
Motherduck build data pipeline
motherduckdb/agent-skills/plugins/motherduck-skills/skills/motherduck-build-data-pipeline Skill
no license53★ repoOpinionated AI agent skills for building applications with MotherDuck
BagelHole/DevOps-Security-Agent-Skills/devops/ai/ai-pipeline-orchestration Skill
43★ repoAgent-ready DevOps, security, infrastructure, and compliance knowledge base with 80+ skills across Kubernetes, Terraform, AWS/Azure/GCP, AI platform operations, container hardening, SOC2/ISO27001, and incident response—plus ready-to-run scripts, templates, and playbooks for SRE, platform, and security teams.
saskinosie/weaviate-claude-skills/weaviate-data-ingestion Skill
39★ repoClaude Skills for connecting Claude.ai to local Weaviate vector databases - manage collections, ingest data, and query with RAG
abhishekbhakat/airflow-mcp-server/io.github.abhishekbhakat/airflow-mcp-server MCP server
33★ repoAn MCP server for Apache Airflow
proyecto26/system-design-skills/skills/messaging-streaming Skill
32★ repoA divide-and-conquer wiki of system-design skills for Claude Code — reason about scalable systems, don't memorize architectures.
saeed-vayghan/gemini-agent-skills/.gemini/skills/data-engineer Skill
29★ repoA comprehensive collection of expert AI Agent Skills for Google Gemini, featuring a powerful CLI to convert and optimize Claude Code agents for the Gemini ecosystem.
mjunaidca/mjs-agent-skills/.claude/skills/kafka Skill
no license26★ repoA curated collection of Agent Skills — reusable units of intelligence that teach AI General Agents how to perform specific tasks autonomously.
charlieviettq/awesome-agent-skill/.cursor/skills/asgard-ai-platform/tech-data-pipeline Skill
22★ repoCurated skill pack for LLM agents in engineer and science workflow (Cursor & Claude ready).
charlieviettq/awesome-agent-skill/.cursor/skills/voltagent/data-ai/data-engineer Skill
22★ repoCurated skill pack for LLM agents in engineer and science workflow (Cursor & Claude ready).
charlieviettq/awesome-agent-skill/.claude/skills/tech-data-pipeline Skill
22★ repoCurated skill pack for LLM agents in engineer and science workflow (Cursor & Claude ready).
charlieviettq/awesome-agent-skill/.claude/skills/va-data-engineer Skill
22★ repoCurated skill pack for LLM agents in engineer and science workflow (Cursor & Claude ready).
magnus919/agent-skills/data-engineering Skill
25 days old21★ repoCurated collection of AI agent skills for Hermes and other agent frameworks
Airflow and workflow orchestration
vaquarkhan/data-engineering-agent-skills/skills/airflow-and-workflow-orchestration Skill
21★ repoProduction-grade Agent Skills for data engineering AI agents: 73 workflows, platform presets, safe backfill/replay, Kafka & Spark reliability, MCP observability, and VS Code/JetBrains installers.
vaquarkhan/data-engineering-agent-skills/skills/dbt-and-analytics-engineering Skill
21★ repoProduction-grade Agent Skills for data engineering AI agents: 73 workflows, platform presets, safe backfill/replay, Kafka & Spark reliability, MCP observability, and VS Code/JetBrains installers.
Debezium and kafka connect cdc
vaquarkhan/data-engineering-agent-skills/skills/debezium-and-kafka-connect-cdc Skill
21★ repoProduction-grade Agent Skills for data engineering AI agents: 73 workflows, platform presets, safe backfill/replay, Kafka & Spark reliability, MCP observability, and VS Code/JetBrains installers.
Enterprise etl and data integration modernization
vaquarkhan/data-engineering-agent-skills/skills/enterprise-etl-and-data-integration-modernization Skill
21★ repoProduction-grade Agent Skills for data engineering AI agents: 73 workflows, platform presets, safe backfill/replay, Kafka & Spark reliability, MCP observability, and VS Code/JetBrains installers.
Etl elt and modernization strategy
vaquarkhan/data-engineering-agent-skills/skills/etl-elt-and-modernization-strategy Skill
21★ repoProduction-grade Agent Skills for data engineering AI agents: 73 workflows, platform presets, safe backfill/replay, Kafka & Spark reliability, MCP observability, and VS Code/JetBrains installers.
Kafka resilience and schema evolution
vaquarkhan/data-engineering-agent-skills/skills/kafka-resilience-and-schema-evolution Skill
21★ repoProduction-grade Agent Skills for data engineering AI agents: 73 workflows, platform presets, safe backfill/replay, Kafka & Spark reliability, MCP observability, and VS Code/JetBrains installers.
Reverse etl and operational data serving
vaquarkhan/data-engineering-agent-skills/skills/reverse-etl-and-operational-data-serving Skill
21★ repoProduction-grade Agent Skills for data engineering AI agents: 73 workflows, platform presets, safe backfill/replay, Kafka & Spark reliability, MCP observability, and VS Code/JetBrains installers.
Streaming and messaging systems
vaquarkhan/data-engineering-agent-skills/skills/streaming-and-messaging-systems Skill
21★ repoProduction-grade Agent Skills for data engineering AI agents: 73 workflows, platform presets, safe backfill/replay, Kafka & Spark reliability, MCP observability, and VS Code/JetBrains installers.
inbharatai/claude-skills/skills/etl-pipeline Skill
19★ repoCurated Claude and Codex skill collection for local AI workflows, automation, research, and product-building.
Chipagosfinest/enterprise-team/enterprise-team/skills/data-catchall Skill
no license18★ repoHire a whole company with one plugin. 75 specialized AI agents across Engineering, Product, Infrastructure, Data, Security, Marketing, Sales, Finance, Legal, and People.
Raishin/vanguard-frontier-agentic/skills/gcp/gcp-data-pipeline-engineer Skill
18★ repoCurated marketplace of AI skills, agents, and rules for cloud, zero-trust, and compliance-aware engineering - works with Claude Code, Codex, Cursor, Copilot, and more.
Huawei drs data replication operator
Raishin/vanguard-frontier-agentic/skills/huawei/huawei-drs-data-replication-operator Skill
18★ repoCurated marketplace of AI skills, agents, and rules for cloud, zero-trust, and compliance-aware engineering - works with Claude Code, Codex, Cursor, Copilot, and more.
Raishin/vanguard-frontier-agentic/skills/java/java-kafka-reliability Skill
18★ repoCurated marketplace of AI skills, agents, and rules for cloud, zero-trust, and compliance-aware engineering - works with Claude Code, Codex, Cursor, Copilot, and more.
zakelfassi/skills-driven-development/examples/data-pipeline/skills/pipeline-stage Skill
17★ repoAgents that learn by doing — and remember how they did it. A methodology for AI agents to create, evolve, and share reusable skills.
Benkapner/claude-code-basecamp/skills/data-pipeline-patterns Skill
14★ repoA meta-repository for Claude Code users that includes workspace setup (skills, commands, hooks)
Multiome data ingestion paired modalities
HolobiomicsLab/asb-skill-collections/collections/epigenomics/v1/skills/multiome-data-ingestion-paired-modalities Skill
14★ repoCurated, evidence-grounded skill and software-tool collections for scientific AI agents, generated by the AgenticScienceBuilder
Custom code execution in data pipeline
HolobiomicsLab/asb-skill-collections/collections/metabolomics/v1/skills/custom-code-execution-in-data-pipeline Skill
14★ repoCurated, evidence-grounded skill and software-tool collections for scientific AI agents, generated by the AgenticScienceBuilder
HolobiomicsLab/asb-skill-collections/collections/metabolomics/v2/skills/multi-format-data-ingestion Skill
14★ repoCurated, evidence-grounded skill and software-tool collections for scientific AI agents, generated by the AgenticScienceBuilder
Data ingestion pipeline design
HolobiomicsLab/asb-skill-collections/collections/metabolomics/v2/skills/data-ingestion-pipeline-design Skill
14★ repoCurated, evidence-grounded skill and software-tool collections for scientific AI agents, generated by the AgenticScienceBuilder
Custom code execution in data pipeline
HolobiomicsLab/asb-skill-collections/collections/metabolomics/v2/skills/custom-code-execution-in-data-pipeline Skill
14★ repoCurated, evidence-grounded skill and software-tool collections for scientific AI agents, generated by the AgenticScienceBuilder
Raw mass spectrometry data ingestion
HolobiomicsLab/asb-skill-collections/collections/metabolomics/v2/skills/raw-mass-spectrometry-data-ingestion Skill
14★ repoCurated, evidence-grounded skill and software-tool collections for scientific AI agents, generated by the AgenticScienceBuilder
Data lineage preservation in etl pipelines
HolobiomicsLab/asb-skill-collections/collections/metabolomics/v2/skills/data-lineage-preservation-in-etl-pipelines Skill
14★ repoCurated, evidence-grounded skill and software-tool collections for scientific AI agents, generated by the AgenticScienceBuilder
Python sqlite migration and etl
HolobiomicsLab/asb-skill-collections/collections/metabolomics/v2/skills/python-sqlite-migration-and-etl Skill
14★ repoCurated, evidence-grounded skill and software-tool collections for scientific AI agents, generated by the AgenticScienceBuilder
Lc ms data pipeline architecture
HolobiomicsLab/asb-skill-collections/collections/metabolomics/v2/skills/lc-ms-data-pipeline-architecture Skill
14★ repoCurated, evidence-grounded skill and software-tool collections for scientific AI agents, generated by the AgenticScienceBuilder
Format agnostic data ingestion
HolobiomicsLab/asb-skill-collections/collections/metabolomics/v2/skills/format-agnostic-data-ingestion Skill
14★ repoCurated, evidence-grounded skill and software-tool collections for scientific AI agents, generated by the AgenticScienceBuilder
Data pipeline module design and integration
HolobiomicsLab/asb-skill-collections/collections/metabolomics/v2/skills/data-pipeline-module-design-and-integration Skill
14★ repoCurated, evidence-grounded skill and software-tool collections for scientific AI agents, generated by the AgenticScienceBuilder
Format agnostic data ingestion
HolobiomicsLab/asb-skill-collections/packs/metabolomics/ion-mobility/skills/format-agnostic-data-ingestion Skill
14★ repoCurated, evidence-grounded skill and software-tool collections for scientific AI agents, generated by the AgenticScienceBuilder
Lc ms data pipeline architecture
HolobiomicsLab/asb-skill-collections/packs/metabolomics/lc-ms/skills/lc-ms-data-pipeline-architecture Skill
14★ repoCurated, evidence-grounded skill and software-tool collections for scientific AI agents, generated by the AgenticScienceBuilder
Python sqlite migration and etl
HolobiomicsLab/asb-skill-collections/packs/metabolomics/lc-ms/skills/python-sqlite-migration-and-etl Skill
14★ repoCurated, evidence-grounded skill and software-tool collections for scientific AI agents, generated by the AgenticScienceBuilder
HolobiomicsLab/asb-skill-collections/packs/metabolomics/ms-generic/skills/multi-format-data-ingestion Skill
14★ repoCurated, evidence-grounded skill and software-tool collections for scientific AI agents, generated by the AgenticScienceBuilder
Raw mass spectrometry data ingestion
HolobiomicsLab/asb-skill-collections/packs/metabolomics/ms-generic/skills/raw-mass-spectrometry-data-ingestion Skill
14★ repoCurated, evidence-grounded skill and software-tool collections for scientific AI agents, generated by the AgenticScienceBuilder
Data ingestion pipeline design
HolobiomicsLab/asb-skill-collections/packs/metabolomics/ms-generic/skills/data-ingestion-pipeline-design Skill
14★ repoCurated, evidence-grounded skill and software-tool collections for scientific AI agents, generated by the AgenticScienceBuilder
Data pipeline module design and integration
HolobiomicsLab/asb-skill-collections/packs/metabolomics/nmr/skills/data-pipeline-module-design-and-integration Skill
14★ repoCurated, evidence-grounded skill and software-tool collections for scientific AI agents, generated by the AgenticScienceBuilder
ivanshamaev/de-agent-skills/skills/airbyte Skill
no license13★ repoПрофессиональные Data Engineering Agent Skills для разработки AI Agentic Data Platform
ivanshamaev/de-agent-skills/skills/airflow_dag_factory Skill
no license13★ repoПрофессиональные Data Engineering Agent Skills для разработки AI Agentic Data Platform
ivanshamaev/de-agent-skills/skills/airflow_dags Skill
no license13★ repoПрофессиональные Data Engineering Agent Skills для разработки AI Agentic Data Platform
ivanshamaev/de-agent-skills/skills/apache_kafka Skill
no license13★ repoПрофессиональные Data Engineering Agent Skills для разработки AI Agentic Data Platform
ivanshamaev/de-agent-skills/skills/datahub_catalog Skill
no license13★ repoПрофессиональные Data Engineering Agent Skills для разработки AI Agentic Data Platform
ivanshamaev/de-agent-skills/skills/dbt_core Skill
no license13★ repoПрофессиональные Data Engineering Agent Skills для разработки AI Agentic Data Platform
ivanshamaev/de-agent-skills/skills/dbt_macros Skill
no license13★ repoПрофессиональные Data Engineering Agent Skills для разработки AI Agentic Data Platform
ivanshamaev/de-agent-skills/skills/dbt_trino Skill
no license13★ repoПрофессиональные Data Engineering Agent Skills для разработки AI Agentic Data Platform
ivanshamaev/de-agent-skills/skills/de_rca Skill
no license13★ repoПрофессиональные Data Engineering Agent Skills для разработки AI Agentic Data Platform
ivanshamaev/de-agent-skills/skills/docker_data_envs Skill
no license13★ repoПрофессиональные Data Engineering Agent Skills для разработки AI Agentic Data Platform
ivanshamaev/de-agent-skills/skills/github_actions_dataops Skill
no license13★ repoПрофессиональные Data Engineering Agent Skills для разработки AI Agentic Data Platform