Llm aiops
a kinds of skills can run in any agents, totally free.
npx -y skills add renky1025/agent-skills --skill llm-aiopsAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
One thing to look at
- 4 stars4 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.
What its author says it does
Copied from the file, not written here
Use when working on AIOps (AI for IT Operations) tasks — incident management, root cause analysis, log parsing/anomaly detection, cloud infrastructure management, or applying LLMs to IT operations. Also use when researching LLM-based solutions for incident response, fault localization, or infrastructure-as-code.
SKILL.md
4.8 KB, as published. Nobody here has run it
LLM AIOps Reference
Overview
Curated reference applying Large Language Models to AIOps, based on the awesome-LLM-AIOps research taxonomy. Covers incident management, log analysis, and infrastructure management.
When to Use
- Incidents: triaging, diagnosing, mitigating cloud incidents
- RCA: root cause analysis of system failures
- Logs: parsing, anomaly detection, logging statement generation
- Infrastructure: IaC, performance diagnosis, LLM training platforms
- AIOps QA: domain-specific question answering for operations
Taxonomy & Key Approaches
1. LLM for Incident Management
| Area | Key Approaches | Notable Systems |
|---|---|---|
| Surveys & Benchmarks | Evaluation frameworks for AIOps agents | AIOpsLab (MLSys 2025), ITBench (ICML 2025) |
| Incident Diagnosis | LLM agents + diagnostic tools | D-Bot (VLDB 2024), NetAssistant (NSDI 2024), FLASH (Preprint 2024) |
| Incident Reporting | Fine-tuning, CoT prompting for summarization | Oasis (FSE 2023), COLA (ICSE-SEIP 2024), MonitorAssistant (FSE 2024) |
| Root Cause Analysis | Agents with tool augmentation, CoT, ICL, multi-agent | RCAgent (CIKM 2024), mABC (EMNLP 2024), Flow-of-Action (WWW 2025), OpenRCA (ICLR 2025), COCA (ICSE 2025), RCACopilot (EuroSys 2024) |
| Incident Mitigation | Agent-based with troubleshooting guides, RAG | Nissist (ECAI 2024), LLexus (SIGOPS 2024), STRATUS (NeurIPS 2025) |
| Postmortem Analysis | Fine-tuning for fault profiling | FaultProfIT (ICSE-SEIP 2024), FAIL (ASE 2024) |
| AIOps Q&A | Instruction tuning, RAG, domain-specialized LLMs | OWL (ICLR 2024), iKnow (ASE 2025), OpsEval (Preprint 2023), MSQA (EMNLP 2023) |
2. LLM for Log Analysis
| Area | Key Approaches | Notable Systems |
|---|---|---|
| Log Parsing | ICL prompting, adaptive caching, entropy + CoT merging | DivLog (ICSE 2024), LILAC (FSE 2024), LogBatcher (ASE 2024), LUNAR (FSE 2025), LibreLog (ICSE 2025) |
| Log Anomaly Detection | Prompting, RAG, fine-tuning | LogGPT (Preprint 2023), LogPrompt (ICPC 2024), RAGLog (Preprint 2024), CodeAD (2025) |
| Logging Statement Generation | ICL, fine-tuning, static context analysis | UniLog (ICSE 2024), SCLogger (FSE 2024), FastLog (ISSTA 2024) |
3. LLM for Infrastructure Management
| Area | Key Approaches | Notable Systems |
|---|---|---|
| Infrastructure-as-Code | Agent-based bug discovery, semantic checking | Unearthing IaC Checks (SOSP 2024) |
| LLM Training Platform | Black-box performance diagnosis | LLMPrism (DSN 2025) |
| Benchmarks | IaC code generation | IaC-Eval (NeurIPS D&B 2024) |
Common LLM Techniques Used
- Prompting: ICL (In-Context Learning), CoT (Chain-of-Thought), ToT (Tree-of-Thought)
- Fine-tuning: Instruction tuning, domain-specific adaptation (e.g., OWL for IT ops)
- Agent-based: Tool-augmented LLMs, multi-agent collaboration, SOP-guided agents
- RAG: Retrieval-Augmented Generation for incident resolution recommendations
- Code Synthesis: LLM generates rule code for anomaly detection (CodeAD)
Key Benchmarks
- ITBench (ICML 2025) — Diverse real-world IT automation tasks
- AIOpsLab (MLSys 2025) — Holistic autonomous cloud agent evaluation
- OpsEval — Comprehensive AIOps task benchmark
- OpenRCA (ICLR 2025) — RCA benchmark with fault propagation awareness
- IaC-Eval (NeurIPS 2024) — Infrastructure-as-Code generation benchmark
Common Mistakes
- ❌ Treating log parsing as a solved problem — LLM-based parsers still struggle with unseen log formats; use adaptive caching (LILAC pattern)
- ❌ Applying generic LLMs to ops without domain tuning — Domain-specific models (OWL) or RAG significantly outperform general-purpose prompting for AIOps Q&A
- ❌ Single-agent RCA for complex microservice failures — Multi-agent systems (mABC, Flow-of-Action) with SOP guidance outperform single-agent approaches
- ❌ Ignoring confidence estimation in RCA — Use PACE-LM style confidence calibration before acting on LLM-generated RCA results
Real-World Impact
The field has rapidly evolved from basic prompting (2023) to sophisticated multi-agent systems with tool augmentation (2024-2025), with production-grade systems like Nissist, STRATUS, and iKnow deployed in real cloud environments. Key trend: moving from single LLM calls to autonomous agent loops with verification.