Skip to the rows

agentsclimarketplace

Documentation audit

Skill michellepellon/superpowers/skills/documentation-audit

A curated set of agent skills to support my daily workflows.

Install

npx -y skills add michellepellon/superpowers --skill documentation-audit

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

1 stars1 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Systematically verify documentation claims against codebase reality. Use when auditing docs, before releases, after refactors, or when documentation drift is suspected. Two-pass approach with pattern expansion ensures comprehensive detection of false claims, dead references, and gaps.

SKILL.md

5.7 KB, as published. Nobody here has run it

Documentation Audit Skill

Systematically verify claims in documentation against the actual codebase using a two-pass approach.

When This Skill Activates

Auditing documentation for accuracy
Before release cycles (verify docs match current behavior)
After major refactors (code changed, docs may be stale)
When users report docs don't match behavior
Periodic hygiene (quarterly documentation review)

Trigger phrases: "audit docs", "verify documentation", "check docs", "docs accurate", "documentation drift"

Core Principle

Low recall is worse than false positives — missed claims stay invisible and mislead users.

Two-pass process:

Pass 1: Extract and verify claims directly from docs
Pass 2A: Expand patterns from false claims to find similar issues
Pass 2B: Compare codebase inventory vs documented items (gap detection)

Quick Start

Identify target docs (user-facing only, skip plans/, audits/)
Note current git commit for report header
Run Pass 1 extraction using parallel agents (one per doc)
Analyze false claims for patterns
Run Pass 2 expansion searches
Generate docs/audits/AUDIT_REPORT_YYYY-MM-DD.md

Claim Types

Type	Example	Verification
`file_ref`	`scripts/foo.py`	File exists?
`config_default`	"defaults to 'AI Radio'"	Check schema/code
`env_var`	`STATION_NAME`	In .env.example + code?
`cli_command`	`--normalize` flag	Script supports it?
`behavior`	"runs every 2 minutes"	Check timers/code

Verification Confidence Tiers

Tier 1 (auto): file_ref, config_default, env_var, cli_command
Tier 2 (semi-auto): symbol_ref, version_req
Tier 3 (human review): behavior, constraint

Pass 2 Pattern Expansion

After Pass 1, analyze false claims and search for similar patterns:

Dead script found: diagnose_track_selection.py
  → Search: all script references → Found 8 more dead scripts

Wrong interval: "every 10 seconds"
  → Search: "every \d+ (seconds?|minutes?)" → Found 3 more

Wrong service name: ai-radio-break-gen.service
  → Search: service/timer names → Found naming inconsistencies

Common Patterns to Check

Always verify these patterns across all docs:

Dead scripts: scripts/*.py references
Timer intervals: every \d+ (seconds?|minutes?)
Service names: ai-radio-*.service, *.timer
Config vars: RADIO_* environment variables
CLI flags: --flag patterns in bash blocks

Output Format

Generate docs/audits/AUDIT_REPORT_YYYY-MM-DD.md:

# Documentation Audit Report
Generated: YYYY-MM-DD | Commit: abc123

## Executive Summary
| Metric | Count |
|--------|-------|
| Documents scanned | 12 |
| Claims verified | ~180 |
| Verified TRUE | ~145 (81%) |
| **Verified FALSE** | **31 (17%)** |

## False Claims Requiring Fixes
### CONFIGURATION.md
| Line | Claim | Reality | Fix |
|------|-------|---------|-----|
| 135 | `claude-sonnet-4-5` | Actual: `claude-3-5-sonnet-latest` | Update |

## Pattern Summary
| Pattern | Count | Root Cause |
|---------|-------|------------|
| Dead scripts | 9 | Scripts deleted, docs not updated |

## Human Review Queue
- [ ] Line 436: behavior claim needs verification

Execution Approach

This skill runs in Plan mode with a forked context to keep extraction artifacts separate from your main conversation:

Parallel extraction - Use one Task agent per document for efficiency
Evidence-based verdicts - Record file:line for every claim verification
Pattern matching - After Pass 1, systematically search for similar issues
Gap detection - List actual artifacts vs documented ones

Anti-Patterns to Avoid

Skipping Pass 2 — Pattern expansion catches 10-20% more issues
Trusting "looks correct" — Always verify with evidence
Fixing without evidence — Always cite file:line or "not found"
Auditing design docs — Focus on user-facing docs, skip historical artifacts
Batching verification early — Complete extraction first

Decision Framework

What to Audit

Include	Skip
User-facing README.md	`docs/plans/`
`docs/` directory	`docs/audits/`
Getting started guides	Design documents
Configuration docs	Historical artifacts
API documentation	Internal brainstorms

When to Run

Trigger	Frequency
Before release	Every release
After refactor	As needed
User reports mismatch	Immediate
Periodic hygiene	Quarterly

Real-World Results

From a representative audit:

12 documents scanned
~180 claims verified
31 false claims found (17% error rate)
Common patterns: dead scripts (9), wrong intervals (4), wrong service names (3)

Detailed References

For comprehensive coverage of specific topics, see:

references/checklist.md — Step-by-step execution checklist with TodoWrite integration
references/extraction-patterns.md — Detailed patterns for claim recognition and verification

Keep looking

Skills are one crate of 328,083. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.