agentsclimarketplace

Doc audit

Skill anantham/expansion/skills/doc-audit

Use when docs may have drifted from code, after development sprints, before onboarding, or when unsure if documented guarantees still hold. Surfaces stale docs, undocumented modules, naming conflicts, invalidated proofs, and convention violations.From its SKILL.md

Install
npx -y skills add anantham/expansion --skill doc-audit

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

2 things to look at

  • no licenseNo license file was found in the repository. Code published without one is not open source by default, so using it at work is a question for whoever answers licensing questions where you are.
  • 1 stars1 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

SKILL.md

8.8 KB, as published. Nobody here has run it

Doc Audit

Overview

Read-only diagnostic that compares documented intent against actual code. Produces a prioritized audit report — never auto-fixes. The human decides whether to update docs or change code.

Announce at start: "Running doc audit — comparing code against docs to surface drift, gaps, and inconsistencies."

When to Use

  • After a development sprint (has code drifted from docs?)
  • Before onboarding a contributor (are docs trustworthy?)
  • Periodically (monthly hygiene)
  • After major refactors (did docs follow?)
  • When you suspect a documented guarantee no longer holds
  • When naming feels inconsistent across modules

Not for: Writing docs (use doc-writer), creating proofs (use doc-prover), one-shot code smells (use surface-tech-debt).

Process

digraph audit {
  "Ask scope" [shape=diamond];
  "Scan code structure" [shape=box];
  "Scan existing docs" [shape=box];
  "Compare & classify" [shape=box];
  "Check proofs" [shape=box];
  "Check conventions" [shape=box];
  "Triage by severity" [shape=box];
  "Present report" [shape=doublecircle];

  "Ask scope" -> "Scan code structure";
  "Scan code structure" -> "Scan existing docs";
  "Scan existing docs" -> "Compare & classify";
  "Compare & classify" -> "Check proofs";
  "Check proofs" -> "Check conventions";
  "Check conventions" -> "Triage by severity";
  "Triage by severity" -> "Present report";
}

Phase 1: Scope

Ask the user:

What should I audit?

  1. Full codebase
  2. Specific directory (e.g., src/data/golden/)
  3. Specific module or feature
  4. Proofs only (re-verify all assertions)

For large codebases, recommend starting with a subsystem to avoid context overflow.

Non-interactive mode: If running via subagent or automated pipeline, scope must be passed as a parameter (e.g., the directory path). Skip the scope question and proceed directly to Phase 2.

Phase 2: Scan

Code structure scan

  • List all modules/packages (directories with __init__.py or significant .py/.js files)
  • For each module: list public classes, functions, constants
  • Note file sizes (LOC) — flag >300 LOC files per project conventions
  • Note imports to understand cross-module dependencies

Docs structure scan

  • Read docs/modules/INDEX.md if it exists
  • Read all files in docs/modules/
  • Read all files in docs/proofs/
  • Read docs/CONVENTIONS.md if it exists
  • Read docs/adr/ for feature-level decisions
  • Note Last verified and Code hash staleness markers in each doc

Phase 3: Compare & Classify

For each module found in code, classify its documentation state:

Finding Types

TypeTagMeaning
UndocumentedGAPCode exists, no docs at all
Stale docsSTALEDoc exists but code has changed (hash mismatch or visible drift)
Code driftDRIFTCode has deviated from documented intent (doc describes X, code does Y)
Missing rationaleWHY?Code exists, docs describe what but not why
Orphaned docsORPHANDoc exists but code was removed/renamed
Convention violationCONVENTIONCode violates a rule in CONVENTIONS.md
Naming conflictNAMINGSame concept, different names across modules
Duplicate logicDUPLICATESame thing implemented differently in multiple places
Invalidated proofPROOF!An ASSERTION's cited code lines have changed
Missing assertionASSERT?Complex invariant with no formal ASSERTION

How to detect each type

GAP: Module directory exists in code, no corresponding docs/modules/<name>.md.

STALE: Doc has Code hash: <sha> header. Compare against git log --format=%h -1 -- <module_path>. If different, doc is stale.

DRIFT: Read the doc's description of what the module does. Read the module's actual public API. If they describe different behavior, it's drift. Pay special attention to:

  • Function signatures that changed
  • New public functions not mentioned in docs
  • Removed functions still mentioned in docs
  • Changed return types or error behavior
  • ADR specification drift — compare code against ADR-documented formulas, algorithms, and contracts. If an ADR says "standard Brier score" but code implements a variant, that's DRIFT.

WHY?: Doc describes what functions do but never mentions trade-offs, alternatives considered, or design decisions. Check: does the doc answer "why this way and not another way?"

CONVENTION: Read docs/CONVENTIONS.md. Grep for violations. Common ones:

  • Naming patterns (snake_case vs camelCase mixing)
  • Error handling patterns (some modules raise, some return None)
  • Import conventions (relative vs absolute)
  • Data format conventions (dict keys: snake_case vs camelCase)

Important: Check CONVENTIONS.md for an "Intentional Divergences" section before flagging naming conflicts. Some projects intentionally use different conventions across boundaries (e.g., snake_case in DB columns but camelCase in API responses). If the divergence is documented as intentional, it's NOT a violation.

NAMING: Search for semantic duplicates across modules:

  • get_user vs fetch_user vs retrieve_user
  • account_id vs user_id vs accountId for the same concept
  • Different names for the same data structure

DUPLICATE: Two modules implementing the same algorithm or pattern differently. Often found via similar function names or similar import patterns.

PROOF!: Read each docs/proofs/*.md. Extract cited file:line references. Check if those lines still contain the cited code. If the code at those lines has changed, the proof may be invalidated.

ASSERT?: Look for complex mathematical operations, security boundaries, data invariants that lack a formal ASSERTION. Candidates:

  • Normalization/scaling operations
  • Cryptographic or hashing operations
  • Data validation that enforces invariants
  • Permission/access control checks
  • Privacy boundaries (data that must not leave the system)

Phase 4: Triage

Rate each finding:

SeverityCriteria
P0 — CriticalInvalidated proof, security boundary drift, data invariant broken
P1 — HighCode drift (doc says X, code does Y), missing rationale for complex logic
P2 — MediumStale docs (hash mismatch but likely minor), naming conflicts, convention violations
P3 — LowUndocumented simple modules, orphaned docs, missing assertions for stable code

Phase 5: Report

Present findings in this format:

# DOC AUDIT REPORT
**Scope:** <what was audited>
**Date:** <date>
**Code hash:** <git short sha of HEAD>

## Summary
| Type | Count | Critical | High | Medium | Low |
|------|-------|----------|------|--------|-----|
| GAP  |       |          |      |        |     |
| STALE|       |          |      |        |     |
| DRIFT|       |          |      |        |     |
| ...  |       |          |      |        |     |

## Critical Findings (P0)
### [PROOF!] <assertion name>
- **Assertion:** <what was claimed>
- **Proof file:** `docs/proofs/<name>.md`
- **Invalidated because:** <file:line> changed from `<old>` to `<new>`
- **Action needed:** Re-verify assertion or update proof

## High Priority (P1)
### [DRIFT] <module name>
- **Doc says:** <documented behavior>
- **Code does:** <actual behavior>
- **Question for human:** Should we update the docs or fix the code?

## Medium Priority (P2)
...

## Low Priority (P3)
...

## Recommended Next Steps
1. <action> — use `doc-prover` to re-verify invalidated proofs
2. <action> — use `doc-writer` to document module X
3. <action> — update CONVENTIONS.md to resolve naming conflict

Integration with Other Skills

  • After audit → doc-writer: "Fix finding #3" → creates/updates module docs
  • After audit → doc-prover: "Re-verify assertion X" → re-checks proof
  • After audit → surface-tech-debt: Audit finds doc gaps; tech-debt finds code smells. Complementary.
  • CONVENTIONS.md is ground truth for convention checks. If it doesn't exist, doc-writer should create it first. Run doc-writer to create CONVENTIONS.md BEFORE running a full audit — otherwise convention checks have no baseline and produce noise.

Checklist

  • Asked user for scope
  • Scanned code structure (modules, classes, functions)
  • Scanned existing docs (modules/, proofs/, CONVENTIONS.md, ADRs)
  • Compared code against docs — classified each finding
  • Checked all proof files for invalidated citations
  • Checked for convention violations
  • Triaged by severity (P0-P3)
  • Presented structured report
  • Listed recommended next steps with skill references

Keep looking

Skills are one crate of 608,475. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.