Sumo qa measuring coverage

Skill sumithr/sumo-qa/skills/sumo-qa-measuring-coverage

Use when the user wants coverage or mutation numbers in the local QA report — "run coverage", "measure coverage", "run mutation testing", "record the survivors". Runs the repo's configured coverage/mutation tool, reads its output (any format), and persists a compact summary into the .sumo-qa artifacts the report loads.From its SKILL.md

Install

npx -y skills add sumithr/sumo-qa --skill sumo-qa-measuring-coverage

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

6 stars6 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

SKILL.md

5.7 KB, as published. Nobody here has run it

Measuring coverage & mutation

Turn a repo's coverage/mutation run into evidence the QA report can cite. The report reads .sumo-qa/coverage.json and .sumo-qa/mutation.json; this skill produces them. The host (you) runs the tool and reads its output; the MCP tools only validate + persist.

Announce at start: "Measuring coverage/mutation and recording it for the report."

Output discipline (mandatory)

Inherits the global discipline from using-sumo-qa: never surface internal taxonomy labels; spend output on findings not framing; one question per turn; no preamble or closing pleasantries.

<HARD-GATE> NEVER fabricate a number. A coverage percentage or survivor count goes into an artifact ONLY when it came from a real run (or a report the user pointed you at) in this session. No run / no report ⇒ record nothing, report "not available". </HARD-GATE>

The Iron Law

MEASURED, REPORTED, NEVER GATED. Coverage and mutation are evidence the report shows — they never become a pass/fail gate and never flip a readiness verdict. A 100% coverage signal does not make a risk "covered" or a blocked verdict "ready". If you catch yourself treating a percentage as a safety threshold, stop.

When to Use

"run coverage and put it in the report", "measure coverage", "what's my coverage" (when the report should carry it)
"run mutation testing", "record the survivors", "get mutation into the scorecard"
As a follow-on to a QA rollout, before sumo_qa_generate_qa_report, when you want the Coverage/Mutation rows populated instead of "not available".

NOT for: live PostToolUse mutation-survivor routing (that is #127's router — a different concern; see sumo-qa-strengthening-tests). Reading a saved report ≠ running the router.

Checklist

Detect (configured tooling only). Inspect for tooling the project already pins — coverage: pyproject [tool.coverage] / --cov in pytest addopts / .coveragerc, package.json jest --coverage / nyc / c8, go test -cover, a committed coverage.xml (Cobertura) / coverage.json (coverage.py) / lcov.info; mutation: [tool.mutmut] / mutmut, Stryker (stryker.conf.*, mutation.json), PIT (mutations.xml). Never pip install / npm i a tool. Nothing configured → stop, report "not available" (step 5). Configured but unrun → offer a scoped run; if declined, fall back.
Run the detected command, scoped. Run the project's own command, preferring a machine-readable report and scoping to changed code when a diff is in play (e.g. pytest --cov=<pkg> --cov-report=json; mutmut run + its results summary). A run that errors (missing dep, misconfig) → report it plainly and fall back; never mask it.
Collect (LLM reads any format). Read the output — JSON, XML, lcov, or console text — and extract: coverage → line_percent + a compact detail naming uncovered/weak changed files/functions (tie gaps to .sumo-qa/diff-impact.json when present — changed code, not global vanity totals); mutation → survivors + killed + a detail on where notable survivors live. Set freshness="fresh" (you just ran it), source_tool = what you ran, generated_at = an ISO-8601 timestamp.
Persist. Coverage → sumo_qa_record_coverage(root, coverage={...}); mutation → sumo_qa_record_mutation(root, mutation={...}). Each validates and writes the artifact, or returns an error envelope (fix the payload, don't hand-write the file). Then tell the user to (re)run sumo_qa_generate_qa_report to see the Coverage/Mutation rows populate.
Fallback (no tool / no run). Report concisely: "Coverage/mutation not available — no coverage tool is configured in this repo." Record nothing. A first-class outcome, never a workflow failure, never a reason to invent a number.

Process Flow

See the Checklist above — that's the flow.

Red Flags — STOP

Thought	Reality
"Coverage is 100%, so this risk is covered / it's safe to merge"	Iron Law. Coverage is reported, never a gate. It cannot discharge a risk or flip a verdict.
"No coverage tool here, I'll just `pip install` one"	Never. Configured tooling only; absent tool ⇒ "not available". Adoption is a separate decision (`sumo-qa-strategising`), repo-pinned, never ad-hoc.
"I'll estimate the percentage from reading the code"	Fabrication. A number comes from a real run or a real report, or it doesn't get recorded.
"I'll re-run the suite to generate a mutation report for the live router"	Wrong skill. Reading a SAVED report here ≠ #127's PostToolUse router. Don't conflate them.
"I'll hand-edit `.sumo-qa/coverage.json`"	Use `sumo_qa_record_coverage` — it validates the shape. A bad payload is an error to fix, not a file to forge.
"The run errored, I'll record the last known number"	Stale/fabricated. Report the failure, fall back to "not available".

Example

Repo pins pytest --cov in addopts. User: "get coverage into the QA report."

Detect: [tool.pytest] addopts carry --cov=src/pkg. ✓
Run: pytest --cov=src/pkg --cov-report=json → coverage.json, 96.4% lines.
Collect: line_percent=96.4, detail="changed-file gaps: pkg/api.py L40-44 (error branch uncovered)", freshness="fresh", source_tool="pytest-cov (coverage.py json)".
Persist: sumo_qa_record_coverage(root=".", coverage={...}) → .sumo-qa/coverage.json. Tell the user to regenerate the report.