Mutation test runner
Skill SkillMedev/qa-test-automation/skills/mutation-test-runner
Build a test suite that actually catches bugs — and stops flaking.
npx -y skills add SkillMedev/qa-test-automation --skill mutation-test-runnerAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
One thing to look at
- 0 stars0 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.
What its author says it does
Copied from the file, not written here
Runs mutation testing on already-covered code and turns each surviving mutant into a specific missing assertion, exposing tests that execute code but verify nothing. Use when line coverage is high yet a bug slipped through, when reviewing a critical module (pricing, permissions, auth, state machines) before release, or when you are asked to run Stryker, PIT, mutmut, or cosmic-ray. Do NOT use when the goal is to find which code paths are untested at all - use coverage-gap-finder instead.
SKILL.md
3.3 KB, as published. Nobody here has run it
Mutation Test Runner
Prove whether covered code is actually verified: deliberately break a line, and check that a test notices. A surviving mutant is proof that the line is executed but unasserted.
Workflow
- Pick the target. Scope to high-risk logic where a silent bug is expensive - pricing, permissions, auth, state machines, financial math. Skip generated code, trivial DTOs, and glue; they have nothing worth mutating.
- Pick the runner: Stryker (JS/TS/C#), PIT (JVM), mutmut or cosmic-ray (Python). Configure it to run against the target's existing test suite.
- Run scoped, not global. In CI, mutate only changed files (Stryker incremental / diff mode); reserve a full sweep for a nightly job. Enable coverage analysis so only tests touching a mutated line rerun, and parallelize across cores.
- Read each surviving mutant as a named missing assertion, not a vague gap:
<mutated to<=survives -> the boundary case is unasserted; add an assertion at the exact edge value.- A removed function call survives -> no test checks that side effect; assert the side effect.
- A negated condition survives -> the two branches are indistinguishable to the suite; assert the differing outcome of each branch.
- A return value replaced with a constant survives -> the return is unchecked; assert it.
- Triage equivalent mutants. Some mutants cannot change observable behavior (a change in dead code,
i++vs++iin isolation). Distinguishing them is undecidable in general - mark them ignored in the tool's config rather than chasing 100%. - Set a meaningful floor on the targeted modules only (e.g. 70-80% mutation score), where score = killed / (total non-equivalent mutants). Never set one global threshold.
Quality bar
- Every reported survivor maps to one concrete, named assertion a developer can write - no "add more tests."
- Mutation runs are scoped (diff/incremental in CI, full sweep nightly); no PR blocks on a full-repo run.
- Equivalent mutants are explicitly ignored, not silently inflating or deflating the score.
- The score is presented as a quality signal for already-covered code, never as a coverage substitute.
Do NOT
- Do NOT block every PR on a full-repo mutation run - it reruns the suite per mutant and will stall the pipeline.
- Do NOT chase a 100% mutation score; equivalent mutants make it unreachable and the marginal mutants are noise.
- Do NOT run mutation testing on generated code, DTOs, or glue - spend the compute where a silent failure hurts.
- Do NOT use a single global threshold; floors belong on critical modules.
- Do NOT treat mutation testing as a substitute for deciding what to test. It measures assertion strength on existing code; a missing feature has no code to mutate. To find untested paths and inputs, use coverage-gap-finder.