Coverage strategy

Coverage strategy

Coverage is a good flashlight and a terrible scoreboard. Aimed at the code, it shows which branches no test has ever run, which is real information. Turned into a target, it rots: people write assertion-free tests that execute lines without checking anything, the number climbs, and the software stays fragile. Read it the other way, as a map of where untested risk sits.

Method

1. Measure branch coverage, not just lines. Run pytest --cov --cov-branch or coverage run --branch. Line coverage calls an if with only its true arm tested fully covered; branch coverage exposes the untaken else, which is exactly where the unhandled case usually hides.
2. Read the report by file, not by headline percent. Open the HTML report and scan for high-risk files sitting low: auth, money, parsing, error paths. A red branch in a refund path is a finding worth a test; one in a debug log is noise worth ignoring. The aggregate number hides both.
3. Rank gaps by blast radius, then write top-down. An uncovered branch in the payment flow outranks an uncovered getter. Order the red spots by how much breaks if they are wrong and how often the code runs.
4. Gate on diff coverage, not a global floor. Enforce coverage on changed lines with diff-cover or a Codecov patch status, so new code arrives tested without demanding a heroic sprint over legacy nobody is touching. A repo-wide percentage gate punishes whoever edits an old file.
5. Reject coverage that comes without an assertion. A test that calls code and checks nothing turns lines green while proving nothing. Catch it in review, and confirm the tests actually bite with mutation-testing: if flipping > to >= leaves the suite green, that covered line is run but not tested.
6. Exclude the deliberately untested so real gaps show. Mark generated bindings, trivial getters, and unreachable defensive branches with # pragma: no cover, so the remaining number reflects code that genuinely should be covered.

Litmus tests

• Can you name the three riskiest files under 60 percent branch coverage now?
• Does your gate fire on new untested code while staying quiet about untouched legacy?
• Would a mutation in a covered line actually fail a test, or merely run?

Boundaries

Coverage measures execution, never whether the assertion was right or the case worth testing, and it says nothing about concurrency or integration seams. Use it to find gaps, then let testing-strategy and human judgment decide which to close. Whether covered code is truly tested is mutation-testing's question, and the strength of the assertions themselves is test-review's.