Coverage ratchet

Coverage Ratchet

A build/test gate proves the tests that exist pass. It says nothing about whether tests were written — a phase with no test task ships green on build success alone. The coverage ratchet closes that hole deterministically: line coverage is measured on every verify, compared against a committed baseline, and may never drop. It only rises.

Why a Ratchet, Not a Target

An absolute threshold ("80%") fails one of two ways on a real codebase: it fails day one (so it gets disabled), or it gets set below current reality (so it gates nothing). A ratchet starts from wherever the project actually is and only tightens. Nobody argues with "don't get worse."

When This Skill Activates

Use this skill when:

• A project has a test target but coverage has never been measured
• Agent-written features keep arriving with build-passing, test-free code
• Setting up the deterministic gauntlet (with testing/fitness-functions/ and swift/code-size/)

Setup

1. Copy templates/coverage-gate.sh into the project (e.g. Scripts/coverage-gate.sh), make it executable, and fill in the config block (scheme, destination, app target name).
2. Bootstrap the baseline:

Scripts/coverage-gate.sh --init     # runs tests with coverage, writes .coverage-baseline

3. Commit both the script and .coverage-baseline.

.coverage-baseline holds a single decimal fraction (e.g. 0.6231) — the app target's line coverage. Trailing whitespace ignored; nothing else in the file.

The Gate

Scripts/coverage-gate.sh            # exit 0 = pass, exit 1 = coverage dropped

• Runs the test suite with coverage enabled, extracts the app target's line coverage from the xccov JSON report.
• Fails if current < baseline − 0.0025 (the epsilon absorbs measurement jitter from unchanged code).
• Suggests a ratchet-up when current > baseline + 0.01: update .coverage-baseline to the new number in the same commit as the tests that earned it. That's the ratchet clicking forward.
• Fails loudly on an unexpected report shape or a missing target — a broken measurement must never read as a pass.

Ratchet-up etiquette: raising the baseline is routine and encouraged; lowering it requires a written reason in the commit message (e.g. deleting a well-covered module) — treat it like widening a fitness-function allowlist.

CRAP Report (Advisory)

templates/crap-report.sh combines per-function cyclomatic complexity (SwiftLint JSON output) with per-function coverage (the same xccov report) into the CRAP score — Change Risk Anti-Patterns:

CRAP(f) = complexity(f)² × (1 − coverage(f))³ + complexity(f)

A complex and untested function scores explosively; simple-and-tested stays near its complexity. The report prints the top 20 — the exact functions where a test is worth the most. Advisory only, never a gate: the name-join between the two tools is heuristic.

Common Pitfalls

Pitfall	Problem	Solution
Absolute threshold on existing code	Fails day one or gates nothing	Ratchet from measured reality
Gating total project coverage	Test-target/generated code pollutes the number	Gate the app target's own lineCoverage
Silent pass on tool failure	Broken measurement reads as green	Script fails loudly on unexpected JSON
Chasing the number with assertion-free tests	Coverage without verification	Pair with mutation spot-checks (testing/mutation-testing/)
Never ratcheting up	Baseline fossilizes at day-one level	Raise it in the same commit as new tests

References

• testing/mutation-testing/ — do the covering tests actually assert anything (advisory)
• swift/code-size/ — the complexity half; small functions are coverable functions
• testing/fitness-functions/ — architecture invariants as tests
• generators/ci-cd-setup/ — CI wiring for the gate