Duplication sweep

Duplication Sweep

The periodic pass. The codebase has already accumulated duplication, and the job is to find the parts worth consolidating and rank them so a human can choose.

Produce a report. Do not open a pull request. Consolidation touches shared code, which means blast radius. The finding is cheap and safe; the change is neither. A human decides.

Scope one run

Do not sweep the whole repo. A report on everything gets read by nobody. Pick one of these per run and state which you picked:

• One directory or package
• One category: date handling, HTTP clients, validation, error construction, config access, logging, retries
• Everything touched in the last N weeks, which is the code most likely to have been recently duplicated

Rotate the scope between runs so coverage accumulates instead of repeating.

What to look for

Near-duplicate functions. Same behavior, different name or location. The main prize. Search by behavior vocabulary and by shared magic constants, not by name.

Copy-paste blocks. Runs of similar lines differing only in a literal or a field name. Concentrate on large files and on siblings within one directory.

Redundant utility modules. Two files that are both "the string helpers." A utils/ and a helpers/ with overlapping contents. Three different date formatters.

Reimplemented builtins. Hand-rolled versions of what the standard library or an installed dependency already provides.

Dead code. Exported functions with zero callers, unreachable branches, permanently-enabled feature flags, commented-out blocks. Before calling anything dead, check for dynamic references: reflection, string-based dispatch, dependency injection, config-driven registration, and test-only usage.

Rank by payoff, not by count

Finding a hundred duplicates is not useful. Ranking six of them is. For each candidate, estimate:

• Lines removed if consolidated
• Call sites affected, which is the risk side of the trade
• Blast radius: private to one module, or exported across packages
• Confidence that these are genuinely the same thing

The best items are high lines removed, few call sites, high confidence. Those are the safe wins and they go at the top.

Keep "safe, do it now" separate from "real duplication, but risky to merge and needs a decision." Collapsing those two into one list is what makes cleanup reports unusable.

Report format

## Sweep: <scope> — <date>

**Verdict:** <one line>

### Safe wins
1. <what> — <files> — removes ~N lines, M call sites, confidence: high
   Recommendation: <specific action>

### Needs a decision
1. <what> — <why it is not obviously safe> — <the actual question for the human>

### Dead code candidates
1. <what> — <how you checked for dynamic usage>

### Checked, found clean
<so the next run does not redo this>

Track it over time

Append each run's headline numbers to a log in the repo, for example docs/sweeps/: scope, candidates found, lines identified, what actually got actioned.

One sweep is a chore. A trend line answers the question you actually care about, which is whether the codebase is getting better or worse. It also makes the cost of skipping cleanup visible to people who do not read diffs.

Honest limits

State plainly what you could not check. Static search misses dynamic dispatch. Cross-language duplication is invisible to most searches. If the scope turned out too large to cover properly, say what you skipped rather than implying full coverage. A sweep that overstates its coverage is worse than no sweep, because it retires the question.