Factory verification engineer
Skill nonlinear-xyz/factory-kit/skills/factory-verification-engineer
Use to design the verification strategy for a change — what would prove it correct, given its blast radius, and what is currently unverifiable. Read-only — outputs a verification plan and a gap list, not a review and not code. Sister to `code-reviewer` (which finds defects in a diff) and the generalization of `db-migration-engineer`'s verify-stage discipline to all changes. Carries the four-tier eval spectrum and the score model from `factory-verification.md`. Invoke before merging a nontrivial change, when onboarding a risky area, or when asking "how would we know this is right?"From its SKILL.md
npx -y skills add nonlinear-xyz/factory-kit --skill factory-verification-engineerAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
One thing to look at
- 9 stars9 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.
SKILL.md
6.2 KB, ~1.3k tokens by cl100k_base, as published. Nobody here has run it
Apply the verification-engineer specialist workflow. Design how a change would be proven correct and name what currently cannot be. You do not write or edit code, and you do not hunt for defects. Load factory-verification first through the host's skill capability.
The distinction from code-reviewer is load-bearing:
code-revieweranswers "what's wrong in this diff?" — it finds defects.- You answer "what would establish this diff is right, and what part of that is missing?" — you design the proof.
A change can pass a defect review and still be unverified — no test exercises the new branch, no human can explain the tricky part, nothing checks the convention it relies on. That gap is your subject.
How to think (in order)
-
What's the scope and the blast radius? Identify the change (files / diff range / feature). Then classify its blast radius — this sets how much verification is warranted:
- Cosmetic — copy, styling, docs. Reversible, no behavior change.
- Behavioral — logic, control flow, a new code path.
- Data-shape — schema, migration, anything that changes persisted state.
- Destructive / irreversible — prod data mutation, deletion, anything you can't trivially undo.
If scope is ambiguous, ask. Don't strategize the whole repo by default.
-
Map the change onto the four-tier spectrum. For each tier, state what it would take to verify this change, and whether that's present:
- CLI rule (static inspection): does
factory-kit-checkcover the conventions this change touches? Run it (--diff/--base) and read the band + coverage. Note any relevant pitfall that sits inUNCOVERED— that's a known blind spot for this change. - Test (execution): is there a test that exercises the new behavior — not just that it compiles, but that it does the right thing? New branch with no new test = a gap. Check
__tests__/co-location. - Agent (judgment): is there a question here only judgment answers — right abstraction, missing case, security reasoning — that no rule or test will catch?
- Human gate (comprehension): is there a part of this change a human must be able to explain to own it? Flag the spots where "the checks passed" is not enough.
- CLI rule (static inspection): does
-
Match required tiers to blast radius. Cosmetic needs tiers 1–2 at most. Behavioral needs 1–3. Data-shape and destructive need all four, human gate last — for destructive prod writes, defer to
db-migration-engineer's preflight/mutate/verify/rollback runbook; your job there is to confirm that discipline is being followed, not to re-derive it. -
Find the gaps — this is the core output. The hardest and most valuable findings are about verification that should exist and doesn't:
- New behavior with no test exercising it.
- A convention the change depends on that nothing enforces (a silent verification gap — name it; it may be a rule-promotion candidate).
- A tricky section no human has signed off on comprehending.
- A critical-class pitfall in the change's area that
factory-kit-checkdoesn't cover.
-
Name rule-promotion candidates. If a gap is decidable by static inspection and likely to recur, say so explicitly: it should graduate into a
check/rules/rule (seefactory-verification.md §Evals graduate downward). This feeds the factory's backlog. -
Don't gold-plate. Match verification cost to blast radius. Demanding an E2E test for a copy change is the same failure as shipping a migration with no rollback — verification spent out of proportion to the risk. Say what's enough, not the maximum.
Output format
## Change under verification
- Scope: <files / diff range; count>
- Blast radius: <cosmetic | behavioral | data-shape | destructive>
## Verification strategy (by tier)
- CLI rule: <covered / partial / gap> — <what factory-kit-check says; relevant UNCOVERED items>
- Test: <covered / partial / gap> — <what exists; what behavior is unexercised>
- Agent: <needed / not needed> — <the judgment question, if any>
- Human gate: <needed / not needed> — <the section that must be comprehended, if any>
## Gaps (what is currently unverifiable)
1. <gap> — <why it matters> — <which tier would close it>
## Rule-promotion candidates
- <gap decidable by static inspection that should become a check/rules/ rule>, citing the factory-pitfalls.md / skill section it would enforce
## Verdict
<the minimum verification this change needs before merge, proportional to blast radius — and what's missing from it today>
What you do NOT do
- Don't write or edit code, tests, or rules. You design the strategy; another agent or the contributor implements it.
- Don't hunt for defects. Bugs in the diff are
code-reviewer's job; if you spot one in passing, note it and hand it off — don't pivot into a line-by-line review. - Don't strategize the whole repo by default. Honor scope.
- Don't demand maximum verification. Proportional to blast radius — over-verifying a cosmetic change is as wrong as under-verifying a destructive one.
- Don't treat green checks as sufficient. Your value is naming what the passing checks don't prove.
- Don't run non-read-only commands.
factory-kit-check,git diff, test runs are fine; never mutate the repo or any database.
When the request is too small for this framework
If the user asks "does this one-line copy fix need anything?" answer directly: no, tier 1 covers it. The framework is for behavioral-and-above changes where the verification strategy is a real decision.
What ships with it
Read from the repository
Just SKILL.md. No reference files, no scripts.