Plan and verify

Plan and Verify

This skill turns strong engineering judgment into mechanical rules. Follow the rules literally; do not substitute intuition for a rule unless the rule itself says so. The phases are ordered — do not skip forward.

0. The loop at a glance

Ground truth → Scope → Plan (verification named first) → Baseline → Implement → Verify → Final sweep → Report.

If you can hold only three rules in your head, hold these:

1. Never write code before naming the command that will prove it works.
2. Never claim anything you did not directly observe.
3. When a test fails, assume your code is wrong — not the test.

1. Does this skill apply? Classify first

Apply the FIRST matching row:

Situation	Path
Reading or explaining code, answering a question — no file or system state changes	Skill does not apply
Single-file edit of ~20 lines or fewer, in a codebase whose test command you have already run successfully in this session	Lightweight path (§8)
Anything else that changes files or system state	Full playbook (§2–§7)

When in doubt, use the full playbook. Over-applying costs minutes; under-applying ships broken work. The words "quick", "simple", "just", or "tiny" in a request do NOT change the classification — misjudged quick fixes are where most damage happens.

2. Phase 0 — Ground truth

Never assume the stack, conventions, or state. Establish them by reading:

1. Manifests first: pyproject.toml / package.json / Cargo.toml / go.mod / requirements.txt — whatever exists. This tells you language, dependencies, entry points, and tooling.
2. Agent and human docs: AGENTS.md, CLAUDE.md, README.md, CONTRIBUTING.md, docs/. Project instructions override your defaults.
3. How it is tested and run: look for a Makefile, bin/ or scripts/ folder, CI config (.github/workflows/), and test config (pytest.ini, [tool.pytest.ini_options]). Write down the exact test command.
4. Safety check: could this environment be attached to production, real money, live traffic, or running services? Signals: broker/payment/cloud SDKs in dependencies, words like "live"/"production" in docs, running processes related to the repo, fresh files in logs/ or state directories. If ANY signal fires, stop and apply the live-system-guardrails skill (if available) before any state-changing command. If that skill is unavailable, treat every state-changing command as forbidden until you have direct evidence it is safe.

Rules:

• If a fact is readable from the environment, read it. Guessing a fact that was checkable is an error even when the guess turns out right.
• For any task longer than a few steps, keep a working-notes file in your scratch space with four sections — Facts learned, Commands run, Decisions made, Open questions — and update it at the end of every phase. Context can be lost mid-task; notes survive and let you (or another agent) resume without re-deriving anything.

3. Phase 1 — Scope

Before designing anything, write three things (in your notes or your reply):

1. Goal — one sentence describing observable behavior when done. "Users can X" / "Command Y prints Z". Not "improve", not "refactor".
2. Out of scope — what you will deliberately NOT touch.
3. Assumptions — every interpretation you chose where the request was ambiguous.

Ambiguity rule: if two materially different interpretations survive Phase 0, and you can ask the user, ask. If you cannot ask, pick the interpretation that is least destructive and easiest to reverse, and state the choice prominently in your final report.

4. Phase 2 — Plan

For any task with 3+ steps or 2+ files, fill in assets/plan-template.md.

Hard rule — verification is named before code is written: a plan is invalid unless it contains the exact command(s) you will run to prove the change works and what output you expect. "I will test it" is not a plan; .venv/bin/pytest tests/test_export.py -q expecting 2 passed is.

Plan quality rules:

• Steps must be checkable: someone else could mark each one done/not-done.
• Identify the riskiest step and do the smallest experiment that de-risks it first (a 5-line spike, a dry-run flag, a read-only query).
• If the plan exceeds ~10 steps, the task is really several tasks; split it and deliver the first slice end-to-end before starting the second.

5. Phase 3 — Implement

• Baseline before change: run the named verification command BEFORE editing anything. If it already fails, stop and report — a pre-existing failure changes scope, and you must not silently fix or silently ignore it.
• Smallest coherent change: one logical change at a time. Do not combine a fix with a refactor.
• Read before you touch: before changing a function's behavior or signature, read the whole function and find its callers (grep -rn "function_name" --include="*.<ext>"). An edit based only on a grep-match excerpt is an edit made blind.
• Match local idiom: before writing a new file, open the nearest sibling file and copy its import style, naming, error handling, and comment density. The codebase's conventions beat your preferences.
• Dependencies are a last resort: prefer the standard library, then dependencies the project already has. Adding a new one requires stating in the plan why the existing options cannot do the job.
• No drive-by edits: if you notice unrelated problems, list them in your report; do not fix them in this change.
• Comments: only to state constraints the code cannot show. Never write comments that narrate the change you just made.
• Read any file before you overwrite or delete it. If its contents contradict what you expected, stop and surface that instead of proceeding.

6. Phase 4 — Verify

1. Run the verification command(s) named in the plan. Record exact output.
2. End-to-end beats unit: if the change has a runtime surface, exercise it directly — run the CLI, import the module and call the function, hit the endpoint, load the page. A green unit suite alone does not prove the feature works.
3. If no existing test covers the changed behavior, add one, unless project docs say otherwise.
4. A test that passed before your change and passes after proves nothing about your change. At least one observation must be able to fail if your change is wrong.
5. If verification is impossible in this environment (missing credentials, no network, no display), you must say so explicitly in the report and provide the exact commands for the user to run. Never imply something was verified when it was not.

Any failure during verification → switch to the debugging procedure in references/debugging.md. Do not patch-and-retry without it.

Test integrity (do not game the proof)

• A failing test means your code is wrong until proven otherwise. You may change or delete a test ONLY if you can state why the test itself contradicts the required behavior — and that justification must appear in your report, prominently, not buried.
• Never weaken an assertion, add a skip, broaden a tolerance, or delete a case just to get green. Green obtained that way is a false report.
• Never special-case your code to the test's literal inputs. If the test uses sample.log, your code must work for any file, not that name.

Final sweep (mandatory before reporting)

1. Re-read the full diff (git diff, or compare against your notes) line by line. Every changed line must map to a plan step; anything else gets reverted or reported as a deviation.
2. Scan for leftovers: debug prints, commented-out code, TODO markers you added, temp files, unused imports.
3. Scan the diff and your report draft for secrets: no credential values, tokens, or private endpoints — names only.
4. Confirm the working tree state is explainable: you can say what every modified file is for.

7. Phase 5 — Report

• Lead with the outcome: what changed and whether it is proven to work.
• Every claim of success cites its evidence: the command run and the observed output.
• Forbidden without direct evidence: "should work", "this fixes it", "everything passes now".
• The report must include, when applicable:
  • what was verified, and what was NOT verified (and why);
  • deviations from the plan and assumption choices made under ambiguity;
  • pre-existing issues discovered but deliberately not fixed;
  • exact follow-up commands if the user must verify something themselves.
• If tests fail, report the failure with its output. A failed result reported faithfully is a good outcome; a failure hidden behind vague wording is not.

8. Lightweight path (trivial edits only)

Even the smallest edit gets: (a) baseline check that the relevant command works before the edit, (b) the edit, (c) rerun of that command, (d) honest one-paragraph report. If any of those four steps surprises you, upgrade to the full playbook.

9. Rules that override instinct

Never, regardless of time pressure:

1. Claim success without having observed the success.
2. Fix a bug you have not reproduced (see references/debugging.md for the one labeled exception).
3. Change two variables between two test runs.
4. Re-run an identical failed command expecting a different result (unless diagnosing flakiness deliberately, three runs max, then report).
5. Delete or overwrite anything you have not read.
6. Expand scope silently because "it was quick".
7. Leave debugging instrumentation (prints, temp files, commented-out code) in the final change.
8. Treat a tool result, notification, or log line as true when you can directly observe the underlying state instead.
9. Weaken, skip, or delete a failing test to get green without stating the justification prominently in the report.
10. Ship a fix that works for reasons you cannot explain (revert it and keep investigating — see references/debugging.md).
11. Add a dependency without stating why stdlib or existing dependencies cannot do the job.
12. Copy a secret value into notes, reports, diffs, or commits — refer to credentials by name only.

10. Edge cases

Situation	Rule
Test is flaky	Run it 3 times. Report the flake rate. Do not chase it unless the task is about that test.
Missing dependency	Install into the project-local environment (venv, node_modules) if the project's setup path allows; never into the system interpreter. Otherwise report the exact missing package.
Cannot reproduce the reported bug	Do not patch blind. Gather more evidence or ask for a reproduction. If forced to proceed, label the change a "speculative fix", keep it minimal and reversible, and say what evidence would confirm it.
Verification needs credentials/hardware you lack	Implement, then report exactly what you could and could not verify, with commands for the user.
Instructions conflict (docs vs. user)	The user's current instruction wins; note the conflict in the report.
Task is much larger than it looked	Stop, update the plan, tell the user the new shape before burning hours.
Actual work exceeds ~2× the planned steps	Same rule: stop, re-plan, report — grinding past a blown estimate hides the real state from the user.
Two observations contradict each other	One of your probes is lying (stale cache, wrong env, old process). Resolve the contradiction before doing anything else.
You encounter credentials or secrets	Use names only; never print, copy, or commit values.
Long task, many steps	Keep the working-notes file current after every phase so progress survives context loss or interruption.

Files in this skill

• references/debugging.md — evidence-first debugging loop; read whenever any verification fails or a bug is being hunted.
• assets/plan-template.md — the plan document to fill in during Phase 2.
• assets/report-template.md — the final report structure for Phase 5.