Post change verification

Post-change verification

This exists because of a repeated pattern: a change gets made, the changed files themselves are confirmed correct, and it's reported as done — but something else in the system that reads the same data was still broken, silently, and got discovered later by accident. Fixing the direct target and verifying the whole system actually reflects that fix are two different steps. This skill is the second one.

The concrete incident that motivated it: I restructured a memory vault's folder layout. A local dashboard that reads that same vault to render a live graph kept showing the old, incomplete data — not because my fix was wrong, but because the dashboard was a long-running process that had loaded the old code before I fixed it, and a source-level fix doesn't reach a process that's already running. Nothing errored. It just quietly kept being wrong until someone looked at the actual numbers and noticed they hadn't moved.

Process

• Identify what changed — a file, a convention, a piece of shared state (not just "I edited X," but "X affects Y and Z too").
• Check a running "dependents map" (a living doc: "if you change X, these things also need checking") for known dependents of what you just changed. If nothing's listed for this exact case, ask the general question: what else reads or depends on this same file/folder/config?
• For each dependent found, check the LIVE state, not just the source. A source-code fix is not the same as a running system reflecting it:
  • Is there a long-running process (a local server, a background script) that already loaded the old version into memory? Source being correct doesn't mean a running instance picked it up — it may need an actual restart.
  • Is there a cache, an already-open file handle, or an external process that could silently re-apply the old state after you fix it?
• Actually execute or observe each dependent — don't just read its code and reason about whether it should still work. Run the function directly, take a screenshot, hit the actual endpoint, open the actual file. Reading code and concluding "this looks like it would work" is not verification.
• If a dependent is a long-running process, verify by restarting it and re-observing — a stale process serving old behavior looks identical to a correctly-updated one until you actually force it to reload.
• If the thing you fixed has ever silently reverted before, don't treat one clean check as permanent — note that it should be spot-checked again later, and say so when reporting.
• Update the dependents map with anything newly discovered, so the next session doesn't have to rediscover it by reading source code from scratch.
• Report per-component, not just "done" — say what was checked, what was actually run/observed (not just read), and what's still uncertain.

Quick Reference

Signals that you're at risk of declaring something done prematurely:

• You edited a file and then only re-read that same file to confirm — you didn't check anything that reads it.
• The thing you fixed backs a process that might already be running from before your fix.
• You're relying on "the code looks right" instead of an actual screenshot, function call, or request/response.
• The bug you just fixed is one that's already recurred once — a single clean check right now doesn't rule out recurrence.

Ground rules

• ALWAYS ask "what else depends on this" before declaring a structural/shared-state change done — never assume the files you directly touched are the whole blast radius.
• ALWAYS check the dependents map first, and add to it when you find something new.
• ALWAYS verify long-running processes by observing their actual live output after a restart, not by reading their source and assuming a restart would fix it.
• NEVER treat "I read the code and it looks correct" as equivalent to "I ran it and confirmed."
• NEVER assume a previously-fixed, previously-recurring issue is now permanently fixed just because it currently checks out clean.