Verifiable criteria
Skill fbsmna-coder/karpathy-pro-max/skills/verifiable-criteria
Stop Claude Code from hallucinating — Karpathy-grade discipline in 8 skills
npx -y skills add fbsmna-coder/karpathy-pro-max --skill verifiable-criteriaAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
One thing to look at
- 0 stars0 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.
What its author says it does
Copied from the file, not written here
Define how success will be verified before writing code. Use on non-trivial tasks to replace vague "should work now" with concrete artifacts — a failing test, a curl command, a log line, a UI screenshot — that prove the change works.
The file declares its own license as MIT. That is the author’s claim about this one file, and it is not the same thing as the license GitHub reports for the repository, which is listed with the other numbers below.
SKILL.md
2.9 KB, as published. Nobody here has run it
Verifiable Success Criteria
"Make it work" is not a success criterion. It is a wish. Without a concrete check, there is no honest way to know when you are done, and bugs will slip through claimed-as-fixed.
The rule
Before writing implementation code, state the exact thing that, when true, proves the task is complete.
Prefer concrete artifacts in this order:
- A test that fails now and will pass after the change.
- A specific shell command (curl, query, script) and its expected output.
- A specific log line that should appear when the code runs.
- A specific UI interaction and what the user should see.
"Tests should pass" is not a criterion. Which tests, demonstrating what?
Mapping common requests
| Vague request | Verifiable form |
|---|---|
| "Add validation" | "Write a test that calls f(invalid_input) and asserts it raises ValidationError. Make it pass." |
| "Fix the bug" | "Write a test that reproduces the bug (currently fails). Fix the code. Same test now passes." |
| "Refactor X" | "All existing tests pass before AND after. No behavior change." |
| "Speed it up" | "Benchmark X currently takes Yms. Target: ≤Ym/2s. Show before/after numbers." |
| "Make the UI nicer" | "Screenshot before. Specific change description. Screenshot after." |
Multi-step tasks
State a brief plan with a verification per step:
1. Add migration for `users.locale` column → verify: `\d users` shows column
2. Update User model with locale field → verify: tests/test_user.py::test_locale_default passes
3. Wire up locale to API response → verify: curl /api/users/1 returns "locale":"en"
Strong criteria let the loop self-terminate. Weak criteria force constant clarification.
When NOT to apply
- Trivial edits: typo, single-line config change, adding a debug print.
- Pure exploration: "What does this code do?" / "Find all places that use X."
- Research tasks where the deliverable is a written analysis, not code.
- Creative work where success is subjective (copywriting, naming).
Match verification to project reality
If the project has no test suite, do not invent test infrastructure to satisfy this rule. Use runtime checks: "Run the bot, send /start, expect welcome message in <2s." A real check that exists is worth more than an idealized check that requires rebuilding the project.
Example
Bad:
Agent: "I added input validation. It should work now."
Good:
Agent: "Added validation. Verified:
pytest tests/test_signup.py::test_invalid_emailwas failing before, passes now. Manual check:curl -X POST /signup -d email=notanemailreturns 400 with{"error": "invalid_email"}instead of 500."