Ml result review
Use when concluding an ML experiment - produce a sober conclusion that separates confirmed results from exploratory observations and states the highest verified rung.From its SKILL.md
npx -y skills add pengzhangzhi/superpowers-ml --skill ml-result-reviewAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
One thing to look at
- 6 stars6 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.
SKILL.md
3.9 KB, 824 tokens by cl100k_base, as published. Nobody here has run it
ML Result Review
Overview
You are concluding an ML experiment, not selling it. Passing tests showed the code path runs; they never showed the method works. Your job here is a SOBER conclusion that resists over-claiming, written so your human partner can decide on real evidence.
Core principle: Confirmed, exploratory, and failed are different things. Keep them apart.
This is downstream of superpowers-ml:ml-feedback-ladder (which owns the rungs) and superpowers-ml:verification-before-completion (evidence before claims). Cite the highest rung you actually reached, never the one you hoped for.
What "Confirmed" Means
A result is CONFIRMED only when the required full or equivalent evaluation completed under the declared primary metric and met the locked success criterion.
- Met the locked criterion under the locked metric = confirmed.
- Better-looking but not the locked test = exploratory, not confirmed.
- Operational interruptions (preemption, requeue, node failure) are not scientific failures. A requeued run that still completed the locked evaluation is confirmed. Note the interruption; do not downgrade the result for it.
Never claim the method beats a baseline without the full or equivalent evaluation under the locked primary metric.
The Review Format
Write the conclusion under these explicit headings, in order:
CONFIRMED
What met the locked success criterion under the primary metric. One line per claim, each tied to the evaluation that proved it. If nothing is confirmed, write "None."
EXPLORATORY
Interesting observations that were NOT the locked test: secondary metrics, partial runs, eyeballed curves, single seeds. Label clearly so no one mistakes these for results.
FAILED / INCOMPLETE
Runs that did not finish the locked evaluation, and WHY. Distinguish scientific failure (method/config wrong) from operational interruption (infra). An operational interruption belongs here only if it actually prevented the evaluation from completing.
HIGHEST VERIFIED RUNG
State the top rung (R0-R7) actually reached and the artifact that proves it (log path, metric value, checkpoint, decision memo). Use the canonical wording from superpowers-ml:ml-feedback-ladder.
EVIDENCE GAPS
What remains unproven. What a skeptic would still doubt. Which rungs are not yet climbed.
RECOMMENDED NEXT
The single next experiment that closes the most important gap. Name its target rung. This is what you hand your human partner to decide on.
Force The Supported / Not-Supported Split
Every conclusion must separate what the evidence supports from what it does not. Model the language exactly:
Supported: implementation trains stably through R5 pilot.
Not supported yet: variant beats baseline at full scale.
Exploratory: early loss looked better, but this was not the
locked primary metric.
If you cannot phrase a claim as "Supported:" with its evidence, it is not supported. Demote it.
Reporting Line
Close with the standard rung statement, naming the highest GREEN rung and what is still unverified above it, e.g.:
Verified through R5 (short pilot). Not yet verified by full study.
Red Flags - STOP
- "It works" / "the method wins" without the locked full evaluation
- Quoting a secondary or early metric as if it were the locked one
- Treating a preemption or requeue as a scientific failure
- Claiming a rung you did not reach, or omitting the artifact for it
- An empty EVIDENCE GAPS section (there are always gaps)
- A conclusion with no RECOMMENDED NEXT
The Bottom Line
Separate confirmed from exploratory from failed. State the highest rung you actually climbed, with its artifact. Name what is still unproven.
A sober "not supported yet" is a real result. An over-claim is a liability.
What ships with it
Read from the repository
Just SKILL.md. No reference files, no scripts.
Gives 0 of the 12 instructions most review quality skills give in 824 tokens
Counted across 1,273 of the 2,403 authors here whose files we hold, read 2026-09-06
- Ask one question at a timein 63 of 1273, across 62 files
- Provide a recommended answer for each questionin 47 of 1273, across 45 files
- Rank findings by severityin 44 of 1273
- Use parameterized queries for database accessin 38 of 1273, across 20 files
- Validate all user input with schemasin 33 of 1273, across 15 files
- Store secrets in environment variablesin 32 of 1273, across 14 files
- Explore the codebase to answer questionsin 31 of 1273, across 29 files
- Store tokens in httpOnly cookiesin 30 of 1273, across 12 files
- Implement rate limiting on API endpointsin 30 of 1273, across 12 files
- Sanitize user-provided HTMLin 29 of 1273, across 11 files
- Return generic error messages to usersin 28 of 1273, across 10 files
- Cite file and line for every findingin 28 of 1273, across 25 files
Said here and by no other author read
- cite the highest verified rung reached
- use the required headings in order
- label exploratory observations clearly
- distinguish scientific failure from operational interruption
- phrase claims as supported or not supported
- include an evidence gaps section
Grouped from the skills themselves: near-identical wordings counted once, and counted by distinct author, so one author publishing three of these counts once. Length counted with cl100k_base; the agent that loads this file may tokenize it differently.