Ml or not

/ml-or-not

Overview

Most teams reach for ML before checking whether the problem actually needs it. This skill runs a structured pre-check across the seven gates that usually predict ML success or failure, and returns a verdict the user can defend in a sprint review.

Core principle: ML earns its place when (a) the rules are unknown or unstable, (b) labeled data is available, (c) errors are tolerable and recoverable, and (d) the unit economics survive ongoing model maintenance. If any of those fails, ship something simpler.

When to Use

• User typed /ml-or-not <problem description>.
• User is scoping a feature and asked "should this be ML", "do we need a model", "would AI help here".
• A PRD or spec mentions ML as the proposed solution but does not justify it.
• A team is choosing between a rules engine, a heuristic, a search index, and a model.

Do NOT use for: problems where the user has already shipped ML and is asking how to improve it. Use /refine instead.

The Process

Step 1. Restate the problem in one sentence

What is the input, what is the desired output, who acts on the output, and what happens if the output is wrong? If the user has not said all four, ask once.

Step 2. Run the seven gates

For each gate, answer with one of: pass (ML clears this gate), fail (this gate kills ML for this problem), or unknown (the user does not yet know). Be honest. Most "ML projects" fail one or more of these but no one ran the check.

Gate	The question	Fail signal
1. Rules	Can the problem be solved with rules a domain expert could write down?	If yes, ship rules. ML is overkill.
2. Stability	Are the rules expected to change frequently?	If rules are stable, hardcode. If they change weekly, ML or a configurable engine.
3. Data	Is there labeled data available, or a cheap way to label it? How much?	No labels and no path to labels = no ML.
4. Volume	Is the prediction volume high enough that automation pays for the model lifecycle?	Low volume = humans + spreadsheet beats ML.
5. Tolerance	Is the cost of a wrong answer recoverable? Is there a human in the loop or a fallback?	High-stakes irreversible decisions usually do not survive ML failure modes.
6. Drift	Will the input distribution change over time, and is there a monitoring plan?	Drift without monitoring becomes silent failure.
7. Economics	Does the unit economics survive ongoing labeling, retraining, monitoring, and on-call cost?	If the lifetime cost exceeds the lifetime value, do not start.

Step 3. Render the verdict

Pick one and defend it in 2 to 4 sentences:

• Yes, use ML. All gates pass or are clearly addressable. State which approach (classification, regression, retrieval, generation) and why.
• No, do not use ML. One or more gates clearly fail. State the cheaper alternative (rules engine, heuristic, search, deterministic check, human review queue) and why it is enough.
• Not yet. ML could work, but a load-bearing gate is unknown (usually data or economics). State exactly what to learn before committing, and how to learn it cheaply.

Step 4. If verdict is "no" or "not yet", propose the alternative

Be concrete. "Use a rules engine" is not enough. Specify: which rules, how they are stored, how they are changed, who owns them. The point of this skill is to send the user away with a real plan, not a gentle no.

Step 5. Output

# /ml-or-not verdict: [problem]

## Restated problem
- Input: ...
- Desired output: ...
- Who acts on it: ...
- Cost of being wrong: ...

## Gate evaluation
| Gate | Status | Note |
|---|---|---|
| 1. Rules | pass / fail / unknown | ... |
| 2. Stability | ... | ... |
| 3. Data | ... | ... |
| 4. Volume | ... | ... |
| 5. Tolerance | ... | ... |
| 6. Drift | ... | ... |
| 7. Economics | ... | ... |

## Verdict
**[YES / NO / NOT YET]**

[2 to 4 sentences defending the call]

## Recommendation
[If yes: which ML approach + first milestone. If no: the cheaper alternative, concretely. If not yet: the specific thing to learn first and how.]

Common failure modes

Failure	Fix
Rationalized away a failing gate to keep ML on the table	Failing gates kill ML, not "the team will figure it out".
Gave a soft "yes" when "not yet" was the honest answer	Most "yes" verdicts should be "not yet" with one or two unknowns to resolve.
Said "no" without proposing the alternative	A no without an alternative is useless. The user still has to ship something.
Treated low-volume problems as ML candidates	Below ~thousands of predictions per month, ML almost never pays off vs. humans + tooling.
Ignored maintenance cost	The model is the cheap part. Labeling pipelines, drift monitoring, retraining, and on-call are the long tail.
Used vague labels like "high stakes" without quantifying	Tie tolerance to actual user or business impact, not adjectives.

When to chain to /design-ai-experiment

If the verdict is "yes" or "not yet, with a learnable unknown", suggest /design-ai-experiment to turn the chosen approach into a falsifiable v0 with a kill criterion. Do not just hand the team a "yes" and walk away.