Validation experiments
Skill eltntawy/idea-discovery-playbook/skills/validation-experiments
Designs smoke tests, fake doors, concierge and Wizard of Oz MVPs, pre-sales, and design-partner experiments with ILI metrics, success and fail thresholds. Use when the user asks to test demand, run a smoke test, pretotype, validate an idea, or design an experiment.From its SKILL.md
npx -y skills add eltntawy/idea-discovery-playbook --skill validation-experimentsAssembled from the repository path, not quoted from the project. Check it against their README if it does not work.
One thing to look at
- 0 stars0 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.
SKILL.md
4.3 KB, 717 tokens by cl100k_base, as published. Nobody here has run it
Validation Experiments
Link each experiment to one row in the assumption-map ledger. Map results to evidence levels in idea-scorecard (Intent = 3; payment = 5).
Loop
- Assume — state the riskiest assumption
- Design — choose the smallest test that can falsify it
- Test — run the experiment
- Measure — collect quantitative data (ILI, CTR, conversion, payment)
- Learn — extract insights
- Decide — proceed, pivot, or kill
Technique picker
| Technique | Best for | Not for |
|---|---|---|
| Fake door | Demand for a specific offer/feature; priced CTA | Proving retention or full workflow |
| Smoke test / landing | Message–market fit, segment + offer at scale | Deep workflow discovery |
| Concierge MVP | Learning what to build; pains and language (5–10 customers) | Scalable unit economics |
| Wizard of Oz | Testing UX and workflow without backend | First read on abstract demand |
| Pre-sale / design partner | Willingness to pay and delivery commitment | Anonymous traffic only |
Recommended sequence: fake door (priced) → concierge (5–10) → Wizard of Oz at small scale → build only if assumptions falsify the kill path.
ILI (Initial Level of Interest)
From pretotyping: ILI = actions_taken / opportunities_offered
- Opportunities: visitors who saw the offer (or segment reached)
- Actions: paid pre-orders, deposits, qualified signups with stated intent, or explicit purchase clicks per your threshold
Record ILI with segment, offer, and date. Label as proxy — not audited revenue.
Experiment plan template
| Field | Value |
|---|---|
| Riskiest assumption | |
| Linked assumption ID | (from assumption-map) |
| Method | fake door / smoke test / concierge MVP / Wizard of Oz / pre-sale / design partner |
| Audience | |
| Duration | |
| Price shown? | yes / no — must be yes for WTP tests |
| Success threshold | |
| Fail threshold | |
| Evidence quality target | (scorecard level 3–5) |
| ILI target | optional numeric |
Ethical fake door
After the CTA, be honest: the product is not built yet. Offer waitlist, founder call, or refund policy — do not imply shipping software that does not exist.
Benchmarks (calibrate per segment)
- Landing visitor → signup: 5–15% can be healthy for warm/niche traffic; <1% on cold paid traffic is often a kill signal for that offer.
- Pre-sales and design-partner time commitments beat email-only signups.
- Do not count views, likes, or unqualified clicks as validation.
Output
Save plans and results to discovery/<topic-slug>/experiments/YYYY-MM-DD-<topic-slug>-experiment.md.
Default sections: Facts, Assumptions, Evidence, Analysis, Decision, Next experiment.
What ships with it
Read from the repository
Just SKILL.md. No reference files, no scripts.