Bayesian reasoning

Bayesian Reasoning

Concept of the skill

Bayesian reasoning treats belief as a state that changes when evidence arrives. The primitives are a hypothesis, prior probability or base rate, evidence, likelihood of seeing that evidence if the hypothesis were true, likelihood of seeing it if the hypothesis were false, posterior belief, residual uncertainty, and update history.

Concept Card

What it is: Bayesian reasoning is a method for updating belief under uncertainty. It starts from a prior or base rate, evaluates how expected the new evidence is under competing hypotheses, updates toward the hypothesis that better predicts the evidence, and preserves residual uncertainty.

Mental model: Confidence is not reset by each new clue. A belief has an existing level, evidence applies pressure to that level, and the posterior becomes the new prior for the next update.

Why it exists: Agents tend to overreact to vivid recent evidence, ignore base rates, and answer uncertain questions as yes/no. Bayesian reasoning forces the belief state, evidence strength, and update size into the open.

What it is not: It is not an expected-value decision table, a statistical modeling workflow, a generic prioritization method, a strategy framework, or a requirement to fabricate exact probabilities when inputs are weak.

Adjacent concepts: base rates, priors, likelihood ratios, posterior probability, diagnostic reasoning, forecasting, calibration, expected value, hypothesis testing, evidence independence.

One-line analogy: Bayesian reasoning is a confidence ledger: every new piece of evidence is posted against the prior balance before the new balance is reported.

Common misconception: The method is not "new evidence says X, therefore X." Evidence matters by how differently it is predicted by X versus not-X, and by how plausible X was before the evidence arrived.

Coverage

This skill teaches agents to:

1. State the hypothesis and plausible alternatives before updating.
2. Make priors and base rates explicit.
3. Separate likelihood from posterior probability.
4. Estimate evidence strength by comparing competing explanations.
5. Update confidence directionally when exact numbers are unjustified.
6. Avoid double-counting correlated evidence.
7. Use natural frequencies for rare-event and diagnostic examples.
8. Report residual uncertainty and the evidence that would change the belief.

Philosophy of the skill

Bayesian reasoning is useful because it makes uncertainty inspectable. A confident answer can hide a weak prior, a diagnostic clue can look decisive while being common under multiple explanations, and a vivid example can overwhelm a large base rate. The Bayesian discipline forces those hidden weights into the answer.

The method is not a demand for spreadsheet precision. In many product, strategy, debugging, and research tasks, the honest output is qualitative: "this evidence raises confidence from low to moderate, but not high, because the base rate is low and the evidence is not independent." That is stronger than an invented 73 percent.

Workflow

1. Define the belief being updated

Name one hypothesis at a time, plus the alternatives.

Hypothesis:
Alternatives:
Decision or question this belief affects:
Current confidence:

Do not update a vague claim such as "this is promising." Rewrite it as a belief that can be supported or weakened.

2. Establish the prior

Use the best available prior source:

Prior source	Use when	Example
Base rate	Similar cases exist	"Only a small share of signups convert without activation."
Historical frequency	The system has logs or repeated runs	"This error has usually been config-related."
Reference class	No direct data exists	"Comparable B2B onboarding changes have mixed results."
Stated assumption	No evidence exists	"Assume low prior and mark it as a placeholder."

If the prior is uncertain, say so. Do not hide the prior by starting from the new evidence.

3. Compare likelihoods

Ask how expected the evidence is under each hypothesis.

Evidence:
If hypothesis is true, how expected is this evidence?
If hypothesis is false or an alternative is true, how expected is this evidence?
Likelihood direction:
Evidence independence:

Evidence is strong only when it is much more expected under one hypothesis than under plausible alternatives.

4. Update the belief

Move confidence in proportion to prior strength and evidence strength.

Situation	Update discipline
Strong prior, weak evidence	Small update
Weak prior, strong diagnostic evidence	Moderate or large update, but still state uncertainty
Rare hypothesis, noisy evidence	Small update unless the evidence is highly diagnostic
Multiple independent signals	Update more than once, but only if independence is credible
Correlated signals	Treat as one evidence cluster, not many independent confirmations

When inputs are rough, use bands: very low, low, moderate, high, very high. Prefer bands to fake decimals.

5. Report the posterior and next evidence

The useful answer includes the belief state and what would change it.

Bayesian update
- Prior/base rate:
- New evidence:
- Likelihood comparison:
- Update:
- Posterior confidence:
- Residual uncertainty:
- Evidence that would change the posterior next:

Natural-Frequency Check

For rare events, translate percentages into counts. This catches base-rate neglect.

Out of 10,000 cases:
- Prior/base-rate cases where the hypothesis is true:
- Cases where the evidence appears if true:
- Cases where the evidence appears if false:
- Total cases with evidence:
- Share of evidence-positive cases where the hypothesis is true:

If a rare event has a 1 percent base rate and a test is 90 percent accurate with a 10 percent false-positive rate, most positive tests may still be false positives. The exact result depends on the numbers, but the lesson is stable: low base rates require very diagnostic evidence.

Anti-Patterns

Anti-pattern	Why it fails	Repair
Base-rate neglect	Treats a vivid clue as if the prior were neutral	State the reference class and prior before the update
Likelihood-posterior swap	"Evidence is likely if H is true" becomes "H is likely"	Compare evidence under H and not-H, then update from the prior
Anecdote overweighting	One case gets treated as representative	Ask whether the evidence is diagnostic or merely salient
Double-counting correlated evidence	Many signals from one source masquerade as independent confirmation	Cluster correlated signals and update once
Hidden prior	The answer smuggles in confidence without naming it	Write the prior or mark it as an assumption
Fake precision	Weak inputs produce precise probabilities	Use confidence bands and evidence gaps
Binary conclusion	An uncertain belief becomes yes/no	Report posterior confidence and next evidence

Boundaries

Use Bayesian reasoning when the task is to update a belief, diagnosis, forecast, or assumption after evidence arrives.

Use another tool when the task is narrower or downstream:

Need	Better owner
Choose the option with highest probability-weighted payoff	Expected value skill when available; otherwise state that EV is downstream
Rank backlog items by impact, urgency, effort, or confidence	prioritization
Turn a vague business strategy into integrated choices	playing-to-win
Diagnose industry structure and profit-pool pressure	porters-five-forces
Choose which reasoning method to apply first	problem-approach-router
Ground factual claims to citations and modality	epistemic-grounding

Verification

Before finishing, verify:

• The hypothesis and alternatives are explicit.
• The prior or base rate is stated, or the absence of one is labeled.
• Likelihood is not confused with posterior probability.
• Evidence strength is judged against competing explanations.
• Correlated evidence is not double-counted.
• Probability precision matches evidence quality.
• The posterior is reported as an update from the prior.
• Residual uncertainty and next evidence are named.
• The answer does not present expected value, strategy cascade, industry analysis, or generic prioritization as Bayesian reasoning.

References

• skills/reasoning-strategy/bayesian-reasoning/references/bayesian-reasoning-sources.md
• skills/reasoning-strategy/bayesian-reasoning/references/upstream-displacement-2026-05-26.md

Do NOT Use When

Use another skill when the task falls outside the declared scope, matches an anti_examples prompt, or is owned by a more specific related skill.