Allocating effort

Allocating effort

Deciding what to work on next is a bandit problem: several options, each with an unknown payoff rate, and every unit of effort spent on one is a unit not spent learning about the others. The two failure modes are symmetric and both are common - committing everything to the current leader before you know it is the leader, and spreading effort evenly long after you do.

Thompson sampling resolves both without tuning, and its allocation has a plain-English meaning: each option gets the share of effort equal to the probability it is the best one.

Workflow

- [ ] 1. Define the arms and what counts as a win
- [ ] 2. Count wins and losses honestly, including the zeros
- [ ] 3. Set priors where you genuinely know something
- [ ] 4. Run the allocation
- [ ] 5. Read the under-explored list before cutting anything
- [ ] 6. Commit the split for a fixed period, then recount

1. Define the arms and the win

An arm is anything you allocate to: a product, a channel, an ad creative, a landing page, a content format. The win must be:

• the same event for every arm - "signup" for one arm and "trial start" for another makes the comparison meaningless;
• as close to revenue as you can measure quickly - clicks are a fast proxy that frequently ranks arms in the opposite order to paid conversion;
• countable per unit of exposure - wins and losses, not a rate someone computed.

If arms have wildly different payoff sizes, this model is the wrong one: it compares rates, not values. Weight the wins by value first, or use the framing-decisions skill instead.

2. Count honestly, including the zeros

The arm with zero wins and fourteen tries goes in the table with wins: 0, losses: 14. Leaving it out because "it obviously doesn't work" is the mistake this whole skill exists to prevent: with a uniform prior that arm's posterior mean is 6%, not 0%, and 14 observations is nowhere near enough to distinguish 6% from 2%.

3. Set priors where you know something

Default is Beta(1,1), uniform, which is right when you truly have no idea. Where you do have a view, encode it as pseudocounts:

{ "name": "paid-search", "wins": 3, "losses": 40, "priorWins": 2, "priorLosses": 18 }

That prior says "roughly a 10% rate, believed about as strongly as 20 observations". Say the strength out loud - it is the part people fudge. And never use a prior of 0 pseudocounts on one side; it asserts an outcome is impossible.

4. Run the allocation

node scripts/calc.js allocate arms.json --budget 40 --floor 0.05

Input shape: examples/allocate.json. --budget is whatever you are splitting (hours per week, currency, traffic share). --floor reserves a minimum share for every arm so nothing is ever fully starved of evidence.

Alternative strategies, when you need them:

Flag	Use when
(default) --strategy thompson	almost always; no tuning, converges to the right split
--strategy ucb1 --c 1	you want a deterministic, explainable ranking for a document
--strategy quantile --alpha 0.9	you want to rank by an explicit optimistic bound
--strategy greedy	you have genuinely stopped learning and only want to harvest

5. Read the under-explored list before cutting anything

The output flags every arm with fewer than 30 observations. Do not kill those. An arm that has not been tried enough times to distinguish "bad" from "unlucky" has not been tested, and cutting it is a decision made on noise.

This is the R-MAX idea: treat an under-explored option as maximally promising until it has had its m attempts. It costs a little and it prevents the expensive error of abandoning something that would have worked.

When you do want the confidence behind a lead:

node scripts/calc.js compare arms.json

A lead below 90% probability is not a winner. The tool says so.

6. Commit, then recount

Fix the split for a defined period - a week, a sprint, a campaign flight - and do not re-derive it daily. Re-allocating continuously means every arm is permanently in transition and none of them accumulate the observations that make the model work.

Output template

## Arms and the win event
Win = <the single event, measured the same way for every arm>
Denominator = <what counts as an attempt>

| Arm | wins | losses | prior | posterior mean |
|---|---|---|---|---|

## Allocation
<paste the allocate output>

Read as: <arm> gets <share> because there is a <share> chance it is the best arm.

## Not yet judgeable
<arms under the observation threshold, with their counts, and the explicit
instruction not to cut them>

## Confidence in the lead
<paste calc compare for the top pair; state plainly if the lead is under 90%>

## Commitment
This split holds until <date or trigger>. Recount then.

Gotchas

• A zero-win arm never has a zero estimate. If you find yourself reasoning from a 0% conversion rate, you have dropped the prior.
• Thompson sampling is stochastic. Reruns give slightly different shares. Pass --seed for a reproducible number to paste into a document.
• Rates are not values. An arm converting at 2% on 500-a-month contracts beats one converting at 20% on 5-a-month ones. This model does not know that. Weight the wins.
• Do not mix time scales. Six months of data on one product against two weeks on another is handled correctly by the posteriors, but only if the underlying rates were stable over those windows. If the product changed materially, discard the old counts rather than letting them dominate.
• Non-stationarity is real and unmodelled here. A channel that worked in January may not work in July. If you suspect drift, discard observations older than a couple of cycles, or track the rate with the tracking-beliefs skill instead.
• The floor is not decoration. Setting --floor 0 recovers pure exploitation and reintroduces the failure mode of never revisiting an arm that had a bad start.
• Arms must be simultaneously available. If you can only work on one product at a time, the shares are a schedule over the period, not a parallel split.

Reference

• references/arms-and-wins.md - choosing arms and win events that make the comparison valid
• references/exploration-strategies.md - what each strategy does, and when the default is wrong

Related skills

• valuing-information when the question is whether to test at all, rather than how to split
• tracking-beliefs when the rate is drifting rather than fixed
• planning-horizons when the options are sequential moves rather than parallel bets
• learning-from-outcomes to check whether last quarter's split was right