agentsclimarketplace

Lifesight experiment design

Skill lifesight/lifesight/skills/lifesight-experiment-design

Official agent skills for the Lifesight MCP — causal marketing measurement inside Claude and Claude Code. Claude Code plugin + Claude.ai bundle.

Install
npx -y skills add lifesight/lifesight --skill lifesight-experiment-design

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

  • 1 stars1 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Use when the user wants to prove or validate causality with an experiment — "is X actually incremental", "prove this channel works", "design a geo-lift test", "set up a holdout", "run an incrementality test", "what should I test next", "read my experiment results", "did the test work", "validate the optimization before we commit". Covers both designing tests and interpreting results. Routed to from the `lifesight` router after the workspace is calibrated.

SKILL.md

4.4 KB, as published. Nobody here has run it

Lifesight Experiment Design

Incrementality is the ground truth of the whole platform — the difference between "this channel got credit" and "this channel caused growth." This spoke does two jobs: design a test that will answer a causal question, and read the results of one that ran. It's also where you recommend what's worth testing — usually the biggest, most expensive assumption the user is about to act on.

Prerequisites (router handles): workspace calibrated, profile loaded. Operate under lifesight-core; present under lifesight-rendering. Load both.

Designing a test

  1. Pin the causal question. What decision hangs on it? "Is Linear TV worth its spend?" "Will scaling TikTok actually add revenue or just shift it?" A test with no decision attached is wasted budget.
  2. Pick the design (via ask_mia's experiment workflow):
    • Geo holdout — withhold a channel in matched control markets to measure the lift it's currently driving. The default for "is this incremental?"
    • Geo scale-up — increase spend in treatment markets to probe further up the response curve. For "should I spend more here?"
    • Time / segment designs where geo isn't feasible.
  3. Sanity-check feasibility before launching: is there enough spend/volume and enough matched markets to detect a realistic effect? Name the minimum detectable lift and the test duration up front — an underpowered test wastes weeks and answers nothing.

Reading results

Pull results via ask_mia, then interpret in this order — significance gates everything:

  1. Significance first. Below ~90%, the result is inconclusive — do not act on the lift, no matter how big it looks. Say so plainly.
  2. Then lift + direction. Note the holdout nuance explicitly: in a holdout, a negative lift means the marketing was effective (withholding it dropped the metric). Don't misread that as "the channel failed."
  3. Then efficiency. Incremental ROAS / CPA from the experiment is the causal ground truth — it outranks platform-reported numbers and even the MMM estimate.
  4. Power context. A null result on an underpowered test is "we couldn't tell," not "no effect." Compare the observed lift to the minimum detectable lift before concluding anything.

Judgment checks (mandatory)

  • No significance, no conclusion. The most common error is acting on an insignificant lift. Hold the line even under "but the number's big" pressure.
  • Holdout sign convention — negative lift = effective. State it so it can't be misread.
  • Experiment > model > platform. When they disagree, the clean experiment wins; use it to recalibrate, not to rationalize.
  • Tests take time and assume no contamination. Don't imply instant answers; flag spillover risk (media bleeding into control markets, un-geo-targetable national buys).

Output shape

  • Design: the test plan — markets/split, duration, the spend change, what it will detect (minimum detectable lift), and the decision it will settle.
  • Read: significance → lift (with sign explained) → incremental ROAS/CPA → verdict → recommended action (scale, cut, recalibrate the model, or re-test with more power).

Next steps to offer

"Recalibrate the budget with this result" (→ budget-optimization) · "Deep-dive the channel we tested" (→ channel-deep-dive) · "Explain how geo-lift works" (→ measurement-coach) · "Design the follow-up test".

Red flags — STOP

  • Acting on a lift below ~90% significance → inconclusive, say so
  • Reading a holdout's negative lift as failure → it means effective
  • Calling an underpowered null "no effect" → it's "couldn't detect"
  • Implying an experiment gives an instant answer → name the duration
  • Letting platform/MMM numbers override a clean experiment

Keep looking

Skills are one crate of 328,083. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.