Response uplift modeling

Response & Uplift Modeling

Predictive modeling skill for promotion targeting. Covers propensity scoring, two-model and single-model uplift, multinomial four-quadrant classification, ROI threshold optimization, and evaluation (lift chart, PR curve).

Theoretical Foundation (Katsov Ch. 3)

Response Modeling Framework (§3.4.1)

Expected campaign profit:

G(U) = Σ_{u∈U} [ Pr(R|u,T) · G(u|R) − C ]
     = Σ_{u∈U} E[G|u,T] − C

Selection rule: target customer u iff E[G|u,T] > C
ROI is maximized by finding the propensity threshold where marginal profit = 0 (§3.6.2.2).

Four Response Quadrants (§3.5.4.2, Kane et al. 2014)

Group	T=1	T=0	Strategy
Persuadables (TR)	Respond	Don't respond	TARGET → highest uplift
Sure Things (CR)	Respond	Respond	Skip — no incremental value
Lost Causes (TN)	Don't respond	Don't respond	Skip — waste budget
Do-Not-Disturb (CN)	Don't respond	Respond	Avoid — treatment harms

Uplift = Pr(R|T,x) − Pr(R|C,x)
Only Persuadables deliver positive incremental ROI.

---

Workflow

Step 1 — Assess Data

Confirm the dataset has:

• Profile features (observation window): RFM, behavioral, demographic
• Treatment indicator T ∈ {0,1}
• Response label Y ∈ {0,1} (outcome window)
• Test/control split if uplift modeling is requested

If no control group exists → default to propensity_logistic only and caveat.

Step 2 — Select Model

Objective	Model	Script
Score all customers by P(response)	Propensity (unconditional)	propensity_logistic.py
Estimate incremental lift, simpler setup	Uplift – Two-Model	uplift_two_model.py
Estimate incremental lift, better calibration	Uplift – Single Model + interactions (Lo, 2002)	uplift_single_model.py
Full quadrant classification	Uplift – Multinomial	multinomial_uplift.py
Find optimal targeting depth	ROI Threshold Optimizer	roi_threshold_optimizer.py

→ Read references/model_selection.md for guidance on choosing between two-model vs single-model.

Step 3 — Train & Score

Run the selected script. Output: score per customer + model metrics.

Step 4 — Optimize Threshold (§3.6.2.2)

Run roi_threshold_optimizer.py with:

• uplift_scores (or propensity scores)
• promo_cost (C — cost per promotion distributed)
• response_value (G — net profit per conversion)

Output: optimal score cutoff, targeting depth (% of population), expected ROI at optimum.

Key insight from Katsov §3.6.2.2: Maximum budget ≠ maximum ROI. Targeting beyond the breakeven threshold destroys value. The ROI curve peaks before budget exhaustion.

Step 5 — Evaluate

• Run lift_chart.py → compare targeted vs random cumulative response curve
• Run pr_curve.py → precision-recall for model quality
• Report: Lift at top decile, AUC, breakeven targeting depth

---

Scripts Reference

Script	Inputs	Key Output
propensity_logistic.py	X_train, y_train, X_score	propensity_score ∈ [0,1] per customer
uplift_two_model.py	X_treat, y_treat, X_ctrl, y_ctrl, X_score	uplift_score = P(R
uplift_single_model.py	X, T, y, X_score	uplift_score via interaction terms (Lo, 2002)
multinomial_uplift.py	X, T, y, X_score	Pr per quadrant (TR, CR, TN, CN) + uplift
roi_threshold_optimizer.py	scores[], promo_cost, response_value	optimal_threshold, targeting_depth, max_roi
lift_chart.py	scores[], y_actual	Lift chart PNG + lift table by decile
pr_curve.py	scores[], y_actual	PR curve PNG + AUC-PR

→ Read individual script headers for full parameter documentation.

---

Output Format — Visualization First

Rule: always render an inline HTML artifact as primary output. Markdown tables are secondary.

Visualization Layer (must-have for every analysis)

After computing results (from scripts or from user-provided data), render a dynamic HTML dashboard using show_widget. The dashboard must include:

1. Four-Quadrant Map — SVG or canvas showing Persuadables / Sure Things / Lost Causes / Do-Not-Disturb with customer density per quadrant
2. Lift Chart — dual-line chart (targeted vs random) with decile markers
3. ROI Curve — cumulative ROI by targeting depth with peak annotated
4. Score Distribution — histogram of uplift/propensity scores with breakeven line
5. KPI Summary Bar — top-line metrics: optimal threshold, targeting depth %, expected ROI, lift@decile1

Use the HTML template in references/dashboard_template.html as the base. Pass computed data as inline JSON.

When No Script Output is Available (user pastes data / describes scenario)

Compute values analytically inline (Python in bash_tool or by hand) then render the dashboard with those values. Never output only a markdown table when a visualization is possible.

Output Sequence

1. [bash_tool] Run scripts if data is available → capture JSON output
2. [show_widget] Render HTML dashboard with results
3. [text] 3–5 line executive recommendation
4. [text] Caveats if no control group or small sample

Fallback (if show_widget unavailable)

Produce markdown tables in this order:

1. Score table (top 10): customer_id | score | decile | quadrant
2. Lift table (Katsov Table 3.7): decile | responses_random | responses_targeted | lift
3. ROI table (Katsov Table 3.8): decile | cost | profit_random | profit_targeted | cumulative_roi
4. Recommendation text

---

Key Caveats

• Never present propensity score as uplift. High propensity ≠ high incremental lift. Sure Things inflate propensity models without adding ROI (Katsov §3.5.4.2).
• Two-model approach risk: separately trained models may have incomparable scales and select non-uplift-predictive features (Radcliffe & Surry, 2011).
• Uplift variance: uplift = difference of two random variables → higher variance than propensity. Requires larger pilot sample for reliable estimates.
• Retention campaigns: use savability (−uplift on churn) × LTV as targeting score instead of raw churn propensity (Katsov §3.6.4, eq. 3.40).

---

Reference Files

• references/model_selection.md — When to use two-model vs single-model vs multinomial
• references/katsov_ch3_excerpts.md — Key equations and tables from Katsov Ch. 3
• references/roi_optimizer_math.md — Full derivation of ROI threshold formula
• references/dashboard_template.html — Reusable HTML artifact with lift chart, ROI curve, quadrant map, score distribution. Load this and substitute SKILL_DATA_JSON placeholder with computed results.

---

Integration with Agency Growth OS

Upstream skill	Handoff
audience-segmentation-brief	Segment-level propensity models (Katsov §3.5.5)
measurement-incrementality	Uplift scores as incrementality proxy pre-experiment
crm-journey-architect	Savability scores feed retention journey triggers
margin-simulation	roi_threshold_optimizer.py output feeds margin model
media-routing-planner	Channel-level response rates for budget allocation