Forest funnel plots

Version Compatibility

Reference examples tested with: metafor 4.4+, forestplot 3.1+, ggforestplot 0.1+ (subgroup forests), ggforest from survminer 0.4.9+, MendelianRandomization 0.10+.

Before using code patterns, verify installed versions match. If versions differ:

• R: packageVersion('<pkg>') then ?function_name

If code throws ImportError, AttributeError, or TypeError, introspect the installed package and adapt the example to match the actual API rather than retrying.

Forest and Funnel Plots

"Summarize effects across studies / subgroups" -> Render each effect estimate (HR, OR, RR, β) as a square (size = inverse variance / weight), horizontal bar (95% CI), and label, with an optional summary diamond at the bottom from a meta-analysis pool (fixed-effect or random-effects). The funnel plot diagnoses publication bias by plotting effect size vs precision; asymmetry indicates missing small-study-with-null-result publications (Egger 1997).

• R: metafor::forest, metafor::funnel, forestplot::forestplot, survminer::ggforest (Cox HR forests), MendelianRandomization::mr_forest

The Single Most Important Modern Insight -- Heterogeneity Is the First Question

A pooled effect estimate is meaningless if the underlying studies are heterogeneous. The Higgins I² statistic (Higgins-Thompson 2002 Stat Med 21:1539) quantifies between-study variance; the conventional 25/50/75% interpretation tiers come from the Cochrane Handbook §10.10.2 (Higgins et al editors), NOT the original Higgins-Thompson paper which cautioned against rigid cutoffs. A meta-analysis with I² > 75% and a pooled effect must explain the heterogeneity (subgroup analysis, meta-regression) — pooling without explanation is statistically defensible but biologically unhelpful.

A forest plot's bottom must report: pooled estimate + 95% CI + I² + τ² (between-study variance) + Q-test p-value. Without these, the plot is a list of effects, not a meta-analysis.

Decision Tree by Analysis Type

Analysis	Tool	Pooling model	Forest method
Single-trial subgroup HRs	survminer::ggforest	None (subgroup display)	Coxph object
Meta-analysis of binary outcomes	metafor::rma -> forest()	DerSimonian-Laird or REML random-effects	Standard forest
Meta-analysis of continuous outcomes	metafor::rma(yi, vi)	REML random-effects	Standard forest
Mendelian randomization	MendelianRandomization::mr_forest	Multiple MR methods	MR-specific forest
Subgroup forest with interaction p	metafor::rma + addpoly + interaction model	Subgroup REML	Nested forest
Network meta-analysis	netmeta::forest.netmeta	Bayesian or frequentist	Network forest
Cumulative meta-analysis (over time)	metafor::cumul + forest	–	Cumulative forest

Fixed-Effect vs Random-Effects Meta-Analysis

Model	Assumption	When appropriate	Pooled estimate weight
Fixed-effect (Mantel-Haenszel, IVS)	All studies estimate the same true effect	Single mechanism, homogeneous design	1 / within-study variance
Random-effects (DerSimonian-Laird, REML)	Studies estimate distinct true effects from a common distribution	Heterogeneous designs / populations	1 / (within + between variance)

Use random-effects by default. Fixed-effect assumes all studies estimate the same parameter, which is almost never true across multi-center trials with different populations. REML is the modern default (Viechtbauer 2005); DerSimonian-Laird is the older default still commonly seen.

metafor::rma + forest -- The Reference Implementation

Goal: Pool study-level effect estimates with random-effects meta-analysis; render a forest plot with study weights, individual effect+CI, and pooled summary diamond.

Approach: Compute per-study yi (effect) and vi (sampling variance); fit REML random-effects model; pass to forest() with prediction interval if heterogeneity is non-trivial.

library(metafor)

# Input: per-study effect (yi) and variance (vi)
# For OR: yi = log(OR), vi = SE(log(OR))^2
# For HR: yi = log(HR), vi = SE(log(HR))^2
res <- rma(yi = log_or, vi = log_or_se^2,
           data = studies, slab = paste(author, year),
           method = 'REML')

# I^2 and tau^2 in the summary
summary(res)
# I^2 (residual heterogeneity)
# tau^2 (estimated amount of (residual) heterogeneity)
# Q-test for heterogeneity

forest(res,
       atransf = exp,                           # display OR on natural scale
       at = log(c(0.25, 0.5, 1, 2, 4)),         # ticks at meaningful OR values
       refline = 0,                              # log(1) for OR/HR/RR
       xlab = 'Odds Ratio (95% CI)',
       header = c('Study', 'OR [95% CI]'),
       mlab = bquote(paste('RE Model (Q = ', .(round(res$QE, 2)),
                            ', df = ', .(res$k - 1),
                            ', p = ', .(format.pval(res$QEp, digits = 2)),
                            '; ', I^2, ' = ', .(round(res$I2, 1)), '%)')),
       addpred = TRUE)                          # prediction interval per Higgins 2009

addpred = TRUE adds a 95% prediction interval — where a new study's effect is expected to fall (Higgins-Thompson-Spiegelhalter 2009 JRSS-A). This is the most honest summary when I² > 30%.

ggforest for Cox Subgroup Forests

library(survminer)
fit <- coxph(Surv(time, status) ~ treatment + age + sex + stage, data = df)
ggforest(fit,
         data = df,
         main = 'Subgroup HRs',
         cpositions = c(0.02, 0.22, 0.4),
         fontsize = 0.7,
         refLabel = 'Reference',
         noDigits = 2)

ggforest produces a publication-ready subgroup forest from a coxph object. For pre-specified subgroup analyses (treatment × subgroup interaction), test interaction explicitly and annotate the p-value.

Small-k Regime -- When Meta-Analysis Asymptotics Break

For k < 5 studies, the REML-based 95% CI from metafor::rma() is severely anti-conservative — it relies on chi-square asymptotics that fail with few studies. Use Hartung-Knapp-Sidik-Jonkman (HKSJ) adjustment (test = 'knha'):

res_hksj <- rma(yi = log_or, vi = log_or_se^2, data = studies,
                method = 'REML', test = 'knha')           # HKSJ for k<5

HKSJ uses a t-distribution with k-1 degrees of freedom and adjusts SE via the Q-statistic — well-calibrated even at k=3. For k < 3 a meta-analysis is not advisable; report individual study effects in a forest plot without a pooled summary.

Also: I² is uninterpretable below k = 5 (Borenstein 2017 Res Synth Methods 8:5); the point estimate has wide CI dominated by k itself, not heterogeneity. Do not report I² for k < 5.

Funnel Plot and Egger Test

Goal: Diagnose publication bias by visual asymmetry of effect size vs precision.

Approach: Plot effect (x) vs SE (inverted y); under no bias, points form a symmetric inverted funnel with the pooled estimate at the apex. Asymmetry suggests missing small-N null-result studies. Egger's regression test (Egger 1997 BMJ 315:629) formalizes the asymmetry.

funnel(res,
       xlab = 'log(OR)',
       refline = res$b)

# Egger's test
regtest(res, model = 'lm', predictor = 'sei')
# significant p indicates asymmetry; suggests publication bias

# Trim-and-fill (Duval-Tweedie 2000) -- adjusts for asymmetry
res_tf <- trimfill(res)
forest(res_tf)
funnel(res_tf)

Contour-enhanced funnel plot (Peters 2008 J Clin Epidemiol 61:991) overlays significance contours (p < 0.10, < 0.05, < 0.01); asymmetry concentrated in "non-significant" regions indicates publication bias more specifically than generic asymmetry.

funnel(res, level = c(90, 95, 99), shade = c('white', 'gray55', 'gray75'),
       refline = 0, legend = TRUE)

Per-Method Failure Modes

Pooling under high heterogeneity without explanation

Trigger: Random-effects meta-analysis pooled with I² > 75%; no subgroup or meta-regression.

Mechanism: Pooled estimate is a weighted average across substantively different effects; biologically meaningless.

Symptom: Pooled OR = 1.5 with 95% CI (1.2-1.8) but per-study effects range 0.3-5.0.

Fix: Run subgroup analysis or meta-regression to explain heterogeneity; report I², τ², and prediction interval; do NOT report a single pooled effect as the answer.

Fixed-effect when studies are heterogeneous

Trigger: Default fixed-effect model on multi-population data.

Mechanism: Fixed-effect weights = 1/within-study variance, ignoring between-study variance.

Symptom: CI is misleadingly narrow; reviewer asks "why fixed effect with high I²?"

Fix: Switch to REML random-effects (method = 'REML'); document the choice.

Egger test p-value over-interpreted

Trigger: k < 10 studies; significant Egger p taken as definitive publication bias.

Mechanism: Egger's test is underpowered with few studies; sensitive to single outliers.

Symptom: Conclusion "publication bias" from k=6 trials.

Fix: Egger requires k ≥ 10 (Sterne 2011 BMJ 343:d4002); for fewer studies, visual funnel + contour-enhanced funnel is more reliable.

Trim-and-fill imputed studies presented as data

Trigger: Reporting trim-and-fill adjusted estimate as "the answer."

Mechanism: Trim-and-fill is a sensitivity analysis; imputed studies are hypothetical.

Symptom: Original pooled OR = 2.0; trim-and-fill adjusted to 1.5; report says "adjusted estimate is 1.5."

Fix: Present original AND trim-and-fill side-by-side; trim-and-fill is sensitivity, not primary.

Subgroup forest without interaction test

Trigger: Subgroup HRs plotted; conclusion "treatment works in subgroup X."

Mechanism: Visual differences across subgroups don't establish significant interaction.

Symptom: Subgroup forest shows HR=0.5 in subgroup A, HR=1.0 in subgroup B; no formal test.

Fix: Add treatment × subgroup interaction term to the model; report interaction p; cite Brookes 2001 / Wang 2007 for subgroup analysis caveats.

Forest plot axis on linear scale for ratios

Trigger: OR/HR/RR plotted with linear x-axis.

Mechanism: Ratios are multiplicatively symmetric; linear axis compresses < 1 effects.

Symptom: OR = 0.5 (halving) appears smaller than OR = 2 (doubling) on a linear scale, even though they are biologically equivalent.

Fix: Always log-scale the x-axis for ratios. metafor's atransf = exp + at = log(c(0.25, 0.5, 1, 2, 4)) is the canonical pattern.

Weights not visible (point sizes uniform)

Trigger: Default forestplot package without weight encoding.

Mechanism: Reader cannot tell study influence on pool.

Symptom: A 5-patient pilot looks visually equivalent to a 5000-patient trial.

Fix: Use metafor forest() which auto-encodes weight via box size. forestplot package needs boxsize = argument.

Reconciliation: When Methods Disagree

Pattern	Cause	Action
Fixed-effect significant; random-effects n.s.	High heterogeneity inflates RE variance	Trust random-effects when I² > 30%
Egger n.s. but funnel looks asymmetric	k < 10 -> Egger underpowered	Trust visual; report contour-enhanced funnel
Trim-and-fill imputes many studies	Severe asymmetry	Caution; sensitivity, not primary
Subgroup forest suggests effect modification; interaction test n.s.	Visual difference does not establish formal interaction	Trust interaction test
MR forest shows divergent estimates across methods	Pleiotropy or weak instruments	Run MR-Egger, weighted median, mode-based (sensitivity); cite Bowden 2015

Quantitative Thresholds

Threshold	Value	Source
I² substantial heterogeneity	> 50%	Cochrane Handbook §10.10.2 (Higgins et al editors)
I² considerable heterogeneity	> 75%	Cochrane Handbook §10.10.2 (Higgins et al editors)
Egger test min k	≥ 10	Sterne 2011 BMJ
Prediction interval (where new study lands)	report when I² > 30%	Higgins-Thompson-Spiegelhalter 2009
Forest x-axis	log scale for ratios	Convention

Common Errors

Error / symptom	Cause	Solution
Pooled estimate with I² = 90%	No heterogeneity exploration	Subgroup / meta-regression
Linear x-axis for OR forest	Ratios should be log-symmetric	atransf = exp, at = log(...)
Uniform point sizes	Weights not encoded	metafor::forest auto-encodes; forestplot needs boxsize
Egger from k=5	Underpowered	k ≥ 10 for Egger
Trim-and-fill as primary	Sensitivity, not primary	Present both; document
Subgroup effect without interaction test	Visual ≠ test	Add interaction term
MR forest with single method	Pleiotropy risk	Triangulate methods

References

• Bowden J, Davey Smith G, Burgess S. 2015. Mendelian randomization with invalid instruments: effect estimation and bias detection through Egger regression. Int J Epidemiol 44(2):512-525.
• Brookes ST, Whitley E, Peters TJ, et al. 2001. Subgroup analyses in randomised controlled trials: quantifying the risks of false-positives and false-negatives. Health Technol Assess 5(33):1-56.
• DerSimonian R, Laird N. 1986. Meta-analysis in clinical trials. Control Clin Trials 7(3):177-188.
• Duval S, Tweedie R. 2000. Trim and fill: a simple funnel-plot–based method of testing and adjusting for publication bias in meta-analysis. Biometrics 56:455-463.
• Egger M, Davey Smith G, Schneider M, Minder C. 1997. Bias in meta-analysis detected by a simple, graphical test. BMJ 315(7109):629-634.
• Higgins JPT, Thompson SG. 2002. Quantifying heterogeneity in a meta-analysis. Stat Med 21(11):1539-1558.
• Higgins JPT, Thompson SG, Spiegelhalter DJ. 2009. A re-evaluation of random-effects meta-analysis. JRSS-A 172(1):137-159.
• Peters JL, Sutton AJ, Jones DR, Abrams KR, Rushton L. 2008. Contour-enhanced meta-analysis funnel plots help distinguish publication bias from other causes of asymmetry. J Clin Epidemiol 61(10):991-996.
• Sterne JAC, Sutton AJ, Ioannidis JPA, et al. 2011. Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ 343:d4002.
• Viechtbauer W. 2010. Conducting meta-analyses in R with the metafor package. J Stat Softw 36(3):1-48.
• Hartung J, Knapp G. 2001. A refined method for the meta-analysis of controlled clinical trials with binary outcome. Stat Med 20(24):3875-3889.
• IntHout J, Ioannidis JPA, Borm GF. 2014. The Hartung-Knapp-Sidik-Jonkman method for random effects meta-analysis is straightforward and considerably outperforms the standard DerSimonian-Laird method. BMC Med Res Methodol 14:25.
• Borenstein M, Higgins JPT, Hedges LV, Rothstein HR. 2017. Basics of meta-analysis: I² is not an absolute measure of heterogeneity. Res Synth Methods 8(1):5-18.
• Higgins JPT, Thomas J, Chandler J, et al (editors). Cochrane Handbook for Systematic Reviews of Interventions (current version). Section 10.10.2 — interpretation tiers for I².

Related Skills

• clinical-biostatistics/effect-measures - HR / OR / RR / NNT definitions
• clinical-biostatistics/subgroup-analysis - Interaction tests for subgroup HRs
• causal-genomics/mendelian-randomization - MR-specific forest + sensitivity
• clinical-biostatistics/trial-reporting - CONSORT and meta-analysis reporting
• data-visualization/color-palettes - Palette for multi-study or subgroup forests