Robustness

Robustness Test Checklist

Systematic checklist of robustness tests for empirical research. Use this to ensure comprehensive testing before submission.

---

Core Robustness Categories

1. Alternative Samples

Test	Description	When to Use
Exclude outliers	Winsorize/trim at different levels (0.5%, 2%, 5%)	Always
Drop financial firms	Exclude SIC 6000-6999	If not already excluded
Drop regulated industries	Exclude utilities, telecoms	Industry-specific effects
Different time periods	Split sample pre/post crisis, early/late	Results may be period-specific
Geographic subsamples	By region, state, country	External validity
Size subsamples	Small vs. large firms	Heterogeneous effects
Balanced panel	Require continuous observations	Survivorship concerns

2. Alternative Specifications

Test	Description	When to Use
Different fixed effects	Firm, industry×year, state×year	Control for unobservables
Additional controls	Add variables referees might suggest	Omitted variable concerns
Drop controls	Verify not over-controlling	Mediator concerns
Different clustering	Firm, industry, state, two-way	Inference robustness
Different standard errors	Bootstrap, Newey-West, Driscoll-Kraay	Serial/cross-sectional correlation
Nonlinear specifications	Quadratic terms, splines	Linearity assumption
Log vs. level	Transform dependent variable	Skewed distributions

3. Alternative Measures

Test	Description	When to Use
Alternative dependent variable	Different proxy for same concept	Measurement concerns
Alternative treatment measure	Continuous vs. binary, different threshold	Treatment definition
Alternative control measures	Different proxies for size, leverage, etc.	Standard practice
Scaled differently	By assets, sales, employees	Scaling choice matters

4. Identification Tests

Test	Description	When to Use
Placebo/Falsification
Placebo timing	Fake treatment 1-3 years before actual	DiD parallel trends
Placebo outcome	Effect on outcome that shouldn't be affected	Specificity of mechanism
Placebo treatment	Random assignment of treatment	Rule out spurious correlation
Pre-trends
Event study plot	Coefficient for each pre/post period	Visual parallel trends
Joint F-test	Test pre-period coefficients = 0	Statistical parallel trends
Endogeneity
Instrumental variables	Find exogenous variation	Selection concerns
Heckman selection	Model selection explicitly	Sample selection
Propensity score matching	Match treated/control	Observable selection
Entropy balancing	Reweight to balance covariates	Covariate imbalance
Regression discontinuity	If threshold exists	Sharp identification

5. Inference Robustness

Test	Description	When to Use
Wild cluster bootstrap	Small number of clusters	<50 clusters
Randomization inference	Permutation-based p-values	Few treated units
Conley standard errors	Spatial correlation	Geographic data
Multiple hypothesis correction	Bonferroni, FDR	Many outcomes tested

---

DiD-Specific Tests

For difference-in-differences designs:

• Event study with pre-treatment coefficients
• Parallel trends test (formal)
• Bacon decomposition (staggered treatment)
• Callaway-Sant'Anna or Sun-Abraham estimator (heterogeneous treatment effects)
• Placebo treatment timing
• Vary treatment window
• Triple-difference if possible
• Exclude always-treated or never-treated

---

IV-Specific Tests

For instrumental variables:

• First-stage F-statistic (>10, prefer >100)
• Weak instrument robust inference (Anderson-Rubin)
• Overidentification test (if multiple IVs)
• Exclusion restriction discussion
• Reduced form results
• Compare OLS vs. IV magnitudes

---

Quick Commands

• "robustness checklist" - Full checklist for current paper
• "DiD robustness" - DiD-specific tests only
• "what tests for [method]" - Tests for specific identification strategy
• "referee-proof" - Most commonly requested tests
• "prioritize tests" - Rank by importance for your setting

---

Referee-Proof Minimum

At minimum, most papers should include:

1. Sample robustness: Exclude outliers, alternative time periods
2. Specification robustness: Alternative fixed effects, controls
3. Measurement robustness: Alternative variable definitions
4. Inference robustness: Alternative clustering
5. Identification test: At least one placebo or pre-trend test

---

Reporting Template

For robustness tables:

Table X: Robustness Tests
Panel A: Alternative Samples
  (1) Baseline
  (2) Exclude financial firms
  (3) Exclude 2008-2009
  (4) Winsorize at 5%

Panel B: Alternative Specifications
  (5) Add industry×year FE
  (6) Control for firm age
  (7) Cluster by industry

Panel C: Alternative Measures
  (8) Alternative dependent variable
  (9) Continuous treatment measure