Eursr research design

Research Design (eursr-research-design)

ESR is a quantitative journal exacting about whether the comparative or longitudinal design actually identifies the mechanism from eursr-theory-building and rules out the leading confound. The design must connect the cross-level hypothesis to evidence that a single cross-section could not provide.

When to trigger

• Specifying the comparative frame, the panel structure, sampling, or the identification strategy
• A reviewer questioned causal claims, generalization, selection, measurement comparability, or a confound
• Justifying why your design adjudicates the rival account from eursr-literature-positioning

Comparative / cross-national

• Justify the country set by design logic (institutional contrast, regime types, most/least-similar), not by data availability alone; say what variation each context contributes.
• Measurement equivalence is the first reviewer demand: establish that constructs mean the same across countries (configural/metric/scalar invariance for latent scales; harmonized coding for education via ISCED/CASMIN, occupation via ISCO/ISEI/EGP).
• Macro N is small. With ~20-30 countries, country-level effects rest on few degrees of freedom — design the macro hypothesis so it does not over-claim from a handful of clusters (see eursr-data-analysis).

Panel / longitudinal / event-history

• State what the panel buys. Within-person change (fixed effects), duration/timing (event history), or growth (latent growth) — match the estimator to the theoretical quantity.
• Attrition and selection into and out of the panel must be addressed (weights, IPW, sensitivity).
• For staggered policy exposure, use heterogeneity-robust DiD (Callaway-Sant'Anna, Sun-Abraham, Borusyak et al.), not naive TWFE.

Causal inference where feasible

• Much of ESR is observational; distinguish description, association, and causation honestly. If causal, state the assumptions (ignorability, parallel trends, exclusion) and defend them; report a sensitivity bound (how strong an unobserved confounder would have to be).

Multilevel / SEM

• Specify the level structure (individuals in countries/regions/cohorts), the random effects, and why a multilevel model is warranted; for measurement, build the latent model before the structural one.

The adjudication test (ESR-specific)

For the single strongest rival explanation: "If the rival were true rather than my argument, the cross-national (or over-time) pattern would look like ___; instead it looks like ___." If you cannot write it, the comparative/panel design does not yet identify the contribution.

What ESR referees demand of each design

Design	Referee's first demand	Satisfying move
Comparative cross-national	"Are the measures equivalent?"	invariance / harmonized coding; justified country set
Panel / fixed-effects	"What does within-person change identify?"	match estimator to the quantity; handle attrition
Event history	"Right risk set and time scale?"	defined onset, censoring, time-varying covariates
Causal (DiD/IV/RDD)	"Assumption defended?"	state + test the assumption; sensitivity bound
Multilevel / SEM	"Enough clusters; measurement first?"	macro df honesty; fit the latent model before structure

Worked micro-example (illustrative)

A comparative study argues that vocational specificity smooths the school-to-work transition.

Country set: most-different welfare/training regimes (e.g., dual-system vs. general-education systems),
  chosen for institutional contrast, not convenience
Measurement: education harmonized via ISCED; vocational specificity coded from program-level data
Design: cross-national + cohort variation; cross-level interaction (specificity × individual track)
Disconfirming pattern sought: if signaling (not skills) drove it, the advantage would vanish once firms
  learn quality → instead it persists across the early career, as the specificity argument predicts
Macro-N caution: ~24 countries → country-level claim kept modest; SEs / df handled in data-analysis

The country set is design-driven, the measures are comparable, and the design specifies what pattern would falsify the argument.

Referee pushback → ESR-specific fix

• "Measures aren't comparable across countries." → Test invariance; report partial invariance and what it permits; use harmonized coding schemes.
• "You infer too much from ~20 countries." → Re-state the macro claim modestly; use df-appropriate inference (see eursr-data-analysis).
• "Association dressed as causation." → Restate what the design identifies; add a sensitivity bound or placebo; drop causal verbs you cannot defend.

Calibration anchors

• Measurement equivalence is the comparative gate. A cross-national claim built on non-equivalent scales is the most common fatal design flaw at ESR.
• The adjudication sentence is the test. If you can't write "if the rival were true the pattern would look like ___," the comparison/panel does not yet earn the contribution.
• Identification honesty travels. Stating plainly what observational European data can and cannot establish reads as strength to a quantitative panel.

Execution bridge (StatsPAI / Stata MCP)

Estimate and audit the design, don't only describe it. Full map: execution-with-mcp. ESR is comparative quantitative sociology; cross-country panels with confounded institutions — foreground fixed effects and clustering.

• detect_design → recommend → fit with as_handle=true → audit_result.
• Observational causal claims: staggered DiD (callaway_santanna / sun_abraham + bacon_decomposition + honest_did_from_result); IV (effective_f_test + anderson_rubin_ci); RDD (rdrobust + mccrary_test).
• Experiments: randomization-based inference, romano_wolf for many-outcome family-wise control, and mediate for mediation (not naive controlling-away).
• Sensitivity: oster_delta / sensemakr for observational claims.

Report the effect size in interpretable units; route the full battery to the appendix/supplement. A run end-to-end (synthetic data, real returns) is in the JF execution walkthrough.

Anti-patterns

• A country set chosen by data availability and dressed up as theory-driven
• Cross-national latent comparisons with no measurement-invariance check
• Over-claiming country-level effects from a handful of clusters
• Naive TWFE on staggered policy timing; ignoring panel attrition
• A design that cannot distinguish your mechanism from the leading alternative

Output format

【Design】comparative / panel / event-history / causal / multilevel-SEM
【What it identifies】description / association / causation
【Comparability / assumption】invariance or key assumption + how defended
【Rival ruled out】the adjudication sentence
【Macro-N / attrition / sensitivity】planned
【Next】eursr-data-analysis

Supplementary resources

• ../../resources/external_tools.md — multilevel / SEM / event-history / DiD tooling
• ../../resources/code/ — reproducible Stata + Python causal-inference skeleton (DiD/IV/RDD/DML)
• ../../resources/official-source-map.md — ESR methodological expectations