Jpart research design

Research Design (jpart-research-design)

JPART has moved toward experimental and causal identification, and reviewers expect the design to connect the theory (jpart-theory-building) to evidence credibly. This skill is mode-aware: pick the section that matches your work and defend it against the strongest alternative explanation a public-management reviewer will raise.

When to trigger

• Specifying identification, an experiment, sampling, or measurement
• A reviewer questioned causal claims, common-method bias, endogeneity, or external validity
• Preparing a pre-analysis plan / preregistration (JPART accepts blinded pre-reg reports)
• Justifying why the design adjudicates the rival account from jpart-literature-positioning

Design-choice gate

Start by matching the theoretical claim to the minimum credible design. Do not choose the design by data availability alone.

Claim type	Minimum design burden	Common downgrade
"X causes Y in public organizations"	Identification strategy with an estimand, assignment/variation story, and falsification or sensitivity evidence	Reframe as association or theory-building descriptive evidence
"Mechanism M explains the effect"	Mediating evidence that is temporally and conceptually downstream of treatment/exposure, plus rival-mechanism checks	Reframe as a plausible mechanism to be tested, not demonstrated
"Public employees/citizens respond differently by condition C"	Pre-specified heterogeneity, adequate power, and measurement invariance across groups	Treat as exploratory moderation
"Policy/intervention improves performance"	Implementation fidelity, baseline comparability, outcome validity, and spillover/contamination checks	Reframe as pilot evidence
"Case evidence revises theory"	Case selection logic, process-tracing observations, rival explanations, and explicit scope conditions	Reframe as illustrative theory elaboration

Experiments (the modern JPART workhorse)

• Population matters. Public-management theory often requires public employees or citizens as subjects — defend the sample (e.g., real managers, frontline staff) over a generic MTurk pool.
• Design. Preregister the design and primary analyses; report power/MDE; pre-specify subgroups; use vignette/conjoint/factorial designs where the theory is about trade-offs.
• Validity. Attention/manipulation checks, attrition, realism of treatment, and consent/IRB.
• Replication awareness. PA has an active experimental-replication norm — design so the experiment could be re-run and pre-register to make exploratory vs. confirmatory analyses explicit.

Observational / causal

• Identification first. State the estimand and the assumptions that license a causal reading (ignorability, parallel trends, exclusion, continuity). Defend them, don't assert them.
• Designs: DID/event study (modern staggered-adoption estimators, not naive TWFE), IV (first-stage strength, exclusion, weak-IV-robust inference), RDD (density/manipulation tests, bandwidth), matching /weighting with balance + sensitivity.
• PA-specific confounds: self-selection into public service, common-source/common-method bias when X and Y come from the same survey, endogenous sorting of managers to organizations.

Multilevel / organizational

• Employees nested in agencies nested in jurisdictions — use multilevel models; cluster SEs at the level of treatment/assignment; report ICCs; do not ignore the nesting that PA data almost always has.

Mixed methods

• Make the qualitative and quantitative components answer the same theoretical question; say what each buys and where they corroborate or diverge.

The adjudication test (JPART-specific)

For the single strongest rival explanation (often selection or common-method bias), write one sentence: "If the rival were true rather than my mechanism, the data would look like ___; instead they look like ___." If you cannot, the design does not yet identify the contribution.

Reviewer stress tests

Run these before the manuscript claims JPART-level causal or theoretical leverage:

• Theory-design alignment: the unit of theory, treatment/exposure, outcome, and inference level match. A theory about managers is not proven by citizen vignettes unless the bridge is explicit.
• Measurement separation: key independent/dependent variables are not merely two self-reports from the same respondent at the same time; if they are, build a common-method defense or narrow the claim.
• Assignment credibility: the reader can say why some units received more/less treatment and why that variation is not just latent performance, resources, or managerial quality.
• Organizational nesting: the standard errors, random effects, or design account for agencies, offices, jurisdictions, schools, or teams where treatment and outcomes cluster.
• Generalization boundary: state whether the result generalizes to public employees, citizens, organizations, jurisdictions, or one institutional setting.
• Transparency path: preregistration, data/code release, and any restricted-data exception can be anonymized and reconciled with jpart-transparency-and-data.

Execution bridge (StatsPAI / Stata MCP)

Estimate and audit the design, don't only describe it. Full map: execution-with-mcp. JPART is public management — observational and experimental designs on public organizations; identification + clustered/multilevel inference.

• detect_design → recommend → fit with as_handle=true → audit_result.
• Observational causal claims: staggered DiD (callaway_santanna / sun_abraham + bacon_decomposition + honest_did_from_result); IV (effective_f_test + anderson_rubin_ci); RDD (rdrobust + mccrary_test).
• Experiments: randomization-based inference + romano_wolf for many-outcome control.
• Sensitivity: oster_delta / sensemakr for observational claims.

Report the magnitude in interpretable units; route the full battery to the appendix. A run end-to-end (synthetic data, real returns) is in the JF execution walkthrough.

Anti-patterns

• A "behavioral PA" experiment on a generic online panel when the theory is about public managers
• Common-source bias: X and Y from the same self-report survey, called a causal effect
• Naive TWFE on staggered adoption; clustering at the wrong level; ignoring agency-level nesting
• Treating self-selection into public service as ignorable
• A design that cannot distinguish your mechanism from selection or the leading alternative

Output format

【Mode】experiment / observational-causal / multilevel / mixed
【Population】public employees / citizens / orgs — defended? [Y/N]
【Estimand or claim】what is being identified/shown
【Key assumption(s)】and how each is defended
【Design-choice gate】causal / mechanism / heterogeneity / policy / case-theory burden met?
【Rival ruled out】the adjudication sentence (often selection / common-method)
【Stress-test gaps】theory-design / measurement / assignment / nesting / generalization / transparency
【Preregistered?】confirmatory vs exploratory split
【Next】jpart-data-analysis

Supplementary resources

• ../../resources/external_tools.md — experiment/causal packages (R/Stata/Python) and survey platforms
• ../../resources/code/ — reproducible DiD/IV/RDD/DML skeleton to adapt
• ../../resources/official-source-map.md — preregistration / blinded pre-reg report policy