Alterlab ssci design gate

Design Gate — Pin the Identifying Assumption Before Anything Else

Skill type: DISCIPLINE-ENFORCING. This is the entry point of the social-science methods spine, not an analysis engine. It routes a study to the right design family and forces the one decision every downstream gate depends on: what identifying assumption licenses the causal claim? It does not run models — for execution it hands off to the analysis skills.

The Core Rule

A CAUSAL CLAIM IS ONLY AS GOOD AS ITS IDENTIFYING ASSUMPTION — NAME IT FIRST.

Design is chosen by the question and the data-generating process — who was selected, who was assigned, what varies and when — not by which method is fashionable or convenient. A quasi-experiment is an observational design that earns a causal interpretation only by committing, up front, to an assumption that makes the effect identified. State that assumption before touching an estimator; if you cannot defend one, the claim is associational, not causal.

When to Use This Skill

Trigger the design gate whenever a design is being chosen, defended, or implied by a claim:

• "What research design should I use for this question?"
• "I have observational survey data and want to claim X improves Y." (← pin the assumption)
• "Should this be an experiment or a quasi-experiment?"
• "Can a difference-in-differences / IV / regression-discontinuity design answer this?"
• "Is my before/after comparison enough to claim the program worked?"

Does NOT Trigger

Route these adjacent requests to the real sibling skill. This gate picks the design and pins the assumption; it does not execute, choose the test, or write.

The request is really about…	Route to	Why not this skill
Running the analysis / a specific model once the design is fixed	alterlab-statistical-analysis / alterlab-statsmodels	Execution, not design choice.
Which statistical test to use (t-test vs Mann-Whitney, etc.)	alterlab-test-selection-guard	Test choice is downstream of design.
Deep qualitative design (grounded theory, phenomenology, coding)	alterlab-qualitative-methods	This gate only routes to qual; that skill does it.
Integrating qual + quant strands, joint displays	alterlab-mixed-methods	Mixed-methods design mechanics.
Grading evidence quality, confounders, bias upstream	alterlab-scientific-thinking	Study-validity judgment, not design routing.
Designing the instrument/questionnaire itself	alterlab-survey-design	Measurement instrument, not design family.

The Design Decision Tree

Walk top-down. Each branch is decided by the data-generating process, not the desired claim.

1. Is the CAUSE randomly ASSIGNED by the researcher?
   ├─ YES → TRUE EXPERIMENT (RCT / lab / field experiment)
   │         identifying assumption: randomization → ignorability (assignment ⊥ potential outcomes)
   └─ NO  → 2. Is there exogenous variation you can exploit?
            ├─ a policy/treatment turned on for some units at some time → DIFFERENCE-IN-DIFFERENCES
            │     assumption: PARALLEL TRENDS (treated & control would have moved together absent treatment)
            ├─ an "as-good-as-random" nudge affecting treatment but not the outcome directly → INSTRUMENTAL VARIABLES
            │     assumption: EXCLUSION RESTRICTION + relevance (instrument affects Y only through the treatment)
            ├─ treatment assigned by a threshold on a running variable → REGRESSION DISCONTINUITY
            │     assumption: CONTINUITY of potential outcomes at the cutoff (no manipulation of the score)
            ├─ one unit observed before/after an intervention over many periods → INTERRUPTED TIME SERIES
            │     assumption: the pre-trend + modeled counterfactual would have continued absent the intervention
            ├─ repeated observations, unit/time confounders → FIXED EFFECTS / panel
            │     assumption: no time-varying confounders (selection-on-observables within unit)
            └─ none of the above, adjust for measured confounders only → OBSERVATIONAL / SELECTION-ON-OBSERVABLES
                  assumption: CONDITIONAL IGNORABILITY (no unmeasured confounding) — the weakest, name it as such

3. Is the aim interpretive / theory-building rather than effect estimation?
   ├─ YES → QUALITATIVE design (route to alterlab-qualitative-methods)
   │         "power" is SATURATION / information power, not a sample-size formula (see alterlab-ssci-sampling-gate)
   └─ combine strands → MIXED METHODS (route to alterlab-mixed-methods)

Full branch logic, the five quasi-experimental designs and their threats, and worked routing examples: references/design_decision_tree.md. A stdlib router that prints the design family and its required assumption from your answers: scripts/design_router.py.

"You Buy Credibility by Burning Information"

Every step from a true experiment toward pure observation trades statistical control for an assumption you must defend. Name the trade honestly:

Excuse	Reality
"It's basically an experiment — people just chose their own group."	Self-selection is exactly what randomization removes. This is observational; name the confounding you are assuming away.
"I have before-and-after data, so it's causal."	A single pre/post has no counterfactual. You need parallel trends (DiD) or a modeled counterfactual (ITS) — state which.
"I controlled for everything relevant."	You controlled for what you measured. Conditional ignorability assumes no unmeasured confounders — the strongest, least testable assumption.
"Instrumental variables fix endogeneity."	Only if the exclusion restriction holds. An instrument that affects the outcome through any other path is invalid — defend exclusion, don't assert it.
"Regression discontinuity is quasi-random at the cutoff."	Only if units cannot manipulate the running variable and outcomes are continuous there. Check for sorting/bunching.

The Design Passport (hand-off)

On exit, emit or update the pipeline's YAML Design Passport with: research_question, design_type, identifying_assumption (the named assumption + why it is defensible), and claim_type (causal | associational | descriptive). Downstream, alterlab-ssci-measurement-gate and alterlab-ssci-sampling-gate append to it, and alterlab-ssci-inference-gate audits final claims against design_type + identifying_assumption.

Self-Check Before Advancing

• Is the design chosen by the data-generating process, not the desired conclusion?
• Is exactly one identifying assumption named, with a sentence on why it is defensible here?
• If observational, is the claim explicitly downgraded to associational unless a QED assumption holds?
• For qualitative aims, is "how many" deferred to saturation logic (not a power formula)?
• Is the Design Passport populated for the next gate?

References

• references/design_decision_tree.md — full branch logic, the five QEDs, threats, examples.
• scripts/design_router.py — stdlib router printing the design family + required assumption.

Part of the AlterLab Academic Skills suite.