Skill type: DISCIPLINE-ENFORCING. This is the entry point of the social-science methods spine, not an analysis engine. It routes a study to the right design family and forces the one decision every downstream gate depends on: what identifying assumption licenses the causal claim? It does not run models — for execution it hands off to the analysis skills.
The Core Rule
A CAUSAL CLAIM IS ONLY AS GOOD AS ITS IDENTIFYING ASSUMPTION — NAME IT FIRST.
Design is chosen by the question and the data-generating process — who was selected, who was assigned, what varies and when — not by which method is fashionable or convenient. A quasi-experiment is an observational design that earns a causal interpretation only by committing, up front, to an assumption that makes the effect identified. State that assumption before touching an estimator; if you cannot defend one, the claim is associational, not causal.
When to Use This Skill
Trigger the design gate whenever a design is being chosen, defended, or implied by a claim:
- "What research design should I use for this question?"
- "I have observational survey data and want to claim X improves Y." (← pin the assumption)
- "Should this be an experiment or a quasi-experiment?"
- "Can a difference-in-differences / IV / regression-discontinuity design answer this?"
- "Is my before/after comparison enough to claim the program worked?"
Does NOT Trigger
Route these adjacent requests to the real sibling skill. This gate picks the design and pins the assumption; it does not execute, choose the test, or write.
| The request is really about… | Route to | Why not this skill |
|---|---|---|
| Running the analysis / a specific model once the design is fixed | alterlab-statistical-analysis / alterlab-statsmodels |
Execution, not design choice. |
| Which statistical test to use (t-test vs Mann-Whitney, etc.) | alterlab-test-selection-guard |
Test choice is downstream of design. |
| Deep qualitative design (grounded theory, phenomenology, coding) | alterlab-qualitative-methods |
This gate only routes to qual; that skill does it. |
| Integrating qual + quant strands, joint displays | alterlab-mixed-methods |
Mixed-methods design mechanics. |
| Grading evidence quality, confounders, bias upstream | alterlab-scientific-thinking |
Study-validity judgment, not design routing. |
| Designing the instrument/questionnaire itself | alterlab-survey-design |
Measurement instrument, not design family. |
The Design Decision Tree
Walk top-down. Each branch is decided by the data-generating process, not the desired claim.
1. Is the CAUSE randomly ASSIGNED by the researcher?
├─ YES → TRUE EXPERIMENT (RCT / lab / field experiment)
│ identifying assumption: randomization → ignorability (assignment ⊥ potential outcomes)
└─ NO → 2. Is there exogenous variation you can exploit?
├─ a policy/treatment turned on for some units at some time → DIFFERENCE-IN-DIFFERENCES
│ assumption: PARALLEL TRENDS (treated & control would have moved together absent treatment)
├─ an "as-good-as-random" nudge affecting treatment but not the outcome directly → INSTRUMENTAL VARIABLES
│ assumption: EXCLUSION RESTRICTION + relevance (instrument affects Y only through the treatment)
├─ treatment assigned by a threshold on a running variable → REGRESSION DISCONTINUITY
│ assumption: CONTINUITY of potential outcomes at the cutoff (no manipulation of the score)
├─ one unit observed before/after an intervention over many periods → INTERRUPTED TIME SERIES
│ assumption: the pre-trend + modeled counterfactual would have continued absent the intervention
├─ repeated observations, unit/time confounders → FIXED EFFECTS / panel
│ assumption: no time-varying confounders (selection-on-observables within unit)
└─ none of the above, adjust for measured confounders only → OBSERVATIONAL / SELECTION-ON-OBSERVABLES
assumption: CONDITIONAL IGNORABILITY (no unmeasured confounding) — the weakest, name it as such
3. Is the aim interpretive / theory-building rather than effect estimation?
├─ YES → QUALITATIVE design (route to alterlab-qualitative-methods)
│ "power" is SATURATION / information power, not a sample-size formula (see alterlab-ssci-sampling-gate)
└─ combine strands → MIXED METHODS (route to alterlab-mixed-methods)
Full branch logic, the five quasi-experimental designs and their threats, and worked routing
examples: references/design_decision_tree.md. A stdlib router that prints the design family
and its required assumption from your answers: scripts/design_router.py.
"You Buy Credibility by Burning Information"
Every step from a true experiment toward pure observation trades statistical control for an assumption you must defend. Name the trade honestly:
| Excuse | Reality |
|---|---|
| "It's basically an experiment — people just chose their own group." | Self-selection is exactly what randomization removes. This is observational; name the confounding you are assuming away. |
| "I have before-and-after data, so it's causal." | A single pre/post has no counterfactual. You need parallel trends (DiD) or a modeled counterfactual (ITS) — state which. |
| "I controlled for everything relevant." | You controlled for what you measured. Conditional ignorability assumes no unmeasured confounders — the strongest, least testable assumption. |
| "Instrumental variables fix endogeneity." | Only if the exclusion restriction holds. An instrument that affects the outcome through any other path is invalid — defend exclusion, don't assert it. |
| "Regression discontinuity is quasi-random at the cutoff." | Only if units cannot manipulate the running variable and outcomes are continuous there. Check for sorting/bunching. |
The Design Passport (hand-off)
On exit, emit or update the pipeline's YAML Design Passport with:
research_question, design_type, identifying_assumption (the named assumption + why it is
defensible), and claim_type (causal | associational | descriptive). Downstream,
alterlab-ssci-measurement-gate and alterlab-ssci-sampling-gate append to it, and
alterlab-ssci-inference-gate audits final claims against design_type + identifying_assumption.
Self-Check Before Advancing
- Is the design chosen by the data-generating process, not the desired conclusion?
- Is exactly one identifying assumption named, with a sentence on why it is defensible here?
- If observational, is the claim explicitly downgraded to associational unless a QED assumption holds?
- For qualitative aims, is "how many" deferred to saturation logic (not a power formula)?
- Is the Design Passport populated for the next gate?
References
references/design_decision_tree.md— full branch logic, the five QEDs, threats, examples.scripts/design_router.py— stdlib router printing the design family + required assumption.
Part of the AlterLab Academic Skills suite.