Skill type: ANALYSIS MODULE. Complex-sample surveys (GSS, ANES, ESS, DHS, Eurobarometer) are drawn with stratification, clustering, and unequal probabilities. Analyzing them as if they were a simple random sample underestimates standard errors and yields falsely narrow CIs and wrong tests. The discipline is design-based inference: a declared design object comes first, every estimate flows through it.
Core Mission
YOU MUST WEIGHT (AND DECLARE STRATA + PSUs) BEFORE QUOTING ANY NUMBER FROM A COMPLEX SURVEY.
When to Use This Skill
- "Give me the weighted % who [X] from ANES/GSS/DHS, with correct standard errors."
- "Why are my survey confidence intervals so narrow?" (← design ignored)
- "Post-stratify / rake my sample to census margins."
- "Fit a logistic regression on this weighted, clustered survey."
Does NOT Trigger
| The request is really about… | Route to | Why not this skill |
|---|---|---|
| Writing questionnaire items / choosing a sampling frame | alterlab-survey-design |
Instrument & sampling design, not weighted analysis. |
| Whether the sample size / frame is adequate | alterlab-ssci-sampling-gate |
Sampling-adequacy gate, upstream. |
| Causal identification (DiD/IV/RDD) | alterlab-causal-inference |
Design-based survey SEs ≠ causal identification. |
| Plain unweighted descriptive/inferential stats | alterlab-statistical-analysis |
No survey design to honor. |
The design-object-first rule
Declare weights + strata + PSU/cluster + FPC before any estimate:
- weights — the inverse-inclusion-probability weight; scales the sample to the population.
- strata — variances are computed within each stratum and pooled. Dropping strata leaves point estimates unchanged but inflates SEs (you lose the variance reduction).
- PSU / cluster — the unit of randomization. If whole districts were sampled, the district is the PSU; lower units are not independent. Declaring the PSU is what corrects the SE upward for the clustering.
- FPC — finite-population correction when the sampling fraction is non-trivial.
Domain (subpopulation) estimation: subset the design object, never filter the data frame first — filtering discards the strata/PSU structure needed for correct domain SEs.
Weight-type discipline
Distinguish design weights (selection probability only) from post-stratification / calibration weights (also correct for sampling error and non-response). One or the other must always be used; report which. "Weight before quoting any percentage."
Variance estimation — support both families
- Taylor linearization — the default analytic method.
- Replicate weights — BRR, Jackknife (JKn), Bootstrap. Use these when the data provider ships replicate weights (many public files do); do not re-derive a design they already replicated.
Verified calls (pinned)
Python — svy (maintained successor to samplics; verified on svy 0.29):
import polars as pl, svy
design = svy.Design(stratum="strata", psu="psu", wgt="wt") # FPC: pop_size=svy.PopSize(psu="N_psu")
sample = svy.Sample(data=pl.from_pandas(df), design=design) # svy takes a polars DataFrame
sample.estimation.mean("trust", deff="wor") # Taylor linearization by default
sample.estimation.mean("trust", by="region") # domain estimation, design kept intact
sample.estimation.prop("trust01") # also .total() .ratio() .median()
sample.glm.fit("trust01", x=["age", svy.Cat("educ")], family="binomial")
Provider-shipped replicate weights: svy.Design(wgt="wt", rep_wgts=svy.JackknifeWgts(prefix="wtrep", n_reps=80)) (also BrrWgts, BootstrapWgts, SdrWgts), then method="replication". svy is
still beta — pin the version and re-check signatures after upgrades. samplics (TaylorEstimator)
still runs but is archived and warns on import; keep it for legacy code only.
R — survey / srvyr (field standard, fully verified):
library(survey)
des <- svydesign(ids = ~psu, strata = ~strata, weights = ~wt, fpc = ~fpc,
data = dat, nest = TRUE)
svymean(~trust, des, deff = TRUE)
svyby(~trust, ~region, des, svymean) # domain estimation (keeps structure)
svyglm(trust01 ~ age + educ, design = des, family = quasibinomial())
# replicate weights when provided:
rep <- svrepdesign(weights = ~wt, repweights = "wtrep[0-9]+", type = "JKn", data = dat)
# calibration:
des2 <- rake(des, sample.margins = list(~agecat, ~sex),
population.margins = list(pop.agecat, pop.sex))
Full Taylor-vs-replicate math, calibration (post-stratification / raking / GREG), and the Python
caveats vs the canonical R recipes: references/design_and_variance.md, references/python_vs_r.md.
Reporting checklist (put in every survey result)
DESIGN: weights (type: design | post-strat/calibrated) + strata + PSU + FPC declared
VARIANCE: Taylor linearization | replicate weights (BRR/JKn/Bootstrap)
N: unweighted N vs weighted population estimate
DEFF: design effect per key estimate (how much the design inflates variance)
DOMAINS: subset of the DESIGN object, not a filtered data frame
References
references/design_and_variance.md— Taylor vs replicate variance, calibration math, DEFF, domain estimation.references/python_vs_r.md— svy (and legacy samplics) caveats and the canonical R survey/srvyr recipes.
Part of the AlterLab Academic Skills suite.