er-robustness

Robustness check system (er-robustness)

When to use

• The main regression is finalized and you are preparing robustness checks, but you can only think of a few moves like "change control variables" and "winsorize"
• Reviewers question identification (omitted variables / measurement error / sample selection), and you do not know which type of test to add
• You have already piled up a large number of robustness tables, but none of them directly address the threat the reviewer worries about most
• You mistakenly treat heterogeneity / mechanism as robustness and write it into the "Robustness Checks" subsection
• For a DID paper, you only plot one event study figure and run the placebo test only once

Core distinction: robustness is neither mechanism nor heterogeneity

Type	Question answered	Operation
Baseline result	What is the main effect	Main regression
Robustness	If the main regression is done a different way, does the conclusion change	Same Y, same core explanatory variable, change the measurement / sample / method
Mechanism	Why does this effect exist	Use a mediator as the dependent variable
Heterogeneity	Under what conditions is the effect stronger	Grouping / interaction

Decision rule:The concluding sentence of a robustness check is always "the core conclusion remains unchanged." If the conclusion of a passage is "the effect is stronger in group A" or "it works through M," then it is not robustness; move it to er-mechanism / er-heterogeneity.

Classification and checklist of robustness checks (organized by identification threat)

Threat 1: Omitted variables / endogeneity

• Change the identification strategy (OLS → IV / DID / RDD; see er-identification)
• Instrumental variables / Bartik instrument, report first-stage F
• Gradually add higher-dimensional fixed effects (individual×year, industry×year, region×year) and more control variables, and examine the stability of the coefficient
• Oster (2019) δ and β bounds（psacalc / oster_bounds): When "observed variables have strong explanatory power and you worry about unobservable selection," report δ (the ratio of unobservable to observable selection required to drive the treatment effect to zero) and the restricted β bound. δ > 1 is a commonly used threshold (the idea of Altonji, Elder & Taber, 2005)
• Placebo test (see the dedicated section below)

Threat 2: Measurement error / variable definition

• Replace the core dependent variable (e.g., different algorithms for TFP: OP / LP / ACF)
• Replace the measurement of the core explanatory variable (continuous vs. discrete, different thresholds, different data sources)
• Different index construction methods (principal components vs. entropy weight vs. simple aggregation)

Threat 3: Sample selection

• Exclude special samples: municipalities directly under the central government, the financial industry, ST/*ST, the IPO year, extreme size
• Different time windows (excluding crisis years / policy preheating period / pandemic years)
• Re-estimate after PSM (redo the main regression within the common support region)
• Heckman two-step method (when there is self-selection into treatment)

Threat 4: Estimation method / standard errors

• Change the clustering level (individual → industry / region / treatment level)
• Two-way clustering (individual + time)
• Bootstrap standard errors
• Use wild cluster bootstrap when there are few clusters (number of clusters < ~40)（boottest / wild_cluster_bootstrap）
• Sensitivity to winsorize / winsorization proportion (1% vs. 5% vs. no winsorization)

Threat 5: Functional form

• Add quadratic term of the core variable / test for nonlinearity (U-shaped, inverted U-shaped)
• Quantile regression (whether the effect is consistent across different positions of the distribution)
• Log vs. level, different standardizations

DID-specific robustness

• Parallel trends event study plot(pre-period coefficients insignificant, confidence intervals containing 0)
• Placebo: randomize treatment timing / treated units, run 500–1000 times, plot the distribution of estimated coefficients, and see whether the true coefficient falls in the tail
• Rule out contemporaneous policy interference (control for dummies of other policies taking effect within the window)
• Heterogeneity-robust estimator (csdid Callaway-Sant'Anna / did_imputation Borusyak et al.) replacing TWFE
• Change the definition of the treatment group (different treatment intensity thresholds / dropping marginal treated units)

Two types of placebo tests

(a) Randomize treatment—fabricate the treatment group or treatment timing, repeat 500–1000 times, and argue that "if the effect is caused by a real policy, random placebos should yield a coefficient distribution centered around 0, and the true coefficient should be an outlier."

(b) Fabricate the outcome variable—use a Y that should not be affected by the policy as the dependent variable and run the main regression, arguing that "if the effect is mechanism-driven rather than data noise, there should be no significant effect on the placebo Y."

Placebo writing template:

为排除[不可观测因素 / 偶然性]驱动结果的可能，本文进行安慰剂检验。
本文随机抽取[处理组 / 处理时点]并重复 N 次估计，图 X 报告 N 次估计的系数（核密度）分布。
结果显示，安慰剂系数集中分布于 0 附近，而本文真实估计值（垂直虚线）位于分布尾部，
表明基准结果并非由随机因素或遗漏变量驱动。

Writing template (one paragraph per robustness check, fixed structure)

为回应[某识别威胁]，本文[具体做法：替换/剔除/更换/加入……]。
结果见表 X，核心解释变量系数为 [数值]，[在 X% 水平显著]，
与基准回归相比方向一致、量级相近（基准为 Y），核心结论不变。

Key points:Every paragraph must report the magnitude, explicitly stating "the magnitude is similar to the baseline Y" rather than vaguely saying "the results remain significant."

The "degree" of robustness checks

• More is not better. The main text focuses on responding tothe 2–4 identification threats reviewers are most likely to question
• Keep the most critical checks in the main text (usually: alternative identification / placebo / replacing core variables / Oster bounds)
• Secondary checks (winsorization proportions, clustering levels, dropping subsamples one by one) go in the appendix, with a one-sentence index in the main text
• The number of robustness tables in the main text is limited (rule of thumb about 3–5 tables/figures, subject to the journal's website at the time of submission); do not let them overshadow the main results

Execution bridge (StatsPAI / Stata MCP)

Run the robustness battery for real, rather than merely listing it. See the full mapping in execution-with-mcp. Economic Research Journal is a top Chinese economics journal; identification credibility is usually the constraint; staggered DID, weak-instrument-robust IV, RDD, and mechanism tests.

• Multiple outcomes / multiple specifications:romano_wolf(stepwise FWER) or benjamini_hochberg, reporting corrected thresholds.
• Omitted variable sensitivity:oster_delta / sensemakr。
• **Inference:** with few clusters use wild_cluster_bootstrap; depending on the dependence structure use twoway_cluster / conley。
• Rerun from a single handle:audit_result(result_id) list missing checks and the corresponding suggest_function。
• Produce tables:etable / did_summary_to_latex generated directly from the handle, without manually copying numbers.

Keep decisive checks in the main text and a detailed battery in the appendix. See the execution chain in JF execution walkthrough。

Must-check list

• Omitted variables / endogeneity category: at least one item done (alternative identification or IV or adding fixed effects + controls)
• Measurement error category: each of the core dependent variable and core explanatory variable measures has been replaced
• Sample selection category: at least one of dropping special samples / different time windows / PSM
• Estimation-method category: sensitivity to changing the clustering level or winsorizing has been done; when there are few clusters, wild cluster bootstrap was used
• A placebo test has been done, and it is in distributional form (500–1000 repetitions), not a single draw
• When the problem of observed-variable selection is severe, Oster's δ and β bounds were reported (δ > 1)
• DID: both an event-study parallel-trends test and replacement of TWFE with a heterogeneity-robust estimator were done
• Each robustness paragraph reported results consistent with the main regression inmagnitude(not just significance)
• Mechanism/heterogeneity labeling was not passed off as robustness

Anti-patterns

• A big stack of robustness checks, but none directly addresses the core identification threat
• Mechanism analysis / heterogeneity analysis is labeled "robustness" to pad the paper
• The placebo is randomized only once and then a conclusion is drawn, without plotting the coefficient distribution
• The conclusion flips when the winsorizing percentage or subsample changes, but this is not discussed, and the favorable version is quietly selected for reporting
• It only says "the results remain significant after adding control variables," and never looks at the coefficient sign and magnitude
• It reports Oster δ < 1 and still claims robustness (δ < 1 means a plausible unobservable selection could overturn the result)
• It uses TWFE for the robustness of a staggered DID, but does not switch to a Callaway-Sant'Anna-type estimator

Output format

【已回应的识别威胁】[遗漏变量 / 测量误差 / 样本选择 / 估计方法 / 功能形式]
【正文稳健性检验】[更换识别, 安慰剂分布, 替换核心 Y, Oster 边界, ...]
【附录稳健性检验】[缩尾敏感性, 聚类层次, 剔除子样本, ...]
【安慰剂类型】随机化处理 / 伪造结果变量 / 两者
【Oster δ】[数值，若适用]
【量级一致性】每段是否都报了与基准的量级对比：是 / 否
【下一步】er-tables-figures