SuperAPI AI Directory

er-reproducibility

Writing & research

View repository

Updated 10-15 20:08

About this skill

Reproducible research package (er-reproducibility)

When to use

  • The empirical work is done, and you are preparing to organize the code and data into a form that "others can rerun"
  • A referee or editor asks you to "provide code for verification", while your scripts are currently a mess
  • You receive an R&R and need to reproduce your own results from three months ago in a clean environment, but cannot get them to run
  • After acceptance, the editorial office asks you to submit supplementary materials (data + code)

Why it matters

  • 《经济研究》referees / editors willverify the code and data: if the tables do not match or the code does not run, this directly becomes a major pitfall at the revision stage, or even leads to retraction.
  • Poor reproducibility → at the R&R stage, every change requires manual recalculation from scratch, with huge time costs.
  • The reality is:the first person who cannot reproduce your results a few months later is yourself. A reproducible package is first and foremost for your future self.

Standard directory structure

One root directory per project, raw data read-only, code layered, outputs separated from code:

project/
  ├── data/raw/        原始数据(只读,永不手改)
  ├── data/clean/      清洗后数据(由代码生成,可删可重建)
  ├── code/
  │   ├── 00_master.do     一键运行,顺序调用以下脚本
  │   ├── 01_clean.do      清洗:原始 → clean
  │   ├── 02_descriptive.do 描述性统计 / 平衡性
  │   ├── 03_baseline.do   基准回归
  │   ├── 04_mechanism.do  机制检验
  │   ├── 05_robustness.do 稳健性
  │   └── 06_figures.do    出图
  ├── output/tables/   表格(由代码生成)
  ├── output/figures/  图形(由代码生成)
  └── README.md        运行说明 + 数据可得性声明

Principle:any file in output should be regenerable by code. Manual artifacts do not go into output.

master do-file template

Define the root path only once in master, use relative paths throughout, fix versions and seeds:

*============================================================
* 00_master.do —— 一键复现正文全部表与图
*============================================================
clear all
set more off
version 17                          // 声明 Stata 版本,便于他人对齐
set seed 20240101                  // 全局随机种子,bootstrap/安慰剂可复现

* 根路径:他人只需改这一行
global root "/Users/yourname/project"
global data  "$root/data"
global code  "$root/code"
global out   "$root/output"

* 按顺序运行,从原始数据到最终图表
do "$code/01_clean.do"
do "$code/02_descriptive.do"
do "$code/03_baseline.do"
do "$code/04_mechanism.do"
do "$code/05_robustness.do"
do "$code/06_figures.do"

display "复现完成:见 $out/tables 与 $out/figures"

Script writing standards

  • One script, one task: keep cleaning, description, regression, and plotting separate; do not cram 8,000 lines into one file.
  • Full-chain traceability: from data/raw to output/tables every step is in the code, with no manual breakpoints.
  • Variable construction is commented: each generated variable notes itssource and formula, for example:
    * 资产负债率 = 总负债 / 总资产(CSMAR 财务报表,FI_T1.B001000000 / FI_T1.A001000000)
    gen lev = total_debt / total_asset
    
  • Winsorizing, merging, and sample screening are all documented in the code:
    winsor2 lev roa, cuts(1 99) replace   // 上下 1% 缩尾
    drop if year < 2008 | year > 2022     // 样本期:2008–2022,注明理由
    
  • Relative path references $data/$out,do not hard-code absolute paths(absolute paths appear only once in master’s global root ).

Data availability statement

Commercial databases (CSMAR / Wind / tax survey, etc.)cannot be made directly public, so the README needs a "data availability statement" explaining the source, access method, and what can be shared versus what is restricted:

## 数据可得性声明
本研究数据来自以下来源:
- 上市公司财务数据:CSMAR 数据库(订阅获取,受许可协议限制,不可公开再分发)。
- 城市层面控制变量:《中国城市统计年鉴》(公开出版物,已随包提供清洗后子集 data/clean/city_panel.dta)。
- 政策清单:作者手工整理自地方政府公开文件,随包提供 data/raw/policy_list.xlsx。
可共享:清洗代码(code/)、公开来源数据、变量构造说明。
受限:CSMAR 原始明细不可公开;复现者须自行订阅并按 01_clean.do 注明的字段重建。
联系方式:[作者邮箱],可协助说明字段口径。

Do not write vague statements such as "data come from public sources".

Randomness and reproducibility

  • Any step involving randomness (bootstrap, placebo permutation, ML tuning, random sampling)must set seed, and the seed must be set uniformly in master.
  • Reportsoftware versions and key command versions: Stata version (version 17), dependent user-written commands and versions (such as csdid / reghdfe / winsor2); you can list the version lines of which csdid in the README.
  • When parallel / multicore computing is involved, note that: changes in the number of cores may alter the random number stream, so the runtime environment must be specified in the documentation.

Must-check list

  • The directory is layered as data/raw, data/clean, code, output, with raw read-only
  • exists 00_master.do, run all scripts sequentially with one click
  • All paths relativized, absolute paths appear only once in master
  • Each generated variable has source and formula comments; winsorizing / merging / filtering leave traces in the code
  • All random steps set seed, seed set uniformly in master
  • README contains a data availability statement, distinguishing shareable / restricted
  • README lists software and key command versions
  • In aclean environmentreproduce with one click from 00_master.do , able to reproduce every table / figure in the main text
  • Table / figure numbering in output is fully consistent with main text citations
  • After acceptance, prepare supplementary materials (data + code) according to editorial office requirements, subject to the official website at the time of submission

Note: After acceptance by 《经济研究》, supplementary materials must be submitted according to editorial office requirements—appendix (Word), data files matching the statistical software or Excel, software code files, and (if any) rights holder restriction statements. Whether to submit a complete replication package / README is subject to the official website at the time of submission and editorial office notification; this skill does not replace official requirements.

Anti-patterns

  • Manually changing numbers in Excel → broken chain, untraceable
  • Hardcoded absolute paths, everything errors on another computer
  • No master, relying on "remember which to run first and which next"
  • Random tests (bootstrap / placebo) without a seed, different results every time
  • Cleaning and analysis mixed in one 8000-line script, changing one place affects everything
  • Final main text tables arehand-copiedfrom logs into Word
  • Data availability says "data from public sources", reviewers cannot judge the definition

Output format

【目录结构】合规 / 待整理:[...]
【master】存在并一键可跑 / 缺失
【路径】全相对 / 发现写死绝对路径:[...]
【随机种子】全部已设 / 缺失:[...]
【数据声明】完整(区分可共享 / 受限)/ 含糊待改
【干净环境复现】通过 / 失败:[哪张表跑不出]
【版本记录】已列 Stata 与命令版本 / 缺失
【下一步】整理完成 → er-submission 做投稿前 preflight 与附件清单

References

  • Citations use the author–year system; in-text citations such as (Gentzkow and Shapiro, 2014), not the [1][2] numbered system.

Supplementary resources