statistical-analysis

Statistical Analysis Guide

This skill provides guidance on the selection, execution, and reporting of statistical analyses in academic papers.

I. Selection of Statistical Tests

Comparing Two Groups

Data Characteristics	Recommended Test
Independent, continuous, normal	Independent samples t-test
Independent, continuous, non-normal	Mann-Whitney U test
Paired, continuous, normal	Paired samples t-test
Paired, continuous, non-normal	Wilcoxon signed-rank test
Binary outcome	Chi-square test or Fisher's exact test

Comparing Three or More Groups

Data Characteristics	Recommended Test
Independent, continuous, normal	One-way ANOVA
Independent, continuous, non-normal	Kruskal-Wallis test
Paired, continuous, normal	Repeated measures ANOVA
Paired, continuous, non-normal	Friedman test

Relationship Analysis

Analysis Objective	Recommended Method
Relationship between two continuous variables	Pearson correlation (normal) or Spearman correlation (non-normal)
Continuous outcome and predictor variables	Linear regression
Binary outcome and predictor variables	Logistic regression

II. Hypothesis Testing

Normality Test

from scipy import stats

# Shapiro-Wilk检验（样本量<5000）
stat, p_value = stats.shapiro(data)
print(f"Shapiro-Wilk检验: W={stat:.4f}, p={p_value:.4f}")

if p_value > 0.05:
    print("数据符合正态分布假设")
else:
    print("数据不符合正态分布，考虑使用非参数检验")

Homogeneity of Variance Test

from scipy import stats

# Levene检验
stat, p_value = stats.levene(group1, group2)
print(f"Levene检验: F={stat:.4f}, p={p_value:.4f}")

if p_value > 0.05:
    print("方差齐性假设满足")
else:
    print("方差不齐，使用Welch's t检验")

III. Effect Size

Common Effect Size References

Test	Effect Size	Small	Medium	Large
t-test	Cohen's d	0.20	0.50	0.80
ANOVA	η²_p	0.01	0.06	0.14
Correlation	r	0.10	0.30	0.50
Regression	R²	0.02	0.13	0.26

Calculating Effect Size in Python

import pingouin as pg

# t检验返回Cohen's d
result = pg.ttest(group1, group2)
d = result['cohen-d'].values[0]
print(f"Cohen's d = {d:.2f}")

# ANOVA返回偏η²
aov = pg.anova(dv='score', between='group', data=df)
eta_p2 = aov['np2'].values[0]
print(f"Partial η² = {eta_p2:.3f}")

IV. APA Format Reporting

Independent Samples t-test

A组（n = 48, M = 75.2, SD = 8.5）得分显著高于B组
（n = 52, M = 68.3, SD = 9.2），t(98) = 3.82, p < .001, 
d = 0.77, 95% CI [0.36, 1.18]。

One-way ANOVA

单因素方差分析显示处理条件对测试分数有显著主效应，
F(2, 147) = 8.45, p < .001, η²_p = .10。事后比较使用
Tukey HSD表明，条件A（M = 78.2, SD = 7.3）得分显著
高于条件B（M = 71.5, SD = 8.1, p = .002）。

Multiple Regression

多元线性回归预测考试成绩，整体模型显著，
F(3, 146) = 45.2, p < .001, R² = .48。学习时间
（β = .35, p < .001）和先前GPA（β = .28, p < .001）
是显著预测变量。

V. Common Statistical Pitfalls

1. P-hacking: Do not test multiple ways until significance emerges
2. HARKing: Do not present exploratory findings as confirmatory
3. Ignoring assumptions: Check and report violations
4. Confusing significance with importance: p < .05 ≠ a meaningful effect
5. Failing to report effect size: Crucial for interpretation
6. Cherry-picking results: Report all planned analyses
7. Multiple comparisons: Correct familywise error rate when appropriate
8. Overinterpreting non-significant results: Absence of evidence ≠ evidence of absence

VI. Python Examples

Complete t-test Procedure

import numpy as np
import pingouin as pg
from scipy import stats

# 数据
group_a = np.array([75, 82, 68, 79, 85, 72, 88, 76])
group_b = np.array([65, 70, 62, 68, 75, 60, 72, 66])

# 1. 描述统计
print(f"A组: M={group_a.mean():.2f}, SD={group_a.std():.2f}")
print(f"B组: M={group_b.mean():.2f}, SD={group_b.std():.2f}")

# 2. 正态性检验
_, p_a = stats.shapiro(group_a)
_, p_b = stats.shapiro(group_b)
print(f"正态性: A组 p={p_a:.3f}, B组 p={p_b:.3f}")

# 3. t检验
result = pg.ttest(group_a, group_b)
print(f"t = {result['T'].values[0]:.2f}")
print(f"p = {result['p-val'].values[0]:.4f}")
print(f"Cohen's d = {result['cohen-d'].values[0]:.2f}")

VII. Statistical Analysis Checklist

• Define the research question and hypotheses
• Determine the appropriate statistical test
• Conduct a power analysis to determine sample size
• Check for missing data and outliers
• Verify assumptions (normality, homogeneity of variance)
• Run the primary analysis
• Calculate effect sizes and confidence intervals
• Conduct post hoc tests (if needed)
• Write the results in APA format

VIII. Recommended Resources

Python Libraries

• scipy.stats: Core statistical tests
• statsmodels: Advanced regression and diagnostics
• pingouin: User-friendly statistical tests with effect sizes