This skill is based on the causal inference logic of King, Keohane & Verba (1994), the threat-to-validity framework of Campbell & Stanley (1963), and the review standards of sociological empirical research. It systematically stress-tests research conclusions to help researchers identify and address weaknesses in their arguments before submission.
Core stance: A conclusion that withstands scrutiny is one that still holds after being vigorously refuted, not one that has never been refuted. Actively seeking out competing explanations and addressing them is far more valuable than waiting for reviewers to point them out.
Getting started: Gather the necessary information
Once triggered, collect the following information:
Required 1: Core conclusion or argument
The conclusion the user wants to stress-test, as specific as possible, for example:
"Platform workers, through the 'time freedom narrative,' internalized the platform's transfer of responsibility for labor time management; this is a subjectivation, a subordination of subjectivity, rather than mere ideological indoctrination."
Required 2: Main evidence supporting the conclusion
Briefly explain what data/materials the conclusion is based on:
- Qualitative research: number of interviewees, types, core data sources
- Quantitative research: sample size, variable operationalization, core statistical results
Optional 3: Overview of the research design
- Type of research method
- Case selection strategy
- Data collection method
Optional 4: Weaknesses the user is already aware of
Potential problems the researcher has already noticed (address these first).
Optional 5: Target journal
Different journals have different review cultures, which can influence the focus of the stress test.
Execution process
After receiving the information,automatically and continuously executethe following four stages, without waiting for user confirmation at each step.
Stage One: Dissecting the conclusion
Before the stress test, first precisely parse the logical structure of the conclusion:
- Core claim: The conclusion claims that A causes (or explains) B, where what are A and B respectively?
- Causal direction: Is it one-way causation? Two-way? Or a conditional relationship?
- Scope of application: What kinds of situations, populations, and time periods does the conclusion explicitly or implicitly apply to?
- Mechanism description: What mechanism does the conclusion claim is happening behind the scenes? (Not just "A→B," but "A leads to B through process X")
- Evidence-conclusion gap: How far does the existing evidence directly support? How large an inferential leap is made from the evidence to the conclusion?
Output format：
【结论解剖】
核心主张：A = [ ]，B = [ ]
因果/解释方向：
适用范围（显性）：
适用范围（隐性预设）：
声称的机制：
证据实际覆盖到：[具体到哪里]
推论跨度评估：小 / 中 / 大
Stage Two: Enumerating competing explanations
Systematically enumerate all alternative explanations that may compete with the current conclusion. Search one by one across the following five categories of sources:
Source 1: Confounding Variables
Is there a third variable that simultaneously affects the causal variable and the outcome variable, making the observed relationship spurious?
For the specific context of the study, propose 2-4 most likely candidate confounding variables, and explain:
- How does this variable simultaneously affect both ends?
- Did the study design control for this variable?
- If it was not controlled for, how much is the robustness of the conclusion threatened?
Source 2: Reverse Causation
Is it possible that B is the cause of A rather than the result? Or are the two mutually causal?
Explain the possible paths of reverse causation, and assess:
- Is the temporal order sufficiently clear?
- Are there instrumental variables or other causal identification strategies?
Source 3: Selection Bias
Is the study's sample/case systematically inclined toward certain characteristics, causing the conclusion to be unrepresentative?
Check:
- How were samples selected into the study? Is there endogeneity in the selection process?
- The "silent majority"—what systematic differences would the subjects who did not enter the sample have from the sample?
- In interview research: are there systematic differences between those who agree to be interviewed and those who refuse?
Source 4: Observer Effects
Did the researcher's presence or the research process itself change the phenomenon being studied?
This is especially important in qualitative research:
- Did respondents present behavior or narratives different from their daily ones in front of the researcher?
- Did respondents guess the researcher's expectations and cater to them?
- Reflexivity: how did the researcher's identity and standpoint influence data collection and interpretation?
Source 5: Alternative Mechanisms
Even if the A→B relationship holds, are there mechanisms different from the current explanation that could also produce the same result?
This is the most challenging category: competition among mechanisms. List 2-3 alternative mechanisms, and for each alternative mechanism explain:
- What is the logic of this alternative mechanism?
- Would it produce the same observable pattern as the current conclusion?
- What kind of evidence could distinguish these two mechanisms?
Output Format(for each type of competing explanation):
[竞争性解释类型：混淆变量/反向因果/选择偏差/观察者效应/替代机制]
具体威胁描述：
威胁的严重程度：高 / 中 / 低
当前研究的处理情况：已处理 / 部分处理 / 未处理
如果未处理，对结论可信度的影响：
可能的回应策略：
Phase 3: Scope-of-Conclusion Test
Stress-test the applicable boundaries of the conclusion to prevent overgeneralization:
Test 1: Case Representativeness
Can the cases/samples chosen by the researcher support the claimed degree of generality?
- If the conclusion claims to "generally hold in context X," which subtypes of X does the current evidence cover?
- Are there clearly omitted subtypes, and could those subtypes produce different results?
Test 2: Temporal Robustness
Does the conclusion hold at a specific historical moment, or does it have a certain temporal robustness?
- What is special about the temporal background of the data collection?
- If this study were repeated at another point in time, how might the conclusion change?
Test 3: Context Transfer
To what extent can the conclusion transfer to other contexts?
- Which contextual features are crucial to the conclusion (change these features and the conclusion no longer holds)?
- Which contextual features do not affect the conclusion (these constitute proof of the conclusion's robustness)?
Output Format：
【结论范围检验】
当前证据实际支撑的范围：[比结论声称的范围更窄？还是基本匹配？]
遗漏的重要子类型：
时间特殊性风险：
情境迁移的边界条件：
建议的结论措辞调整：[将当前过宽的结论收窄至证据能支撑的范围]
Phase 4: Simulating reviewer challenges
Simulate 2-3 reviewers with different stances, raise the most likely core challenges, and provide suggested responses:
Reviewer A: Methodological rigorist
Focus: validity of the research design, rigor of causal inference, reasonableness of sampling
"[Simulated challenge content]"
Suggested response strategy:
Reviewer B: Theoretical dialogue advocate
Focus: whether the dialogue with existing literature is accurate, originality of the theoretical contribution, handling of alternative theories
"[Simulated challenge content]"
Suggested response strategy:
Reviewer C: Empirical domain expert
Focus: whether the understanding of the research context is accurate, whether data interpretation aligns with domain common sense
"[Simulated challenge content]"
Suggested response strategy:
Phase 5: Comprehensive recommendations
Based on the above analysis, provide:
- The 2-3 weaknesses that most need to be proactively addressed(ordered by degree of threat)
- Issues that can be handled through textual rhetoric(acknowledging limitations, narrowing the scope of conclusions, explaining the rationale for research design choices)
- Issues that require additional data or analysis to truly resolve(if any)
- Recommendation on conclusion strength: Is the current conclusion too strong? What strength is it recommended to revise it to?
After all output, output the following fixed text:
The purpose of stress testing is to strengthen conclusions, not to destroy them. Please have the researcher judge and inform us of the following questions:
- Which competing explanation do you think poses the greatest threat? Do you have any response materials you have not yet told me about?
- Do you agree with the recommendations for adjusting the scope of the conclusions?
- Is there any conclusion strength you deliberately maintained rather than overlooked? If so, explain the reason.
References
- King, G., Keohane, R. O., & Verba, S. (1994). Designing Social Inquiry: Scientific Inference in Qualitative Research. Princeton University Press.
- Campbell, D. T., & Stanley, J. C. (1963). Experimental and Quasi-Experimental Designs for Research. Houghton Mifflin.
- Mahoney, J. (2010). After KKV: The new methodology of qualitative research. World Politics, 62(1), 120–147.
- Seawright, J. (2016). Multi-Method Social Science: Combining Qualitative and Quantitative Tools. Cambridge University Press.
Language
- Default Chinese
- If the user inputs in English, output in English, and keep frontmatter field names in English