Measure the quality of a technical article from the perspective of "whether readers can reproduce the same thing on their own." This is a separate axis independent of style evaluation (mizchi-blog-style) and logical evaluation.The most important thing in a technical article is whether the reader who read it can reproduce it on their own It stands on this premise.
When to use it
- Final check before publishing a technical article draft
- Hands-on articles / tutorial articles
- Tool introduction articles / setup articles
- Verification of articles that claim "it worked"
Situations not to use it:
- Concept explanation articles (there is nothing to reproduce)
- Poems / opinion articles
- Small tips that are self-contained within the article
Reproducibility check perspectives (10 axes)
Score each axis from 0 to 2 points, total 20 points maximum → converted to 10 points.
| # | Axis | 0 (NG) | 1 (partial) | 2 (OK) |
|---|---|---|---|---|
| 1 | Explicitness of environment prerequisites | No OS / version / required tools listed | Partially listed | All listed (OS, lang version, CLI tools) |
| 2 | Completeness of code | Fragments only, import/setup omitted | Main parts only | Complete form that works by copy-paste |
| 3 | Accuracy of commands | left as placeholder (<your-token> etc. with no explanation) |
Some placeholders | Executable as-is |
| 4 | Explicitness of version dependencies | No mention | Partial | Explicitly states e.g. "works with v3.x", "v2 and earlier is X" |
| 5 | Full posting of config files | Excerpts only | Main keys only | Full minimal working configuration |
| 6 | Presentation of expected output | none | Explained in prose | Actual output / screenshots |
| 7 | Handling of errors | Not addressed | Mentions 1 case | Several major errors + handling |
| 8 | Explicitness of project prerequisites | Author's environment prerequisites are implicit | Partially explicit | path / repo structure / existing settings all explicit |
| 9 | Link health | Broken links or authentication required | Some require authentication | All public and accessible |
| 10 | Explicitness of author-dependent knowledge | Helpers / dotfiles implicit | Partially explicit | All explicit or unnecessary |
Evaluation workflow
Use the same subagent dispatch as empirical-prompt-tuning to evaluate technical articles. The difference is that the subagent is given not the role of an "implementer" but the role-play of "a first-time reader attempting to reproduce it" to be made to do.
- Finalize the target article
- subagent dispatch (template described below)
- Extract "reproduction stumbling points" from the returned evaluation
- Add / revise the descriptions in the article corresponding to the stumbling points
- If necessary, re-evaluate with a new subagent
subagent dispatch template
あなたは <記事のテーマ領域> に興味があるが <技術スタック> は初めての読者です。
この記事を読んで、手元の環境で同じことを再現しようとします。
## 対象記事
<記事ファイルのパス>
## 評価観点 (再現性 10 軸)
各軸を 0〜2 点で採点。`tech-article-reproducibility` skill の判定表参照:
/Users/mz/.claude/skills/tech-article-reproducibility/SKILL.md
1. 環境前提の明示
2. コードの完全性
3. コマンドの正確性
4. バージョン依存の明示
5. 設定ファイル全文掲載
6. 期待される出力の提示
7. エラー時の対処
8. プロジェクト前提の明示
9. リンク健全性 (実際に WebFetch で確認)
10. 著者依存知識の明示
## タスク
1. 記事を読みながら「自分が手元で再現するならどこで詰まるか」を想像する
2. 各軸 0〜2 で採点 + 根拠を引用
3. 詰まりポイント Top 5 を行番号付きで列挙
## レポート構造
- 再現性スコア: X/20 (内訳テーブル)
- 詰まりポイント Top 5: <行番号> <引用> → <なぜ詰まるか>
- 不足情報: 記事に追加すべき情報のリスト
- 総評: この記事を読んで手元で再現できる確率は何 % か (主観)
How to read the score
- 18-20: Publishable as a hands-on, almost no additional information needed
- 14-17: Requires some googling but is reproducible, publishable
- 10-13: Requires information outside the article to reproduce, additional revisions recommended
- 9 or below: Difficult to reproduce, reconsider the article's premises or position it as something other than a hands-on
Pitfalls
- The evaluator's prior knowledge is too advanced: If you don't explicitly give the subagent an "beginner role," it will judge from an expert's perspective that it is "sufficient." Emphasize "first-time" in the prompt
- Ignoring link health: A link may be alive at publication time but break a year later.Alive Separately check whether reproduction is possible using only links
- Inlining all sample code: Reproducibility improves, but the article becomes bloated. A hybrid approach combining this with links to a repository is realistic
- Reproducibility ≠ writing quality: There are articles that are highly reproducible but hard to read.
mizchi-blog-styleUse in combination with etc. to measure on both axes
Related
empirical-prompt-tuning— the meta-skill of subagent dispatch + iterative improvementmizchi-blog-style— evaluation on the writing-style axis (an axis independent of this skill)