paper-replication

Deep Learning Paper Replication Skill

Read a deep learning paper (PDF or text) and transform the mathematical descriptions, network structures, and implementation details in the paper into runnable, verifiable, maintainable PyTorch code and accompanying documentation.

Core Objective

• First understand the paper's core problem, innovations, mathematical formulas, and hyperparameters.
• Then use Mermaid to visualize the key data flow and tensor shapes.
• Finally output an industrial-grade PyTorch implementation, and generate module-level technical documentation when needed.

Workflow

Choose the appropriate execution mode based on user needs and paper complexity:

• Full mode(default): Execute all 4 Phases, suitable for complete reproduction of complex models.
• Quick mode: When the user explicitly only needs code, skip Phase 4 (documentation generation).
• Code-first mode: When the user already understands the paper, simplify the deconstruction in Phase 1 and focus on executing Phase 3.

For simple models (such as MLP, single-layer networks), automatically simplify:

• The Mermaid diagram in Phase 2 uses a simplified version.
• The documentation structure in Phase 4 is trimmed as needed, and not all subdirectories are forcibly generated.

Phase 1: Paper audit and architecture deconstruction

Output a minimal but complete deconstruction result:

• Paper overview: One-sentence problem definition + 2-4 core innovations.
• Mathematical principles: Core formulas, symbol explanations, Loss Function analysis.
• Architecture details: Number of layers, number of heads, dimensions, convolution parameters, dropout, activation functions, normalization methods.
• Implementation gaps: Separately mark implementation details not explicitly stated in the paper; do not disguise them as the original text of the paper.

Recommended output skeleton:

## 论文概述
**问题 (Problem)**: ...

**核心创新 (Contribution)**:
1. ...
2. ...

## 数学原理
### 核心公式
$$ ... $$

### Loss Function
$$ ... $$

## 架构细节
| 组件 | 参数 | 值 | 来源 |
|------|------|-----|------|

Phase 2: Architecture flow visualization

Use Mermaid to show the paper's data flow, covering at least:

• Input and output nodes.
• Key paths in Backbone / Encoder / Decoder / Neck / Head / Loss.
• Input/Output Tensor Shape of key nodes, uniformly using [B, C, H, W]、[B, N, D] and other standard symbols.
• Use module-level granularity for deep networks; do not mechanically expand all layers.

Color specification:

• Blue #e1f5fe: Input/output nodes.
• Orange #fff3e0: Core processing modules.
• Green #e8f5e9: Feature extraction or caching modules.
• Pink #fce4ec: Attention mechanisms.
• Purple #f3e5f5: Loss or aggregation nodes.

Simple models may use simplified diagrams; for complex models, prioritize drawing the module relationship diagram, then supplement it with a tensor shape change diagram.

Phase 3: PyTorch implementation

Follow these points when implementing code:

• Inherit torch.nn.Module, and add clear docstrings to core classes.
• Strictly use type hints.
• Explicitly annotate tensor shape changes after key operations.
• Split complex submodules into independent classes or modules/ subfiles.
• Provide _init_weights or equivalent initialization logic, and note the source.
• Provide set_seed or an equivalent reproducibility configuration.
• At the end of the file, include minimal runnable verification code: forward pass, shape check, parameter count check, and gradient flow check.
• For hyperparameters not specified in the paper, explicitly mark Paper did not specify, and also state the recommended value, reference source, alternative values, and potential impact.

For the complete coding standards, verification templates, and Paper did not specify protocol, see references/CODING_STANDARD.md。

Phase 4: Module documentation generation

Generate technical documentation for each core module, including:

• Overview: functionality, architectural position, input/output specifications.
• Mathematical principles: formulas, symbol explanations, and derivations when necessary.
• Data flow diagram: Mermaid flowchart + tensor shape changes.
• Implementation details: key code, hyperparameters, design decisions.
• Usage examples: invocation methods and input/output examples.
• Notes: common issues, performance considerations, and differences from the paper.

For detailed documentation standards and Mermaid templates, see references/DOCUMENTATION_GUIDE.md。

Tensor Shape Notation Standard

The entire reproduction process uniformly uses the following dimension symbols:

Symbol	Meaning
B	Batch size
C	Channels
H, W	Height / Width
T	Time or sequence length
D	Feature dimension
N	Number of elements / tokens
E	Embedding dimension
K	Kernel size

The comment format is unified as:

# x: [B, C, H, W] <- Input tensor
# x: [B, 64, H//2, W//2] <- Conv2d(C, 64, k=3, s=2, p=1)

Output Structure

Adapt the output structure according to model complexity:

Simple model(can be expressed in a single file, such as an MLP or simple CNN):

output/
├── model.py
└── README.md

Medium model(requires splitting into submodules):

output/
├── model.py
├── modules/
└── README.md

Complex model(full reproduction, such as a Transformer or detection network):

output/
├── README.md
├── docs/
├── model.py
├── modules/
├── config.py
└── requirements.txt

If the paper has three clear boundaries for training, inference, and configuration, you may further split out train.py、infer.py、configs/, but do this only when the complexity truly requires it.

Reading Strategy

• Read the main paper first, then read the Appendix or Supplementary as needed.
• Prioritize textual descriptions, formulas, and pseudocode, then use figures and tables for cross-validation.
• If official code exists, prioritize using the official code to correct ambiguities in the text, and explain the differences in comments.
• For separate Supplementary Material / Appendix files, prioritize reading the main paper, then supplement by looking for detailed hyperparameters, additional ablations, and complete pseudocode; for any implementation details from supplementary materials, mark Reference: Supplementary Section X。

For more PDF reading points and common pitfalls, see references/COMMON_PITFALLS.md。

Constraints

Tool Stack

• Python 3.10+
• PyTorch 2.0+
• Mermaid
• einops(optional)
• timm(optional)

Language Rules

• Use Simplified Chinese for analysis and explanation.
• Use English for code comments.
• Use English for variable naming and follow PEP 8.

Verification Checklist

Before finishing, confirm:

Phase 1

• The core problem and innovations are clear.
• Key formulas, loss functions, and symbol explanations have been extracted.
• The hyperparameter configuration is complete, and unspecified items are explicitly marked.

Phase 2

• The Mermaid diagram covers the main components.
• Key nodes are annotated with Tensor Shapes.
• Appropriate granularity is used for deep networks.

Phase 3

• The implementation inherits nn.Module。
• Type hints are complete.
• Dimension comments follow key layers.
• Submodule boundaries are clear.
• Initialization, random seeds, and verification code are complete.

Phase 4 (if enabled)

• Each core module has corresponding documentation.
• The documentation covers overview, mathematics, flowchart, implementation, examples, and notes.
• The document structure matches the model complexity, without over-generating.

Runtime Validation

• Random Tensor forward propagation works normally.
• Output Shape matches expectations.
• Parameter count is basically consistent with the paper or official implementation (if comparable).
• Gradient flow check passed.
• Key implementation pitfalls and framework differences have been investigated.

References

• references/CODING_STANDARD.md When to read: before starting to write code, or when needing to complete the verification template.
• references/DOCUMENTATION_GUIDE.md When to read: when needing to generate docs/, module documentation, or Mermaid templates.
• references/COMMON_PITFALLS.md When to read: when reading PDFs, comparing papers with code, handling framework differences, or checking for missing details.