modeling-paper-rubric-and-model-selector

Scoring-Aligned Paper Structure and Model Selection (Paper Rubric & Model Selector)

Global Workflow Collaboration Constraints (Long-Conversation Anti-Drift)

• This skill must not be used as an isolated entry point. When the user requests a complete paper, Word generation, workflow continuation, or the stage is uncertain, first return to paper-workflow-orchestrator to determine the current S0-S8 stage.
• Before starting or continuing the formal tasks of this skill, you must run:

  python .claude/skills/paper-workflow-orchestrator/scripts/workflow_guard.py --skill modeling-paper-rubric-and-model-selector
  

• If the output is [WORKFLOW FAIL] or the report status != "PASS", stop this skill, and backfill the prerequisite stages according to the failure items in paper_output/qa/workflow_guard_report.json ; do not continue from memory.
• This skill only writes paper_output/ artifacts within its own contract scope; after completion, you must return to paper-workflow-orchestrator to determine the next step, and use context-memory-keeper to record completed artifacts, blockers, and the next step.
• In a long conversation, if the context grows long, the stage is uncertain, or the user invokes skills separately, first run:

  python .claude/skills/paper-workflow-orchestrator/scripts/workflow_guard.py --status
  

  Then read paper_output/qa/workflow_guard_report.json、paper_output/preflight_report.json、paper_output/input_manifest.json、paper_output/results/run_manifest.json and this skill's upstream JSON contract, and continue according to the recommended_skill and next_action in the report.
• Before continuing the workflow, you must treat paper_output/context/workflow_memory.json as a long-term breakpoint record; if its current_step、next_step、recommended_skill and workflow_guard.py --status are inconsistent, the guard report shall prevail.
• After completing this skill's artifacts each time, first return to paper-workflow-orchestrator or run workflow_guard.py --status, then update the workflow memory:

  python .claude/skills/context-memory-keeper/scripts/update_workflow_memory.py
  

  After updating, read paper_output/context/workflow_memory.json / .mdto confirm that the next step and recommended skill have been recorded.

Execution Contract

• Upstream input: preferentially read paper_output/step1/problem_analysis.json。
• Must output:paper_output/plan/model_route.json、rubric_alignment.json、scoring_strategy.md。
• Downstream handoff:quality-assurance-auditor read the model route to generate tasks.json；paper-micro-unit-generator through tasks.json use the model, validation, and scoring fields.
• Recommended next step: if external data is needed, enter authoritative-data-harvester; otherwise enter data-cleaning-and-visualization. For a complete paper goal, return to paper-workflow-orchestrator to determine subsequent stages.
• Failure fallback: if problem_analysis.json is missing, first run problem-doc-model-selector; if this step fails in the complete workflow, QA should fall back to problem_analysis.json。

Goal

Fuse a "score-earning" writing structure with a "problem-aligned and actionable" model selection into a reusable workflow, and output:

• A paper outline that can be directly applied (customized to the problem's questions)
• A scoring-point alignment table (each scoring point corresponds to the location of evidence in your paper)
• Model selection and comparative experiment route (including baseline, improvements, validation, and interpretation)

When to invoke

• When you need to quickly build a paper structure and writing order that meets the scoring criteria
• When you are unsure whether the model is aligned with the problem, overly complex, or insufficiently explained
• When you already have a plan but worry about "answering the wrong question/inconsistent metric definitions/insufficient validation"

Inputs (provide as much as possible)

• Competition name/year/problem number (optional)
• Original problem statement or key requirements (required)
• Your understanding of each question (optional, but strongly recommended)
• Data situation: source, fields, time span, sample size, missing rate (optional)
• Constraints and output: requires prediction/optimization/evaluation/classification/clustering/simulation/scheduling, etc. (required)
• Highlights you want to emphasize: innovation/interpretability/reproducibility (optional)

Input contract (recommended)

Read first before execution:

• paper_output/step1/problem_analysis.json

If this file exists, its questions[] must be used as the sole source of sub-questions; you must not fabricate, merge, or discard sub-questions on your own.

Output contract (recommended)

This skill should generate:

• paper_output/plan/model_route.json: the model route, validation plan, chart evidence, and section placement for each question.
• paper_output/plan/rubric_alignment.json: the mapping of scoring points, evidence forms, and QA rules.
• paper_output/plan/scoring_strategy.md: a scoring closed-loop explanation for humans and Agents to read.

These JSON are project-custom workflow contracts, not platform built-in standards. For detailed rules, see docs/workflow-contracts.md。

Script entry point (recommended)

python .claude/skills/modeling-paper-rubric-and-model-selector/scripts/build_model_route.py

This script will read paper_output/step1/problem_analysis.json, and write the model route and scoring closed loop to paper_output/plan/. If this file does not exist, you should first run problem-doc-model-selector。

External "paper structure prompt" resource (optional)

Supports attaching your pre-prepared paper structure prompt file (or text) to this skill for generation and evaluation.

Layered directory convention (recommended)

• SKILL.md: skill definition and usage instructions
• references/: reusable reference materials and prompt resources (default templates, scoring points checklist, etc.)

Usage methods (choose one)

1. Path reference (recommended)

• Provide at call time:paper_prompt_path
• Rule: If this path is provided, read its content first as the "paper structure prompt"; if not provided, use the default prompt file included with this skill.

1. Direct paste

• Provide at call time:paper_prompt_text
• Rule: If this text is provided, use this text first; otherwise use paper_prompt_path; if neither is available, use the default prompt file.

Call examples

• Method 1 (path):
  • paper_prompt_path: <项目根目录>\paper_output\plan\paper_prompt.md
• Method 2 (paste):
  • paper_prompt_text: <把你的论文结构提示词全文粘贴在这里>

Runtime integration mechanism (recommended reference)

Before generating any of the A/B/C/D outputs,It is recommended to read references/paper_prompt_default.md as a "structural reference":

1. Reference priority: This file serves as a reference for the writing template, butdoes not forcibly blockgeneration. The generated paper outline should correspond as much as possible to the chapter requirements of this file, but the model is allowed to adapt based on the actual context.
2. Anti-forgetting mechanism：
   • It is recommended to mention the core structure of this file in the Prompt (such as a six-paragraph abstract) to guide model generation.
   • If the user provides an additional prompt, the user's requirements take priority, and this file is secondary.
3. Encourage expansion: On the premise of ensuring logical coherence,encouragethe use of academic "all-purpose" phrases (such as "With the development of...", "In summary") to enrich the length and enhance the coherence and sense of volume of the article.

Then use PAPER_PROMPT as a strong constraint and source of writing style, integrating it into:

• The chapter naming, writing order, word count allocation, and style requirements of the paper outline
• The "evidence forms" in the scoring point alignment table (which figures, which tables, which tests)
• Model route and controlled experiment design (especially the requirements for "model validation/robustness/sensitivity/reproducibility")

Backward compatible

When not provided paper_prompt_path / paper_prompt_text , this skill maintains the original default behavior and outputs normally.

Constraints (must be followed)

• Memory Interaction (required):
  • Before starting, check memoryskill.md in External Resources / Literature, if relevant literature exists, it must be incorporated into the "References" section and the background review of the "Model Formulation" part.
  • After completion, update the generated "Paper Outline" and "Model Route" to memoryskill.md。

Outputs (fixed structure)

A. One-page problem alignment

• For each question: input, output, evaluation metrics, key constraints, boundary conditions, feasible validation methods

B. Paper outline (in scoring-friendly order)

Must include and adjust proportions according to the problem:

1. Abstract (Chinese and English as required): problem, methods, results, contributions, keywords
2. Problem restatement: rephrase each question into a computable task in your own words
3. Model assumptions: necessary and defensible, explain the reasonableness and impact of each item
4. Notation and variable descriptions: tabulated, including units and ranges
5. Data description and preprocessing: sources, cleaning rules, missing/outlier handling, reproducible steps
6. Model formulation (subsections by question): objective function/constraints/loss, derivation or structural explanation
7. Solution methods and implementation: algorithm steps, complexity/convergence, parameter settings
8. Results and analysis: main results, visualizations, comparisons, error/benefit explanations
9. Model validation: controlled experiments, sensitivity analysis, robustness testing, extreme scenarios
10. Conclusions and recommendations: provide conclusions and actionable recommendations for each question
11. Limitations and prospects: honest but not self-defeating, point out directions for improvement
12. References: standard citations of the problem statement, data sources, and key methods
13. Appendix: key code, additional figures and tables, supplementary notation (choose according to competition requirements)

C. Scoring point alignment table (centered on "evidence")

Output a table: scoring point → evidence you provide → paper location (chapter/figure/table/experiment). Common scoring point mapping (can be added or removed according to the competition):

• Accurate understanding of the problem: one-page problem alignment + restatement of each question as computable
• Reasonableness of the model: objectives/constraints consistent with the problem statement, assumptions defensible
• Method innovation/improvement: clear improvement points over the baseline, and explain why they are effective
• Credible results: validation, comparison, error analysis, or proof of constraint satisfaction
• Clear expression: consistent notation, self-explanatory figures and tables, conclusions corresponding to each question
• Reproducibility: complete data sources and processing, parameter settings, and algorithm steps

D. Model selection and route (start with the problem, then advance)

For each question, output:

• Task type determination: prediction/classification/clustering/evaluation/optimization/simulation/mechanistic modeling
• Minimum viable baseline: runnable, interpretable, comparable
• One to two improvement routes: improve accuracy/robustness/efficiency/interpretability
• Validation plan: metrics, cross-validation/hold-out/backtesting, ablation, sensitivity
• Risk points: insufficient data, inconsistent definitions, overfitting, unexplainability, computational timeout

Model Selection Quick Reference (by common question types)

1) Prediction (time series/regression)

Applicable: given history, predict the future or estimate parameters.

• Baseline: moving average/exponential smoothing/linear regression/ARIMA (can explain trend and seasonality)
• Improvement: feature engineering + tree models; or LSTM/Transformer (only when sample size is sufficient)
• Validation: rolling backtest, MAPE/RMSE, confidence intervals/error decomposition

2) Classification/Discrimination

Applicable: determine category, risk level, whether it occurs.

• Baseline: logistic regression/naive Bayes (interpretable)
• Improvement: random forest/gradient boosting; cost-sensitive learning (class imbalance)
• Validation: AUC/F1/PR curve, confusion matrix, threshold sensitivity

3) Evaluation/Ranking/Composite Index

Applicable: multi-indicator scoring, ranking, selection.

• Baseline: normalization + weighted sum (weights can come from the problem statement or experts)
• Improvement: entropy weight/CRITIC/AHP/TOPSIS/VIKOR (clarify weight source and meaning)
• Validation: weight sensitivity, ranking stability, comparison with known facts

4) Optimization/Scheduling/Site Selection/Routing

Applicable: resource allocation, cost minimization/benefit maximization, satisfying constraints.

• Baseline: linear programming/integer programming (write objectives and constraints clearly)
• Improvement: multi-objective (weighted/ε-constraint), heuristics (genetic/simulated annealing) for large scale
• Validation: feasibility check, comparison with baseline strategy, constraint violation rate, complexity and time

5) Clustering/Grouping/Profiling

Applicable: unlabeled grouping, pattern discovery.

• Baseline: K-means/hierarchical clustering
• Improvement: GMM/DBSCAN (when noise and shapes are complex)
• Validation: silhouette coefficient/stability, interpretable description of group differences

6) Mechanism/Simulation/System Dynamics

Applicable: emphasize mechanism explanation, scenario deduction.

• Baseline: differential equations/difference equations/system dynamics
• Improvement: parameter calibration + uncertainty analysis; comparison with data-driven models
• Validation: historical fitting, scenario consistency, parameter sensitivity

Hard rules to prevent going off track (must check)

• For each question, provide at least one "quantifiable output" and one "verifiable indicator/test method"
• Every key variable in the model can find its definition and unit in the problem statement/data
• Figures/tables and conclusions correspond question by question; avoid "did a lot but did not answer the question"
• At least one baseline comparison: prove your method improves on or is more reasonable than a simple approach

Final deliverable (I will provide it when you ask me to generate)

• Outline and section titles that can be pasted directly into the paper (numbered by question type)
• Scoring point alignment table (including the list of figures/tables/experiments you should add)
• Model route for each question: baseline + improvement + validation + risks and alternatives

Default prompt file (reusable)

When not provided paper_prompt_path / paper_prompt_text , by default read the paper structure prompt from the following file:

• references/paper_prompt_default.md

Directory conventions (aligned with the project globally)

• Competition problems and attachments are uniformly placed in problem_files/, supplementary data is placed in crawled_data/。
• It is recommended to archive this skill's output (outline/scoring point alignment/model route) to paper_output/plan/, for reference when generating the main text later.
• The above paper_prompts/... is only a historical path example; the current project recommends archiving custom prompt files also to paper_output/plan/, or directly using this skill's built-in references/paper_prompt_default.md。

connection between preceding and following

• Prerequisite: none (usable as soon as the problem statement is available).
• Follow-up:problem-doc-model-selector(more detailed per-question analysis) or return to paper-workflow-orchestrator to continue the paper workflow.

Constraints (must be followed)

• Memory Interaction (required):
  • After completing the planning, must call context-memory-keeper, to update the "paper outline structure" and "core scoring points" to memoryskill.md。
• This skill must output a mapping list of "scoring point → evidence → paper location"; when generating the paper later, each item must be traceable to a specific chapter/figure/table, otherwise it is considered unaligned.
• If the output requires "data preprocessing/visualization evidence", then subsequently must call data-cleaning-and-visualization to produce paper_output/figures/, otherwise that scoring point lacks evidence.
• If the user's goal is "complete paper production", after this skill ends, the next step must be made explicit: enter the data/figure stage, or return to paper-workflow-orchestrator to continue the complete workflow.