modeling-paper-rubric-and-model-selector

Scoring-Aligned Paper Structure and Model Selection (Paper Rubric & Model Selector)

Global Process Collaboration Constraints (Long Conversation Drift Prevention)

• This skill must not be used as an isolated entry point. When the user requests a complete paper, generates Word, continues the process, or is uncertain about the stage, first return to paper-workflow-orchestrator to determine the current S0-S8 stage.
• Before starting or continuing the formal tasks of this skill, you must run:

  python .claude/skills/paper-workflow-orchestrator/scripts/workflow_guard.py --skill modeling-paper-rubric-and-model-selector
  

• If the output is [WORKFLOW FAIL] or the report status != "PASS", stop this skill, and according to the failure items in paper_output/qa/workflow_guard_report.json , go back and supplement the prerequisite stages; do not continue based on memory.
• This skill only writes the paper_output/ artifacts within its own contract scope; after completion, you must return to paper-workflow-orchestrator to determine the next step, and use context-memory-keeper to record the completed artifacts, blockers, and next steps.
• In long conversations, if the context becomes long, the stage is uncertain, or the user invokes skills separately, first run:

  python .claude/skills/paper-workflow-orchestrator/scripts/workflow_guard.py --status
  

  then read paper_output/qa/workflow_guard_report.json、paper_output/preflight_report.json、paper_output/input_manifest.json、paper_output/results/run_manifest.json and this skill's upstream JSON contract, and continue according to the recommended_skill and next_action in the report.
• Before continuing the process, you must treat paper_output/context/workflow_memory.json as a long-term breakpoint record; if its current_step、next_step、recommended_skill and workflow_guard.py --status are inconsistent, the guard report shall prevail.
• After completing this skill's artifacts each time, first return to paper-workflow-orchestrator or run workflow_guard.py --status, then update workflow memory:

  python .claude/skills/context-memory-keeper/scripts/update_workflow_memory.py
  

  after updating, read paper_output/context/workflow_memory.json / .md, and confirm that the next step and recommended skill have been recorded.

Execution Contract

• Upstream input: preferentially read paper_output/step1/problem_analysis.json。
• Must output:paper_output/plan/model_route.json、rubric_alignment.json、scoring_strategy.md。
• Downstream handoff: data, modeling, and evidence gates read the model route; S7's paper-formal-writer writes the model, validation, and scoring fields into the formal writing plan.tasks.json is only for legacy/quickstart.
• Recommended next step: if external data is needed, enter authoritative-data-harvester; otherwise enter data-cleaning-and-visualization. The complete paper goal should return to paper-workflow-orchestrator to determine subsequent stages.
• Failure fallback: if problem_analysis.json is missing, first run problem-doc-model-selector; when this step fails in the complete workflow, QA should fall back to problem_analysis.json。

Goal

Fuse the "score-earning" writing structure with "on-topic and actionable" model selection into a reusable process, outputting:

• A paper outline that can be directly applied (customized according to the problem statement)
• Scoring point alignment table (each scoring point corresponds to the evidence location in your paper)
• Model selection and comparative experiment route (including baseline, improvement, validation, and explanation)

When to Invoke

• When you need to quickly build a paper structure and writing order that meets the scoring criteria
• When you are unsure whether the model is on-topic, overly complex, or insufficiently explained
• When you already have a plan but are worried about "not answering the question/mismatched metric definitions/insufficient validation"

Input (provide as much as possible)

• Competition name/year/problem number (optional)
• Original problem statement or key requirements (required)
• Your understanding of each question (optional, but strongly recommended)
• Data situation: source, fields, time span, sample size, missing ratio (optional)
• Constraints and output: requires prediction/optimization/evaluation/classification/clustering/simulation/scheduling, etc. (required)
• Highlights you want to emphasize: innovation points/interpretability/reproducibility (optional)

Input contract (recommended)

Read in priority before execution:

• paper_output/step1/problem_analysis.json

If this file exists, its questions[] must be used as the sole source of sub-questions; you must not fabricate, merge, or discard sub-questions on your own.

Output contract (recommended)

This skill should generate:

• paper_output/plan/model_route.json: the model route, validation plan, chart evidence, and section placement for each question.
• paper_output/plan/rubric_alignment.json: scoring points, evidence forms, and QA rule mapping.
• paper_output/plan/scoring_strategy.md: a scoring closed-loop explanation for humans and Agents to read.

These JSON files are project-custom workflow contracts, not platform built-in standards. For detailed rules, see docs/workflow-contracts.md。

Script entry point (recommended)

python .claude/skills/modeling-paper-rubric-and-model-selector/scripts/build_model_route.py

This script will read paper_output/step1/problem_analysis.json, and write the model route and scoring closed loop to paper_output/plan/. If this file does not exist, you should first run problem-doc-model-selector。

External "paper structure prompt" resource (optional)

Supports attaching a paper structure prompt file (or text) that you have prepared in advance to this skill, for generation and evaluation.

Layered directory conventions (recommended)

• SKILL.md: skill definition and usage instructions
• references/: reusable reference materials and prompt resources (default templates, scoring point checklists, etc.)

Usage (choose one)

1. Path reference (recommended)

• When calling, provide:paper_prompt_path
• Rule: if this path is provided, read its content in priority as the "paper structure prompt"; if not provided, use the default prompt file bundled with this skill.

1. Direct paste

• When calling, provide:paper_prompt_text
• Rule: if this text is provided, use this text in priority; otherwise use paper_prompt_path; if neither is available, use the default prompt file.

Calling examples

• Method 1 (path):
  • paper_prompt_path: <项目根目录>\paper_output\plan\paper_prompt.md
• Method 2 (paste):
  • paper_prompt_text: <把你的论文结构提示词全文粘贴在这里>

Runtime integration mechanism (recommended reference)

Before generating any of the A/B/C/D outputs,it is recommended to read references/paper_prompt_default.md as a "structural reference":

1. Reference priority: this file serves as a reference for the writing template, butdoes not forcibly blockgeneration. The generated paper outline should correspond as much as possible to the chapter requirements of this file, but the model is allowed to adapt according to the actual context.
2. Anti-forgetting mechanism：
   • It is recommended to mention the core structure of this file in the Prompt (such as the six-part abstract) to guide model generation.
   • If the user provides an additional prompt, the user's requirements take precedence, and this file is secondary.
3. Encourage expansion: under the premise of ensuring logical coherence,encouragethe use of academic "all-purpose" phrases (such as "With the development of...", "In summary") to enrich the length and enhance the coherence and sense of volume of the article.

Then use PAPER_PROMPT as a strong constraint and source of writing style, integrating it into:

• the chapter naming, writing order, word count allocation, and stylistic requirements of the paper outline
• the "evidence forms" in the scoring point alignment table (which figures, which tables, which tests)
• Model route and controlled experiment design (especially the requirements for "model validation/robustness/sensitivity/reproducibility")

Backward compatibility

When not provided, paper_prompt_path / paper_prompt_text this skill maintains its original default behavior and outputs normally.

Constraints (must be followed)

• Memory Interaction (required):
  • Before starting,, check the memoryskill.md in External Resources / Literature, and if relevant literature exists, it must be incorporated into the "References" section and the background review in the "Model Formulation" part.
  • After completion,, update the generated "Paper Outline" and "Model Route" to memoryskill.md。

Outputs (fixed structure)

A. One-page problem alignment

• For each question: inputs, outputs, evaluation metrics, key constraints, boundary conditions, feasible validation methods

B. Paper outline (in scoring-friendly order)

Must include and adjust the weight according to the problem:

1. Abstract (Chinese and English as required): problem, methods, results, contributions, keywords
2. Problem restatement: turn each question into a computable task in your own words
3. Model assumptions: necessary and defensible, with justification and impact explained item by item
4. Notation and variable descriptions: tabulated, including units and ranges
5. Data description and preprocessing: sources, cleaning rules, missing/outlier handling, reproducible steps
6. Model formulation (subsections by question): objective function/constraints/loss, derivation or structural explanation
7. Solution method and implementation: algorithm steps, complexity/convergence, parameter settings
8. Results and analysis: main results, visualization, comparison, error/benefit explanation
9. Model validation: controlled experiments, sensitivity analysis, robustness testing, extreme scenarios
10. Conclusions and recommendations: provide conclusions and actionable recommendations for each question
11. Limitations and outlook: honest but not self-defeating, point out directions for improvement
12. References: properly cite the problem statement, data sources, and key methods
13. Appendix: key code, additional figures and tables, supplementary notation (choose according to competition requirements)

C. Scoring point alignment table (with "evidence" as the core)

Output a table: scoring point → evidence you provide → paper location (chapter/figure/table/experiment). Common scoring point mappings (can be added or removed according to the competition):

• Accurate understanding of the problem: one-page problem alignment + problem restatement making each question computable
• Model reasonableness: objectives/constraints consistent with the problem statement, assumptions defensible
• Method innovation/improvement: clear improvement over the baseline and explanation of why it is effective
• Credible results: validation, comparison, error analysis, or proof of constraint satisfaction
• Clear expression: unified notation, self-explanatory figures and tables, conclusions corresponding to each question
• Reproducibility: complete data sources and processing, parameter settings, and algorithm steps

D. Model selection and route (stick to the problem first, then go advanced)

For each question, output:

• Task type determination: prediction/classification/clustering/evaluation/optimization/simulation/mechanistic modeling
• Minimum viable baseline: runnable, interpretable, comparable
• One to two improvement routes: improve accuracy/robustness/efficiency/interpretability
• Validation plan: metrics, cross-validation/holdout/backtesting, ablation, sensitivity
• Risk points: insufficient data, inconsistent definitions, overfitting, uninterpretability, computation timeout

Model Selection Quick Reference (by common question types)

1) Prediction (time series/regression)

Applicable: Given history, predict the future or estimate parameters.

• Baseline: Moving average/exponential smoothing/linear regression/ARIMA (can explain trend and seasonality)
• Improvement: Feature engineering + tree models; or LSTM/Transformer (only when sample size is sufficient)
• Validation: Rolling backtest, MAPE/RMSE, confidence intervals/error decomposition

2) Classification/Discrimination

Applicable: Determine category, risk level, whether it occurs.

• Baseline: Logistic regression/Naive Bayes (interpretable)
• Improvement: Random forest/gradient boosting; cost-sensitive learning (class imbalance)
• Validation: AUC/F1/PR curve, confusion matrix, threshold sensitivity

3) Evaluation/Ranking/Composite Index

Applicable: Multi-indicator scoring, ranking, selection of the best.

• Baseline: Normalization + weighted sum (weights can come from the problem statement or experts)
• Improvement: Entropy weight/CRITIC/AHP/TOPSIS/VIKOR (clarify weight source and meaning)
• Validation: Weight sensitivity, ranking stability, comparison with known facts

4) Optimization/Scheduling/Site Selection/Routing

Applicable: Resource allocation, cost minimization/benefit maximization, satisfying constraints.

• Baseline: Linear programming/integer programming (state objectives and constraints clearly)
• Improvement: Multi-objective (weighted/ε-constraint), heuristics (genetic/simulated annealing) for large scale
• Validation: Feasibility check, comparison with baseline strategies, constraint violation rate, complexity and time

5) Clustering/Grouping/Profiling

Applicable: Unlabeled grouping, pattern discovery.

• Baseline: K-means/hierarchical clustering
• Improvement: GMM/DBSCAN (when noise and shapes are complex)
• Validation: Silhouette coefficient/stability, interpretable description of group differences

6) Mechanism/Simulation/System Dynamics

Applicable: Emphasis on mechanism explanation, scenario deduction.

• Baseline: Differential equations/difference equations/system dynamics
• Improvement: Parameter calibration + uncertainty analysis; comparison with data-driven models
• Validation: Historical fitting, scenario consistency, parameter sensitivity

Hard rules to prevent going off track (must check)

• For each question, provide at least one "quantifiable output" and one "verifiable indicator/test method"
• Every key variable in the model can find its definition and unit in the problem statement/data
• Figures/tables and conclusions correspond question by question; avoid "did a lot but did not answer the question"
• At least one baseline comparison: prove that your method improves on or is more reasonable than a simple approach

Final deliverables (I will provide when you ask me to generate)

• Outline and section titles that can be pasted directly into the paper (numbered by question type)
• Scoring point alignment table (including the list of figures/tables/experiments you should add)
• Model route for each question: baseline + improvement + validation + risks and alternatives

Default prompt file (reusable)

When not provided paper_prompt_path / paper_prompt_text , by default read the paper structure prompt from the following file:

• references/paper_prompt_default.md

Directory conventions (aligned with the project globally)

• Competition problems and attachments are uniformly placed in problem_files/, supplementary data is placed in crawled_data/。
• It is recommended to archive this skill's output (outline/scoring point alignment/model route) to paper_output/plan/, for reference when generating the main text later.
• The above paper_prompts/... is only a historical path example; for the current project, it is recommended to also archive custom prompt files to paper_output/plan/, or directly use this skill's built-in references/paper_prompt_default.md。

Contextual continuity

• Prerequisite: none (usable once you have the problem statement).
• Follow-up:problem-doc-model-selector(more detailed per-question analysis) or return to paper-workflow-orchestrator to continue the paper workflow.

Constraints (must be followed)

• Memory Interaction (required):
  • After completing the plan, you must call context-memory-keeper, and update the "paper outline structure" and "core scoring points" to memoryskill.md。
• This skill must output a mapping list of "scoring point → evidence → paper location"; when generating the paper later, it must be possible to map each item to a specific chapter/chart/table, otherwise it is considered unaligned.
• If the output requires "data preprocessing/visualization evidence", you must subsequently call data-cleaning-and-visualization to produce paper_output/figures/, otherwise that scoring point lacks evidence.
• If the user's goal is "complete paper production", after this skill ends, the next step must be clearly stated: enter the data/chart stage, or return to paper-workflow-orchestrator to continue the complete workflow.