icml-review

ICML Pre-Submission Review

Two jobs: (1) catch everything that could get the paper desk-rejected or quietly penalized for formatting issues, and (2) read the paper like an ICML reviewer, then report what they would say. This skill does not edit the paper. Keeping review separate from writing both keeps the review objective and clearly records the problems found.

Communicate with the user in their language; the review report itself is written in English (it simulates the OpenReview form) unless the user requests otherwise.

Independence is crucial

The agent that wrote the paper knows the intent of every sentence, so it will not notice where readers get confused. Farquhar also points out that LLMs tend to agree with others' views and require repeated pushing to engage in genuine critique. Therefore:

• Preferably run this skill in anew session or sub-agent that has not seen the drafting conversation . Give it the PDF (or LaTeX), and at most also provide .icml/venue_facts.md. Do not give it claims.md or review notes—reviewers will not have these materials.
• Read like a busy expert with 5–10 papers to review: title, abstract, Figure 1, introduction, figures and tables, then the rest. Note where you would stop reading.
• Assume a bold claim is wrong, then look for holes (Nanda). Then check whether the paper has already preemptively addressed it.
• Be specific. Every weakness must point to a location and explain how to fix it.

Step 0 — Preparation

Run python scripts/init_workspace.py --root <latex-root> --skill icml-review. Read .icml/venue_facts.md(if there is no workspace, read references/icml-venue-facts.md); if the target year is different, refresh it. If possible, compile the paper so that what you review is the PDF the reviewers will see.

Part A — Compliance Audit (Mechanical Checks)

python scripts/check_submission.py --tex main.tex --pdf main.pdf \
    --names "First Last,First Last" --affils "University,Lab,Company" \
    [--pristine-sty /path/to/official/icml2026.sty] [--position-track]

If you do not know the author names and affiliations (as well as lab/cluster names, usernames), ask the user; without this information, the anonymity check will be weak. Then do the checks that scripts cannot do, listed in references/compliance-checks.md : open the figures to check for embedded names, check supplementary code for identity leaks, read every self-citation, confirm whether the OpenReview abstract matches, and confirm that nothing reviewers need exists only in the appendix.

If icml-cite is available, also run its check_bib.py. Otherwise at least confirm that the PDF has no ?? or (?), and no placeholder citation keys.

Classify findings:Desk-reject risk(page limit, anonymity, missing impact statement, hidden text, template modifications),Must fix、Suggested fix。

Part B — Simulated ICML Review

Read references/review-rubric.md and fill in its template. Briefly explain:

1. Summary: Write in your own words a summary of the content that the authors would agree with.
2. Claims check: List the paper's claims (from the abstract, introduction, and contribution list); for each claim, check whether the provided evidence is sufficient. This is the core of rigor. Check whether the textual descriptions of figures and tables are completely accurate.
3. Strengths and weaknesses: Covers rigor, presentation, importance, originality—using ICML's broad definition of originality (new insights into existing methods also count), and judging rigor separately from impact.
4. Scores: Following ICML's scoring criteria (1–4 per dimension, 1–6 overall, 1–5 confidence), giving a reason for every "fair"/"poor".
5. Key questions(3–5, numbered): questions whose answers would change the score. These are the most useful outputs: they predict the rebuttal.
6. Limitations Evaluation.
7. Reviewer-type variants: Briefly explain how a skeptical theorist, a practitioner who wants baselines and computational details, and a reviewer from an adjacent subfield would each react.

Calibration: most submissions are not accepted. Do not give 5–6 unless the paper is genuinely strong in every dimension; if you find yourself praising everything, go back and find the weakest claim.

Part C — Reporting and Handoff

Write the report to .icml/reviews/simulated-<YYYY-MM-DD>.md(format in the rubric file), including: compliance findings ordered by severity, simulated reviews, and a prioritized fix list mapping each issue to its location in the paper and fix type (writing, experiment, citation, human decision). Add items that can only be handled by a human to open_issues.md. Update state.md. Tell the user the three most important things in plain language, and suggest handing the fix list to icml-write; leave critical issues to icml-rebuttal.

In this skill, do not edit the paper even for the smallest fix—list them instead.

Files

• scripts/check_submission.py — mechanical compliance check (shared with icml-camera-ready).
• scripts/init_workspace.py — creates the shared .icml/ workspace.
• references/review-rubric.md — ICML review form, scoring criteria, reading protocol, report template.
• references/compliance-checks.md — what each check means, how to fix it, human check items.
• references/icml-venue-facts.md、references/workspace-contract.md — shared file.