score-optimizer

Score Optimizer

This skill is responsible for only one thing: optimizing the article scoring system itself.

What it is suitable for:

• Adjusting the article viral-score Prompt
• Scoring the sample corpus in batches
• Evaluating scoring accuracy against annotated data
• Studying whether the scoring dimensions, weights, and few-shot examples are more reasonable

What it is not responsible for:

• Directly editing the article body
• Platform rewriting
• Publishing packaging

Working Scope

Applicable scenarios:

• The user says "optimize the article scoring prompt"
• The user says "make this viral-score system more accurate"
• The user says "run a round of scoring experiments on the sample corpus"
• The user says "calibrate the scoring rules for viral score / composite score"

Inapplicable scenarios:

• The user says "just change this article to a high score"
• The user says "optimize the article content according to this scorer"

The latter should instead use article-optimizer。

Required Reads

Read before starting:

1. references/score_program.md
2. references/score_prompt.md

Read again when you need to see the implementation:

• scripts/run_scoring.py
• scripts/evaluate.py
• assets/articles/labels.example.json —— annotation schema example

Sample corpus location (provided by the user, this skill does not include samples):

• assets/articles/samples/ —— put in the .md articles you have annotated yourself, then cp labels.example.json labels.json fill in your scores. See details in assets/articles/samples/README.md

Workflow

1. First confirm that the current task is optimizing the scorer, not editing articles.
2. Read references/score_program.md, and execute according to its experimental constraints.
3. By default, only change references/score_prompt.md, do not touch the evaluation script.
4. Run the batch scoring script, then run the evaluation script.
5. Compare composite_score、mae、spearman、classification。
6. Record the results and decide whether to keep or roll back.

Commands

Execute in the skill root directory:

python3 scripts/run_scoring.py > scoring.log 2>&1
python3 scripts/evaluate.py > eval.log 2>&1
grep "^composite_score:\|^mae:\|^spearman:\|^classification:" eval.log

If charts are needed:

python3 scripts/generate_score_chart.py

Guardrails

• By default, do not change scripts/evaluate.py
• By default, do not change scripts/run_scoring.py
• By default, do not install new dependencies
• If you just want to raise the score of a single article, do not use this skill