cheat-bump

/cheat-bump — Rubric / Bucket Upgrade

Two modes:

Mode	Trigger	What it does	Verification strength
Full rubric bump	--propose "<新公式>"	Change formula / dimensions / weights	5 steps + cross-model review (mandatory)
bucket-only recalibration	--bucket-only	Only re-derive bucket boundaries	Automatically derived from data, no review

A full rubric bump strictly follows shared-references/bump-validation-protocol.md 's 5 steps. bucket-only takes the lightweight path—see Phase B below.

Overview

入口：用户触发 /cheat-bump
  ↓
[Phase A0: 检测调用模式]
  ↓
  ├─ --bucket-only  →  [Phase B: 轻量 bucket 重校]
  └─ --propose      →  [Phase 0~6: 完整 rubric bump]

Phase A0: Invocation mode triage (do this first)

Read user arguments:

• contains --bucket-only → take Phase B(lightweight recalibration)
• contains --propose "<...>" → take Phase 0~8(full rubric bump)
• neither → ask the user: "What do you want to do? 1) Adjust the rubric formula / add or remove dimensions → --propose; 2) Only re-derive bucket boundaries → --bucket-only"

If the user says "I think ER is too low and I want to adjust it" → that is the --propose path. If the user says "my account has grown, the bucket is no longer accurate" → that is the --bucket-only path. The two paths cannot be mixed—one operation does only one thing.

---

Full rubric bump process

[用户：升级 rubric --propose "ER×1.5→2.0，砍 NA，加 MS"]
  ↓
[Phase 0: 前置门槛检查]
  ↓
[Phase 1: 写出新公式完整方程]
  ↓
[Phase 2: 校准池全量重打分]
  ↓
[Phase 3: 计算排序一致性]
  ↓
[Phase 4: 跨模型独立审核（强制）]
  ↓
[Phase 5: 落地 + cleanup pass]
  ↓
[Phase 6: 更新所有校准样本的 prediction 文件底部追加 Re-scored 行]

Constants

• READINESS_HEURISTIC —
  • Default reference: calibration pool ≥ 5 samples + at least 1 cross-sample observation supported by ≥3 samples
  • but Claude may propose a bump(even with few samples) if the observed signal is especially strong:
    • N=3 but a strong counterexample appears that completely overturns the current rubric assumption (a ≥3x deviation like composite 8.5 vs actual 5w)
    • 1 post shows a single but extremely strong phenomenon (e.g., a single meme with ≥2000 likes in the comments)
  • Claude may also refuse a bump(even with sufficient samples) if the evidence is weak:
    • N=10 but the observations are all low-confidence fragmentary patterns with no clear direction
    • Many non-serious judgments of "just looked at it casually" during the user's review
  • When writing in the prediction header or cheat-bump output, must state: whether this proposal is default-aligned or judgment-driven, giving the user a basis for scrutiny
• THRESHOLD = 0.8 — the consistency threshold between the new ranking and the actual-performance ranking (4/5). This one ishard-coded—the statistical rigidity of bump validation
• CROSS_MODEL_AUDIT = true — call an external LLM for independent review. false is only for offline
• REQUIRE_CONFIRM = true — before landing, require the user to explicitly say "yes, bump"

Inputs

Required	Source
--propose text	User arguments; if missing, ask
rubric_notes.md	User project root
predictions/*.md full set	Calibration pool data
.cheat-state.json	Status

Workflow

Phase 0: Precondition gate check

According to bump-validation-protocol.md 's "when prohibited" section, check item by item:

Check	Failure handling
Calibration pool total sample count vs observation strength	Claude judgment——According to READINESS_HEURISTIC: default ≥5 samples but exceptions allowed (strong counterexample / strong meme). If the default is not met, Claude mustexplicitly statewhy a bump is still proposed ("although only N=3 samples, this one X occurrence shows composite Y vs actual Z, which is a W-fold deviation"), so the user can scrutinize
New calibration count since last bump vs observation maturity	Claude judgment——Default recommendation is ≥3 new samples, but if 3 consecutive samples all show strong evidence pointing in the same direction → no need to wait further
in_progress_session == null	Reject: "You have in-progress predictions unfinished. Finish that process first or clear the state"
Trigger conditions met (systematic deviation / cross-sample new observation / sufficient evidence for new dimension)	Warn but do not block——ask the user why bump now

Pass → enter Phase 1.

Phase 1: Write out the complete equation of the new formula

Cannot just accept the user's brief description. Expand it into a complete equation:

当前：v2  composite = (ER×1.5 + SR×1.5 + HP×1.5 + QL + NA + AB + SAT) / 8.5 × 2.0
提议：v2.1  composite = (ER×2.0 + HP×1.5 + MS×1.5 + QL + SR + TS + SAT) / 9.0 × 2.0

变化总结：
- ER ×1.5 → ×2.0（升）
- SR ×1.5 → ×1.0（降）
- 新增 MS ×1.5（Memetic Shareability）
- 新增 TS ×1.0（Topic Shareability）
- 删除 NA（与 HP 重叠）
- 删除 AB（被 TS 替代）
- 归一化常数 8.5 → 9.0
- 公式总维度数：7 → 7（净变化 0）

If the user's proposal is vague (e.g. "raise the ER weight a bit") → ask for the specific value,guessing is forbidden。

Phase 2: Full re-scoring of the calibration pool (mandatory blind sub-agent）

Glob predictions/*.md all files with a complete review section → calibration pool.

bump is the tool's highest-risk action——all re-scoring must go through cheat-score-blind sub-agent. inline re-scoring = the main Claude has already seen the actual results, rank consistency becomes overfit rather than a true signal.

Mandatory constraints

• Do not accept self-scored fallback——/cheat-predict has a --skip-blind flag, but /cheat-bump does not. If the Task tool is unavailable → abort bump, report to the user "resolve the Task tool first, then bump"
• Do not accept "I'll only recompute composite without re-scoring dims" —— even if the new formula only adjusts weights without adding dimensions, every dim of every prediction must have its script re-reviewed by the sub-agent. Reason: the old dim scores themselves may be contaminated; once weights change, there is no guarantee the old dims still hold

For each prediction:

1. Parse the prediction file to get the corresponding scripts/<id>.md path (from the Script Path header field)
2. Verify the script file exists + hash matches the header Script Hash ; if inconsistent → warn (script has been modified) but still spawn the sub-agent
3. Spawn the cheat-score-blind sub-agent via the Task tool：

   Spawn cheat-score-blind sub-agent.
   
   Input:
     script_path: <prediction header 的 Script Path>
     rubric_notes_path: rubric_notes.md
     sidecar_path: .cheat-cache/bump-rescores/<prediction-id>.json
   
   Task: 按 rubric_notes 当前公式（已是新版 vN+1）给 script 打分。
   返回严格 JSON。写 sidecar 文件用于 bump 主流程批量读取。
   
   不要读 state file / predictions/ / videos/ 任何其他文件。
   不要询问用户 —— 你没有用户。
   不要读这份 prediction 文件本身 —— 你只看 script + rubric。
   

4. Wait for the sub-agent to finish → read the sidecar JSON → the main process computes composite with the new formula
5. Write the "re-score table" to .cheat-cache/bump-rescores.json(summary).Mark each entry as blind: true —— during bump phase 5 cleanup, write this field along with the new score to the prediction file's Re-scored under v<N+1> line

Also honestly annotate contamination that was not contaminated

Even when going through the sub-agent,there are still two types of residual contamination to honestly annotate in the bump report：

Type	Source	Annotation field
Model prior contamination	the sub-agent is still Claude, RLHF is shared	model_prior_warning: true(default true, cannot be disabled)
User's own rubric design bias	rubric_notes.md is written by the user, so it naturally fits their own content	rubric_self_designed: true(default true, cannot be turned off)

These two points remind the user that channel C (cross-model audit) is indispensable. The end of the bump report must print: "The rank consistency above is consistency within channel A.The final decision must wait for the channel C audit to pass。"

Failure mode

Symptom	Handling
The script file for a certain prediction is missing	The sub-agent skips that item, and the main process summarizes in the report "N items were excluded due to missing scripts." If the remaining valid pool < MIN_SAMPLES → abort bump
The sub-agent returns refusal != null	Resend the Task at most 3 times; if it still fails → mark that item as rescore_failed: true exclude it from the calibration pool
The Task tool is entirely unavailable	abort bump, and prompt the user "The Task tool is a hard dependency of bump. If it is truly an offline environment, run /cheat-bump --bucket-only to take the lightweight branch"
The sub-agent output contains contamination_signal	Mark suspicious: true but do not exclude—list these suspicious items at the end of the bump report for the user to review

Phase 3: Compute ranking consistency

每个样本：
  new_composite_rank: 用新公式排序的 rank
  actual_plays_rank: 用实际播放排序的 rank
  delta: |new_rank - actual_rank|

输出对照表：
| 样本 | composite (v2) | composite (v2.1) | rank (new) | actual | rank (actual) | delta |
|---|---|---|---|---|---|---|
| 仓鼠 | 9.41 | 9.55 | 1 | 124.8w | 1 | 0 |
| 停止期待 | 8.24 | 9.11 | 2 | 71.1w | 2 | 0 |
| 老板废话 | 7.65 | 8.11 | 4 | 39.6w | 3 | 1 |
| 求职悖论 | 8.47 | 7.56 | 5 | 16.8w | 4 | 1 |
| 谁问你了 | 8.24 | 7.00 | 6 | 11.7w | 5 | 1 |

排序一致性：4/5 在 |delta| ≤ 1
Pairwise no-regression：旧公式做对的所有 pair 在新公式下未颠倒 ✓

Judgment:

• Ranking consistency < THRESHOLD (default 0.8) → local rejection, and clearly report the failure before moving to Phase 4
• A regression appears in pairwise → local rejection

THRESHOLD is hardcoded in the protocol—temporarily lowering it is not allowed (that itself is another meta-decision that requires a bump).

Phase 4: Cross-model independent review (mandatory, unless escape hatch)

CROSS_MODEL_AUDIT=true(default):

Call mcp__llm-chat__chat：

prompt:
你是一个独立审稿人。下面是一个内容创作者准备升级的 rubric 公式。
请独立判定两件事：
1. 排序一致性：新公式给样本的排序与实际表现排序，是否真的在 ≥80% 样本上一致？
2. 解释力：新公式相比旧公式，是否更好地解释了校准池的实绩分布？

数据：
旧公式：(ER×1.5 + SR×1.5 + HP×1.5 + QL + NA + AB + SAT) / 8.5 × 2.0
新公式：(ER×2.0 + HP×1.5 + MS×1.5 + QL + SR + TS + SAT) / 9.0 × 2.0

校准池：
[Phase 2 重打表的完整 JSON]

排序对照：
[Phase 3 表格的完整 JSON]

输出格式：
- 判定：PASS 或 REJECT
- 理由：≥100 字
- 关键风险：[如有，列出新公式的潜在问题]

Receive the external LLM reply → parse the judgment.

Judgment logic:

• Local PASS + external PASS → pass, enter Phase 5
• Local PASS + external REJECT → treat as REJECT. A conflict means at least one side's interpretation is unstable
• Local REJECT → already terminated in Phase 3
• mcp__llm-chat__chat unavailable → gracefully degrade to CROSS_MODEL_AUDIT=false, mark the state file as last_bump_self_audited: true

CROSS_MODEL_AUDIT=false：

• rely only on local judgment
• The state file keeps marking it, and cheat-status keeps prompting the user "this bump is self-review; it is recommended to configure mcp__llm-chat__chat"

Phase 5: Land + cleanup pass

After passing review,REQUIRE_CONFIRM=true → ask the user: "The new formula PASSes local and external review. Final confirmation: execute the bump landing? This will modify rubric_notes.md + rubric-memo.md and delete several observations that have been absorbed. Only answer 'yes, bump' to execute."

After the user confirms:

5a. Update rubric_notes.md（only put general language, without video names / actual results）

• Top metadata update:
  • **当前版本**: vN+1
  • **Last bumped at**: <ISO 8601>
  • **Upgrade memos**: 见 [rubric-memo.md](rubric-memo.md)(pointer, do not copy Memo content)
• Add a row to the version quick-reference table (only version number + formula signature,withoutevidence samples)
• Update the "current scoring dimensions" section (delete NA / AB, add MS / TS)
• Derived evidence section If the new dimension needs anchor explanation → use general language：
  • ✅ Allowed: "Derived evidence: high abstraction density sample → CC=1 → low reach"
  • ❌ Forbidden: "Derived evidence: "Stop expecting" CC=1 → actual performance 13.7w" (video name + actual performance number)
  • Hit a forbidden pattern → extract that passage into the "Derived evidence" subsection of rubric-memo.md, and replace it in place with generic language

5b. Write Memo to rubric-memo.md（append mode, do not overwrite history）

According to bump-validation-protocol.md Step 5 + templates/rubric-memo.template.md format, append a Memo to the end of the file:

• Trigger observation (including real observation ID)
• Evidence data (calibration pool re-scoring table + ranking comparison, including real video names + actual performance）
• Derived evidence (including real sample names + actual performance）
• Diagnosis
• New formula
• Cross-model review conclusion citation (including model name + verdict + reason excerpt)
• Known limitations

Neveroverwrite existing content in rubric-memo.md—bump memos accumulate in chronological order.

5c. cleanup pass (according to observation-lifecycle.md 's "cleanup pass mandatory timing")

Within rubric_notes.md execute (do nottouch rubric-memo.md):

• Observations already absorbed as new dimensions → delete (e.g., observation E absorbed as MS → delete observation E)
• Observations overturned by new data → delete
• Observations still unresolved → migrate to the new version's "hypotheses to verify" section
• "Patterns" already verified → move to the "pattern sedimentation area"

5d. Organize + self-check

• Re-read rubric_notes.md in full, ensuring readers can understand the current rules within 60 seconds—exceeding 600 lines triggers additional cleanup
• Self-check leak guard: for rubric_notes.md run grep -E '\\d+\\s*[wWmMkK万]|播放|实绩|实际' → if there is a hit → abort bump + rollback, prompt the user that "rubric_notes.md contains forbidden content (actual performance / play count)". This content should be in rubric-memo.md, not rubric_notes.md

Phase 6: Batch update of calibration samples

For each calibration sample's prediction file,append at the bottom(do not touch the prediction section, do not touch the review section):

---
**Re-scored under v2.1 on 2026-05-04**: composite=8.24 → 9.11 (blind: true)
（rubric bump 时全量重算，由 cheat-score-blind sub-agent 独立打分；详见 rubric-memo.md 的 v2 → v2.1 升级 Memo）

blind: true fieldrequired—tell future readers of this record "this is a channel B isolated scoring, not the main Claude's self-assessment". If a prediction was excluded in Phase 2 due to sub-agent failure → there will be no Re-scored line (keep as is).

Use the Edit tool, matching the very end of each file.

Phase 7: Update state file

{
  "rubric_version": "v2.1",
  "last_bump_at": "<ISO timestamp>",
  "last_bump_self_audited": false,
  "consecutive_directional_errors": [],
  "calibration_samples_at_last_bump": <current value>
}

Clear consecutive_directional_errors—the new rubric recounts from scratch.

Phase 8: Console report

✅ Rubric 已升级 v2 → v2.1

变化：
- ER ×1.5 → ×2.0
- SR ×1.5 → ×1.0
- 新增 MS / TS
- 删除 NA / AB

校准池重打：5/5 通过排序检查（4/5 一致 + 0 pairwise 回归）
跨模型审核：✅ PASS
Cleanup pass：删除观察 D 和 E（已吸收为 QL 重定义和 MS 维度）

下一篇预测起按 v2.1 公式打分。
所有历史预测文件已追加 Re-scored 标记。

---

Phase B: bucket-only recalibration (lightweight branch)

/cheat-bump --bucket-only [--scheme ratio|absolute|percentile]

Essential difference from a full bump: bucket boundaries are not part of the rules, they are data-derived quantities. Re-deriving themdoes not require cross-model review—the derivation algorithm is deterministic, with no "judgment" component.

B1: Select algorithm (automatically derived based on the number of available samples,state does not store scheme）

Algorithm	Applicable	Boundary derivation method
ratio(default N=1-4)	Small sample	Previous post / median of the most recent 3 posts × {0.3 / 1 / 3 / 10 / 30}
absolute(default N=5-9)	Medium sample	Calibration pool median × {0.3 / 1 / 3 / 10 / 30}, fixed boundaries
percentile(default N≥10)	Large sample	Calibration pool actual percentile {30 / 60 / 85 / 95 / 100}

--scheme Parameters allow the userexplicitly override the default：

• --scheme ratio Force ratio (even if N≥5)
• --scheme absolute Force absolute
• --scheme percentile Force percentile (requires N≥3, otherwise error)

Not specified --scheme → automatically derived per the table above.

The old design had a bucket_scheme state field—removed in v1.1. All skills derive the algorithm in real time from calibration_samples; there is no need to persist "which one is currently in use." This avoids the state inconsistency problem of "forgetting to sync after switching schemes."

B2: Derive new boundaries

Read predictions/*.md all samples in actual_plays that have

ratio mode：

baseline = median(最近 3 篇 actual_plays)
buckets = {
  "退步": (-inf, baseline * 0.3),
  "持平": (baseline * 0.3, baseline * 1),
  "命中": (baseline * 1, baseline * 3),
  "小爆": (baseline * 3, baseline * 10),
  "大爆": (baseline * 10, +inf),
}

absolute mode：

baseline = median(全部校准池 actual_plays)
buckets = {
  "底部": (-inf, baseline * 0.3),
  "基础盘": (baseline * 0.3, baseline * 1),
  "命中": (baseline * 1, baseline * 3),
  "爆款": (baseline * 3, baseline * 10),
  "现象级": (baseline * 10, +inf),
}

percentile mode：

sorted_plays = sorted(全部校准池 actual_plays)
buckets = {
  "底部":   ≤ p30,
  "基础盘": p30 - p60,
  "命中":   p60 - p85,
  "小爆":   p85 - p95,
  "大爆":   ≥ p95,
}

B3: Report changes + user confirmation

当前 bucket scheme: ratio
proposed scheme: absolute
baseline: 4.2w 中位数（基于 5 篇校准样本）

新边界：
- 底部:   < 1.3w
- 基础盘: 1.3w - 4.2w
- 命中:   4.2w - 12.6w
- 爆款:   12.6w - 42w
- 现象级: > 42w

派生说明：
- 5 篇实绩：1.5w / 3.8w / 4.2w / 5.6w / 18w
- 中位数 4.2w，新桶按 ×{0.3, 1, 3, 10} 派生

确认应用？(yes / no)

B4: Apply

After user confirmation:

1. Edit rubric_notes.md the "Bucket scheme" section, replacing it with the new table
2. Update .cheat-state.json the baseline_plays field (bucket scheme is not persisted—it is derived in real time on the next cheat-predict)
3. At rubric_notes.md the top of the bucket section, append a one-line change record:v2 buckets recalibrated on YYYY-MM-DD: scheme=absolute, baseline=4.2w (基于 N=10 个样本)
4. Do notmodify any prediction file—the bucket labels of historical predictions remain as-is (judgments made under the scheme in effect when that sample was written)

B5: Impact on future predictions

The next /cheat-predict will derive per the new bucket. Bucket labels in historical prediction filesare not recalculated—buckets are semantic judgments made at prediction time; rewriting them after the fact would break blindness.

What Phase B does not do

• Does not re-score composite (the formula has not changed)
• Does not re-review the observation section (the rubric has not changed)
• Does not invoke cross-model review (deterministic derivation requires no judgment)
• Does not require a strict sample-count threshold (judged by Claude per READINESS_HEURISTIC; ratio mode can run with N=1)

---

Key Rules

1. 5 steps cannot be skipped(full rubric bump only). Any request to "simplify and just run it first" is rejected
2. THRESHOLD is hardcoded(full rubric bump only). Dynamic adjustment is not allowed
3. Cross-model review is the default(full rubric bump only). Turning off review requires an explicit marker in the state file
4. The cleanup pass is part of the bump(full rubric bump only). It is not allowed to finish a bump without cleaning up the observation section
5. REQUIRE_CONFIRM(required in both modes). Before final application, the user must explicitly say "yes, bump" or "yes, recalibrate"
6. Bucket recalibration does not touch historical predictions. Buckets are semantic at prediction time; rewriting them after the fact breaks blindness

Refusals

• "Skip re-scoring the calibration pool, just swap the formula" → Reject. Principle #2
• "Skip the cheat-score-blind sub-agent, the main Claude can just re-score directly" → Reject. bump not acceptedAny self-scored fallback — sub-agent unavailable → abort bump, do not accept "self-review"
• "Skip external LLM review" → only when CROSS_MODEL_AUDIT=false is explicitly set
• "This time adjust THRESHOLD to 3/5 to let it pass" → refuse. Changing THRESHOLD is a meta-level bump
• "Keep all old observations as history" → violates principle #3
• "Bump first, do cleanup next time" → refuse. cleanup is part of the bump
• "Only recompute composite without re-scoring dims" → refuse. New weights × old dims is still old contamination. Every dim must have its script re-reviewed by a sub-agent
• "Write the full Memo at the top of rubric_notes.md so it's easy for me to read" → refuse. rubric_notes.md is the blind sub-agent whitelist — containing video names / actual results → leaks through the whitelist. Write the Memo to rubric-memo.md (whitelistoutside), rubric_notes.md only holds formulas + generic-language dimension definitions + pointers
• "Keep real video names in the derived-evidence section so the rubric reads more concretely" → refuse. In rubric_notes.md you must use generic language ("high-abstraction-density sample"); derived evidence with video names goes in rubric-memo.md

Integration

• Upstream:/cheat-retro detects ≥3 same-direction deviations → proposes running /cheat-bump
• Dependencies:mcp__llm-chat__chat(if configured) + Task tool (spawn cheat-score-blind)
• Modify:
  • rubric_notes.md(structural update,neverwrite real video names / actual results)
  • rubric-memo.md（new— append the full Memo, including evidence + derived evidence)
  • All predictions/*.md(append a Re-scored line, do not touch the prediction section)
  • .cheat-state.json
• Downstream: the next /cheat-predict automatically scores by the new rubric_version