SuperAPI AI Directory
Partner中文AI newsFavorites
Skillsverifying-stream-ground-truth

verifying-stream-ground-truth

Writing & research

View repository

Updated 10-16 09:44

About this skill

Verifying Stream Ground Truth

Overview

Visual ground truth (on-screen game banners, browser tabs, infocards) and real-time LiveChat listener reactions always supersede spoken ASR fluffs, streamer slip-ups, and LLM pre-training assumptions.

THE IRON RULE OF STREAM VERIFICATION:
Never guess from spoken slurs or pre-training memory when on-screen frames can be inspected.

When to Use

flowchart TD
    A[Disputed Number / Proper Noun in Transcript?] -->|Yes| B{LiveChat Canary Burst?}
    A -->|No| C[Normal ASR Topic Flow]
    B -->|Viewer Teases/Corrects Number| D[High Suspicion: Streamer Misspoke]
    B -->|Normal Banter| E{Astronomical or East Asian Unit?}
    E -->|Numbers like 万, 億, พันล้าน| D
    E -->|Standard Numbers| C
    D --> F[Run slice_stream_frame.py for Targeted Window]
    F --> G[Inspect Frame with Vision: Read Banner / UI Text]
    G --> H[Convert Units & Emit Ground-Truth Timestamp]

Mandatory Triggers:

  1. Disputed or Astronomical Numbers: Streamer quotes impossible figures (e.g. "3,500 million downloads", "over 30,000 million").
  2. LiveChat Correction Canary: Viewers actively tease, challenge, or correct the streamer in real time ("35 million is enough, grandpa. That's more than the world population.").
  3. East Asian Unit Dissonance: Dialogue references Japanese/Korean game campaigns where 万 (10,000) or 億 (100,000,000) was translated incorrectly.
  4. Ambiguous Deictic Pronouns: Streamer says "Look at this one", "Which cabinet is good?", "This one is very strong" without naming the entity within ±30s.
  5. Explicit User Verification: User prompts /btw use vision, --vision, or "Try using vision to check it".
  6. Generative AI Outputs vs Hyperbolic Roasting: Streamer generates images/videos with AI or evaluates creative tools, and uses hyperbolic political/satirical roasts (e.g. "Did the Communist Party win the election?", "Just like the Chinese government", "Haunting like a hungry ghost").
    • Iron Rule: NEVER report the streamer's hyperbolic roast as literal image content (e.g., do NOT write "TH AI generated an image... it turned into a Communist government cabinet").
    • Action: Extract the targeted frame via slice_stream_frame.py or inspect storyboard frames, ground what is literally depicted (e.g. "A cyber girl with white hair standing next to a person in a suit"), and clearly frame the streamer's reaction as a roast under [Reaction].

When NOT to Use:

  • Static podcast/radio segments where the streamer merely chats over a static background image or webcam.
  • Verbatim news article reading where the text matches known RSS/news transcripts.

Quick Reference: 3-Step Verification Protocol

Step Action Tool / Command
1. Spot Canary Check LiveChat comments around timestamp for audience corrections. Grep LiveChat / view transcript overlay
2. Targeted Slice Download ONLY a 1–2 minute slice (instant ~2-3s download). python3 scripts/slice_stream_frame.py "<URL>" "HH:MM:SS"
3. Vision Grounding Inspect on-screen banner, calculate canonical number/name. view_file on frame image or agy --print

Targeted Frame Extraction

Do NOT download multi-hour streams or wait for whole storyboard unpacks when verifying a single event.

Automated Helper (Recommended)

Use the included helper script:

# Slices a 60-second window around 00:14:45 and outputs exact frame
python3 <skill-dir>/scripts/slice_stream_frame.py \
  "https://www.youtube.com/watch?v=VIDEO_ID" \
  "00:14:45" \
  -o "frames/banner_check.jpg"

Manual CLI Alternative

# 1. Download targeted 720p clip (instant ~2-3s download via Chrome cookies)
yt-dlp --cookies-from-browser chrome \
  --download-sections "*00:14:15-00:15:15" \
  -f "bestvideo[height<=720]+bestaudio/best[height<=720]" \
  -o "target_slice.mp4" "https://www.youtube.com/watch?v=VIDEO_ID"

# 2. Extract exact frame at offset
ffmpeg -ss 00:00:30 -i target_slice.mp4.webm -frames:v 1 -q:v 2 target_frame.jpg -y

Japanese & East Asian Unit Conversion Trap

Streamers often misread Japanese kanji units on the fly when sleep-deprived:

Japanese Kanji Numeric Value Thai Equivalent Streamer Misspeak Trap Correct Translation
ten thousand (man) 10,000 10 thousand (หมื่น) Reads 3500万 as "3,500 million" or "30 thousand million" 3,500 × 10,000 = 35,000,000 (35 million / 35M)
30 million 30,000,000 30 million Confused with 30,000 million 30 million (30M)
hundred million (oku) 100,000,000 100 million Confused with billion 100 million (100M)

VLM Prompt Engineering & Anti-Hallucination Discipline

When dispatching video frames to local VLMs (Gemma 4 VLM, Qwen-VL):

  1. Domain Prior Injection: Always supply candidate games commonly played by the streamer (e.g. Deadlock, Baldur's Gate 3, FGO, Uma Musume, Wuthering Waves). Unconstrained VLMs default to generic Western titles (Valorant, Lethal Company, GTFO, horror walking simulators).
  2. Game Modding & HUD Precedence: Streamers frequently use custom 3D model/skin mods (e.g. anime skins in Deadlock such as Ellen Joe). HUD elements (top 6v6 team hero roster, souls score, ability icons, mini-map, lane icons) STRICTLY take precedence over character model appearances.
  3. Main Display vs Chat Isolation: Explicitly instruct the model to inspect the central screen area (game HUD, video player title, article header) and ignore viewer chat on the side. Never attribute chat comments to stream content.
  4. Zero-Thinking JSON Enforcement: Local VLMs will waste token budget generating verbose English chain-of-thought unless instructed: Respond ONLY with a valid JSON object. No preface, no reasoning explanation, no markdown outside the JSON block.
  5. Application Switch Boundary Precedence: When the visual frame transitions to a new full-screen application or game (e.g., launching Honkai: Star Rail, Deadlock, or Steam):
    • The topic boundary MUST immediately break at the transition second.
    • Never bleed previous game entities or topics (e.g., FGO) into the newly opened game's UI or gameplay.
    • Specific game terms, such as the Thai endgame term "หอรี", "Apocalyptic Shadow", "เลือดมอนเฟ้อ" (inflated enemy health), "เพล่า" (Pela), or "ฮั่วฮั่ว" (Huohuo), belong exclusively to the active on-screen game.
  6. Donation Alert Media vs. Product Showcase: When character art, VTuber avatars, or figure photos abruptly appear on screen:
    • Check Donation Overlay: Inspect bottom screen corners for donation alerts, donor handles (e.g., SymboliRudolf), and currency tickers (THB ...).
    • Cross-Reference Audio: If the streamer reads a pun/joke, laughs, or says "Thank you very much [name] for the donation", classify as [Donation] or [SuperChat], NEVER [Talk] โชว์ฟิกเกอร์ or [Review].
    • Capture Joke / Lore Interaction: Note the donor's joke and streamer/chat reaction (e.g., A failed joke by the tea boy that made chat mood down).

Rationalization Table & Red Flags

Agents under pressure find excuses to avoid visual inspection. Every excuse below is a violation:

Rationalization / Excuse Reality & Counter
"Audio clearly says 30,000 so it must be 30M DL." Streamers misspeak constantly when tired. The on-screen banner is the authoritative source.
"I remember FGO 30M DL happened in October, so there is no DL campaign here." Pre-training memory is fallible and dates shift across servers. Never let memory overrule the screen.
"Downloading video takes too long and wastes bandwidth." slice_stream_frame.py uses --download-sections to fetch a tiny ~10MB slice in 2 seconds.
"The transcript is enough to guess the topic." Guessing produces hallucinations that degrade audience trust. When challenged, verify with frames.

Red Flags - STOP and Run Vision

  • Quoting download numbers > 1,000 million for a mobile game.
  • Seeing LiveChat laugh (555, ถถถ) or type numbers immediately following streamer statements.
  • Removing campaign titles entirely because "the streamer sounded confused".
  • Attributing endgame mechanics, described with Thai expressions such as "หอรี" (an endgame term) or "เฟ้อกว่าเวเนซุเอลา" (more inflated than Venezuela), to a previously discussed game after a new game has launched.
  • Hallucinating unmentioned lore/chapter titles (e.g., "Lostbelt") solely due to pre-training association with a franchise.
  • Labeling viewer donation alerts with custom avatars as literal product reviews or figure showcases ([Talk] โชว์ฟิกเกอร์).

Common Mistakes

Mistake Prevention / Fix
Downloading full stream video Always pass --download-sections "*START-END" to yt-dlp.
HTTP 403 Forbidden on video stream Always pass --cookies-from-browser chrome to authenticate yt-dlp.
Subagents hallucinating from fluffed speech Inject extracted frame OCR keywords into subagent prompts before timestamping.
Ignoring LiveChat correction timestamps Cross-reference LiveChat comments within ±30 seconds of the controversial statement.

Real-World Impact (Case Study)

In Anibon stream mJGd2XW8Tlg:

  • Streamer Spoken Words: "Celebrating 3,500 million downloads... 30,000 million cabinets, I said it wrong"
  • Baseline Failure: Subagents hallucinated 30M DL. Manual review falsely deduced no DL campaign existed at all and removed the tag.
  • Visual Ground Truth: Running slice_stream_frame.py at 00:14:45 revealed the official banner: 3500万DL突破キャンペーン (35M DL) + LiveChat correcting "35 million is enough, grandpa. 30 thousand million is more than the world population.".
  • Result: Corrected to 35M DL with 100% precision.