Extract traceable evidence screenshots from screen recordings. By default, first perform temporal cluster selection, content increment refinement, and high-confidence loading state filtering locally; only when the current model supports image input and the basic results still need to be streamlined, prepare a small amount of multimodal audit material.
Workflow
1. Confirm input and output
Obtain a local video path. Supports .mp4、.mov、.avi、.mkv、.webm、.flv、.wmv、.ts. By default outputs to the video's sibling <视频名>_frames/; if that directory may contain other materials, explicitly specify a new -o directory.
The script must first complete parameter checking and video probing, then generate the complete result in a hidden staging at the same level as the target directory. Only after frames, reports, ownership markers, and optional archives are all successful, replace the old outputs of this tool in one go; if any step fails, keep the old results. If the output root directory or any file in it is a symbolic link, the directory contains unknown files, or _evidence_leads、_vision_audit、_curated、_vision_review.json has already been generated, refuse in-place overwrite and require use of a new -o. Legacy results without ownership markers are allowed compatible replacement only when _report.json exactly matches the actual base frame list.
2. Run basic refinement
# 默认：场景候选 + 自适应时间簇择优 + 图像去重
uv run scripts/extract.py -i <视频路径>
uv run --with rapidocr-onnxruntime scripts/extract.py -i <video path> --ocr-dedup
Only save discarded candidates when debugging missed frames; do not use as default input for multimodaluv run scripts/extract.py -i <video path> --keep-drop-candidates --drop-candidate-limit 80
默认流程：
1. 用 ffmpeg 场景变化和 2 秒保底采样生成高召回候选；
2. 计算候选帧清晰度、内容质量、相似度、滚动重叠、加载态和拼接风险；
3. 按前后帧形成稳定段与连续运动段；
4. 稳定段选择清晰、完整、停留时间更长的代表帧，不机械保留第一张；
<!-- skill-lint:constraint BASE-HIGH-RECALL -->
5. 前后均有稳定页的短运动段至少保留一张低置信代表帧；只有三帧像素拼合能直接证明是前后页面合成时才自动删除，持续滚动段依据相邻内容重叠自适应调整保留跨度；
6. 自动丢弃高置信居中加载浮层；疑似未完成页只有在 1.25 秒内出现同一页面骨架、主内容明显增加且完整后帧最终被基础层保留时才删除；
7. 用三帧像素拼合与分区覆盖共同估计横向、纵向或缩放切换风险；分区风险只用于提权审计，不单独自动删除；
8. 单帧 `transition` 质量标签只记录风险，不单独删除；再执行 SHA256、dHash、像素差异、SSIM、可选滚动合并和 OCR 内容增量判断。OCR 对短运动段最多检查 24 张高风险图片，并在新增金额、长编号或足量正文时否决近似去重和最小间隔删除；启用 OCR 时关闭未完成页的纯视觉自动删除。
需要调参时读取 `references/strategy-and-params.md`。依赖安装见 `references/setup.md`。
### 3. 检查基础结果
检查：
- `_report.json` 中 `frames` 与实际 `frame_*.jpg` 是否一致；
- `temporal_selection` 的稳定段、运动段、自适应密度和切换丢弃统计；
- 联系表总览中是否仍有左右两页拼接、上下切换中间态或明显重复页；
- 视频开头、结尾、长滚动和关键主体/金额/地址页面是否有覆盖。
不要只因帧数减少就判定准确率提高。基础结果不调用大模型，并在报告中保留 `vision_audit_status=not_prepared`。
### 4. 生成高价值证据线索包
基础结果确认完整后，先用代码生成一个独立的高价值索引；这一步也适用于纯文字模型：
```bash
# 推荐：本地 RapidOCR 多锚点 + 无文字图像主体 + 时序信号
uv run --with rapidocr-onnxruntime scripts/prepare_evidence_leads.py \
-i <基础输出目录>
# 未安装 OCR 时仍可运行，自动保留图像主体与时序排序
uv run scripts/prepare_evidence_leads.py \
-i <基础输出目录> --no-ocr
The script generates _evidence_leads/evidence_index.json, a 4-page large-image contact sheet composed of at most 24 clues, and vision_template.jsonthat a weak model can directly fill in. The code only records subject/qualification, product/work/service, transaction performance, communication commitments, public statements, comment controversies, propagation timelines, documentary records, etc.category and combination hit counts; it does not save OCR original text, company names, product names, prices, or communication content. A single broad word such as "product" or "message" must not independently trigger a category.
OCR is not the master switch for value. Text-sparse images such as product appearance, packaging/marks, work frames, defect states, business premises, and human behavior still enter clue ranking through continuous image subject area; the specific meaning is left to multimodal or human judgment. The evidence clue stage only performs ranking, classification, and summarization, and never deletes, replaces, or overwrites the base frame_*.jpg, nor does it determine authenticity, legality, relevance, or probative force.
If the current model can read images, view page by page evidence_sheet_NNN.jpg, fill in the template in place according to _evidence_leads/VISION_INSTRUCTIONS.md , then apply:
python3 scripts/apply_evidence_review.py \
-i <基础输出目录> \
-r <填写完成的 vision_template.json>
The applier only accepts closed categories, visible facts, and "possible uses", rejects legal conclusions such as "sufficient to prove", and generates _evidence_leads/evidence_review.json. Weaker models use the default 2-column large images and do not expand the image budget; when unable to read images, deliver the code ranking results and clearly state that evidence_review was not executed. For the detailed contract see references/evidence-leads.md。
5. Select a deduplication multimodal branch based on capability
If the current model or tool cannot read images, stop at the base results and clearly state that the visual audit was not executed. Do not fabricate image judgments based on file names or algorithm scores.
If images are supported and the user wants further trimming, generate an economy audit package:
uv run scripts/prepare_vision_audit.py \
-i <基础输出目录> \
--max-groups 8 \
--max-images 24
The script first requires the target and at least one context to pass strict near-pixel consistency or high-quality scroll overlap verification, then selects a small number of groups by risk score and temporal coverage, and generates _vision_audit/audit_manifest.json and a contact sheet. The multimodal layer only subtracts; it does not restore images missed or already discarded by the base layer; when there is no safely removable item, generating 0 groups is allowed. Follow references/vision-audit.md to view the before/after frames of the same group and output _vision_review.json. By default, do not exceed 8 groups or 24 unique images; if expansion is truly needed, first explain the cost and privacy impact to the user.
When the model's visual capability is weak or output stability is average, prioritize using the weak model tier:
uv run scripts/prepare_vision_audit.py \
-i <基础输出目录> \
--profile weak
weak By default, at most 6 groups and 18 unique images; each contact sheet has only one red-box judgment target, A/B/C roles are clearly labeled, and generate review_template.json and MODEL_INSTRUCTIONS.md. View them group by group, image by image, and fill in the template in place. Deletions or replacements proposed by weak and balanced take effect only when "confidence at least 0.90 + local approval covering the candidate + local risk signal + coverage frame ultimately survives" are all satisfied simultaneously; coverage chains, coverage loops, or any other insufficiency safely degrade to retention.
6. Apply the deduplication visual audit
python3 scripts/apply_vision_review.py \
-i <基础输出目录> \
-r <基础输出目录>/_vision_review.json
The application script must recompute the base report, manifest, and image SHA256. On success, generate _curated/ and _curated/_curated_report.json; it must not delete or overwrite the base frame_*.jpg. Invalid references, expired manifests, missing reasons, hash mismatches, or empty curated sets must fail closed.
Main outputs
| File | Description |
|---|---|
frame_NNN_MMmSSs.jpg |
Base frames retained by the local algorithm |
_report.json |
Inputs, parameters, time cluster statistics, discard statistics, frame timestamps, and SHA256 |
_video_screenshot_output.json |
Output ownership marker without source paths, bound to the report and directory content hash and the root directory manifest, for safe rerun verification |
_review_candidates/ |
Algorithm discard candidates saved only when explicitly enabled |
_evidence_leads/evidence_index.json |
Evidence clue ranking, categories, signal counts, privacy statement, and per-frame hashes; does not include OCR raw text |
_evidence_leads/evidence_sheet_NNN.jpg |
High-value contact sheets with large images in 2 columns by default; at most 24 images and 4 pages |
_evidence_leads/vision_template.json |
Closed categories, visible facts, and possible-use templates |
_evidence_leads/evidence_review.json |
Non-destructive visual classification results verified by the applier |
_vision_audit/audit_manifest.json |
Budget-limited visual audit checklist and decision contract |
_vision_audit/contact_sheet_NNN.jpg |
Contact sheets of before/after frames in the same group; in the weak tier, one large image per group with one target |
_vision_audit/review_template.json |
Restricted answer template prefilled in the weak tier |
_vision_audit/MODEL_INSTRUCTIONS.md |
Per-group judgment explanations in the weak tier |
_vision_review.json |
Structured audit results from the multimodal model; weak uses template schema 1.1 |
_curated/ |
Visually curated copy that does not modify the base results |
_curated/_curated_report.json |
Sources of curated frames, exclusion list, and full-chain hashes |
Parameter selection
- Use by default
scene, suitable for mobile apps, web pages, and chat screen recordings. - Enable by default
--temporal-select; use--no-temporal-select。 - When scrolling continuously and quickly and worrying about missing pages, lower
--motion-chunk-seconds; if the results are still too dense, raise it again. The default value will also be automatically multiplied by 0.8, 1.25, or 1.45 according to scroll overlap. - OCR is an optional enhancement; when it is not installed, there must be a clear prompt and graceful degradation; it may retain a few more frames than pure visual mode in order to protect newly added amounts or body text, and "fewer frames" cannot be taken as the sole objective.
--keep-drop-candidatesIt is only used for manual troubleshooting of missed frames in the base layer and is not equivalent to multimodal auditing; the multimodal production path will not restore these images.- By default, only
_report.jsonandextraction_meta.jsonare archived to Skillarchive/(screenshots and videos are not duplicated, to avoid redundant disk usage); for batch regression or temporary verification use--no-archive, without changing the base report and the raw output directory. - When model capability is weak, use
--profile weak; do not compensate for model capability by increasing the number of audit groups; prioritize keeping tasks small, targets high-definition, and no-op conservative.
Dependencies
System dependencies
| Dependency | Installation method |
|---|---|
| ffmpeg ≥ 5.0 | macOS: brew install ffmpegLinux: sudo apt-get install ffmpeg |
| Python ≥ 3.10 | macOS: brew install python |
| uv | macOS: brew install uv |
Python packages
| Package name | Purpose | Installation command |
|---|---|---|
Pillow>=10.0.0 |
Image metrics, temporal cluster analysis, and contact sheets | uv run scripts/extract.py --help Auto-prepare |
rapidocr-onnxruntime |
Optional OCR content increments and text deduplication | uv run --with rapidocr-onnxruntime scripts/extract.py -i <视频> --ocr-dedup |
The same optional dependency is also used for multi-anchor classification of evidence cues:uv run --with rapidocr-onnxruntime scripts/prepare_evidence_leads.py -i <基础输出目录>. When missing, it clearly degrades to visual subjects and temporal ordering, without affecting base frame extraction.
Acceptance and security boundaries
- Run
uv run scripts/check_pipeline.py --case all, requiring all domain regressions to pass and outputDOMAIN_CHECKS_PASSED。 - Output transactions can be run separately
uv run scripts/check_pipeline.py --case transactional-output, and must cover bad videos, invalid parameters, symbolic links, unknown files, downstream artifact protection, successful replacement, and commit rollback. - Behavior changes must be retested with representative videos and visually spot-checked via contact sheets or frame-by-frame viewing; static checks or a reduction in frame count alone cannot prove accuracy.
- Do not modify the original video, and do not upload complete screen recordings to the cloud by default.
- Retain base frames and full-chain SHA256; the evidence cue layer may only rank, classify, and summarize, and the visual deduplication layer may only generate non-destructive curated copies.
- When it is uncertain whether amounts, identities, addresses, commitments, and key conversations in legal evidence are covered, choose to retain them and prompt for manual review.
Required permissions and security notes
- Local file access: read videos explicitly provided by the user; images and review JSON are written only to the explicit output directory, and Skill
archive/writes only to_report.jsonandextraction_meta.jsonmetadata copies; do not scan unrelated directories. - Local process execution: invoke the local
ffmpeg、ffprobeand Python scripts with argument arrays, without using shell concatenation to execute user input. - Transactional replacement: new results are written only to staging at the same level as the target; the old directory is not modified before a successful commit. The old directory must pass an ownership marker, or a consistency check between the legacy report and the actual frame manifest; if evidence cues, visual reviews, curated results, symbolic links, or unknown files are found, refuse to overwrite and prompt to use a new directory.
- Network and dependencies: frame extraction and OCR can both run offline;
uv runOn first run, when Pillow is missing, dependencies may be downloaded over the network. Whether multimodal auditing uploads contact sheets depends on the current model provider; before processing unmasked evidence, first confirm its privacy policy. - Credentials and environment variables: This Skill does not read API keys, tokens, passwords, or cloud credentials, nor does it call external vision APIs on its own.