SuperAPI AI Directory
Partner with us中文AI newsFavorites
Skillsbyted-mediakit-voiceover-editing

byted-mediakit-voiceover-editing

Design & creation

View repository

Updated 10-15 17:30

About this skill

I. Modes and Credentials

1.1 Three Execution Modes

Mode Description Required Environment Variables ASR Method
apig SkillHub gateway proxy, Bearer Token authentication ARK_SKILL_API_BASE + ARK_SKILL_API_KEY(container-injected) + VOLC_SPACE_NAME + ASR_API_KEY + ASR_BASE_URL Doubao Speech Large Model
cloud Direct connection to Volcano Engine OpenAPI, HMAC signing VOLC_ACCESS_KEY_ID + VOLC_ACCESS_KEY_SECRET + VOLC_SPACE_NAME + ASR_API_KEY + ASR_BASE_URL Doubao Speech Large Model
local Fully local execution, no cloud service required None (optional EXECUTION_MODE=local） Qwen3-ASR local inference

Priority：apig > cloud > local. Auto-detection checks environment variables in this order; when parameters are missing, it prints .env path and the list of missing variables and automatically downgrades.

1.2 Credential Configuration

  • .env file location：<SKILL_DIR>/.env
  • The script first readsprocess environment variables, then uses .env fills in items that are not set(does not override container-injected values)
  • ARK_SKILL_* is usually injected by the deployment container,no needto manually write to .env
  • Missing parameters do not block: terminal input() interaction is not used; when parameters are missing, a prompt is printed and it automatically downgrades to an available mode
  • Agent recommends that users configure variables throughediting .env fileorAgent file writing toolto avoid terminal paste issues
  • Security: create in the consolekeys containing only the required permissions; for testing, usean independent VOD space；.env do not commit to the repository

1.3 Mode Intent Recognition (Agent Must Read)

When the user expresses an intent to switch modes in the conversation, the Agent should recognize and execute it:

User Expression Recognized As Operation
"use local mode" / "do not use cloud" / "offline processing" EXECUTION_MODE=local Write .env or pass --mode local
"use cloud" / "use Volcano Engine" / "use AK/SK" EXECUTION_MODE=cloud Write .env or pass --mode cloud
"use gateway" / "use apig" / "use SkillHub" EXECUTION_MODE=apig Write .env or pass --mode apig

Isolation Requirement: The mode selection of each task is independent of the others. Switching to local in one task should not affect the mode of another task. The Agent should re-detect at the start of each task or follow the user's specification for that instance.

II. Execution Constraints (STRICT — NO EXCEPTIONS)

  1. Use only the following scripts, creating new scripts is prohibited
  2. Execute all steps in order, skipping a step means task failure
  3. Before executing any script, you must first cd <SKILL_DIR>/scripts
  4. Each CHECKPOINT must verify the artifact before continuing
  5. Any step failure muststop immediately and report, and must not continue
  6. Step 4/5/6 outputprohibitedsimplified versions, placeholders, examples, or omitted content
  7. The final output must remain as-is and cannot be modified in any way
  8. Whether the review page opens automatically is determined by TALKING_VIDEO_AUTO_EDIT_REVIEW_AUTO_OPEN (forced): The Agent must not additionally execute open/start/xdg-open and other commands
  9. Timeout settings: Audio/video processing takes a long time, so the host timeout should be set to 60 minutes

Division of responsibilities

Component Responsibilities Does not do
Script (rule engine) ASR optimization, candidate generation (marking positions + rule confidence + deletion suggestions) Does not make the final delete/keep decision
Host Agent (you) Semantic sentence segmentation, confirmation of verbal tic identification, candidate review, final delete/keep decision Does not modify the script

Core principle: the script provides candidates (including deleted_parts + cleaned_text), and the Agent makes the final decision.

3. Path rules

3.1 SKILL_DIR and Output-dir

  • SKILL_DIR：byted-mediakit-voiceover-editing directory path
  • PROJECT_ROOT: derived from scripts/project_paths.py :
    1. Environment variable VOICEOVER_EDITING_PROJECT_ROOT If set, use it
    2. Otherwise it is <SKILL_DIR> of parents[2](move up 3 levels along the parent chain, does not depend on intermediate directory naming)
  • Output-dir：<PROJECT_ROOT>/output/<素材名>/; if the directory already exists, it automatically increments to <素材名>_01、<素材名>_02...
  • When the script starts, it prints path derivation logs for easy debugging confirmation

3.2 Material name derivation

Source Derivation rule Example
URL Take the last segment and remove the extension https://x.com/video.mp4 → video
Local file Take the filename and remove the extension /path/Test_Video_720p.mp4 → Test_Video_720p
DirectUrl Take the FileName and remove the extension test.mp4 → test
Vid Take the Vid value v0xxx → v0xxx

3.3 Derive output-dir from context

  • Derivation priority(try in order):
    1. If the conversation history/command parameters already explicitly pass --output-dir output/<子目录> → use it directly
    2. If it cannot be obtained from the conversation history → ask the user to specify
  • Agent must notscan the repository to infer output-dir

3.4 Repeated processing

Before writing any output file/directory, if the target already exists,you must prompt the user：

  • Directory already exists: "Delete the original directory? [Delete/Keep and create new (01)]"
  • File already exists: "Delete/Overwrite/Keep?"
  • After a 20-second timeout, default to "Keep and create new (01)"

4. Script list

Before execution, you must cd <SKILL_DIR>/scripts

Script Purpose
./scripts/setup.sh Environment check and dependency installation
./scripts/step2_confirm_config.py Step 2: Generate a checkpoint after confirming filler words/stutter words is complete (prevents skipping steps)
./scripts/pipeline_url_to_asr.py Step 3: URL → ASR pipeline (supports --mode local/cloud/apig）
./scripts/merge_asr_words.py Step 4: When the output is missing words, merge from raw
./scripts/prepare_export_data.py Step 6a: Data preprocessing (--width --height --write-step6）
./scripts/serve_review_page.py Step 6b: Review page static service + data saving + export proxy
./scripts/export_server.py Export service (independent process, receives review page POST)
./scripts/vod_direct_export.py Step 6c: VOD export task submission and query

5. Required steps

Execution order: Execute all steps in order; skipping steps means the task fails For the complete checklist for each Step, see references/执行步骤/ under the step-by-step documents.

Step Description Document
Step 1 Environment Check and Dependency Installation 1. Environment Check.md
Step 2 Filler Word/Pause Word Confirmation and Rule Update 2. Filler Word Prompts and User Behavior Update.md
Step 3 URL → ASR Pipeline and Candidate Generation 3. URL to ASR Pipeline and Candidate Generation.md
Step 4 ASR Semantic Correction (Agent Execution) 4. ASR Semantic Correction.md
Step 5 Spoken Word Editing (Agent Execution) 5. Spoken Word Editing.md
Step 5.5 Review Logic Confirmation 5.5 Review Logic Confirmation.md
Step 6a Data Preprocessing 6a. Data Preprocessing.md
Step 6b Review and Export 6b. Review and Export.md
Step 6c VOD Export Task Submission and Query 6c. VOD Export Task Submission and Query.md

VI. Artifact Comparison Table

Artifact File Generation Step Description
step1_preuploaded.json Step 3 Material upload/registration result (including _execution_mode）
step3_voice_separation_result.json Step 3 Vocal separation result
step5_asr_raw_*.json Step 3 ASR raw transcription
step5_asr_optimized.json Step 4 ASR after semantic correction
step6_speech_cut.json Step 5 Spoken word editing decisions
review_import_data.json Step 6a Review page data (including _execution_mode、track、sentences）
export_request.json Step 6a / Review Save Export request (this file will be updated synchronously after "Save" on the review page)
export_submit_*.json Step 6b/6c Final submitted export data

VII. Review Page and Data Linkage

7.1 Mode Awareness

The review page automatically identifies the current mode through the review_import_data.json field in _execution_mode , and in the interface:

  • Displaymode badge(APIG blue/cloud green/local orange)
  • Adjustexport button text(local mode displays "Local Export Video")
  • Adjustexport success message(local mode displays the output file path, cloud displays OutputVid + PlayURL)

7.2 Local Mode Review Page

Local modefully supports the review page. The Source field uses the http://127.0.0.1:<port>/local-media/<绝对路径> format, and local files are accessed through the serve_review_page.py route proxy of /local-media/ .

7.3 Data Linkage (Review Modification ↔ Direct Export Synchronization)

The review page provides two action buttons:

Button Function Data Flow
💾 Save Review Persist modifications to disk POST /api/save-review → Update review_import_data.json + Regenerate export_request.json
Export Directly trigger video export POST /export → apply_review_to_export → export_submit_*.json → ffmpeg/VOD

Key: After the user makes modifications on the review page, clicking "💾 Save Review" will synchronize the modifications to disk. After that, even if the review page is closed, when the Agent exports directly through vod_direct_export.py --output-dir <输出目录> submit --wait , it will also read the updated export_request.json, ensuring data consistency.

⚠️ Key Constraint: When calling vod_direct_export.py ,--output-dir must be written before the submit/query subcommandbefore. One-line invocation format:

cd SKILL_DIR/scripts && source .venv/bin/activate && python vod_direct_export.py --output-dir <绝对路径> submit --wait

7.4 Review Page Service Endpoints

Endpoint Method Description
/ GET Review page HTML
/api/review-data GET Return review_import_data.json
/api/mode GET Return the current execution mode
/api/save-review POST Save review changes (write back to review_import_data + regenerate export_request)
/export POST Trigger export (local: ffmpeg; cloud/apig: vod_direct_export)
/local-media/<path> GET Local mode media file proxy

VIII. FAQ

Symptom Handling
Local file went through DirectUrl mode The local file must be passed asthe first positional argument;--directurl Only used for existing FileName in the VOD space
step5 write failed Must write to output/<文件名>/step5_asr_optimized.json, writing to the output root directory is prohibited
concat rule should be deleted but audio is still playing actionTime must be looked up from step5 wordsKeep onlythe ms of the part
Duplicate file not prompted Before writing, you must check whether the target exists, and handle it according to rule 3.4
step6 correction not taking effect Ensure the top level of step6 is optimized_segments or sentences; run --write-step6 to write back
segment start/end times inaccurate Step 6a will correct based on step5 words
delete not in deleted_parts Each action: delete segmentmusthave a corresponding entry in deleted_parts
Review page changes lost after closing Click "💾 Save Review" before closing to persist to disk
Review page local resource 404 Confirm the Source field is in /local-media/ URL format; check whether serve_review_page.py is running normally
Blocked after missing parameter prompt No longer use input(), automatically degrade when parameters are missing and print the .env path prompt

IX. Subtitle Visibility (Alpha)

  • Field：textElement.Extra[transform].Alpha（0～1）
  • Meaning：0 hidden (not rendered to canvas),1 shown
  • Deleted state: Alpha set to 0;Restore: Alpha set to 1