SuperAPI AI Directory
Partner with us中文AI newsFavorites
SkillsVideo Producer

Video Producer

Design & creation

View repository

Updated 10-15 17:32

About this skill

Skill: Video Producer (Industrial-grade short video automation workflow)

Prerequisites

  • ffmpeg (handles video aspect ratio, camera movement, splicing, audio mixing)
  • edge-tts (generates a continuous main audio track with emotion)
  • Built-in tool generate_video (only used to generate text-free, non-hands-on emotional/character B-roll)

Workflow (workflow steps)

When the user requests generating a short video, strictly follow the steps below,and you must wait for user confirmation at key nodes：

Step 0: Asset inventory and confirmation (Asset Check)

  • When the user only provides a script, **first [mandatory pause]** and ask the user: "Is there any reference material (images, screen recordings, official videos, webpage URLs, etc.) that needs to be interspersed in the video?"
  • If the user answers "no", then plan it as a storyboard script of100% purely AI-generated visuals.
  • If the user answers "yes" or has already uploaded materials, first analyze the content of these original materials.For video materials uploaded by the user, you must proactively use ffprobe to extract the exact physical duration (accurate to the second). If a webpage URL is provided, the Agent needs to prepare to call browser automation tools for screenshots or recording.

Step 1: Refine storyboard and script confirmation (Human-in-the-loop mandatory wait)

  • Based on the asset inventory in Step 0, split the script into specific storyboard scripts (Blocks).
  • [Word count constraint based on duration]: When planning narration, you must strictly refer to the physical duration of the original video material to match the number of dialogue words. Calculate according to the normal Chinese TTS speaking rate (about 4.5 characters/second). For example: for a 10-second screen recording clip, the assigned dialogue must be controlled between 40-45 characters to prevent serious audio-visual disconnection.
  • [Fast pace and time limit for static visuals]: To maintain the fast pace of short videos, the display time of static screenshots or images must bestrictly limited to within 3 seconds. If the narration voiceover for that segment exceeds 3 seconds,you must split the storyboard: show the screenshot for the first 3 seconds, and use related AI-generated dynamic visuals (B-roll) to seamlessly fill the remaining time. Never cut the original narration text or let a single image stay frozen for too long.
  • [Dynamic ending CTA]: The Call to Action (CTA) or Logo display at the end of the video cannot use just a static image. It must be planned as a 3D dynamic video clip generated using generate_video combined with Logo assets (as an attachment), to enhance the commercial quality of the ending.
  • [Standard storyboard table formatting]: The storyboard script must be output in a clear Markdown table format. Never omit the richness of the visual prompts just to be concise. The table must include the following columns, and the content must satisfy:
    • Storyboard number: e.g., Block 1.
    • Narration (VO) / sentence breaks: strictly split into lines within 15 characters, and absolutely do not add performance directions or emotional filler.
    • Duration: the estimated or physically extracted exact duration.
    • Visual assets and visual presentation (Visuals & Prompts)：
      • For hands-on materials, specify the extraction interval and processing method (e.g., proportional centering + Gaussian blur base plate).
      • For AI-generated visuals, write an extremely detailed English Prompt, must include: subject, action, environment, lighting (e.g., volumetric lighting, cinematic), camera movement (e.g., fast-paced dolly zoom), and style (e.g., Pixar 3D animated style).
  • [Mandatory pause]: send this detailed storyboard table to the user, and explicitly ask: "Please confirm whether the storyboard script and visual planning need modifications?"
  • Note: only after the user explicitly replies "confirm" or confirms there are no issues may you continue with the subsequent steps. It is absolutely forbidden to skip this step and start rendering or final assembly without authorization.

Step 2: Generate Master Audio & Subtitles (Master Audio & Subtitles)

  • It is strictly forbidden to generate audio in segments!
  • [Long sentence splitting constraint]: before generating speech, if it is vertical screen or 3:4 ratio, long sentence copy must be cut into extremely short sentence fragments (a single sentence should preferably not exceed 12-15 characters), for example splitting "它自带云端电脑Yolo模式下写代码全包了" into multiple segments. This prevents a single subtitle line from overflowing the screen edge and makes the video rhythm appear more crisp.
  • Use edge-tts to generate the complete copy into one audio file in one click (e.g., master_audio.mp3），and be sure to add the --write-subtitles subtitles.srt parameter, extract the subtitle file perfectly aligned with the sound。
  • It is recommended to use emotionally rich timbres, such as zh-CN-XiaoxiaoNeural or zh-CN-YunxiNeural, and appropriately increase the speech rate (e.g., --rate=+10%) to match the fast pace of short videos.

Step 3: Original quality asset processing (UI / screenshots / URL) - FFmpeg / Playwright

For all screenshots or videos with text, code, or UI interfaces,absolutely do not throw them to AI for generation：

  • Ordinary images/screenshots: based on the number of words in the voiceover corresponding to that storyboard, calculate how long the image needs to stay (for example, 3-5 seconds). Use FFmpeg to convert it into a static video.[Any motion effects are strictly forbidden]: absolutely do not make any subtle motion effects (such as a slow center zoom-in or breathing effect). The processing method is very simple: just display the original image centered, and place a Gaussian Blur background of that image underneath to fill the extra areas, completely preserving the pure stability and minimalist beauty of the original image.
  • Webpage URL assets (fast scrolling recording): if the user provides a URL, call a browser automation tool (such as Playwright) to open the page. When recording the scrolling video,maintain the regular widescreen or adaptive ratio(it is strictly forbidden to directly force-record to 3:4 or a narrow-side ratio, to avoid the left and right information being violently cropped). Principles for implementing the recording:
    1. Zero waiting on the first screen: ignore network loading and long pauses, and get straight to the point.
    2. High-frequency fast scrolling: when writing automation scripts (e.g., setInterval control), you must increase the scroll step size (e.g., 12 pixels) and shorten the interval (e.g., 16ms) to achieve a rapid and silky scrolling effect.
    3. Trim the head and tail: after recording is complete, be sure to use FFmpeg/ffprobe to precisely cut off the Loading or white-screen time at the beginning, keeping only the high-density informationpure scrolling substantive footage, with the duration controlled within 5 seconds, to adapt to the extremely fast pace of short videos.
  • Video and screen recording asset processing (Gaussian blur base plate): For practical screen recordings with mismatched dimensions or the scrolling videos generated from URLs above, when converting to the target aspect ratio (e.g., 3:4),never use black bar padding (Pad black bars). You must handle it like screenshots: scale the original video proportionally and center it, while placing a Gaussian Blur of that video frame underneath as a dynamic background to fill the excess areas, fully maintaining the visual continuity and premium feel.

Step 4: AI Ambient Clip Generation (B-roll / Characters) - generate_video

For emotional shots such as hooks, transitions, and people drinking coffee that do not contain complex text:

  • [Strict time control, crisp and clean]: When calling generate_video , thedurationSeconds parameter muststrictly matchthe physical duration required for that segment's voiceover in the storyboard script (API allowed range 4-15 seconds). The visuals should stop as soon as the voiceover finishes; never generate overly long, dragged-out, useless footage.
  • [Cost and resolution control]: To strictly control API costs and greatly improve generation efficiency, when calling generate_video you must set the resolution parameter to the lowest quality (i.e., 480p), unless the user has an explicit high-definition delivery requirement.
  • [Concrete Prompt writing]: AI video generation Prompts must be highly detailed, andthe visual presentation must precisely correspond to the voiceover concept. The use of vague abstract terms is strictly prohibited. For example: when the voiceover mentions "WeChat and Feishu", you cannot just write "social media integration"; you must write "a glowing green chat bubble and a blue paper plane icon floating in a 3D tech space"; when it mentions "AI brain", you must be specific to "a glowing futuristic mechanical brain pulsing with neon blue lights".
  • [Action rhythm constraints]: Actively add fast-paced, decisive action, quick motion and similar words to the Prompt to force the AI to speed up action execution and avoid the "slow-motion sluggishness" common in AI videos.
  • Set generateAudio: false(silent generation).
  • To avoid real-person privacy review and maintain character consistency, add "Pixar style 3D animated character" to the Prompt, and pass the user's "character card" as attachments .
  • Set the corresponding aspectRatio。

Step 5: Forced Normalization & Assembly

  • [Absolute format uniformity]: Before final assembly, all generated assets (whether still images, screenshots, or generated videos) must be forcibly reshaped via FFmpeg into a unifiedtarget resolution (e.g., 720x960) and target frame rate (e.g., -r 30). If the parameters are inconsistent, subsequent compositing will inevitably cause black screens or audio breaks.
  • Create concat_list.txt, arrange all normalized video clips in order, and concatenate them silently into a single main video.

Step 6: Final Mix & Hardcoded Subtitles

  • BGM (Fully automatic soundtrack)：BGM must be added automatically; never ask the user for it. Use curl/wget to download real BGM with a clear rhythm from a public library, trim it to the corresponding duration, and add audio fade in/out (afade).
  • Global mixing: Use the FFmpeg amix filter to combine the original main audio track at 100% volume (master_audio.mp3) with BGM reduced to 15%-25% volume for a smooth mix, ensuring it does not overpower the vocals.
  • Commercial subtitle parameters: Call the FFmpeg subtitles filter to burn the .srt extracted in Step 2 into the bottom of the frame.
    • Idiot-proof settings that never overflow: In vertical short videos, you must set a relatively moderate font size (such as FontSize=14), leave a bottom safe zone to avoid the comment bar (such as MarginV=35), and enable stroke and shadow to ensure visibility against any background.
    • Recommended template：subtitles=subtitles.srt:force_style='Fontname=SimHei,FontSize=14,Bold=-1,PrimaryColour=&H00FFFFFF,OutlineColour=&H00000000,BorderStyle=1,Outline=1,Shadow=1,MarginV=35,Alignment=2'
  • Use -shortest to trim excess audio or video and output the final MP4 with subtitles and polished background music.

Notes

  • Always remember: AI is only responsible for generating "atmosphere" and "non-specific faces"; FFmpeg is responsible for ensuring "information clarity (text/UI)" and "global alignment".
  • When processing materials, strictly use force_original_aspect_ratio=decrease to prevent any material from being stretched or distorted.