What is OpenStoryline
OpenStoryline is the first open-source video editing Agent from Xiaohongshu with a director's mindset. It enables full-process intelligent creation through natural language dialogue, from material retrieval and script writing to rendering output. The tool can accurately understand the emotions in the visuals, automatically match BGM and beat sync, and supports stylized imitation writing and fully conversational fine-tuning. OpenStoryline supports one-click saving of editing logic to train a personalized style avatar. Built on the MCP protocol, with zero GPU dependency and one-click Docker deployment, it makes AI a professional editor available on call.
Main Features of OpenStoryline
- Semantic-level material retrieval: Just describe the theme or atmosphere in one sentence, and the Agent can automatically search for and download matching online materials without manual searching.
- Intelligent segmentation and understanding: Supports automatic material cleaning, deduplication, and highlight moment extraction.
- Stylized copywriting: Feed it any reference text to precisely replicate the tone, sentence patterns, and rhythm.
- Audio-visual synchronization: First analyze the video content and editing rhythm, then generate high-quality commentary that matches the transitions and builds emotional progression.
- Emotional soundtrack: Recommends BGM from a private playlist based on the video's emotional goals, automatically performs music beat sync, and makes the visuals move with the drumbeat.
- Aesthetic matching: Just describe the feeling, such as "documentary style" or "emotional Vlog style," and the system automatically matches suitable voiceover timbres and font combinations.
- Natural language modification: Directly type your needs, such as "make this segment shorter," and the Agent can accurately understand unstructured instructions and take effect immediately.
- Interrupt and correct at any time: You can interrupt at any time during the creation process, issue new instructions in real time, and do not need to wait for rendering to finish.
- Non-destructive fine-tuning: Precisely performs local adjustments while maintaining the overall narrative rhythm, so modifications do not require starting over.
- Skill accumulation: Supports one-click saving of the complete editing logic of satisfactory works as a dedicated "Editing Skill." The next time it is called, the style can be instantly replicated for efficient batch production.
Technical Principles of OpenStoryline
- Agent Client (decision-making hub): The agent brain is composed of LLM/VLM. After receiving user instructions, it dynamically routes: casual chat is answered directly, while editing tasks are automatically decomposed and tools are invoked. Storyline Middleware serves as the key intermediate layer, managing context memory, adaptively completing missing parameters, and filtering redundant information to ensure stable and efficient system operation. Agent Memory is responsible for recording execution results offline and supports historical rollback.
- MCP Server (tool execution layer): Uses the Model Context Protocol to encapsulate capabilities such as scene segmentation, content understanding, timeline planning, and video rendering as atomic tool nodes. The standardized interface supports hot-pluggable extensions; developers can extend the Agent's capabilities by mounting new Python functions.
- Resources & Input (resource layer): Supports mixed input of text and multimedia, and can dynamically configure the LLM API. In addition to BGM and fonts, the resource library also stores user-accumulated Skills, enabling style replication and batch production.
OpenStoryline Project Address
- GitHub Repository：https://github.com/FireRedTeam/FireRed-OpenStoryline
- Online Experience Demo：https://fireredteam-firered-openstoryline.hf.space/
OpenStoryline Use Cases
-
Vlog Production: After uploading album materials, the Agent automatically completes script writing, editing, music scoring, and subtitle generation, producing visually striking videos without the need to learn professional software.
-
Trend Tracking: Quickly obtain matching materials through semantic search, greatly shortening the content production cycle from topic planning to finished product release.
-
Product Recommendations: Automatically identify product visual selling points and generate sales copy with matching rhythm, combined with emotional BGM and beat-syncing to increase conversion rates.
-
Documentary Production: Describe abstract requirements such as "restrained documentary feel," and the Agent automatically matches a professional audiovisual style with calm voiceover, minimalist fonts, and a soothing rhythm.
-
Knowledge Popularization: After complex knowledge points are input, the Agent automatically breaks them down into explainer videos with clear visual logic and moderate pacing, lowering the comprehension threshold for knowledge dissemination.