ShortGPT is an experimental AI framework focused on automated content creation for short-video channels such as YouTube Shorts and TikTok. It aims to simplify tasks such as video creation, asset sourcing, voiceover synthesis, and editing. Its most popular use cases include YouTube automation and TikTok creativity program automation.
Main Capabilities
- Automated editing framework: Simplifies the video production process through a video editing language oriented toward large language models.
- Scripts and prompts: Provides ready-to-use scripts and prompts for various LLM automated editing processes.
- Voiceover and content creation: Supports multiple languages, including English, Spanish, Arabic, French, Polish, German, Italian, Portuguese, Russian, Mandarin Chinese, Japanese, Hindi, Korean, and more, with support for over 30 languages when using EdgeTTS.
- Subtitle generation: Automatically generates video subtitles.
- Asset sourcing: Fetches images and video assets from the internet, connecting to the network and the Pexels API when necessary.
- Memory and persistence: Uses TinyDB to ensure long-term persistence of automated editing variables.
Framework Overview
- ContentShortEngine: Used to create short videos, handling tasks from script generation to final rendering, including adding YouTube metadata.
- ContentVideoEngine: Suitable for longer videos, responsible for generating audio, automatically sourcing background video assets, synchronizing subtitles, and preparing background assets.
- ContentTranslationEngine: Used for dubbing and translating an entire video, from major languages to more specific target languages. It accepts a video file or YouTube link, transcribes the audio, translates the content, dubs it in the target language, adds subtitles, and returns a video in the new language.
- EditingEngine: An automated editing engine that uses an editing markup language and JSON to break the editing process into manageable, customizable blocks that are easier for large language models to understand.
ShortGPT offers customization options, from language selection to adding watermarks, to meet different needs. As a framework, it is adaptable and flexible, enabling efficient and creative content creation.
Usage
Quick Start via Google Colab
If you do not want to install prerequisites on your local system, you can use Google Colab notebook. This option is free and requires no installation or setup. After clicking the link to enter the notebook, simply run the cells from top to bottom in order.
Colab link:https://colab.research.google.com/drive/1_2UKdpF6lqxCqWaAcZb3rwMVQqtbisdE?usp=sharing
Running Locally
To run ShortGPT locally, Docker is required. For detailed steps, see "installation-notes.md" in the repository. After running the Dockerfile, execute the runShortGPT.py script, and the Gradio interface will open on localhost port 31415 (http://localhost:31415）。
Tech Stack
- Moviepy: Used for video editing and rendering.
- OpenAI: Used to automate the entire process, including generating scripts and prompts.
- ElevenLabs: Used for speech synthesis, supporting multilingual voiceovers.
- EdgeTTS: Microsoft's free speech synthesis service, supporting more languages than ElevenLabs.
- Pexels: Used to collect background materials, accessing images and videos.
- Bing Image: Used to collect images, providing relevant visual materials.
Use Cases and Limitations
ShortGPT is suitable for content creators who want to automate the operation of short video channels, especially users who need to batch generate, translate, or dub videos. As an experimental framework, it may still be under active development, with more documentation coming soon. The project is open source, and contributions of new features, infrastructure improvements, or documentation enhancements are welcome.
