VideoCrafter is an open-source video generation and editing toolbox designed to help users create video content. It currently includes two models: Text-to-Video (Text2Video) and Image-to-Video (Image2Video).
Purpose:
- General text-to-video generation: generate videos based on text descriptions.
- General image-to-video generation: generate videos based on input images.
Key capabilities:
- VideoCrafter2 shows significant improvements over VideoCrafter1 under limited data, providing better motion and concept combination capabilities.
- Provides high-resolution video generation, such as a 640x1024 I2V model, as well as a 512x320 VideoCrafter2 checkpoint.
- Supports downloading pretrained models via Hugging Face.
How to use:
- Set up the environment: installation via Anaconda is recommended.
- Text-to-video: download the pretrained T2V model (via Hugging Face), place model.ckpt into checkpoints/base_512_v2/model.ckpt, then enter the corresponding command in the terminal.
- Image-to-video: download the pretrained I2V model (via Hugging Face), place model.ckpt into checkpoints/i2v_512_v1/model.ckpt.
- Local Gradio demo: download the pretrained T2V and I2V models and place them into the corresponding directories according to the above guidelines.
Applicable scenarios:
- Research purposes, such as academic research on video generation and editing.
- Video content creation in personal or non-commercial projects.
Limitations:
- This repository is for research purposes only and is limited to personal, research, or non-commercial use.
- The technical report is still being prepared and is not yet available.
- Other specific features, fees, pricing, platforms, or performance information are not yet available.
