Today let's share something suitable for doing AI video locally.
To be honest, current AI videogeneration technology is already quite mature, with only one drawback: expensive. Billing is basically per second, and it's hard to guarantee getting it right in one take. Your wallet really can't take it.
Today this GitHub Alibaba Wan2.1 project is quite ingenious. It can run locally with as little as 8G of VRAM, which I believe most computers nowadays can meet.
01. Project Introduction
Wan2.1 is a complete set of video generation foundation models open-sourced by Alibaba's Tongyi team.
Project address:https://github.com/Wan-Video/Wan2.1
Wan2.1 includes not only text-to-video, but also image-to-video, first-last frame interpolation, video editing, reference subject generation, and text-to-image tasks.
Although Wan2.2 has now been released, Wan2.1 is relatively complete in low-VRAM usage, especially suitable for local experiments and secondary development.
Many video generation projects release only a single model, such as only text-to-video, or only image-to-video. Wan2.1 is more like a toolbox of video models, with different models handling different tasks.
- For text-to-video, you can start with Wan2.1-T2V-1.3B, focused on 480P, suitable for local testing and low-VRAM devices;
- if you want better image details and motion performance, choose Wan2.1-T2V-14B, which supports 480P and 720P.
- Image-to-video corresponds to Wan2.1-I2V-14B, and the official release provides 480P and 720P versions respectively.
- To control the starting and ending frames of a video, you can use Wan2.1-FLF2V-14B-720P.
- Wan2.1-VACE-1.3B leans toward video creation and editing, aimed at 480P scenarios, with a lower barrier to entry
- Wan2.1-VACE-14B supports 480P and 720P, suitable for reference subject generation, local modifications, video subject replacement, and more complex video editing.
The official release provides integration methods such as Hugging Face, ModelScope, Diffusers, ComfyUI, and Gradio.
This time I chose Wan2.1-T2V-1.3B, which can run with 8G of VRAM. On an RTX 4090, generating about 5 seconds of 480P video takes about 4 minutes according to the official reference time. The 1.3B model can also try 720P, but the official documentation states that 720P is less stable than 480P.
After the user inputs a prompt, Wan2.1 first uses the T5 encoder to turn the text into semantic vectors. The semantic vectors describe the image content, subject actions, scene, camera movement, visual style, lighting, and color.
Each subsequent Transformer Block injects the text semantics into the video generation process through Cross-Attention. Wan2.1 supports multilingual input, and the official release especially emphasizes Chinese and English prompt capabilities.
Wan2.1 also supports prompt expansion. For example, if you write:
A cat is making a cake in the kitchen.
The expansion model can supplement the cat's appearance, the kitchen environment, the making actions, camera distance, lighting, style, and action sequence.
But when actually writing prompts, the recommended structure is:
subject + action + scene + camera + lighting + style + motion rhythm.
For example:
A girl in a yellow raincoat stands on a rainy night street, slowly turning her head to look at the camera, with vehicles passing in the distance and neon lights reflected in the puddles. The camera slowly pushes forward from a medium shot, shallow depth of field, realistic film style, cool blue tones.
Wan2.1 uses a Diffusion Transformer, and the generation process can be roughly understood as:
- Create a segment of random noise.
- The model determines what should appear in the noise based on the prompt.
- Iteratively denoise multiple times.
- Gradually form the subject, background, and action.
- VAE decodes the latent variables into video.
02. Project testing
Before running Wan2.1, you need to prepare the corresponding Python environment first. The official dependencies mainly include:
- PyTorch ≥ 2.4.0
- torchvision ≥ 0.19.0
- Diffusers ≥ 0.31.0
- Transformers ≥ 4.49.0
- Accelerate
- Flash Attention
- Gradio
- OpenCV
- NumPy 1.x
The basic installation method is as follows, using Windows PowerShell as an example:
1 Set-Location "想安装 Wan 2.1的文件夹路径"
2 git clone https://github.com/Wan-Video/Wan2.1.git
3 Set-Location .\Wan2.1
4
5 python -m venv .venv
6 .\.venv\Scripts\python.exe -m pip install --upgrade pip
7 .\.venv\Scripts\python.exe -m pip install -r requirements.txtThe .venv here is the Python virtual environment for the Wan2.1 project itself. For subsequent dependency installation and script execution, it is recommended to use the Python in .venv to avoid polluting the system Python environment.
Then download the 1.3B model:
1 Set-Location "Wan 2.1的文件夹路径"
2 .\.venv\Scripts\python.exe -m pip install "huggingface_hub[cli]"
3
4 .\.venv\Scripts\huggingface-cli.exe download Wan-AI/Wan2.1-T2V-1.3B `
5 --local-dir .\Wan2.1-T2V-1.3BAfter installation, enter the .venv virtual environment in the Wan2.1 project directory and you can use it. In PowerShell, enter it in the following format:
Set-Location "Wan 2.1的文件夹路径"
# 进入 Wan2.1 项目目录。后面的模型路径、输出路径、generate.py 都基于这个目录执行。
New-Item -ItemType Directory -Force .\outputs
# 创建 outputs 输出文件夹。如果文件夹已经存在，-Force 会让 PowerShell 继续执行，不报错。
.\.venv\Scripts\python.exe generate.py `
# 使用 Wan2.1 项目里的 .venv 虚拟环境 Python，运行 generate.py 生成脚本。末尾的 ` 表示命令还没结束，下一行继续。
--task t2v-1.3B `
# 指定任务类型。t2v 表示文生视频，1.3B 表示使用 13 亿参数版本。
--size 832*480 `
# 设置视频分辨率。这里是宽 832、高 480，也就是 480P 横屏。
--frame_num 65 `
# 设置生成帧数。Wan2.1 通常按 16 FPS 导出，65 帧大约是 4 秒。
--ckpt_dir .\Wan2.1-T2V-1.3B `
# 指定模型权重目录，也就是下载好的 Wan2.1-T2V-1.3B 文件夹。
--offload_model True `
# 开启模型卸载，把暂时不用的模型部分放到 CPU 内存里，降低显存压力，但会变慢。
--t5_cpu `
# 让 T5 文本编码器在 CPU 上运行，省显存。T5 负责理解提示词。
--sample_steps 20 `
# 设置采样步数。步数越高通常画面更稳，但耗时更久。20 步比较适合质量和速度平衡。
--sample_shift 8 `
# 控制采样过程的时间分布。1.3B 常用 8 到 12，8 是比较稳的默认选择。
--sample_guide_scale 6 `
# 控制提示词引导强度。数值越高越听提示词，但太高可能让画面变硬或变形。1.3B 用 6 比较稳。
--save_file .\outputs\moonlit_train_window.mp4 `
# 指定输出视频文件名和保存位置。这里会保存到 outputs 文件夹里。
--prompt "提示词放这里"
# 视频提示词。描述主体、场景、动作、镜头、光线、风格和运动节奏。这个也要这种格式When we did the first case, we first used a smaller number of frames to slowly test where the upper limit of our machine is.
Wan2.1 default export is usually 16 FPS, so: 65 frames ÷ 16 FPS ≈ 4 seconds.
Common reference: 33 frames ≈ 2 seconds, 49 frames ≈ 3 seconds, 65 frames ≈ 4 seconds, 81 frames ≈ 5 seconds.
case 1 33 frames, sampling steps 12
PowerShell input:
Set-Location “D:\360MoveData\Users\win\Desktop\蒸馏\Wan2.1”
.\.venv\Scripts\python.exe generate.py `
–task t2v-1.3B `
–size 832*480 `
–frame_num 33 `
–ckpt_dir .\Wan2.1-T2V-1.3B `
–offload_model True `
–t5_cpu `
–sample_steps 12 `
–sample_shift 8 `
–sample_guide_scale 6 `
–save_file .\outputs\quick_test.mp4 `
–prompt “A small robot walking through a rainy neon street, cinematic lighting.”
It can be seen that at 33 frames and 12 steps, the subject is already clear, the robot's form is basically stable, and the neon street and wet-ground reflections have appeared. The problem is that the image is relatively empty, the sense of rain is not obvious, and the robot's movements are very stiff.
case 2 33 frames, sampling steps 20
PowerShell input:
.\.venv\Scripts\python.exe generate.py `
–task t2v-1.3B `
–size 832*480 `
–frame_num 33 `
–ckpt_dir .\Wan2.1-T2V-1.3B `
–offload_model True `
–t5_cpu `
–sample_steps 20 `
–sample_shift 8 `
–sample_guide_scale 6 `
–save_file .\outputs\robot_neon_street.mp4 `
–prompt “A small friendly robot walking alone through a rainy neon street at night, wet asphalt reflecting red and blue signs, soft steam rising from street vents, cinematic cyberpunk atmosphere, low-angle tracking shot, slow camera movement, detailed metal body, glowing eyes, realistic rain, volumetric lighting, shallow depth of field, high detail, smooth motion, 35mm film look”
The image is clearly better than before. The red and blue neon, wet road surface, and night scene atmosphere are all more stable, the robot is centered, and the subject recognition is very good. But 2 seconds is too short, the range of motion is not enough, and the robot looks more like it is standing on the street and moving slightly, not really like “walking through”.
case 3 49 frames, sampling steps 20
PowerShell input:
Set-Location “D:\360MoveData\Users\win\Desktop\蒸馏\Wan2.1”
.\.venv\Scripts\python.exe generate.py `
–task t2v-1.3B `
–size 832*480 `
–frame_num 49 `
–ckpt_dir .\Wan2.1-T2V-1.3B `
–offload_model True `
–t5_cpu `
–sample_steps 20 `
–sample_shift 8 `
–sample_guide_scale 6 `
–save_file .\outputs\robot_neon_street_v2.mp4 `
–prompt “A cinematic 3-second video of a small friendly robot walking alone through a rain-soaked neon street at night. The robot has a compact metal body, round glowing eyes, and subtle reflections on its wet surface. Red, blue, and purple neon signs reflect on the asphalt. Light rain falls through the frame, steam rises from street vents, and distant shop lights glow softly in the background. Low-angle tracking shot, slow forward camera movement, realistic rain, volumetric lighting, shallow depth of field, smooth walking motion, detailed cyberpunk street, moody atmosphere, 35mm film look, high detail.”
This version already has a sense of space. The fog, ground reflections, and distant shop lights are all more complete, and the robot blends more naturally with the environment. 3.06 seconds feels more comfortable in motion than the previous two, and the image looks more like a complete shot.
case 4 65 frames, sampling steps 20
PowerShell input:
Set-Location “D:\360MoveData\Users\win\Desktop\蒸馏\Wan2.1”
.\.venv\Scripts\python.exe generate.py `
–task t2v-1.3B `
–size 832*480 `
–frame_num 65 `
–ckpt_dir .\Wan2.1-T2V-1.3B `
–offload_model True `
–t5_cpu `
–sample_steps 20 `
–sample_shift 8 `
–sample_guide_scale 6 `
–save_file .\outputs\moonlit_train_window.mp4 `
–prompt “A cinematic 4-second video seen from inside an old night train. A young traveler sits beside a rain-covered window, watching a quiet countryside pass by under moonlight. Reflections of warm carriage lamps shimmer on the glass. Outside the window, distant trees, small houses, and silver mist slide slowly across the frame. The camera begins with a close-up of raindrops on the window, then gently shifts focus to the traveler’s calm face and the moving landscape beyond. Soft film grain, realistic rain, warm interior light, cool blue moonlight outside, slow emotional camera movement, shallow depth of field, natural motion, detailed train cabin, quiet poetic atmosphere, 35mm film look, high detail.”
This one is longer and the sense of narrative is also better. The warm light inside the carriage, the cold blue outside the window, the raindrops, and the traveler's side profile are all very well done, the character has no obvious distortion, and the overall result is very stable. But it ran for a long time, which shows that the upper limit of my machine is basically here.
03. Digging a little
AI video is no longer just a tool for trying something new.
Goldman Sachs predicts that the creator economy may grow from about 250 billion USD to 480 billion USD by 2027. Demand for video content is rising, but for creators, the biggest pressure is still cost, the number of trial-and-error attempts, and production speed.
Wan2.1 cannot replace a professional video team, nor can it guarantee that every video will be finished in one take, but Wan2.1-T2V-1.3B pulls text-to-video from a platform where you pay by the second back into a workflow where it can be repeatedly tested on a local graphics card. Prompts can be modified slowly, parameters can be adjusted by yourself, and the cost of failure is also much lower.
Grand View Research predicts that the global AI video generation market will grow from 788.5 million USD in 2025 to 3.4416 billion USD in 2033, with a compound annual growth rate of about 20.3%.
Wan2.1 is suitable for placement at the front end of a real production pipeline: first use low-cost generation for sample clips, validating shots, testing prompts and styles, then decide which clips are worth refining, editing, interpolating frames, or handing off to more expensive commercial models.
Wan2.1 turns the video trial and error that used to be expensive, low-frequency, and built on piled-up budgets into a daily production process that ordinary computers can also take part in.
Original link:GitHub racked up 16K Stars, local text-to-video with just 8G VRAM