remotion-video

Remotion Video

A framework for creating MP4 videos programmatically with React.

Core Concepts

Composition - Definition of the video (dimensions, frame rate, duration)
useCurrentFrame() - Get the current frame number to drive animations
interpolate() - Map the frame number to any value (position, opacity, etc.)
spring() - Physics-based animation effects
- Arrange components on the timeline

Quick Start

Create a New Project

npx create-video@latest

After selecting a template:

cd <project-name>
npm run dev  # 启动 Remotion Studio 预览

Project Structure

my-video/
├── src/
│   ├── Root.tsx           # 注册所有 Composition
│   ├── HelloWorld.tsx     # 视频组件
│   └── index.ts           # 入口
├── public/                # 静态资源（音频、图片）
├── remotion.config.ts     # 配置文件
└── package.json

Basic Component Example

Minimal Video Component

import { AbsoluteFill, useCurrentFrame, useVideoConfig } from "remotion";

export const MyVideo = () => {
  const frame = useCurrentFrame();
  const { fps, durationInFrames } = useVideoConfig();

  return (
    <AbsoluteFill style={{ backgroundColor: "white", justifyContent: "center", alignItems: "center" }}>
      <h1 style={{ fontSize: 100 }}>Frame {frame}</h1>
    </AbsoluteFill>
  );
};

Register Composition

// Root.tsx
import { Composition } from "remotion";
import { MyVideo } from "./MyVideo";

export const RemotionRoot = () => {
  return (
    <Composition
      id="MyVideo"
      component={MyVideo}
      durationInFrames={150}  // 5秒 @ 30fps
      fps={30}
      width={1920}
      height={1080}
    />
  );
};

Animation Techniques

interpolate - Value Mapping

import { interpolate, useCurrentFrame } from "remotion";

const frame = useCurrentFrame();

// 0-30帧：透明度 0→1
const opacity = interpolate(frame, [0, 30], [0, 1], {
  extrapolateRight: "clamp",  // 超出范围时钳制
});

// 位移动画
const translateY = interpolate(frame, [0, 30], [50, 0]);

spring - Physics Animation

import { spring, useCurrentFrame, useVideoConfig } from "remotion";

const frame = useCurrentFrame();
const { fps } = useVideoConfig();

const scale = spring({
  frame,
  fps,
  config: { damping: 10, stiffness: 100 },
});

Sequence - Time Orchestration

import { Sequence } from "remotion";

<>
  <Sequence from={0} durationInFrames={60}>
    <Intro />
  </Sequence>
  <Sequence from={60} durationInFrames={90}>
    <MainContent />
  </Sequence>
  <Sequence from={150}>
    <Outro />
  </Sequence>
</>

AI Voice Narration Integration

Add AI voice narration to your video to achieve audio-video synchronization. Two approaches are supported:

Approach	Pros	Cons	Hardware Requirements	Recommendation
MiniMax TTS	Cloud cloning, extremely fast (<3 seconds), excellent audio quality	Billed per character	None	⭐⭐⭐ Top choice
Edge TTS	Zero configuration, free	Fixed voice, cannot be customized	None	⭐⭐

Approach Selection Flow

1. 首选 MiniMax TTS
   - 检测 API Key 是否配置
   - 测试调用是否正常（余额充足）
   - 如果成功 → 使用 MiniMax

2. MiniMax 不可用时
   → 退回 Edge TTS（使用预设音色 zh-CN-YunyangNeural）

Approach 1: MiniMax TTS (Recommended)

Cloud API approach, no local GPU required, extremely fast generation, excellent voice cloning results.

Configuration

Register at https://www.minimax.io (International version) or https://platform.minimaxi.com (China version)
Get an API Key
Upload audio to MiniMax Audio to clone a voice and get the voice_id

API Differences

Version	API Domain	Description
International version	`api.minimax.io`	Recommended, stable
China version	`api.minimaxi.com`	Requires a China account

⚠️ Common Errors：api.minimax.chat isthe wrong domain, and will return "invalid api key". Please make sure to use the correct domain from the table above.

Generate Script

Use scripts/generate_audio_minimax.py to generate audio, supporting:

Resume from breakpoint: existing audio files are automatically skipped
Real-time progress: shows generation progress to avoid waiting blindly
Automatic config update: automatically updates the Remotion scene configuration after generation completes

# 设置环境变量
export MINIMAX_API_KEY="your_api_key"
export MINIMAX_VOICE_ID="your_voice_id"

# 运行脚本
python scripts/generate_audio_minimax.py

Pricing Reference (2025)

Model	Price
speech-02-hd	¥0.1/thousand characters
speech-02-turbo	¥0.05/thousand characters

⚠️ MiniMax TTS Pitfall Experience

Problem	Cause	Solution
`invalid api key`	Used the wrong API domain	Use `api.minimax.io`for the international version, and `api.minimaxi.com`
config.ts syntax error `Syntax error "n"`	Python script in an f-string uses `",\\n".join()` produces a literal `\n` instead of an actual newline	See "Python TypeScript Generation Notes" below
No progress display for a long time	Running a command in the background shows no output	Run the script in the foreground, or use `tail -f` to view logs in real time

Python TypeScript Generation Notes

❌ Incorrect usage: using \n in an f-string produces a literal character

# 这会在生成的文件中写入字面的 \n 字符串，而非换行！
content = f'export const SCENES = [{",\\n".join(items)}];'

✅ Correct usage: handle string concatenation separately

# 先用真正的换行符拼接
scenes_content = ",\n".join(items)  # 在 f-string 外部拼接
# 再放入模板
content = f'''export const SCENES = [
{scenes_content}
];'''

Option 2: Edge TTS

No special hardware required, completely free, suitable for scenarios that do not require voice cloning.

Installation

pip install edge-tts

Recommended voice

Voice ID	Name	Style
zh-CN-YunyangNeural	Yunyang	Professional broadcast tone (recommended)
zh-CN-XiaoxiaoNeural	Xiaoxiao	Warm and natural
zh-CN-YunxiNeural	Yunxi	Sunny youth

Generate script

Use scripts/generate_audio_edge.py to generate audio:

python scripts/generate_audio_edge.py

Remotion audio synchronization

import { Audio, Sequence, staticFile } from "remotion";

// 音频配置（根据生成的时长）
const audioConfig = [
  { id: "01-intro", file: "01-intro.mp3", frames: 450 },
  { id: "02-main", file: "02-main.mp3", frames: 600 },
];

// 计算起始帧
const sceneStarts = audioConfig.reduce((acc, _, i) => {
  if (i === 0) return [0];
  return [...acc, acc[i - 1] + audioConfig[i - 1].frames];
}, [] as number[]);

// 场景渲染
{audioConfig.map((scene, i) => (
  <Sequence key={scene.id} from={sceneStarts[i]} durationInFrames={scene.frames}>
    <SceneComponent />
    <Audio src={staticFile(scene.file)} />
  </Sequence>
))}

Tutorial video architecture (scene-driven)

Core architecture for tutorial and explainer videos:Audio-driven scene switching。

Architecture overview

音频脚本 → TTS 生成 → audioConfig.ts → 场景组件 → 视频渲染

Key idea:

Audio determines duration: the duration of each scene is determined by the audio length
Scenes are chapters: one concept = one scene = one audio segment
Configuration is truth：audioConfig.ts is the single source of truth for audio-video synchronization

audioConfig.ts template

See templates/audioConfig.ts, including:

SceneConfig interface definition
SCENES array
getSceneStart() calculation function
TOTAL_FRAMES and FPS constants

Scene switching Hook

import { useCurrentFrame } from "remotion";
import { SCENES } from "./audioConfig";

// 根据当前帧号返回场景索引
const useCurrentSceneIndex = () => {
  const frame = useCurrentFrame();
  let accumulated = 0;
  for (let i = 0; i < SCENES.length; i++) {
    accumulated += SCENES[i].durationInFrames;
    if (frame < accumulated) return i;
  }
  return SCENES.length - 1;
};

// 使用
const sceneIndex = useCurrentSceneIndex();
const currentScene = SCENES[sceneIndex];

Main scene component pattern

import { AbsoluteFill, Audio, Sequence, staticFile, useVideoConfig } from "remotion";
import { ThreeCanvas } from "@remotion/three";
import { SCENES, getSceneStart, TOTAL_FRAMES } from "./audioConfig";

export const TutorialVideo: React.FC = () => {
  const { width, height } = useVideoConfig();
  const sceneIndex = useCurrentSceneIndex();
  const currentScene = SCENES[sceneIndex];

  return (
    <AbsoluteFill style={{ backgroundColor: "#1a1a2e" }}>
      {/* 3D 内容 */}
      <ThreeCanvas width={width} height={height} camera={{ position: [0, 0, 4], fov: 50 }}>
        {/* 根据 sceneIndex 渲染不同场景 */}
        {sceneIndex === 0 && <Scene01Intro />}
        {sceneIndex === 1 && <Scene02Concept />}
        {sceneIndex === 2 && <Scene03Demo />}
      </ThreeCanvas>

      {/* 音频同步 - 每个场景一个 Sequence */}
      {SCENES.map((scene, idx) => (
        <Sequence key={scene.id} from={getSceneStart(idx)} durationInFrames={scene.durationInFrames}>
          <Audio src={staticFile(`audio/${scene.audioFile}`)} />
        </Sequence>
      ))}

      {/* UI 层：标题 + 进度 */}
      <div style={{ position: "absolute", top: 40, left: 0, right: 0, textAlign: "center" }}>
        <h1 style={{ color: "white", fontSize: 42 }}>教程标题</h1>
      </div>
      <div style={{ position: "absolute", bottom: 60, left: 60 }}>
        <span style={{ color: "white" }}>{currentScene?.title}</span>
      </div>
      {/* 进度条 */}
      <div style={{ position: "absolute", bottom: 30, left: 60, right: 60, height: 4, backgroundColor: "rgba(255,255,255,0.2)" }}>
        <div style={{ width: `${((sceneIndex + 1) / SCENES.length) * 100}%`, height: "100%", backgroundColor: "#3498DB" }} />
      </div>
    </AbsoluteFill>
  );
};

Root.tsx uses dynamic frame count

import { Composition } from "remotion";
import { TutorialVideo } from "./TutorialVideo";
import { TOTAL_FRAMES } from "./audioConfig";

export const RemotionRoot: React.FC = () => {
  return (
    <Composition
      id="Tutorial"
      component={TutorialVideo}
      fps={30}
      durationInFrames={TOTAL_FRAMES}  // 从 audioConfig 动态获取
      width={1920}
      height={1080}
    />
  );
};

⚠️ Lessons learned from tutorial videos

Problem	Cause	Solution
Scene switching is abrupt	Direct switching with no transition	Use spring/interpolate to add entrance animations
3D content is out of sync with audio	Hardcoded frame counts	All durations are read from audioConfig
WebGL crashes during rendering	Multiple ThreeCanvas instances exist at the same time	Use sceneIndex for conditional rendering, with only one 3D scene at a time
The video is too brief	There is only one large scene	One concept = one scene component, layered explanation

Scene component design principles

Single responsibility: each scene component is responsible for only one concept
Independent animation: each scene has its own useCurrentFrame(), and animation starts from 0
Delayed appearance: use the delay parameter to control elements appearing in sequence
Camera adaptation: Different scenarios may require different camera positions

// 场景组件示例
const Scene02Input: React.FC = () => {
  const frame = useCurrentFrame();
  const { fps } = useVideoConfig();

  // 入场动画
  const gridScale = spring({ frame, fps, config: { damping: 15 } });

  return (
    <group>
      <PixelGrid position={[0, 0, 0]} scale={gridScale * 1.5} />
    </group>
  );
};

Camera Controller Modes

import { useThree } from "@react-three/fiber";

// ✅ 推荐写法：直接设置相机位置，避免插值导致的持续抖动
const CameraController: React.FC<{ sceneIndex: number }> = ({ sceneIndex }) => {
  const { camera } = useThree();

  const cameraSettings: Record<number, [number, number, number]> = {
    0: [0, 0, 4],      // 开场：正面
    1: [0, 0, 3],      // 输入层：靠近
    2: [-0.5, 0, 3.5], // 卷积：偏左
    3: [0, 0, 5],      // 总结：拉远全景
  };

  const target = cameraSettings[sceneIndex] || [0, 0, 4];

  // 直接设置位置，不用插值
  camera.position.set(target[0], target[1], target[2]);
  camera.lookAt(0, 0, 0);

  return null;
};

⚠️ Do not use position += (target - position) * factor this syntax, it will never converge precisely and will cause the image to keep shaking. See "🚨 Common Pitfalls in 3D Scenes - Pitfall 1".

Common Features

Add video/audio

import { Video, Audio, staticFile } from "remotion";

// 使用 public/ 目录下的文件
<Video src={staticFile("background.mp4")} />
<Audio src={staticFile("music.mp3")} volume={0.5} />

// 外部 URL
<Video src="https://example.com/video.mp4" />

Add image

import { Img, staticFile } from "remotion";

<Img src={staticFile("logo.png")} style={{ width: 200 }} />

Parameterized video (dynamic data)

// 定义 props schema
const myCompSchema = z.object({
  title: z.string(),
  bgColor: z.string(),
});

export const MyVideo: React.FC<z.infer<typeof myCompSchema>> = ({ title, bgColor }) => {
  return (
    <AbsoluteFill style={{ backgroundColor: bgColor }}>
      <h1>{title}</h1>
    </AbsoluteFill>
  );
};

// 注册时传入默认值
<Composition
  id="MyVideo"
  component={MyVideo}
  schema={myCompSchema}
  defaultProps={{ title: "Hello", bgColor: "#ffffff" }}
  ...
/>

Render output

CLI rendering

# 渲染为 MP4
npx remotion render MyVideo out/video.mp4

# 指定编码器
npx remotion render --codec=h264 MyVideo out/video.mp4

# WebM 格式
npx remotion render --codec=vp8 MyVideo out/video.webm

# GIF
npx remotion render --codec=gif MyVideo out/video.gif

# 仅音频
npx remotion render --codec=mp3 MyVideo out/audio.mp3

# 图片序列
npx remotion render --sequence MyVideo out/frames

# 单帧静态图
npx remotion still MyVideo --frame=30 out/thumbnail.png

Common rendering parameters

Parameter	Description
`--codec`	h264, h265, vp8, vp9, gif, mp3, wav, etc.
`--crf`	Quality (0-51, lower is better, default 18)
`--props`	Pass props in JSON format
`--scale`	Scale factor
`--concurrency`	Number of parallel renders

Advanced Features

Subtitles (@remotion/captions)

npm i @remotion/captions @remotion/install-whisper-cpp
npx remotion-install-whisper-cpp  # 安装 Whisper

import { transcribe } from "@remotion/install-whisper-cpp";

const { transcription } = await transcribe({
  inputPath: "audio.mp3",
  whisperPath: whisperCppPath,
  model: "medium",
});

Embed player in web apps

npm i @remotion/player

import { Player } from "@remotion/player";
import { MyVideo } from "./MyVideo";

<Player
  component={MyVideo}
  durationInFrames={150}
  fps={30}
  compositionWidth={1920}
  compositionHeight={1080}
  style={{ width: "100%" }}
  controls
  inputProps={{ title: "Dynamic Title" }}
/>

AWS Lambda rendering

npm i @remotion/lambda
npx remotion lambda policies role   # 设置 IAM
npx remotion lambda sites create    # 部署站点
npx remotion lambda render <site-url> MyVideo  # 渲染

3D video production (@remotion/three)

Use React Three Fiber to create 3D animated videos in Remotion.

Applicable scenarios

Scenario	Description	Example
Product showcase	3D model rotation and exploded animations	Phone product promotional video
Character animation	Cartoon character explanations, storytelling	Parenting educational videos
Data visualization	3D charts, spatial data	Geographic information, architectural display
Logo animation	Brand 3D Logo entrance	Opening and ending sequences

Installation

npm i three @react-three/fiber @remotion/three @types/three

Official template(recommended for beginners):

npx create-video@latest --template three

Basic example

import { ThreeCanvas } from "@remotion/three";
import { useCurrentFrame, useVideoConfig, interpolate, spring } from "remotion";
import { useEffect } from "react";
import { useThree } from "@react-three/fiber";

// 3D 场景组件
const My3DScene = () => {
  const frame = useCurrentFrame();
  const { fps, durationInFrames } = useVideoConfig();
  const camera = useThree((state) => state.camera);

  // 设置相机
  useEffect(() => {
    camera.position.set(0, 0, 5);
    camera.lookAt(0, 0, 0);
  }, [camera]);

  // 旋转动画
  const rotation = interpolate(frame, [0, durationInFrames], [0, Math.PI * 2]);

  // 弹性入场
  const scale = spring({ frame, fps, config: { damping: 10, stiffness: 100 } });

  return (
    <mesh rotation={[0, rotation, 0]} scale={scale}>
      <boxGeometry args={[1, 1, 1]} />
      <meshStandardMaterial color="royalblue" />
    </mesh>
  );
};

// 视频组件
export const My3DVideo = () => {
  const { width, height } = useVideoConfig();

  return (
    <ThreeCanvas width={width} height={height}>
      <ambientLight intensity={0.5} />
      <pointLight position={[10, 10, 10]} />
      <My3DScene />
    </ThreeCanvas>
  );
};

Load GLTF model

import { useGLTF } from "@react-three/drei";
import { useCurrentFrame, interpolate } from "remotion";

const Model = () => {
  const frame = useCurrentFrame();
  const { scene } = useGLTF("/models/character.glb");

  const rotation = interpolate(frame, [0, 150], [0, Math.PI * 2]);

  return <primitive object={scene} rotation={[0, rotation, 0]} scale={0.5} />;
};

Install drei(React Three Fiber utility library):

npm i @react-three/drei

Video as 3D texture

import { ThreeCanvas, useVideoTexture } from "@remotion/three";
import { staticFile, useVideoConfig } from "remotion";

const VideoOnMesh = () => {
  const { width, height } = useVideoConfig();
  const videoTexture = useVideoTexture(staticFile("/video.mp4"));

  return (
    <ThreeCanvas width={width} height={height}>
      <mesh>
        <planeGeometry args={[4, 3]} />
        {videoTexture && <meshBasicMaterial map={videoTexture} />}
      </mesh>
    </ThreeCanvas>
  );
};

Use during rendering useOffthreadVideoTexture() to ensure frame accuracy:

import { useOffthreadVideoTexture } from "@remotion/three";

const texture = useOffthreadVideoTexture({ src: staticFile("/video.mp4") });

3D character composition tips

Compose characters using basic geometries (no professional modeling required):

// 简单卡通角色：头 + 身体 + 四肢
const CartoonCharacter = ({ emotion = "happy" }) => {
  const frame = useCurrentFrame();

  // 表情控制
  const eyeScale = emotion === "happy" ? 1 : 0.5;
  const mouthRotation = emotion === "happy" ? 0 : Math.PI;

  // 走路动画：腿部摆动
  const legSwing = Math.sin(frame * 0.2) * 0.3;

  return (
    <group>
      {/* 头部 - 球体 */}
      <mesh position={[0, 1.5, 0]}>
        <sphereGeometry args={[0.5, 32, 32]} />
        <meshStandardMaterial color="#FFE4C4" />
      </mesh>

      {/* 身体 - 胶囊体 */}
      <mesh position={[0, 0.5, 0]}>
        <capsuleGeometry args={[0.3, 0.8, 16, 32]} />
        <meshStandardMaterial color="#4169E1" />
      </mesh>

      {/* 左腿 */}
      <mesh position={[-0.15, -0.3, 0]} rotation={[legSwing, 0, 0]}>
        <cylinderGeometry args={[0.08, 0.08, 0.6]} />
        <meshStandardMaterial color="#333" />
      </mesh>

      {/* 右腿 */}
      <mesh position={[0.15, -0.3, 0]} rotation={[-legSwing, 0, 0]}>
        <cylinderGeometry args={[0.08, 0.08, 0.6]} />
        <meshStandardMaterial color="#333" />
      </mesh>
    </group>
  );
};

⚠️ Pitfall experience

WebGL context overflow

Problem: Multiple 3D scenes rendering at the same time throws the error Error creating WebGL context

Cause: Browsers limit the number of WebGL contexts (usually 8-16)

Solution：

Rendering configuration: Use the angle OpenGL engine

// remotion.config.ts
export default {
  chromiumOptions: {
    gl: "angle",  // 或 "angle-egl"
  },
};

During CLI rendering:

npx remotion render --gl=angle MyVideo out.mp4

Lazy-load scenes: Only render 3D content near the current frame

import { useCurrentFrame } from "remotion";

const LazyScene = ({ sceneStart, sceneDuration, children }) => {
  const frame = useCurrentFrame();
  const buffer = 30; // 缓冲 30 帧

  // 只在场景时间范围 ± buffer 内渲染
  const shouldRender =
    frame >= sceneStart - buffer &&
    frame <= sceneStart + sceneDuration + buffer;

  if (!shouldRender) {
    return null; // 不渲染，释放 WebGL 上下文
  }

  return <>{children}</>;
};

// 使用
<Sequence from={0} durationInFrames={150}>
  <LazyScene sceneStart={0} sceneDuration={150}>
    <Scene1 />
  </LazyScene>
</Sequence>
<Sequence from={150} durationInFrames={150}>
  <LazyScene sceneStart={150} sceneDuration={150}>
    <Scene2 />
  </LazyScene>
</Sequence>

Server-side rendering configuration

Server-side rendering (SSR) must configure the gl option:

// renderMedia() / renderFrames() / getCompositions()
await renderMedia({
  composition,
  serveUrl,
  outputLocation: "out.mp4",
  chromiumOptions: {
    gl: "angle",
  },
});

useCurrentFrame inside Sequence

<Sequence> The useCurrentFrame() inside returnsthe frame number relative to the start of the Sequence, not the global frame number.

<Sequence from={60} durationInFrames={90}>
  <MyScene />  {/* 这里 useCurrentFrame() 从 0 开始，不是 60 */}
</Sequence>

Advanced resources

Resource	Purpose	Link
Mixamo	Free skeletal animation library	https://www.mixamo.com
Sketchfab	Free/paid 3D models	https://sketchfab.com
Ready Player Me	Virtual character generation	https://readyplayer.me
Spline	Online 3D design tool	https://spline.design
gltfjsx	GLTF to React Component	`npx gltfjsx model.glb`

Advanced Directions

Blender → GLTF: Model in Blender, export in GLTF format, load with useGLTF load
Mixamo Animation: Download FBX animations, convert to GLTF, play with useAnimations play
Spline Design: Design 3D scenes in Spline, import with @splinetool/r3f-spline import

3Blue1Brown Style Guide (Tutorial Videos)

For tutorial and explainer videos, draw on 3Blue1Brown's visualization design principles.

Core Philosophy

3B1B 内核：让观众「自己发现」，而不是「被告知答案」

Principle	Description	Example
Why → What	Ask why first, then show what	"How to recognize handwritten digits?" → Show a neural network
Build step by step	Elements appear one by one, don't fade in as a whole	Neurons light up in sequence, not all at once
Colors have semantics	Colors convey information, not decoration	Blue = positive, red = negative, yellow = highlight
Make values concrete	Show specific numbers to ground abstract concepts	Pixel value 0.7, activation value 0.92
2D first	Clarity over flashiness, use 3D only when necessary	Use 2D for network structures, 3D for spatial data

Color Scheme

// 3B1B 风格配色（语义化）
const COLORS_3B1B = {
  background: "#000000",     // 纯黑背景
  positive: "#58C4DD",       // 蓝色 - 正权重/正向
  negative: "#FF6B6B",       // 红色 - 负权重/负向
  highlight: "#FFFF00",      // 黄色 - 当前焦点/高亮
  result: "#83C167",         // 绿色 - 结果/正确
  text: "#FFFFFF",           // 白色 - 文字
  neutral: "#888888",        // 灰色 - 中性/未激活
  accent: "#FF8C00",         // 橙色 - 强调
};

// 使用示例
<meshStandardMaterial
  color={weight > 0 ? COLORS_3B1B.positive : COLORS_3B1B.negative}
  emissive={isHighlighted ? COLORS_3B1B.highlight : "#000"}
  emissiveIntensity={isHighlighted ? 0.3 : 0}
/>

2D/3D Hybrid Strategy

Content Type	Recommended Dimension	Reason
Network structure diagram	2D	Clear hierarchy, easy to annotate
Data flow	2D + animated arrows	Emphasize order and causality
Convolution operation	2D top-down view	Grid alignment, values visible
Feature map stacking	2.5D (perspective)	Show depth/number of channels
3D object recognition	3D	The content itself is 3D

2D mode implementation: Use an orthographic camera + flat geometry

import { OrthographicCamera } from "@react-three/drei";

// 正交相机 = 无透视变形 = 2D 感觉
<OrthographicCamera makeDefault position={[0, 0, 10]} zoom={100} />

// 扁平几何体
<mesh>
  <planeGeometry args={[1, 1]} />  {/* 2D 平面 */}
  <meshBasicMaterial color={color} />
</mesh>

Step-by-step build animation

Core: Use delay parameters to control elements appearing in sequence

// 批量元素逐个出现
const StaggeredGroup: React.FC<{
  children: React.ReactNode[];
  delayPerItem?: number
}> = ({ children, delayPerItem = 8 }) => {
  const frame = useCurrentFrame();
  const { fps } = useVideoConfig();

  return (
    <>
      {React.Children.map(children, (child, i) => {
        const delay = i * delayPerItem;
        const progress = spring({
          frame: frame - delay,
          fps,
          config: { damping: 12, stiffness: 100 },
        });

        if (frame < delay) return null;

        return (
          <group scale={Math.max(0, progress)} opacity={progress}>
            {child}
          </group>
        );
      })}
    </>
  );
};

// 使用
<StaggeredGroup delayPerItem={10}>
  <Neuron position={[0, 0, 0]} />
  <Neuron position={[1, 0, 0]} />
  <Neuron position={[2, 0, 0]} />
</StaggeredGroup>

Numeric label component

import { Text } from "@react-three/drei";

const ValueLabel: React.FC<{
  value: number;
  position: [number, number, number];
  fontSize?: number;
}> = ({ value, position, fontSize = 0.15 }) => {
  // 根据值选择颜色
  const color = value > 0.5 ? COLORS_3B1B.positive :
                value < -0.5 ? COLORS_3B1B.negative :
                COLORS_3B1B.neutral;

  return (
    <Text
      position={position}
      fontSize={fontSize}
      color={color}
      anchorX="center"
      anchorY="middle"
      font="/fonts/JetBrainsMono-Regular.ttf"  // 等宽字体
    >
      {value.toFixed(2)}
    </Text>
  );
};

Highlight focus component

// 脉冲高亮框 - 引导注意力
const FocusBox: React.FC<{
  position: [number, number, number];
  size: [number, number];
  label?: string;
}> = ({ position, size, label }) => {
  const frame = useCurrentFrame();
  const pulse = 1 + Math.sin(frame * 0.15) * 0.08;

  return (
    <group position={position}>
      {/* 高亮框 */}
      <mesh scale={[pulse, pulse, 1]}>
        <planeGeometry args={size} />
        <meshBasicMaterial
          color={COLORS_3B1B.highlight}
          transparent
          opacity={0.2}
        />
      </mesh>
      {/* 边框 */}
      <lineSegments>
        <edgesGeometry args={[new THREE.PlaneGeometry(...size)]} />
        <lineBasicMaterial color={COLORS_3B1B.highlight} linewidth={2} />
      </lineSegments>
      {/* 标签 */}
      {label && (
        <Text position={[0, size[1] / 2 + 0.2, 0]} fontSize={0.12} color={COLORS_3B1B.highlight}>
          {label}
        </Text>
      )}
    </group>
  );
};

Script Writing Guide (Tutorial Type)

❌ Announcement style (avoid)：

"首先是输入层。图像是一个数字矩阵。"
"接下来是卷积层。卷积核在图像上滑动。"

✅ Exploratory style (recommended)：

"你能轻松认出这是数字 7，但你能描述你是怎么做到的吗？
（停顿 1 秒）
这正是神经网络要解决的问题。

让我们先看看计算机「看到」的是什么——
（数字网格逐个显示）
不是图像，而是 784 个数字。

那么问题来了：如何从这堆数字中识别出 7？"

Script Structure Template：

1. 🎯 提出问题（10%）
   - 用观众能共鸣的问题开场
   - "你有没有想过..."

2. 🤔 直觉猜测（15%）
   - 引导观众思考可能的方案
   - "也许我们可以..."

3. 🔍 逐步验证（50%）
   - 一步步展示机制
   - 每一步都回答「为什么这样设计」

4. 📐 形式化（15%）
   - 展示数学公式（可选）
   - 将直觉转化为精确描述

5. 🎬 回顾总结（10%）
   - 完整流程快速回放
   - 强调核心洞见

⚠️ Common Pitfalls

Pitfall	Problem	Improvement
3D showmanship	Rotation and perspective distract attention	Express with the simplest viewpoint
Arbitrary colors	Red, green, blue are just decoration	Establish a color-meaning mapping
Appearing as a whole	The audience doesn't know where to look	Element by element + highlight guidance
Only saying What	The audience doesn't understand the design motivation	Ask Why first, then show What
Information overload	Cramming too many concepts into one scene	One scene, one concept

Process Animation Mode

Core Idea: Don't just show "what it is"—show "how it's computed." Let the audience see with their own eyes how data flows and how computation happens.

Applicable Scenarios

Scenario	Description	Example
Algorithm visualization	Show each step of the operation	Sorting, searching, graph traversal
Mathematical formula derivation	Expand the computation term by term	Matrix multiplication, convolution operations
Data processing pipeline	Input → transform → output	CNN forward propagation, data cleaning
Decision process	Compare, filter, final selection	Max pooling, softmax

Animation Mode Classification

静态展示 → 结构动画 → 过程动画
   ↓           ↓           ↓
  截图      元素出现     计算过程
            淡入淡出     数据流动
            相机移动     结果写入

Process Animation Component Library

1. Step-by-Step Computation Display (StepByStep)

// 逐步显示计算过程
const StepByStepCalc: React.FC<{
  steps: string[];      // ["1×0.5", "+ 0×0.3", "+ 1×(-0.2)", "= 0.3"]
  startFrame: number;
  framesPerStep?: number;
}> = ({ steps, startFrame, framesPerStep = 20 }) => {
  const frame = useCurrentFrame();

  return (
    <div style={{ fontFamily: "monospace", fontSize: 24, color: "white" }}>
      {steps.map((step, i) => {
        const stepStart = startFrame + i * framesPerStep;
        const opacity = interpolate(frame, [stepStart, stepStart + 10], [0, 1], {
          extrapolateLeft: "clamp",
          extrapolateRight: "clamp",
        });
        const isResult = i === steps.length - 1;

        return (
          <span
            key={i}
            style={{
              opacity,
              color: isResult ? COLORS.result : COLORS.text,
              fontWeight: isResult ? "bold" : "normal",
            }}
          >
            {step}{" "}
          </span>
        );
      })}
    </div>
  );
};

2. Value Fly-In Animation (ValueFlyIn)

// 计算结果飞入目标位置
const ValueFlyIn: React.FC<{
  value: number;
  from: [number, number, number];
  to: [number, number, number];
  startFrame: number;
  duration?: number;
}> = ({ value, from, to, startFrame, duration = 30 }) => {
  const frame = useCurrentFrame();
  const { fps } = useVideoConfig();

  const progress = spring({
    frame: frame - startFrame,
    fps,
    config: { damping: 15, stiffness: 80 },
  });

  if (frame < startFrame) return null;

  const position: [number, number, number] = [
    from[0] + (to[0] - from[0]) * progress,
    from[1] + (to[1] - from[1]) * progress,
    from[2] + (to[2] - from[2]) * progress,
  ];

  const scale = 1.5 - 0.5 * progress; // 飞行时放大，落地时缩小

  return (
    <Text
      position={position}
      fontSize={0.12 * scale}
      color={COLORS.result}
      anchorX="center"
      anchorY="middle"
    >
      {value.toFixed(1)}
    </Text>
  );
};

3. Region Highlight Comparison (CompareHighlight)

// 多个值依次比较，胜出者高亮
const CompareHighlight: React.FC<{
  values: number[];
  positions: [number, number, number][];
  startFrame: number;
  framesPerCompare?: number;
}> = ({ values, positions, startFrame, framesPerCompare = 15 }) => {
  const frame = useCurrentFrame();

  // 计算当前比较进度
  const compareIndex = Math.floor((frame - startFrame) / framesPerCompare);
  const maxIndex = values.indexOf(Math.max(...values));

  return (
    <>
      {values.map((value, i) => {
        const isComparing = i <= compareIndex && i <= maxIndex;
        const isWinner = compareIndex >= values.length - 1 && i === maxIndex;

        return (
          <group key={i} position={positions[i]}>
            <mesh>
              <boxGeometry args={[0.2, 0.2, 0.02]} />
              <meshStandardMaterial
                color={isWinner ? COLORS.result : isComparing ? COLORS.highlight : COLORS.dim}
                emissive={isWinner ? COLORS.result : "#000"}
                emissiveIntensity={isWinner ? 0.5 : 0}
              />
            </mesh>
            <Text position={[0, 0, 0.02]} fontSize={0.08} color="#000">
              {value}
            </Text>
          </group>
        );
      })}
    </>
  );
};

4. Sliding Window (SlidingWindow)

// 卷积核/池化窗口滑动
const SlidingWindow: React.FC<{
  gridSize: number;         // 输入网格大小
  windowSize: number;       // 窗口大小 (3 for 3x3)
  stride: number;           // 步幅
  currentStep: number;      // 当前步骤 (0, 1, 2, ...)
  onPositionChange?: (row: number, col: number) => void;
}> = ({ gridSize, windowSize, stride, currentStep }) => {
  const outputSize = Math.floor((gridSize - windowSize) / stride) + 1;
  const totalSteps = outputSize * outputSize;
  const step = Math.min(currentStep, totalSteps - 1);

  const row = Math.floor(step / outputSize) * stride;
  const col = (step % outputSize) * stride;

  // 窗口位置（相对于网格中心）
  const pixelSize = 0.12;
  const gap = 0.01;
  const offset = (gridSize / 2 - 0.5) * (pixelSize + gap);
  const windowOffset = (windowSize / 2 - 0.5) * (pixelSize + gap);

  const x = col * (pixelSize + gap) - offset + windowOffset;
  const y = row * (pixelSize + gap) - offset + windowOffset;

  return (
    <mesh position={[x, y, 0.05]}>
      <boxGeometry args={[windowSize * pixelSize + (windowSize - 1) * gap,
                          windowSize * pixelSize + (windowSize - 1) * gap, 0.02]} />
      <meshStandardMaterial
        color={COLORS.negative}
        transparent
        opacity={0.6}
        emissive={COLORS.negative}
        emissiveIntensity={0.3}
      />
    </mesh>
  );
};

Script Writing Guide (Process Animation Edition)

Key Shift: The script needs to match the animation's rhythm and give the animation "breathing room."

❌ Traditional script (information-dense)：

"卷积核在图像上滑动，每到一个位置就做点乘运算，得到一个数值。"
（一句话带过，观众还没看清发生了什么）

✅ Process animation script (with breathing room)：

"让我们看看卷积是怎么计算的。"
（停顿 - 窗口移动到位置）

"卷积核覆盖了这 9 个像素。"
（停顿 - 高亮 3x3 区域）

"我们把每个像素值，和对应的权重相乘..."
（停顿 - 逐步显示乘法）

"然后把所有结果加起来。"
（停顿 - 显示求和过程）

"得到的这个数字，就写入特征图的对应位置。"
（停顿 - 结果飞入）

"第一个位置完成了。接下来，窗口向右滑动一格..."
（加速展示后续步骤）

Time Allocation Suggestions

Level of Detail	First full demonstration	Repeat accelerated	Applicable Scenario
Extremely detailed	3-4 seconds/step	0.5 seconds/step	Core concept appearing for the first time
Medium	2 seconds/step	0.3 seconds/step	Supporting concepts
Fast	1 second/step	Flash by	Repetition of something already explained

Example: Time allocation for a convolution scenario

总时长：~25 秒

0-3s:   引入（"让我们看看卷积是怎么计算的"）
3-12s:  第 1 次卷积（完整详细展示）
        - 窗口移动 (1s)
        - 高亮区域 (1s)
        - 计算过程 (4s)
        - 结果飞入 (2s)
        - 解说旁白 (1s)
12-18s: 第 2-3 次卷积（中等速度，简化解说）
18-23s: 剩余位置（快速滑动，仅显示结果）
23-25s: 展示完整特征图

⚠️ Lessons Learned from Process Animation Pitfalls

Problem	Cause	Solution
Animation too fast to see clearly	Insufficient time allocation	Increase the number of frames for key steps
Narration and animation out of sync	The script has no breathing room	Rewrite the script and add pause markers
Information overload	Showing too much at once	Break into stages: structure first, then process
Repeated content is boring	Showing everything in detail every time	Detailed the first time + accelerated afterward
Values too small to see	3D text rendering issue	Use a 2D HTML overlay
Camera keeps shaking	Interpolation never converges	See "Camera Control Pitfalls" below
Image rotated 90 degrees	Row and column coordinate mapping is reversed	See "Grid Coordinate Pitfalls" below
Progress shows several thousand %	The progress variable is not clamped	`Math.min(1, (frame - start) / duration)`
The feature map only has color patches and no numerical values	The component lacks a numerical display function	Add `values` + `showValues` parameter

Progress variables must be clamped

// ❌ 错误：场景持续时间可能远超预期，progress 会变成 5000%
const calcProgress = frame > 30 ? (frame - 30) / 60 : 0;

// ✅ 正确：限制在 [0, 1] 范围
const calcProgress = frame > 30 ? Math.min(1, (frame - 30) / 60) : 0;

Feature map displays computed results

// FeatureMap 组件应支持显示数值
<FeatureMap
  position={[2, 0, 0]}
  size={0.6}
  count={1}
  color={COLORS.result}
  filledCells={filledCount}
  gridSize={6}
  values={[2, -1, 0, 3, ...]}  // 每个格子的计算结果
  showValues                    // 启用数值显示
/>

🚨 Common Pitfalls in 3D Scenes

Pitfall 1: The camera keeps shaking

Symptoms: The image keeps slightly zooming in and out, shaking

Incorrect写法：

// ❌ 永远无法精确到达目标，导致持续微抖动
const CameraController = ({ targetZ }) => {
  const { camera } = useThree();
  const frame = useCurrentFrame();

  useEffect(() => {
    camera.position.z += (targetZ - camera.position.z) * 0.05;
  }, [frame]);

  return null;
};

Correct写法：

// ✅ 方案A：使用 spring 动画（推荐）
const CameraController = ({ targetZ, transitionFrame = 0 }) => {
  const { camera } = useThree();
  const frame = useCurrentFrame();
  const { fps } = useVideoConfig();

  const z = spring({
    frame: frame - transitionFrame,
    fps,
    from: camera.position.z,
    to: targetZ,
    config: { damping: 20, stiffness: 100 },
  });

  camera.position.z = z;
  return null;
};

// ✅ 方案B：直接设置（无过渡）
const CameraController = ({ targetZ }) => {
  const { camera } = useThree();
  camera.position.set(0, 0, targetZ);
  camera.lookAt(0, 0, 0);
  return null;
};

// ✅ 方案C：插值但加阈值
useEffect(() => {
  const delta = targetZ - camera.position.z;
  if (Math.abs(delta) < 0.001) {
    camera.position.z = targetZ; // 接近时直接设置
  } else {
    camera.position.z += delta * 0.1;
  }
}, [frame]);

Pitfall 2: The grid image is rotated 90 degrees

Symptoms: An image that should display in the normal orientation (such as the digit 7) is rotated 90 degrees

Root cause: In image processing, row corresponds to the y-axis (top to bottom),col corresponds to the x-axis (left to right), but in the code, the row index is mapped to the x coordinate, and the column index is mapped to the y coordinate.

Incorrect写法：

// ❌ row 映射到 x，col 映射到 y，图像会旋转90度
for (let row = 0; row < size; row++) {
  for (let col = 0; col < size; col++) {
    const x = (row - size/2) * cellSize;  // 错！row 应该是 y
    const y = (col - size/2) * cellSize;  // 错！col 应该是 x
    // ...
  }
}

Correct写法：

// ✅ col 映射到 x，row 映射到 y（且 y 要翻转）
for (let row = 0; row < size; row++) {
  for (let col = 0; col < size; col++) {
    const x = (col - size/2 + 0.5) * cellSize;           // col → x
    const y = ((size - 1 - row) - size/2 + 0.5) * cellSize; // row → y（翻转）
    // ...
  }
}

Mnemonic：

Image coordinates:image[row][col] = image[y][x](rows are y, columns are x)
3D coordinates: x to the right, y upward
Flip row: image row=0 is at the top, 3D y=max is at the top

Workflow Best Practices

Recommended npm scripts configuration

{
  "scripts": {
    "dev": "remotion studio",
    "audio": "python3 scripts/generate_audio.py",
    "render": "remotion render MyVideo out/video.mp4",
    "build": "npm run audio && npm run render"
  }
}

Real-time progress display

Audio generation and video rendering may both take a long time,be sure to run in the foregroundso you can see the progress:

# ✅ 推荐：前台执行，实时显示进度
npm run audio
npm run render

# ✅ 或者用 shell 脚本封装
bash scripts/render.sh

# ❌ 避免：后台执行看不到进度
npm run render &

render.sh example：

#!/bin/bash
cd "$(dirname "$0")/.."
echo "🎬 开始渲染视频..."
npx remotion render MyVideo out/video.mp4
if [ $? -eq 0 ]; then
    echo "✅ 渲染完成!"
    ls -lh out/video.mp4
else
    echo "❌ 渲染失败"
    exit 1
fi

Design principles for resumable work

Long-running tasks (such as batch audio generation) should support resuming:

Check for existing files: Skip completed items
Atomic operations: Failure to generate a single file does not affect completed ones
Progress saving: Preserve completed parts on failure
Idempotent execution: Repeated runs produce the same result

Debugging Tips

Studio hot reload：npm run dev real-time preview
Check frames: In Studio, drag the timeline to check frame by frame
Performance: Avoid heavy computations inside components, use useMemo
Static files: Place them in the public/ directory and reference them with staticFile() Quote

FAQ

Q: Video rendering is very slow?

Use --concurrency to increase parallelism
Test by lowering the resolution:--scale=0.5
Consider AWS Lambda distributed rendering

Q: Fonts are not displayed?

Use @remotion/google-fonts or load locally
Make sure the font is loaded before rendering

Q: Video素材 doesn't play?

Check the video encoding format (H.264 recommended)
Use <OffthreadVideo> instead of <Video> to improve performance

Reference Resources

Official documentation:https://remotion.dev/docs
Template library:https://remotion.dev/templates
GitHub：https://github.com/remotion-dev/remotion

About this skill

Core Concepts

Quick Start

Create a New Project

Project Structure

Basic Component Example

Minimal Video Component

Register Composition

Animation Techniques

interpolate - Value Mapping

spring - Physics Animation

Sequence - Time Orchestration

AI Voice Narration Integration

Approach Selection Flow

Approach 1: MiniMax TTS (Recommended)

Configuration

API Differences

Generate Script

Pricing Reference (2025)

⚠️ MiniMax TTS Pitfall Experience

Python TypeScript Generation Notes

Option 2: Edge TTS

Installation

Recommended voice

Generate script

Remotion audio synchronization

Tutorial video architecture (scene-driven)

Architecture overview

audioConfig.ts template

Scene switching Hook

Main scene component pattern

Root.tsx uses dynamic frame count

⚠️ Lessons learned from tutorial videos

Scene component design principles

Camera Controller Modes

Common Features

Add video/audio

Add image

Parameterized video (dynamic data)

Render output

CLI rendering

Common rendering parameters

Advanced Features

Subtitles (@remotion/captions)

Embed player in web apps

AWS Lambda rendering

3D video production (@remotion/three)

Applicable scenarios

Installation

Basic example

Load GLTF model

Video as 3D texture

3D character composition tips

⚠️ Pitfall experience

WebGL context overflow

Server-side rendering configuration

useCurrentFrame inside Sequence

Advanced resources

Advanced Directions

3Blue1Brown Style Guide (Tutorial Videos)

Core Philosophy

Color Scheme

2D/3D Hybrid Strategy

Step-by-step build animation

Numeric label component

Highlight focus component

Script Writing Guide (Tutorial Type)

⚠️ Common Pitfalls

Process Animation Mode

Applicable Scenarios

Animation Mode Classification

Process Animation Component Library

1. Step-by-Step Computation Display (StepByStep)

2. Value Fly-In Animation (ValueFlyIn)

3. Region Highlight Comparison (CompareHighlight)

4. Sliding Window (SlidingWindow)

Script Writing Guide (Process Animation Edition)

Time Allocation Suggestions

⚠️ Lessons Learned from Process Animation Pitfalls