Overview
MiniMax H3 is a general-purpose omni-modal generation system that supports unified understanding of multimodal contexts composed of text, images, video, and audio, and can generate video with native stereo audio at resolutions up to 2K and durations up to 15 seconds. According to the official description, thanks to its system design oriented toward task generalization, H3 possesses broad multimodal context understanding and generation capabilities from the pretraining stage, and performs outstandingly in following complex multimodal instructions.
System Composition
The complete H3 system consists of three modules:
- H3-Context-IR: For increasingly complex inputs, it deeply understands and refines the input multimodal instructions, converting them into a "Context Intermediate Representation" that H3 can easily understand for generation. The official recommendation strongly advises incorporating it into the generation pipeline, or building a custom context processing system by referring to the Prompting Guidance. This module relies on a multi-stage workflow and multiple hosted models and services, and is not included in this open-source release; the official API is provided to reproduce the official workflow behavior.
- H3-Base: Generates audio and video based on the output of H3-Context-IR, producing results at 768p resolution.
- H3-Regenerate-2K: Feeds the 768p results back into H3 together with the original context to regenerate 2K resolution output, thereby leveraging H3's generation capability and the rich information in the original context to produce high-resolution results with more accurate details and higher fidelity. This module is not yet open-sourced; the official API is provided to verify official results.
Model Architecture
H3-Base uses encoders or VAEs corresponding to each modality to encode different modalities, organizes the encoded representations into a unified packed multimodal sequence, uses RoPE to capture the necessary spatial and temporal relationships between tokens, and then feeds them into H3-Omni-Transformer. Specifically: text is encoded by H3-Encoder; visual input is jointly encoded by H3-Encoder and H3-VisualVAE; audio is encoded solely by H3-AudioVAE. H3-Omni-Transformer jointly predicts video and audio latents, which are then decoded into video and stereo audio respectively.
To reduce the computational overhead of long multimodal sequences, H3 natively supports sparse attention training and inference, but the initial open-source version only provides full-attention inference; the sparse attentionimplementation will be released in a subsequent update.
H3-Encoder uses the complete pretrained weights of Qwen3-VL-32B and provides its 50th-layer hidden states to H3-Omni-Transformer. The tokenizer configuration has newly added <d> and other special tokens; using H3 requires the tokenizer and related configuration files provided by the H3 repository.
H3-VisualVAE is a temporally causal video autoencoder with a spatial compression factor of 16×, a temporal compression factor of 4×, and 24 latent channels, denoted as f16t4d24. Before being fed into the Transformer, the visual latents are further divided into 1×2×2 patches along the (temporal, height, width) dimensions, so the effective spatial downsampling factor of the visual tokens entering the Transformer is 32×, while the temporal downsampling factor remains 4×. Its latent space is optimized simultaneously for reconstruction quality and ease of learning by generative models. After training the encoder, a ViT-based decoder was additionally trained to reduce decoding cost and further improve reconstruction quality.
H3-AudioVAE uses the same encoder and decoder for the left and right channels and processes each channel independently, then merges the decoded results, thereby supporting stereo audio input and output. Each channel compresses 32 kHz audio into a latent token sequence with a temporal rate of 40 Hz; the latent space is optimized with reference to VA-VAE, making it easy for generative models to learn while maintaining audio reconstruction quality.
H3-Omni-Transformer is a 33B-parameter dense single-stream Transformer, of which about 13B parameters are in the AdaLN-related branches. Since the AdaLN modulation outputs can be precomputed and cached, these parameters do not need to be loaded in pure inference deployment. The official release provides the full model weights to support further development, including fine-tuning. Neither the attention layers nor the FFN layers contain modality-specific structures; the modality-specific parameters are limited to the input/output layers and the AdaLN branches; modality-specific AdaLN improves generation quality at relatively low additional training and inference cost. The model uses a three-dimensional multimodal rotary position embedding (MM-RoPE) to represent positional relationships along (t, h, w), i.e., the temporal and two spatial dimensions. In the final stage of training, native sparse attention was introduced to reduce the computational cost of long sequences; this implementation is not included in the initial open-source release.
Usage
Use via API: Global platform platform.minimax.io, China platform platform.minimaxi.com.
Use via applications: WebApp global hailuoai.video, China hailuoai.com; desktopglobal hub.minimax.io, China hub.minimaxi.com.
Local deployment of H3-Base: H3 is released as two task-specific checkpoints, each containing a dedicated Omni Transformer model and the required processor, tokenizer, text encoder, Visual VAE, and independent Audio VAE components, as CFG-distilled Omni Transformer weights. Each checkpoint is released as a self-contained Hugging Face-style repository. The repository also hosts the original checkpoints (FL2VA/, Ref2VA/) and the diffusers format; model_index.json is the repository-level public entry point, and the task-family-specific diffusers indexes are located at FL2VA/model_index.json and Ref2VA/model_index.json. diffusers users do not need to download manually:ModularPipeline.from_pretrained("MiniMaxAI/MiniMax-H3") will fetch the required components on demand.
Recommended inference frameworks: SGLang, vLLM, diffusers, ComfyUI (with R2V template / T2V template). Deployment examples using SGLang as an example cover the three use cases T2VA, FL2VA, and Ref2VA. In the example scripts, conditions[].uri points to the public CDN by default, and can also be changed to file:// Local path; this path is resolved by the SGLang service process rather than the machine running curl, and must point to a file accessible to the server side; if the service runs in a container, the file must be located in a directory already mounted into the container (for example, mount a host directory to /data/minimax-h3 and reference file:///data/minimax-h3/...).
Complete 2K workflow: Combine the locally deployed SGLang service with the official H3-Context-IR and H3-Regenerate-2K APIs to reproduce the 2K video quality generated directly by the MiniMax API. You need to first configure the SGLang endpoint and MiniMax API credentials, and use interfaces such as /video-generation-v2-create, /video-generation-v2-h3-context-ir, and /video-generation-v2-regeneration. The example encodes the local H3-Base output file as a Base64 Data URL; in a production environment, it is recommended to upload the video to a publicly accessible URL and pass it in as base_video.
Prompt Writing Skills
The repository includes nine skills, one of which is the H3 prompt writing skill, containing two prompt guides under skills/h3-prompt-writing/references/: base-en.txt for text/keyframe mode, and ref-en.txt for full-reference (Ref2VA) mode. This skill is pure Markdown + reference files, with no external API calls, and can be used in carriers capable of reading SKILL.md and local files, such as Claude Code, Claude Agent SDK, Cursor, Windsurf, OpenAI-based agents/Codex, LangChain, etc.; the accompanying skills/h3-prompt-writing/agents/openai.yaml only adds optional metadata (display name, description, default prompt) to the ChatGPT/Codex skill UI, and does not restrict the skill to OpenAI agents. The other eight are stylized video generation skills oriented toward MiniMax Hub canvas workflows (hub_generate_video, hub_generate_image, canvas nodes, selection cards, etc.), and cannot be ported to general-purpose agent carriers.
The prompt guide documents in the HuggingFace release have not been copied to this repository.
Security Protection
User-submitted text, images, and videos, as well as enhanced prompts, are subject to automatic review, and content suspected of being illegal, pornographic, or infringing third-party rights may be blocked. The official implementation uses industry-standard filtering measures, but false positives or false negatives cannot be eliminated; these protections do not affect the licensee's obligations under the MiniMax H3 Community License Agreement, especially those regarding lawful use and usage restrictions.
Input/Output and Limitations
H3 supports input/output specifications such as text-to-video, image-to-video, first-and-last-frame-to-video, and text-to-audio-video; the specific complete specifications are subject to the official release. The current initial open-source version only supports full-attention inference; H3-Context-IR and H3-Regenerate-2K are not open-sourced and must be called via API; local deployment of H3-Base outputs 768p, and 2K requires combining with the official API; the model is released under the Community License Agreement, and the contact email is model@minimax.io.
