VeOmni is a model training framework developed and open-sourced by ByteDance's Seed team, with the project hosted on GitHub (https://github.com/ByteDance-Seed/VeOmni）。其核心目标是为单模态与多模态模型的预训练和后训练提供统一支持，使用户能够方便地在各类加速器上扩展任意模态的模型规模，兼顾灵活性与易用性。
[Design Principles]
- Flexible and modular: VeOmni adopts a modular design, allowing users to decouple most components and replace them with their own implementations as needed.
- No Trainer constraints: It supports linear training scripts, avoiding the use of structured, rigid Trainer classes such as PyTorch-Lightning or HuggingFace Trainer, exposing the complete training logic to users for maximum transparency and control. At the same time, VeOmni also provides a base trainer for pure text or VLM/Omni model training, and provides an RL trainer as a training backend for reinforcement learning.
- Native Omni models: It enables users to easily scale any omni model to multiple devices and multiple accelerators.
- Native Torch: It fully utilizes native PyTorch functions to ensure maximum compatibility and performance.
[Main Capabilities]
- Training backend: FSDP2.
- Sequence parallelism: Based on DeepSpeed Ulysses, supporting non-asynchronous and asynchronous modes.
- Expert parallelism: Supports training of large MoE models, such as Qwen3-MoE.
- Efficient MoE GroupGemm kernel: Integrates Liger-Kernel.
- Model compatibility: Compatible with HuggingFace Transformers models, such as Qwen3, Qwen3-VL, Qwen3-MoE, etc.
- Dynamic batching strategy and Omnidata processing.
- Checkpointing: Torch Distributed Checkpoint.
- Hardware support: NVIDIA GPU, AMD ROCm, and Ascend NPU training.
- Experiment tracking: wandb.
[Usage] The official "Quick Start with Qwen3" quick start process is provided. To integrate a new model, you can refer to the "Support New Models" documentation for extension.
[Versions and Updates]
- VeOmni was released in April 2025.
- The VeOmni technical report was released in August 2025, and a WeChat group was opened.
- The first official version v0.1.0 was released in September 2025.
- In November 2025, the paper "OmniScale: Scaling Any Modality Model Training with Model-Centric Distributed Recipe Zoo" was accepted by AAAI 2026.
[Planned Features] VeOmni v0.2 roadmap (issues #268, #271), ViT balance tool (#280), validation dataset during training (#247), RL post-training for omni-modal models based on VeRL (#262).
[Ecosystem Applications] Multiple projects already use VeOmni, including dFactory (dLLM fine-tuning), UI-TARS (native agent automated GUI interaction), OpenHA (Minecraft open-source hierarchical agent model), UI-TARS-2, Open-dLLM (open diffusion large language model), and LingBot-VLA (VLA foundation model). The project welcomes community contributions; please see CONTRIBUTING.md.
[About the Team] ByteDance's Seed team was founded in 2023 and is committed to building industry-leading AI foundation models.
