ChatLearn is a large-scale reinforcement learning training framework developed by the Alibaba Cloud PAI platform, mainly used for alignment tasks of large language models (LLMs). The project is open-sourced on GitHub, and the repository address is https://github.com/alibaba/ChatLearn。
In terms of use, ChatLearn targets reinforcement learning training and alignment for LLMs, supports end-to-end training workflows starting from algorithms such as GRPO, and is suitable for scenarios requiring large-scale distributed reinforcement learning training. ChatLearn was officially released in August 2024.
Main capabilities include:
- User-friendly programming interface: users only need to encapsulate a small number of functions to write a single model, while the system handles resource scheduling, data and control flow transmission, and distributed execution.
- Extensible training methods: supports user-defined model execution workflows, making customized training processes more flexible and convenient.
- Diverse distributed acceleration engines: supports industry-leading training engines (FSDP2, Megatron) and inference engines (vLLM, SGLang), and the inference engine can be controlled through the runtime parameter runtime_args.rollout_engine.
- Flexible parallel strategies and resource allocation: supports different parallel strategies for different model configurations to adapt to each model's computation, memory, and communication characteristics; it also supports scheduling mechanisms for exclusive or shared resources among models, enabling efficient serial/parallel execution and GPU memory sharing optimization.
- High performance: compared with current SOTA systems, performance is improved by 52% at the 7B+7B scale (policy model + reward model) and by 137% at the 70B+70B scale; supports reinforcement learning training with more than 600B parameters. It achieves 115% acceleration under an 8 GPU, 7B+7B configuration, and 208% acceleration under a 32 GPU, 70B+70B configuration (compared with DeepSpeed-Chat and OpenRLHF, and the latter two have already been performance-optimized).
Functional features include: support for training engines such as Megatron and FSDP; support for inference engines such as vLLM and SGLang; support for reinforcement learning algorithms such as GRPO and GSPO; support for wandb and tensorboard experiment monitoring; support for training acceleration technologies such as sequence packing, Ulysses sequence parallelism, and Group GEMM.
The update record shows: officially released in August 2024; in May 2025, support for FSDP and Mcore framework training, and Qwen3 end-to-end GRPO training tutorials based on FSDP+vLLM and Megatron+vLLM; in July 2025, support for FSDP2, providing Qwen3-235B-A22B reinforcement learning training examples based on Mcore and FSDP2, DeepSeek-V3-671B reinforcement learning training examples based on Mcore, as well as sequence packing, sequence parallelism, Group GEMM, etc.; in August 2025, support for GSPO on Mcore; in September 2025, support for Agentic RL tasks and vision-language model RL tasks; in October 2025, improving the RL training stability and efficiency of MLA models (such as Moonlight/DeepSeek-V3) through context parallelism and sequence packing.
In terms of usage, the official documentation provides a quick start guide, including environment and code setup, and gives end-to-end GRPO training workflow tutorials for Qwen3 models based on FSDP+vLLM and Megatron+vLLM. The inference engine is controlled through the runtime_args.rollout_engine parameter.
Applicable scenarios include: large-scale LLM reinforcement learning alignment training, training tasks requiring custom model execution workflows, Agentic RL tasks, vision-language model RL tasks, and scenarios requiring high-throughput distributed training and flexible resource scheduling.
In terms of limitations, the materials do not clearly specify system requirements, minimum hardware configuration, license type, or detailed installation command specifics; this information is not yet provided. The official future plans listed include simplifying configuration settings, supporting MoE model RL training tutorials, supporting more models, performance optimization, and supporting more RL algorithms.
