CPPO (Completion Pruning Policy Optimization) is a method aimed at accelerating the training of reasoning models based on GRPO (Group Relative Policy Optimization), and the paper has been accepted by NeurIPS 2025. The following content is organized based on the project's public materials.
Purpose CPPO addresses the problem of high training costs when GRPO is used to train reasoning models. Although GRPO is effective, it requires sampling multiple completions for each question, which makes training time increase multiplicatively with the number of completions. CPPO reduces training overhead by optimizing the way completions are used, thereby accelerating the training process.
Main capabilities According to the materials, the core mechanism of CPPO includes two points: first, pruning completions with lower absolute advantage, because not all completions contribute equally to policy training, and their contribution depends on their relative advantage; second, introducing a dynamic completion allocation strategy that maximizes GPU utilization by incorporating additional questions. Experimental results show that, compared with the original GRPO, CPPO can achieve up to 8.32x speedup on GSM8K and 3.51x speedup on Math, while maintaining or even improving accuracy.
Usage In the materials, environment preparation information for reproducing the experiments is provided: reproducing the results on GSM8K requires two GPUs with 80G VRAM, and reproducing the results on Math requires four GPUs with 80G VRAM. Model checkpoints (ckpt) can be downloaded from Hugging Face. The project has also released a verl version of CPPO. The specific installation commands, code examples, and running steps are not yet provided in the existing materials.
Applicable scenarios CPPO is suitable for scenarios that require training GRPO-based reasoning models, especially research or engineering tasks with high requirements for training efficiency and a desire to reduce the number of sampled completions to lower training costs. The validation tasks mentioned in the materials include mathematical reasoning tasks such as GSM8K and Math, as well as Out-of-Distribution tasks.
Limitations The existing materials do not provide specific installation commands, complete code examples, the range of supported models, software dependencies beyond hardware, license agreement, or pricing information. Reproducing the experiments has clear requirements for GPU VRAM (GSM8K requires two 80G GPUs, Math requires four 80G GPUs), which may constitutea usage threshold. Other potential limitations are not yet provided.
