Perception-R1 is a reinforcement learning post-training framework for visual perception tasks, proposed by the paper "Perception R1: Pioneering Perception Policy with Reinforcement Learning," with the official code hosted on GitHub. This project aims to use reinforcement learning to improve the performance of multimodal large language models (MLLMs) on visual perception tasks.
In terms of purpose, Perception-R1 focuses on optimizing visual perception policies, using Group Relative Policy Optimization (GRPO) to perform scalable reinforcement learning during the MLLM post-training stage. Its goal is to enhance the model's visual understanding capabilities in perception tasks such as grounding, counting, and detection.
Main capabilities include:
- Perceptual perplexity analysis: proposes a new analytical framework that reveals the key threshold for effective reinforcement learning in perception tasks, providing insights into when and how RL improves visual understanding.
- GRPO optimization: adopts carefully designed rule-based reward shaping to perform scalable policy learning.
- Performance improvement: achieves significant improvements on multiple visual perception benchmarks, especially reaching 31.9% mAP on the COCO2017 validation set, becoming the first 3B-scale MLLM to achieve this performance.
Usage:
- Before training, you need to modify the script to specify the model and data paths, then run the experiments. The training script contains comprehensive configuration of hyperparameters, data loading, and model checkpoints, and parameters such as learning rate and batch size can be adjusted according to custom scenarios.
- Evaluation: download the evaluation data from Hugging Face and extract it to the eval/ folder. Note that COCO images are not included in the package and need to be downloaded separately from the official COCO website and placed in the eval/images/coco/ directory. Evaluation supports tasks such as counting, grounding, and detection.
Applicable scenarios: suitable for academic or industrial scenarios researching the improvement of visual perception capabilities of multimodal large language models, especially for researchers who need to use reinforcement learning to optimize tasks such as grounding, detection, and counting.
Limitations: COCO images need to be downloaded by users themselves; the project does not provide image data; training and evaluation require users to configure model and data paths themselves; more perception tasks (such as OCR) are planned for release but are not yet available.
Project continuously updated: 2025-04-10 First release of the model and evaluation code; 2025-05-27 planned to add more perception tasks.
