R1-Multimodal-Journey (also known as MM-Eureka V0) is an open-source project for exploring reinforcement learning in multimodal large models, and its latest version has been migrated to MM-Eureka (FanqingM/MM-Eureka-V0), with the official also simultaneously at https://github.com/ModalMinds/MM-EUREKA publishing updated code, stronger models, and fully open-source content including data.
Purpose
The project focuses on exploring the effectiveness of R1-style reinforcement learning (RL) on more challenging problems, especially geometry problems, and verifying the performance of this training paradigm on vision-language models (VLMs).
Main capabilities and findings
- Supports vLLM, significantly accelerating the training process, which the official description says is 5-6 times faster than OpenR1-MM.
- Fixed some bugs in Open-R1-Multimodal, such as the get_per_token_logps function.
- Upgraded transformers to 4.49.0.dev0 to support Qwen2.5_VL.
- In terms of training data, 20,000 samples from geo170k are currently used.
- Experiments found that: even when the model is as small as 3B and under the instruct setting, the "aha moment" can also appear at the very beginning of training, and the official provided samples from only 10 training steps.
- VLMs seem to struggle to exhibit a length growth pattern similar to LLMs; the project tested multiple datasets and was unable to reproduce the length growth curve observed in simpleRL-reason, but did observe performance improvements in evaluation.
- Although R1's RL provides rewards based only on answer correctness, it still shows higher data efficiency than SFT, and the official compared this on the mathvista-testmini set.
- The project believes that VLMs struggle to reach an R1 moment similar to LLMs because of the lack of high-quality data; current multimodal reasoning data is significantly less than language data, which makes it difficult for models to show length growth on simple datasets such as geo170k and prone to overfitting.
- The official found that the slowness of OpenR1-Multimodal was caused by slow generation making other processes wait, and after switching to vLLM, training time was significantly reduced (under the conditions of 8 H100s and 20k Geo170k samples).
Usage
- Data preparation: modify the necessary input/output paths in local_scripts/gen_dataset.py; run python local_scripts/gen_dataset.py to prepare the dataset. Unlike Open-R1-Multimodal, which directly stores images in PIL format, this project stores image paths (URL or localpath) to support vLLM.
- Training: modify the necessary input/output paths in local_scripts/train_qwen2_5_3b.sh; run sh local_scripts/train_qwen2_5_3b.sh. Under the default settings, vLLM uses cuda:7 for generation, and the remaining 7 nodes are used for training.
- Evaluation: run python eval/evaluate_mathvista.py --checkpoint ${CHECKPOINT} --datasets MathVista_testmini to evaluate the checkpoint on MathVista_testmini.
Dependencies and Acknowledgements
The project codebase is built on Open-R1-Multimodal and incorporates open-source contributions from vLLM, Open-R1, and trl; thanks also to DeepSeek-R1 and Qwen2.5-VL for providing open-source technology and base models. The core contributors (equal contribution, in alphabetical order) are Lingxiao Du, Xiangyan Liu, Fanqing Meng.
Limitations
The official statement explicitly states that the latest version is located at MM-Eureka and recommends paying attention to its subsequent progress. Regarding fees, pricing, platform support scope, specific license agreements, and other features and effects not mentioned in the materials, no information is currently provided.
