R1-Multimodal-Journey
AI developer platformsR1-Multimodal-Journey (also known as MM-Eureka V0) is an open-source project exploring reinforcement learning (RL) for multimodal large models, aiming to apply R1-style reinforcement learning similar to DeepSeek-R1 to vision-language models (VLMs) and verify its effectiveness on more challenging problems such as geometry. The project is built on Open-R1-Multimodal, integrates open-source achievements such as vLLM, Open-R1, and trl, and adopts the technologies and base models of DeepSeek-R1 and Qwen2.5-VL. Its core improvements include: support for vLLM-accelerated training, which is 5-6 times faster than OpenR1-MM on 8 H100s; fixing some bugs in Open-R1-Multimodal (such as the get_per_token_logps function); and upgrading transformers to 4.49.0.dev0 to support Qwen2.5_VL. The experiment used 20,000 samples from geo170k and found that even a small 3B model under the instruct setting can exhibit an "aha moment" within the first 10 training steps. The project also found that VLMs struggle to show a length growth curve like LLMs, but performance still improves; compared with SFT, R1-style RL is based only on answer correctness rewards, yet exhibits higher data efficiency. The project's latest progress has been migrated to MM-Eureka, and more capabilities, models, and data are also open-sourced in that repository.