SuperAPI AI Directory
Partner with us中文AI newsFavorites
All toolsAI developer platformsMM-EUREKA

MM-EUREKA

MM-EUREKA is an open-source multimodal reasoning research project by ModalMinds that explores the frontiers of multimodal reasoning through rule-based reinforcement learning (RL). The project has released two multimodal reasoning models, MM-Eureka-Qwen-7B and MM-Eureka-Qwen-32B, and has open-sourced the complete training pipeline, code, data, and more. On the MMK12 evaluation set, the 7B model scores 66.1 and the 32B model scores 72.3; on MathVista, they reach 73.0 and 74.8 respectively, surpassing open-source models of the same scale and larger on some metrics. The project also proposes the CPGD reinforcement learning algorithm and the MM-PRM process reward model, and has publicly released the MMK12 dataset containing 15,000 multidisciplinary K12 samples and 2,000 multiple-choice questions for reasoning evaluation in subjects such as mathematics, physics, chemistry, and biology.

Open website

About this tool

MM-EUREKA is an open-source multimodal reasoning research project by ModalMinds, aiming to explore the frontiers of multimodal reasoning through rule-based reinforcement learning. The project is built on OpenRLHF and includes multiple extensions for reinforcement learning training of vision-language models (VLMs).

Purpose and Goals MM-EUREKA aims to extend large-scale rule-based reinforcement learning to the field of multimodal reasoning, improving model performance on multidisciplinary K12 tasks such as mathematics, physics, chemistry, and biology, as well as multimodal mathematical reasoning. The project has released two multimodal reasoning models, MM-Eureka-Qwen-7B and MM-Eureka-Qwen-32B, and has open-sourced the complete training pipeline.

Main Capabilities The project has made improvements in three aspects: models, algorithms, and data:

  • Models: MM-Eureka-Qwen-7B scores 66.1 on the MMK12 evaluation set, only 0.2 points lower than InternVL-2.5-78B; on MathVista (testmini) it reaches 73.0, surpassing InternVL-2.5-78B. MM-Eureka-Qwen-32B scores 72.3 on MMK12, surpassing Qwen2.5-VL-72B (70.3) and Gemini2-Flash, and is second only to o1 (73.9); it scores 73.4 on WeMath and reaches 74.8 on MathVista.
  • Algorithms: The project further iterates on the codebase, supporting algorithms such as Online Filter, ADORA, and DAPO; it proposes CPGD (Clipped Policy Gradient Optimization with Policy Drift), and experiments show that it is more stable and performs better than GRPO; it releases the MM-PRM process reward model for step-level supervision of multimodal mathematical reasoning.
  • Data: It has open-sourced the self-collected MMK12 dataset, containing 15k diverse high-quality samples and 2k multiple-choice questions in mathematics, physics, chemistry, and biology for evaluation.

Technical Enhancements (based on OpenRLHF)

  • Multimodal RFT support: OpenRLHF has been extended to support vision-language models, currently supporting InternVL, and supporting RLOO, REINFORCE++, and GRPO training (using Ray), vLLM integration and distributed training, hybrid engine (--colocate_all_models, --vllm_enable_sleep).
  • Better rule reward support: provides training visualization for rule rewards (format reward, accuracy reward, repetition penalty).
  • Enhanced online filtering: use --enable_accuracy_filter, --freezing_filter_steps, --accuracy_lower_bound, --accuracy_upper_bound to filter experiences based on accuracy reward during training.
  • ADORA: enable adaptive online rollout adjustment via --use_adora and --adora_lamda.
  • DAPO: enable DAPO loss via --use_dapo.
  • CPGD: enable CPGD loss via --use_cpg_loss and --use_policy_drift; --policy_drift_coef controls the weight of the policy drift regularization term, --policy_drift_clip_eps controls the clipping range of policy drift; --use_clip_filter_like_weight enables the clip-filter-like weight proposed by CPGD, and --clip_filter_like_weight_clip_eps controls its clipping range. Example scripts are located in the MM-EUREKA/examples/scripts/ directory.

Key modifications for stable training

  • The base model was upgraded from InternVL2.5-8B-Instruct to Qwen2.5-VL-7B-Instruct;
  • The Vision Transformer (ViT) module is frozen during training;
  • The underlying RL algorithm was replaced from RLOO with GRPO;
  • The data filtering strategy was changed from offline to online;
  • Additional K12 data was collected, expanding the total dataset size to 15,000 samples.

Usage

  1. Data preparation: training data can be downloaded from MMK12; custom datasets need to be formatted as JSONL files.
  2. Start training: ensure the paths in the training script are correct, and configure environment variables such as $MASTER_ADDR and $NODE_RANK. Single-node run sh examples/scripts/train_mm_eureka_qwen_7b_single_node.sh; multi-node run sh examples/scripts/train_mm_eureka_qwen_7b_multi_node.sh.

Applicable scenarios Suitable for multimodal reasoning research, multidisciplinary K12 task evaluation, multimodal mathematical reasoning, reinforcement learning algorithm (GRPO, RLOO, DAPO, CPGD, etc.) experiments, and rule reward training for vision-language models.

Limitations and notes The project is still under active development. Regarding specific commercial use, fees, pricing, platform support scope, performance guarantees, and other information, the materials do not currently provide them. The project acknowledges open-source contributions such as OpenRLHF, LMM-R1, vLLM, DeepSeek-R1, InternVL, and QwenVL.

Related tools

Browse category