LLaVA-Reasoner-DPO is the unofficial implementation repository of the paper "Improve Vision Language Model Chain-of-thought Reasoning", focusing on enhancing the chain-of-thought (CoT) reasoning ability of vision-language models (VLMs).
Main purpose: Through two-stage training with supervised fine-tuning (SFT) and direct preference optimization (DPO), enhance the reasoning performance of vision-language models in tasks such as visual question answering, enabling the model to generate more accurate reasoning chains.
Main capabilities and resources:
- Data: Released the ShareGPT4o-reasoning dataset, containing 193k CoT predictions and filtered direct predictions; released the ShareGPT4o-reasoning-dpo dataset, containing 66k DPO data, covering the three domains of A-OKVQA, math, and ChartQA.
- Models: Provides Open-LLaVA-NeXT as the base model (same as xiaoachen98/Open-LLaVA-NeXT); provides LLaVA-Reasoner-SFT-preview (an SFT model for direct prediction + CoT); LLaVA-Reasoner-SFT (an SFT model that adds math data on top of preview); LLaVA-Reasoner-DPO-preview (a model trained with DPO from SFT-preview).
- Training pipeline: The project includes sft and dpo pipelines, as well as distill gpt, inference, and evaluation pipelines.
Usage: The repository provides code pipelines for training and evaluation, but the specific installation commands, dependency configuration, and usage steps are not yet provided in the existing materials, and reference should be made to the documentation in the GitHub repository and related projects (such as Open-LLaVA-NeXT, LLaVA-Hound-DPO).
Applicable scenarios: Suitable for research and development on chain-of-thought reasoning in vision-language models, especially in fields such as visual question answering, mathematical reasoning, and chart understanding, and can be used by researchers for SFT/DPO training, model distillation, inference, and evaluation.
Limitations: This material does not provide specific information such as fees, pricing, platform support, or performance results; the project is a research-oriented unofficial implementation, and its use requires relevant knowledge of deep learning and model training.
