SuperAPI AI Directory
Partner with us中文AI newsFavorites
All toolsAI videoVideoChat-R1

VideoChat-R1

VideoChat-R1 is a model project focused on video spatio-temporal perception and reasoning, developed by OpenGVLab, including versions such as VideoChat-R1, VideoChat-R1-caption, VideoChat-R1-thinking, and VideoChat-R1.5, with its paper accepted by NIPS2025. Through multi-task joint reinforcement learning fine-tuning, the project enhances the model's spatio-temporal perception and reasoning capabilities for videos, and performs better in benchmarks such as short videos, long videos, temporal grounding, and video reasoning. Its reasoning process simulates the human hierarchical attention mechanism, enabling it to progressively locate regions of interest, so that model performance improves with each step. The project provides Huggingface models, papers, evaluation code, training scripts, and inference instructions, and is suitable for research related to video understanding.

Open website

About this tool

VideoChat-R1 is a video spatio-temporal perception and reasoning model project developed by OpenGVLab and open-sourced on GitHub, and its associated paper was accepted by NIPS2025. The full project title is VideoChat-R1 & R1.5: Enhancing Spatio-Temporal Perception and Reasoning via Reinforcement Fine-Tuning, that is, enhancing video spatio-temporal perception and reasoning capabilities through reinforcement learning fine-tuning.

  1. Purpose This project targets video understanding tasks, with the goal of improving the model's spatio-temporal perception and reasoning capabilities for video content through multi-task joint reinforcement learning, covering scenarios such as short and long videos, temporal grounding, video reasoning, and spatio-temporal perception.

  2. Main Capabilities According to the project description, the model achieves consistent and stronger results in benchmarks such as short and long videos, temporal grounding, video reasoning, and spatio-temporal perception. The project adopts multi-task joint RL to strengthen the model's spatio-temporal perception and reasoning capabilities. During inference, the model simulates the human hierarchical attention mechanism and can progressively locate the Region of Interest (ROI) in the input video. This multi-step perception process makes the model's performance improve with each step.

  3. Versions and Release Timeline

  • 2025/09/26: Released the VideoChat-R1.5 model, including the Huggingface model, paper, and evaluation code.
  • 2025/09/22: VideoChat-R1.5 was accepted by NIPS2025.
  • 2025/04/22: Released VideoChat-R1-caption on Huggingface.
  • 2025/04/14: Released VideoChat-R1 and VideoChat-R1-thinking on Huggingface.
  • 2025/04/10: Released the VideoChat-R1 paper and code.

  1. Usage The project provides demo and inference instructions. For specific inference steps, please refer to the Huggingface README. The related evaluation code is located in the eval_scripts and lmms-eval_videochat directories, and the training-related scripts are located in the training_scripts directory. The project also provides citation information and notes that if there areFor related inquiries, please contact yanziang@pjlab.org.cn.

  2. Applicable Scenarios Suitable for research work related to video spatio-temporal perception and reasoning, including short and long video understanding, temporal grounding, video reasoning, and other directions, and can be used by researchers for reference, evaluation, and training.

  3. Limitations The project materials do not provide information such as fees, pricing, platform services, deployment details, or commercial use terms, and the related content is not currently available.

Related tools

Browse category