VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuning
This work addresses the problem of improving video comprehension for AI agents, offering a data-efficient method that is incremental in applying reinforcement learning to multimodal models.
The paper tackled the challenge of enhancing spatio-temporal perception in video understanding for Multimodal Large Language Models (MLLMs) by integrating reinforcement fine-tuning with rule-based rewards, resulting in VideoChat-R1 achieving state-of-the-art performance with improvements like +31.8 in temporal grounding and +31.2 in object tracking.
Reinforcement Learning (RL) benefits Large Language Models (LLMs) for complex reasoning. Inspired by this, we explore integrating spatio-temporal specific rewards into Multimodal Large Language Models (MLLMs) to address the unique challenges of video understanding, such as long-range temporal associations. This paper investigates how rule-based rewards, particularly temporal ones, can improve video reasoning and their generalizability. Our study proposes Reinforcement Fine-Tuning (RFT) as a data-efficient method to enhance video reasoning on specific tasks without sacrificing original capabilities. Through joint RFT on multiple spatio-temporal perception tasks, we developed VideoChat-R1, a powerful Video MLLM. VideoChat-R1 achieves state-of-the-art spatio-temporal perception, demonstrating significant improvements in tasks like temporal grounding (+31.8) and object tracking (+31.2), while also improving general QA benchmarks. The enhanced perception and preserved chat abilities contribute to a more reliable video dialogue system, leading to our ``Temporal Clue-driven Reasoning" inference schema. This work provides a foundation for developing robust, real-world video comprehension agents.