VideoCap-R1: Enhancing MLLMs for Video Captioning via Structured Thinking
This work addresses the challenge of generating accurate action descriptions in video captions for AI applications, representing an incremental advance by applying existing RL techniques to a new multimodal domain.
The paper tackles the problem of enhancing video captioning in multi-modal large language models by introducing VideoCap-R1, which uses GRPO-based reinforcement learning with structured thinking prompts and specialized reward mechanisms. The approach achieves substantial improvements over baseline models across multiple benchmarks, including +4.4 event F1 on DREAM1K and +4.2 accuracy on VDC.
While recent advances in reinforcement learning have significantly enhanced reasoning capabilities in large language models (LLMs), these techniques remain underexplored in multi-modal LLMs for video captioning. This paper presents the first systematic investigation of GRPO-based RL post-training for video MLLMs, with the goal of enhancing video MLLMs' capability of describing actions in videos. Specifically, we develop the VideoCap-R1, which is prompted to first perform structured thinking that analyzes video subjects with their attributes and actions before generating complete captions, supported by two specialized reward mechanisms: a LLM-free think scorer evaluating the structured thinking quality and a LLM-assisted caption scorer assessing the output quality. The RL training framework effectively establishes the connection between structured reasoning and comprehensive description generation, enabling the model to produce captions with more accurate actions. Our experiments demonstrate that VideoCap-R1 achieves substantial improvements over the Qwen2VL-7B baseline using limited samples (1.5k) across multiple video caption benchmarks (DREAM1K: +4.4 event F1, VDC: +4.2 Acc, CAREBENCH: +3.1 action F1, +6.9 object F1) while consistently outperforming the SFT-trained counterparts, confirming GRPO's superiority in enhancing MLLMs' captioning capabilities.