MMDuet2: Enhancing Proactive Interaction of Video MLLMs with Multi-Turn Reinforcement Learning
This addresses the challenge of real-time proactive interaction in video MLLMs for applications like video assistants, though it appears incremental as it builds on existing proactive interaction concepts.
The paper tackles the problem of enabling video multimodal large language models to proactively decide when to respond during video playback, proposing a multi-turn reinforcement learning method that eliminates the need for precise response time annotations. The result is MMDuet2, which outperforms existing baselines in response timing and quality, achieving state-of-the-art performance on the ProactiveVideoQA benchmark.
Recent advances in video multimodal large language models (Video MLLMs) have significantly enhanced video understanding and multi-modal interaction capabilities. While most existing systems operate in a turn-based manner where the model can only reply after user turns, proactively deciding when to reply during video playback presents a promising yet challenging direction for real-time applications. In this work, we propose a novel text-to-text approach to proactive interaction, where the model autonomously determines whether to respond or remain silent at each turn based on dialogue history and visual context up to current frame of an streaming video. To overcome difficulties in previous methods such as manually tuning response decision thresholds and annotating precise reply times, we introduce a multi-turn RL based training method that encourages timely and accurate responses without requiring precise response time annotations. We train our model MMDuet2 on a dataset of 52k videos with two types of dialogues via SFT and RL. Experimental results demonstrate that MMDuet2 outperforms existing proactive Video MLLM baselines in response timing and quality, achieving state-of-the-art performance on the ProactiveVideoQA benchmark.