Who Can We Trust? Scope-Aware Video Moment Retrieval with Multi-Agent Conflict
This work solves the problem of integrating multiple models for more accurate video moment retrieval, which is incremental as it builds on existing methods by adding conflict resolution.
The study tackled video moment retrieval by addressing conflicts between different models' location results, introducing a reinforcement learning model with a multi-agent system that uses evidential learning to resolve conflicts and detect out-of-scope queries without extra training, achieving effectiveness shown in experiments on benchmark datasets.
Video moment retrieval uses a text query to locate a moment from a given untrimmed video reference. Locating corresponding video moments with text queries helps people interact with videos efficiently. Current solutions for this task have not considered conflict within location results from different models, so various models cannot integrate correctly to produce better results. This study introduces a reinforcement learning-based video moment retrieval model that can scan the whole video once to find the moment's boundary while producing its locational evidence. Moreover, we proposed a multi-agent system framework that can use evidential learning to resolve conflicts between agents' localization output. As a side product of observing and dealing with conflicts between agents, we can decide whether a query has no corresponding moment in a video (out-of-scope) without additional training, which is suitable for real-world applications. Extensive experiments on benchmark datasets show the effectiveness of our proposed methods compared with state-of-the-art approaches. Furthermore, the results of our study reveal that modeling competition and conflict of the multi-agent system is an effective way to improve RL performance in moment retrieval and show the new role of evidential learning in the multi-agent framework.