Moment of Untruth: Dealing with Negative Queries in Video Moment Retrieval
This addresses a practical issue in video analysis for users needing reliable moment localization, but it is incremental as it adapts an existing method to a new task formulation.
The paper tackles the problem of false positives in video moment retrieval when irrelevant queries are given, by proposing Negative-Aware Video Moment Retrieval (NA-VMR) and adapting UniVTG to achieve high negative rejection accuracy (avg. 98.4%) while maintaining moment retrieval scores within 3.87% Recall@1.
Video Moment Retrieval is a common task to evaluate the performance of visual-language models - it involves localising start and end times of moments in videos from query sentences. The current task formulation assumes that the queried moment is present in the video, resulting in false positive moment predictions when irrelevant query sentences are provided. In this paper we propose the task of Negative-Aware Video Moment Retrieval (NA-VMR), which considers both moment retrieval accuracy and negative query rejection accuracy. We make the distinction between In-Domain and Out-of-Domain negative queries and provide new evaluation benchmarks for two popular video moment retrieval datasets: QVHighlights and Charades-STA. We analyse the ability of current SOTA video moment retrieval approaches to adapt to Negative-Aware Video Moment Retrieval and propose UniVTG-NA, an adaptation of UniVTG designed to tackle NA-VMR. UniVTG-NA achieves high negative rejection accuracy (avg. $98.4\%$) scores while retaining moment retrieval scores to within $3.87\%$ Recall@1. Dataset splits and code are available at https://github.com/keflanagan/MomentofUntruth