Dual Encoding for Zero-Example Video Retrieval
It addresses the problem of efficient cross-modal retrieval for video search systems, offering a concept-free approach that improves performance over existing methods, though it is incremental as it builds on deep encoding techniques.
The paper tackles zero-example video retrieval, where users search unlabeled videos using natural language queries without visual examples, by proposing a dual deep encoding network that encodes videos and queries into dense representations, achieving new state-of-the-art results on benchmarks like MSR-VTT, TRECVID 2016, and 2017 Ad-hoc Video Search.
This paper attacks the challenging problem of zero-example video retrieval. In such a retrieval paradigm, an end user searches for unlabeled videos by ad-hoc queries described in natural language text with no visual example provided. Given videos as sequences of frames and queries as sequences of words, an effective sequence-to-sequence cross-modal matching is required. The majority of existing methods are concept based, extracting relevant concepts from queries and videos and accordingly establishing associations between the two modalities. In contrast, this paper takes a concept-free approach, proposing a dual deep encoding network that encodes videos and queries into powerful dense representations of their own. Dual encoding is conceptually simple, practically effective and end-to-end. As experiments on three benchmarks, i.e. MSR-VTT, TRECVID 2016 and 2017 Ad-hoc Video Search show, the proposed solution establishes a new state-of-the-art for zero-example video retrieval.