Domain Adaptation in Multi-View Embedding for Cross-Modal Video Retrieval
This addresses the challenge of cross-modal video retrieval in real-world scenarios where labeled data is scarce, though it is incremental as it builds on existing domain adaptation techniques.
The paper tackles the problem of retrieving uncaptioned videos using text queries by adapting a model trained on a related but different video-caption dataset, introducing a new benchmark for fine-grained actions. Their iterative domain alignment method outperforms baseline approaches, showing consistent gains in retrieval performance.
Given a gallery of uncaptioned video sequences, this paper considers the task of retrieving videos based on their relevance to an unseen text query. To compensate for the lack of annotations, we rely instead on a related video gallery composed of video-caption pairs, termed the source gallery, albeit with a domain gap between its videos and those in the target gallery. We thus introduce the problem of Unsupervised Domain Adaptation for Cross-modal Video Retrieval, along with a new benchmark on fine-grained actions. We propose a novel iterative domain alignment method by means of pseudo-labelling target videos and cross-domain (i.e. source-target) ranking. Our approach adapts the embedding space to the target gallery, consistently outperforming source-only as well as marginal and conditional alignment methods.