Collaborative Quantization for Cross-Modal Similarity Search
This addresses the problem of efficient cross-modal search for applications like image-text retrieval, though it is an incremental extension of quantization methods from single-modal to cross-modal settings.
The paper tackles cross-modal similarity search by proposing a collaborative quantization method that jointly learns quantizers for images and texts, aligning their representations in a common space. Experimental results on three benchmark datasets show the approach achieves state-of-the-art performance.
Cross-modal similarity search is a problem about designing a search system supporting querying across content modalities, e.g., using an image to search for texts or using a text to search for images. This paper presents a compact coding solution for efficient search, with a focus on the quantization approach which has already shown the superior performance over the hashing solutions in the single-modal similarity search. We propose a cross-modal quantization approach, which is among the early attempts to introduce quantization into cross-modal search. The major contribution lies in jointly learning the quantizers for both modalities through aligning the quantized representations for each pair of image and text belonging to a document. In addition, our approach simultaneously learns the common space for both modalities in which quantization is conducted to enable efficient and effective search using the Euclidean distance computed in the common space with fast distance table lookup. Experimental results compared with several competitive algorithms over three benchmark datasets demonstrate that the proposed approach achieves the state-of-the-art performance.