Bayes Merging of Multiple Vocabularies for Scalable Image Retrieval
This work addresses a specific bottleneck in scalable image retrieval for computer vision applications, offering an incremental improvement over existing multi-vocabulary methods.
The paper tackles the problem of vocabulary correlation in image retrieval by proposing a Bayes merging approach to down-weight indexed features in overlapping areas, improving retrieval accuracy while maintaining high recall. Experiments on three benchmark datasets show consistent improvements over baselines and competitive performance with state-of-the-art methods.
The Bag-of-Words (BoW) representation is well applied to recent state-of-the-art image retrieval works. Typically, multiple vocabularies are generated to correct quantization artifacts and improve recall. However, this routine is corrupted by vocabulary correlation, i.e., overlapping among different vocabularies. Vocabulary correlation leads to an over-counting of the indexed features in the overlapped area, or the intersection set, thus compromising the retrieval accuracy. In order to address the correlation problem while preserve the benefit of high recall, this paper proposes a Bayes merging approach to down-weight the indexed features in the intersection set. Through explicitly modeling the correlation problem in a probabilistic view, a joint similarity on both image- and feature-level is estimated for the indexed features in the intersection set. We evaluate our method through extensive experiments on three benchmark datasets. Albeit simple, Bayes merging can be well applied in various merging tasks, and consistently improves the baselines on multi-vocabulary merging. Moreover, Bayes merging is efficient in terms of both time and memory cost, and yields competitive performance compared with the state-of-the-art methods.