Hashing with Mutual Information
This addresses the need for efficient retrieval in large databases for applications like image and video search, representing an incremental improvement with a novel information-theoretic approach.
The paper tackles the problem of learning binary vector embeddings for fast nearest neighbor retrieval by proposing a supervised hashing method that optimizes mutual information to reduce ambiguity in the Hamming space, achieving effective results on four image retrieval benchmarks including ImageNet.
Binary vector embeddings enable fast nearest neighbor retrieval in large databases of high-dimensional objects, and play an important role in many practical applications, such as image and video retrieval. We study the problem of learning binary vector embeddings under a supervised setting, also known as hashing. We propose a novel supervised hashing method based on optimizing an information-theoretic quantity: mutual information. We show that optimizing mutual information can reduce ambiguity in the induced neighborhood structure in the learned Hamming space, which is essential in obtaining high retrieval performance. To this end, we optimize mutual information in deep neural networks with minibatch stochastic gradient descent, with a formulation that maximally and efficiently utilizes available supervision. Experiments on four image retrieval benchmarks, including ImageNet, confirm the effectiveness of our method in learning high-quality binary embeddings for nearest neighbor retrieval.