Efficient Training of Very Deep Neural Networks for Supervised Hashing
This addresses the challenge of efficient deep learning for hashing tasks, offering a practical solution for applications like image retrieval, though it is incremental in improving training methods.
The paper tackles the problem of training very deep neural networks for supervised hashing, which is limited by vanishing gradients and computational inefficiency in existing methods, and achieves state-of-the-art performance on benchmark datasets by training networks with 64 hidden layers in about 3 hours on a single GPU.
In this paper, we propose training very deep neural networks (DNNs) for supervised learning of hash codes. Existing methods in this context train relatively "shallow" networks limited by the issues arising in back propagation (e.e. vanishing gradients) as well as computational efficiency. We propose a novel and efficient training algorithm inspired by alternating direction method of multipliers (ADMM) that overcomes some of these limitations. Our method decomposes the training process into independent layer-wise local updates through auxiliary variables. Empirically we observe that our training algorithm always converges and its computational complexity is linearly proportional to the number of edges in the networks. Empirically we manage to train DNNs with 64 hidden layers and 1024 nodes per layer for supervised hashing in about 3 hours using a single GPU. Our proposed very deep supervised hashing (VDSH) method significantly outperforms the state-of-the-art on several benchmark datasets.