DualHash: A Stochastic Primal-Dual Algorithm with Theoretical Guarantee for Deep Hashing
This work addresses the problem of ensuring convergence in deep hashing methods for researchers and practitioners in large-scale retrieval, though it is incremental as it builds on existing regularization techniques.
The paper tackles the challenge of optimizing deep hashing with W-type regularizations, which lack convergence guarantees, by proposing DualHash, a stochastic primal-dual algorithm that achieves rigorous complexity bounds and demonstrates superior performance on image retrieval databases.
Deep hashing converts high-dimensional feature vectors into compact binary codes, enabling efficient large-scale retrieval. A fundamental challenge in deep hashing stems from the discrete nature of quantization in generating the codes. W-type regularizations, such as $||z|-1|$, have been proven effective as they encourage variables toward binary values. However, existing methods often directly optimize these regularizations without convergence guarantees. While proximal gradient methods offer a promising solution, the coupling between W-type regularizers and neural network outputs results in composite forms that generally lack closed-form proximal solutions. In this paper, we present a stochastic primal-dual hashing algorithm, referred to as DualHash, that provides rigorous complexity bounds. Using Fenchel duality, we partially transform the nonconvex W-type regularization optimization into the dual space, which results in a proximal operator that admits closed-form solutions. We derive two algorithm instances: a momentum-accelerated version with $\mathcal{O}(\varepsilon^{-4})$ complexity and an improved $\mathcal{O}(\varepsilon^{-3})$ version using variance reduction. Experiments on three image retrieval databases demonstrate the superior performance of DualHash.