Deep Robust Multilevel Semantic Cross-Modal Hashing
This work addresses cross-modal retrieval for applications like multimedia search, but it appears incremental as it builds on existing hashing methods with robustness enhancements.
The paper tackles the problem of false codes in cross-modal retrieval due to modality discrepancies and noise by proposing a Robust Multilevel Semantic Hashing (RMSH) method, which achieves state-of-the-art performance on three benchmarks.
Hashing based cross-modal retrieval has recently made significant progress. But straightforward embedding data from different modalities into a joint Hamming space will inevitably produce false codes due to the intrinsic modality discrepancy and noises. We present a novel Robust Multilevel Semantic Hashing (RMSH) for more accurate cross-modal retrieval. It seeks to preserve fine-grained similarity among data with rich semantics, while explicitly require distances between dissimilar points to be larger than a specific value for strong robustness. For this, we give an effective bound of this value based on the information coding-theoretic analysis, and the above goals are embodied into a margin-adaptive triplet loss. Furthermore, we introduce pseudo-codes via fusing multiple hash codes to explore seldom-seen semantics, alleviating the sparsity problem of similarity information. Experiments on three benchmarks show the validity of the derived bounds, and our method achieves state-of-the-art performance.