Simultaneously Learning Robust Audio Embeddings and balanced Hash codes for Query-by-Example
This work addresses a specific bottleneck in audio retrieval systems for applications requiring efficient and scalable query-by-example, though it is incremental as it builds on existing deep learning and indexing methods.
The paper tackles the problem of imbalanced hash codes in audio fingerprinting systems, which degrade retrieval performance, by proposing a self-supervised learning framework that simultaneously learns robust audio embeddings and balanced hash codes, resulting in improved retrieval efficiency and high accuracy, especially under high distortion levels.
Audio fingerprinting systems must efficiently and robustly identify query snippets in an extensive database. To this end, state-of-the-art systems use deep learning to generate compact audio fingerprints. These systems deploy indexing methods, which quantize fingerprints to hash codes in an unsupervised manner to expedite the search. However, these methods generate imbalanced hash codes, leading to their suboptimal performance. Therefore, we propose a self-supervised learning framework to compute fingerprints and balanced hash codes in an end-to-end manner to achieve both fast and accurate retrieval performance. We model hash codes as a balanced clustering process, which we regard as an instance of the optimal transport problem. Experimental results indicate that the proposed approach improves retrieval efficiency while preserving high accuracy, particularly at high distortion levels, compared to the competing methods. Moreover, our system is efficient and scalable in computational load and memory storage.