ALM2Vec: Learning Audio Embeddings for Universal Audio Retrieval with Large Audio-Language Models
For researchers and practitioners in audio retrieval, this work provides a unified embedding model that extends beyond audio-caption matching to support diverse retrieval objectives and natural-language instructions.
ALM2Vec introduces a universal audio embedding framework derived from large audio-language models, enabling instruction-aware retrieval across diverse tasks and domains. It achieves competitive performance on standard benchmarks while demonstrating compositional and controllable retrieval capabilities.
Recent advances in language--audio retrieval have been largely driven by contrastive dual-encoder architectures that align audio and text in a shared embedding space. While effective, existing retrieval embeddings are primarily optimized for audio--caption matching, limiting their ability to support diverse retrieval objectives and controllable retrieval behaviors. We present ALM2Vec, a universal audio embedding framework derived from pretrained large audio--language models (LALMs). By transferring the audio understanding, instruction-following, and reasoning capabilities acquired through large-scale multimodal training, ALM2Vec learns a unified embedding space for retrieval across audio domains and task types. Beyond conventional text--audio retrieval, ALM2Vec incorporates natural-language instructions into the embedding process, enabling instruction-aware retrieval for scenarios such as audio question answering and aspect-conditioned retrieval. Experimental results show that ALM2Vec achieves competitive performance on standard audio and speech retrieval benchmarks while exhibiting promising compositional and controllable retrieval capabilities, highlighting its potential as a unified audio embedding model for retrieval across domains, tasks, and user intents.