MUST-RAG: MUSical Text Question Answering with Retrieval Augmented Generation
This addresses the problem of music-specific knowledge gaps in LLMs for researchers and developers in music AI, though it is incremental as it builds on existing RAG techniques.
The paper tackles the limited effectiveness of large language models in music-related applications by proposing MusT-RAG, a framework that adapts general-purpose LLMs for text-only music question answering using retrieval augmented generation, resulting in significant performance improvements over traditional fine-tuning approaches across benchmarks.
Recent advancements in Large language models (LLMs) have demonstrated remarkable capabilities across diverse domains. While they exhibit strong zero-shot performance on various tasks, LLMs' effectiveness in music-related applications remains limited due to the relatively small proportion of music-specific knowledge in their training data. To address this limitation, we propose MusT-RAG, a comprehensive framework based on Retrieval Augmented Generation (RAG) to adapt general-purpose LLMs for text-only music question answering (MQA) tasks. RAG is a technique that provides external knowledge to LLMs by retrieving relevant context information when generating answers to questions. To optimize RAG for the music domain, we (1) propose MusWikiDB, a music-specialized vector database for the retrieval stage, and (2) utilizes context information during both inference and fine-tuning processes to effectively transform general-purpose LLMs into music-specific models. Our experiment demonstrates that MusT-RAG significantly outperforms traditional fine-tuning approaches in enhancing LLMs' music domain adaptation capabilities, showing consistent improvements across both in-domain and out-of-domain MQA benchmarks. Additionally, our MusWikiDB proves substantially more effective than general Wikipedia corpora, delivering superior performance and computational efficiency.