ArtistMus: A Globally Diverse, Artist-Centric Benchmark for Retrieval-Augmented Music Question Answering
This work addresses the problem of factual and contextual music question answering for researchers and practitioners in music information retrieval and computational musicology, though it is incremental as it applies existing RAG methods to a new domain-specific dataset.
The paper tackled the problem of limited music knowledge in large language models by introducing MusWikiDB, a vector database of 3.2M passages from music-related Wikipedia pages, and ArtistMus, a benchmark of 1,000 questions on 500 diverse artists, to evaluate retrieval-augmented generation for music question answering. The result showed that RAG improved factual accuracy by up to +56.8 percentage points, with open-source models like Qwen3 8B increasing from 35.0 to 91.8, and MusWikiDB provided 6 percentage points higher accuracy and 40% faster retrieval than a general-purpose corpus.
Recent advances in large language models (LLMs) have transformed open-domain question answering, yet their effectiveness in music-related reasoning remains limited due to sparse music knowledge in pretraining data. While music information retrieval and computational musicology have explored structured and multimodal understanding, few resources support factual and contextual music question answering (MQA) grounded in artist metadata or historical context. We introduce MusWikiDB, a vector database of 3.2M passages from 144K music-related Wikipedia pages, and ArtistMus, a benchmark of 1,000 questions on 500 diverse artists with metadata such as genre, debut year, and topic. These resources enable systematic evaluation of retrieval-augmented generation (RAG) for MQA. Experiments show that RAG markedly improves factual accuracy; open-source models gain up to +56.8 percentage points (for example, Qwen3 8B improves from 35.0 to 91.8), approaching proprietary model performance. RAG-style fine-tuning further boosts both factual recall and contextual reasoning, improving results on both in-domain and out-of-domain benchmarks. MusWikiDB also yields approximately 6 percentage points higher accuracy and 40% faster retrieval than a general-purpose Wikipedia corpus. We release MusWikiDB and ArtistMus to advance research in music information retrieval and domain-specific question answering, establishing a foundation for retrieval-augmented reasoning in culturally rich domains such as music.