Advancing the Foundation Model for Music Understanding
This work addresses the problem of specialized models in music understanding for researchers and practitioners, offering a more integrated approach.
The authors tackled the fragmentation in Music Information Retrieval by introducing MuFun, a unified foundation model for holistic music understanding, which significantly outperforms existing audio large language models on the proposed MuCUE benchmark.
The field of Music Information Retrieval (MIR) is fragmented, with specialized models excelling at isolated tasks. In this work, we challenge this paradigm by introducing a unified foundation model named MuFun for holistic music understanding. Our model features a novel architecture that jointly processes instrumental and lyrical content, and is trained on a large-scale dataset covering diverse tasks such as genre classification, music tagging, and question answering. To facilitate robust evaluation, we also propose a new benchmark for multi-faceted music understanding called MuCUE (Music Comprehensive Understanding Evaluation). Experiments show our model significantly outperforms existing audio large language models across the MuCUE tasks, demonstrating its state-of-the-art effectiveness and generalization ability.