Semantic-Aware Interpretable Multimodal Music Auto-Tagging
This addresses the need for transparent and user-centric music tagging systems for researchers and end-users, though it is incremental as it builds on existing foundation models.
The paper tackled the problem of lack of interpretability in music auto-tagging by developing a framework that uses semantically clustered multimodal features and an expectation maximization algorithm, achieving competitive tagging performance.
Music auto-tagging is essential for organizing and discovering music in extensive digital libraries. While foundation models achieve exceptional performance in this domain, their outputs often lack interpretability, limiting trust and usability for researchers and end-users alike. In this work, we present an interpretable framework for music auto-tagging that leverages groups of musically meaningful multimodal features, derived from signal processing, deep learning, ontology engineering, and natural language processing. To enhance interpretability, we cluster features semantically and employ an expectation maximization algorithm, assigning distinct weights to each group based on its contribution to the tagging process. Our method achieves competitive tagging performance while offering a deeper understanding of the decision-making process, paving the way for more transparent and user-centric music tagging systems.