DAIRHuM: A Platform for Directly Aligning AI Representations with Human Musical Judgments applied to Carnatic Music
This work addresses the problem of data scarcity and cultural specificity in music information retrieval for under-represented genres like Carnatic music, though it is incremental as it builds on existing alignment concepts.
The paper tackles the challenge of aligning music AI model representations with human musical judgments by introducing DAIRHuM, a platform that allows users to label similarities in music recordings and analyze alignment with pre-trained models, applied to Carnatic music where it demonstrates significant findings on rhythmic harmony alignment and highlights genre-specific differences.
Quantifying and aligning music AI model representations with human behavior is an important challenge in the field of MIR. This paper presents a platform for exploring the Direct alignment between AI music model Representations and Human Musical judgments (DAIRHuM). It is designed to enable musicians and experimentalists to label similarities in a dataset of music recordings, and examine a pre-trained model's alignment with their labels using quantitative scores and visual plots. DAIRHuM is applied to analyze alignment between NSynth representations, and a rhythmic duet between two percussionists in a Carnatic quartet ensemble, an example of a genre where annotated data is scarce and assessing alignment is non-trivial. The results demonstrate significant findings on model alignment with human judgments of rhythmic harmony, while highlighting key differences in rhythm perception and music similarity judgments specific to Carnatic music. This work is among the first efforts to enable users to explore human-AI model alignment in Carnatic music and advance MIR research in Indian music while dealing with data scarcity and cultural specificity. The development of this platform provides greater accessibility to music AI tools for under-represented genres.