A Hybrid Framework for Song Lyric Annotation Based on Human-LLM Alignment
This work provides a practical solution for improving annotation efficiency in the underexplored domain of lyrics emotion recognition, though it is incremental in nature.
The paper addresses the challenge of emotion annotation in song lyrics, which is subjective and often misaligned with song emotion. It introduces a hybrid framework that combines human and LLM annotation by predicting misalignment, achieving improved annotation quality.
Emotion recognition of song lyrics is a challenging task since lyrics may not necessarily align with the overall emotion of a song. As a result, lyrics annotation remains largely underexplored. Drawing inspiration from research in large language model (LLM) assisted annotation, we examine the alignment between humans and LLMs for annotation of lyrics by creating a new sentence-level dataset of lyrics. Our observations highlight the subjectivity of the task and the inherent challenges. Following this, we present a hybrid annotation framework that optimizes human and LLM annotation by predicting potential misalignment in annotation.