Echoes of Humanity: Exploring the Perceived Humanness of AI Music
This research addresses the need to understand human perception of AI music for educating users and improving models, but it is incremental as it builds on existing listener-focused experiments.
The study tackled the problem of how humans perceive AI-generated music by conducting a blind Turing-like test, finding that listeners' reliability in distinguishing AI music causally increases when song pairs are similar, with a focus on vocal and technical cues in judgments.
Recent advances in AI music (AIM) generation services are currently transforming the music industry. Given these advances, understanding how humans perceive AIM is crucial both to educate users on identifying AIM songs, and, conversely, to improve current models. We present results from a listener-focused experiment aimed at understanding how humans perceive AIM. In a blind, Turing-like test, participants were asked to distinguish, from a pair, the AIM and human-made song. We contrast with other studies by utilizing a randomized controlled crossover trial that controls for pairwise similarity and allows for a causal interpretation. We are also the first study to employ a novel, author-uncontrolled dataset of AIM songs from real-world usage of commercial models (i.e., Suno). We establish that listeners' reliability in distinguishing AIM causally increases when pairs are similar. Lastly, we conduct a mixed-methods content analysis of listeners' free-form feedback, revealing a focus on vocal and technical cues in their judgments.