Enabling Embodied Analogies in Intelligent Music Systems
This work addresses the challenge of enabling embodied analogies in intelligent music systems, which is incremental as it builds on existing multidisciplinary tools.
The paper tackled the problem of cross-modal machine learning for music and dance by creating a dataset using wisdom-of-the-crowd approaches and applying audio/language-informed techniques to identify emotional content, with results including the integration of motion capture data from dancers.
The present methodology is aimed at cross-modal machine learning and uses multidisciplinary tools and methods drawn from a broad range of areas and disciplines, including music, systematic musicology, dance, motion capture, human-computer interaction, computational linguistics and audio signal processing. Main tasks include: (1) adapting wisdom-of-the-crowd approaches to embodiment in music and dance performance to create a dataset of music and music lyrics that covers a variety of emotions, (2) applying audio/language-informed machine learning techniques to that dataset to identify automatically the emotional content of the music and the lyrics, and (3) integrating motion capture data from a Vicon system and dancers performing on that music.