Mugeetion: Musical Interface Using Facial Gesture and Emotion
This addresses the challenge of integrating intangible emotions into algorithmic music creation for users like composers or interactive installation audiences, though it appears incremental in combining existing gesture and emotion-sound modeling techniques.
The authors tackled the problem of capturing and quantifying emotions for music composition by developing Mugeetion, a musical interface that translates users' facial gestures into emotional states and relays this data to musical features, which was tested in an exhibition where audiences heard sound changes based on their emotions.
People feel emotions when listening to music. However, emotions are not tangible objects that can be exploited in the music composition process as they are difficult to capture and quantify in algorithms. We present a novel musical interface, Mugeetion, designed to capture occurring instances of emotional states from users' facial gestures and relay that data to associated musical features. Mugeetion can translate qualitative data of emotional states into quantitative data, which can be utilized in the sound generation process. We also presented and tested this work in the exhibition of sound installation, Hearing Seascape, using the audiences' facial expressions. Audiences heard changes in the background sound based on their emotional state. The process contributes multiple research areas, such as gesture tracking systems, emotion-sound modeling, and the connection between sound and facial gesture.