Visual Response to Emotional State of User Interaction
This work addresses the challenge of making interactive art more responsive to user emotions, though it appears incremental in applying existing emotion detection methods to a new domain.
The authors tackled the problem of creating an interactive art installation that reflects environmental mood by interpreting user language and tone, resulting in a system that adjusts 3D seasonal animations based on detected emotional states using a novel fusion method to minimize demographic disparities.
This work proposes an interactive art installation "Mood spRing" designed to reflect the mood of the environment through interpretation of language and tone. Mood spRing consists of an AI program that controls an immersive 3D animation of the seasons. If the AI program perceives the language and tone of the users as pleasant, the animation progresses through idealized renditions of seasons. Otherwise, it slips into unpleasant weather and natural disasters of the season. To interpret the language and tone of the user interaction, hybrid state-of-the-art emotion detection methods are applied to the user audio and text inputs. The emotional states detected separately from tone and language are fused by a novel approach that aims at minimizing the possible model disparity across diverse demographic groups.