EVOKE: Emotion Enabled Virtual Avatar Mapping Using Optimized Knowledge Distillation
This work addresses the need for immersive and emotionally engaging experiences in virtual environments, but it is incremental as it applies an existing method (knowledge distillation) to a specific domain.
The paper tackled the problem of integrating emotion recognition into 3D avatars for virtual environments by developing a lightweight framework using knowledge distillation, achieving 87% accuracy with a model that has 18 times fewer parameters than the teacher model.
As virtual environments continue to advance, the demand for immersive and emotionally engaging experiences has grown. Addressing this demand, we introduce Emotion enabled Virtual avatar mapping using Optimized KnowledgE distillation (EVOKE), a lightweight emotion recognition framework designed for the seamless integration of emotion recognition into 3D avatars within virtual environments. Our approach leverages knowledge distillation involving multi-label classification on the publicly available DEAP dataset, which covers valence, arousal, and dominance as primary emotional classes. Remarkably, our distilled model, a CNN with only two convolutional layers and 18 times fewer parameters than the teacher model, achieves competitive results, boasting an accuracy of 87% while demanding far less computational resources. This equilibrium between performance and deployability positions our framework as an ideal choice for virtual environment systems. Furthermore, the multi-label classification outcomes are utilized to map emotions onto custom-designed 3D avatars.