Emotion Recognition in Audio and Video Using Deep Neural Networks
This work addresses emotion recognition for applications like human-computer interaction, but it is incremental as it builds on existing deep learning methods.
The paper tackled emotion recognition from audio and video by exploring neural network architectures, resulting in a multi-model approach that achieved 54.0% accuracy for 4 emotions and 71.75% for 3 emotions on the IEMOCAP dataset.
Humans are able to comprehend information from multiple domains for e.g. speech, text and visual. With advancement of deep learning technology there has been significant improvement of speech recognition. Recognizing emotion from speech is important aspect and with deep learning technology emotion recognition has improved in accuracy and latency. There are still many challenges to improve accuracy. In this work, we attempt to explore different neural networks to improve accuracy of emotion recognition. With different architectures explored, we find (CNN+RNN) + 3DCNN multi-model architecture which processes audio spectrograms and corresponding video frames giving emotion prediction accuracy of 54.0% among 4 emotions and 71.75% among 3 emotions using IEMOCAP[2] dataset.