Human-Centered Emotion Recognition in Animated GIFs
This work addresses emotion recognition for social media users by improving performance on GIFs, though it is incremental as it builds on existing methods with a focus on human features.
The study tackled the problem of automated emotion recognition in animated GIFs by emphasizing human-related information, resulting in a proposed Keypoint Attended Visual Attention Network (KAVAN) that outperformed state-of-the-art methods on the MIT GIFGIF dataset.
As an intuitive way of expression emotion, the animated Graphical Interchange Format (GIF) images have been widely used on social media. Most previous studies on automated GIF emotion recognition fail to effectively utilize GIF's unique properties, and this potentially limits the recognition performance. In this study, we demonstrate the importance of human related information in GIFs and conduct human-centered GIF emotion recognition with a proposed Keypoint Attended Visual Attention Network (KAVAN). The framework consists of a facial attention module and a hierarchical segment temporal module. The facial attention module exploits the strong relationship between GIF contents and human characters, and extracts frame-level visual feature with a focus on human faces. The Hierarchical Segment LSTM (HS-LSTM) module is then proposed to better learn global GIF representations. Our proposed framework outperforms the state-of-the-art on the MIT GIFGIF dataset. Furthermore, the facial attention module provides reliable facial region mask predictions, which improves the model's interpretability.