Building Emotional Machines: Recognizing Image Emotions through Deep Neural Networks
This work addresses the challenge of recognizing emotions in images for applications in affective computing and human-computer interaction, but it is incremental as it builds on existing methods by combining object and background features.
The paper tackled the problem of predicting emotions from images by focusing on high-level semantic features like objects and backgrounds, and built a deep neural network that outputs continuous emotion values in valence-arousal space, achieving effective prediction performance as confirmed by experiments.
An image is a very effective tool for conveying emotions. Many researchers have investigated in computing the image emotions by using various features extracted from images. In this paper, we focus on two high level features, the object and the background, and assume that the semantic information of images is a good cue for predicting emotion. An object is one of the most important elements that define an image, and we find out through experiments that there is a high correlation between the object and the emotion in images. Even with the same object, there may be slight difference in emotion due to different backgrounds, and we use the semantic information of the background to improve the prediction performance. By combining the different levels of features, we build an emotion based feed forward deep neural network which produces the emotion values of a given image. The output emotion values in our framework are continuous values in the 2-dimensional space (Valence and Arousal), which are more effective than using a few number of emotion categories in describing emotions. Experiments confirm the effectiveness of our network in predicting the emotion of images.