Multi-task, multi-label and multi-domain learning with residual convolutional networks for emotion recognition
This work addresses the problem of robust emotion recognition for applications in real-world settings, but it is incremental as it builds on existing multi-task approaches with a novel loss function.
The paper tackles the challenge of automated emotion recognition from facial images in uncontrolled environments by proposing a multi-task learning loss function that jointly learns emotion recognition and facial Action Unit detection, resulting in improved performance on two non-controlled datasets and an application for compound facial emotion expressions.
Automated emotion recognition in the wild from facial images remains a challenging problem. Although recent advances in Deep Learning have supposed a significant breakthrough in this topic, strong changes in pose, orientation and point of view severely harm current approaches. In addition, the acquisition of labeled datasets is costly, and current state-of-the-art deep learning algorithms cannot model all the aforementioned difficulties. In this paper, we propose to apply a multi-task learning loss function to share a common feature representation with other related tasks. Particularly we show that emotion recognition benefits from jointly learning a model with a detector of facial Action Units (collective muscle movements). The proposed loss function addresses the problem of learning multiple tasks with heterogeneously labeled data, improving previous multi-task approaches. We validate the proposal using two datasets acquired in non controlled environments, and an application to predict compound facial emotion expressions.