E2-Capsule Neural Networks for Facial Expression Recognition Using AU-Aware Attention
This work addresses facial expression recognition for applications like human-computer interaction, but it is incremental as it builds on existing capsule network techniques.
The paper tackled facial expression recognition by proposing E2-Capsnet, which uses AU-aware attention and enhanced capsule networks, achieving superior performance over state-of-the-art methods on RAF-DB and EmotioNet datasets.
Capsule neural network is a new and popular technique in deep learning. However, the traditional capsule neural network does not extract features sufficiently before the dynamic routing between the capsules. In this paper, the one Double Enhanced Capsule Neural Network (E2-Capsnet) that uses AU-aware attention for facial expression recognition (FER) is proposed. The E2-Capsnet takes advantage of dynamic routing between the capsules, and has two enhancement modules which are beneficial for FER. The first enhancement module is the convolutional neural network with AU-aware attention, which can help focus on the active areas of the expression. The second enhancement module is the capsule neural network with multiple convolutional layers, which enhances the ability of the feature representation. Finally, squashing function is used to classify the facial expression. We demonstrate the effectiveness of E2-Capsnet on the two public benchmark datasets, RAF-DB and EmotioNet. The experimental results show that our E2-Capsnet is superior to the state-of-the-art methods. Our implementation will be publicly available online.