Recognizing Facial Expressions of Occluded Faces using Convolutional Neural Networks
This addresses a practical problem for VR applications where facial expressions need to be recognized despite occlusions, but it is incremental as it adapts existing CNN methods to a specific scenario.
The paper tackles facial expression recognition for occluded faces, specifically in VR settings where the upper face is covered, by training CNNs on intentionally occluded images to focus on the lower face, resulting in up to 13% higher accuracy than baseline models on occluded faces.
In this paper, we present an approach based on convolutional neural networks (CNNs) for facial expression recognition in a difficult setting with severe occlusions. More specifically, our task is to recognize the facial expression of a person wearing a virtual reality (VR) headset which essentially occludes the upper part of the face. In order to accurately train neural networks for this setting, in which faces are severely occluded, we modify the training examples by intentionally occluding the upper half of the face. This forces the neural networks to focus on the lower part of the face and to obtain better accuracy rates than models trained on the entire faces. Our empirical results on two benchmark data sets, FER+ and AffectNet, show that our CNN models' predictions on lower-half faces are up to 13% higher than the baseline CNN models trained on entire faces, proving their suitability for the VR setting. Furthermore, our models' predictions on lower-half faces are no more than 10% under the baseline models' predictions on full faces, proving that there are enough clues in the lower part of the face to accurately predict facial expressions.