Deep and Shallow Covariance Feature Quantization for 3D Facial Expression Recognition
This work addresses facial expression recognition for 3D face scans, which is important for applications like human-computer interaction, but it appears incremental as it builds on existing multi-modal and feature quantization approaches.
The paper tackled 3D facial expression recognition by proposing a multi-modal 2D+3D feature-based method that combines shallow features from 3D images and deep CNN features from transformed 2D images, achieving high classification performances on BU-3DFE and Bosphorus datasets compared to state-of-the-art methods.
Facial expressions recognition (FER) of 3D face scans has received a significant amount of attention in recent years. Most of the facial expression recognition methods have been proposed using mainly 2D images. These methods suffer from several issues like illumination changes and pose variations. Moreover, 2D mapping from 3D images may lack some geometric and topological characteristics of the face. Hence, to overcome this problem, a multi-modal 2D + 3D feature-based method is proposed. We extract shallow features from the 3D images, and deep features using Convolutional Neural Networks (CNN) from the transformed 2D images. Combining these features into a compact representation uses covariance matrices as descriptors for both features instead of single-handedly descriptors. A covariance matrix learning is used as a manifold layer to reduce the deep covariance matrices size and enhance their discrimination power while preserving their manifold structure. We then use the Bag-of-Features (BoF) paradigm to quantize the covariance matrices after flattening. Accordingly, we obtained two codebooks using shallow and deep features. The global codebook is then used to feed an SVM classifier. High classification performances have been achieved on the BU-3DFE and Bosphorus datasets compared to the state-of-the-art methods.