Color histogram equalization and fine-tuning to improve expression recognition of (partially occluded) faces on sign language datasets
This work addresses the problem of improving expression recognition for sign language analysis, particularly for deaf subjects, but is incremental as it builds on existing methods with dataset-specific adjustments.
The study tackled facial expression recognition on sign language datasets, including partially occluded faces, by introducing color histogram equalization and fine-tuning, achieving 83.8% mean sensitivity with low variance and outperforming human accuracy in some cases.
The goal of this investigation is to quantify to what extent computer vision methods can correctly classify facial expressions on a sign language dataset. We extend our experiments by recognizing expressions using only the upper or lower part of the face, which is needed to further investigate the difference in emotion manifestation between hearing and deaf subjects. To take into account the peculiar color profile of a dataset, our method introduces a color normalization stage based on histogram equalization and fine-tuning. The results show the ability to correctly recognize facial expressions with 83.8% mean sensitivity and very little variance (.042) among classes. Like for humans, recognition of expressions from the lower half of the face (79.6%) is higher than that from the upper half (77.9%). Noticeably, the classification accuracy from the upper half of the face is higher than human level.