EmotiEffNet Facial Features in Uni-task Emotion Recognition in Video at ABAW-5 competition
This work addresses emotion recognition in-the-wild for applications like human-computer interaction, but it is incremental as it builds on existing methods and competition frameworks.
The authors tackled emotion recognition in video by using EmotiEffNet for feature extraction and an ensemble classifier with post-processing smoothing, achieving significantly higher F1-scores for facial expression recognition and action unit detection and better correlation coefficients for valence/arousal estimation compared to baseline on the Aff-Wild2 database.
In this article, the results of our team for the fifth Affective Behavior Analysis in-the-wild (ABAW) competition are presented. The usage of the pre-trained convolutional networks from the EmotiEffNet family for frame-level feature extraction is studied. In particular, we propose an ensemble of a multi-layered perceptron and the LightAutoML-based classifier. The post-processing by smoothing the results for sequential frames is implemented. Experimental results for the large-scale Aff-Wild2 database demonstrate that our model achieves a much greater macro-averaged F1-score for facial expression recognition and action unit detection and concordance correlation coefficients for valence/arousal estimation when compared to baseline.