Facial Affect Analysis: Learning from Synthetic Data & Multi-Task Learning Challenges
This work addresses practical problems in facial affect analysis for in-the-wild applications, though it appears incremental as it builds on existing competition frameworks.
The paper tackles two challenges in facial affect analysis: multi-task learning and learning from synthetic data, proposing novel frameworks that outperformed baselines by a large margin on the ABAW competition validation sets.
Facial affect analysis remains a challenging task with its setting transitioned from lab-controlled to in-the-wild situations. In this paper, we present novel frameworks to handle the two challenges in the 4th Affective Behavior Analysis In-The-Wild (ABAW) competition: i) Multi-Task-Learning (MTL) Challenge and ii) Learning from Synthetic Data (LSD) Challenge. For MTL challenge, we adopt the SMM-EmotionNet with a better ensemble strategy of feature vectors. For LSD challenge, we propose respective methods to combat the problems of single labels, imbalanced distribution, fine-tuning limitations, and choice of model architectures. Experimental results on the official validation sets from the competition demonstrated that our proposed approaches outperformed baselines by a large margin. The code is available at https://github.com/sylyoung/ABAW4-HUST-ANT.