Self-Supervised Multi-View Representation Learning using Vision-Language Model for 3D/4D Facial Expression Recognition
This work addresses facial expression recognition for applications in human-computer interaction and mental health analysis, offering a scalable and annotation-efficient solution, though it is incremental as it builds on existing self-supervised and vision-language methods.
The paper tackled the problem of 3D/4D facial expression recognition by proposing SMILE-VLM, a self-supervised vision-language model that unifies multiview visual representation learning with natural language supervision, achieving state-of-the-art performance on multiple benchmarks and matching or exceeding supervised baselines.
Facial expression recognition (FER) is a fundamental task in affective computing with applications in human-computer interaction, mental health analysis, and behavioral understanding. In this paper, we propose SMILE-VLM, a self-supervised vision-language model for 3D/4D FER that unifies multiview visual representation learning with natural language supervision. SMILE-VLM learns robust, semantically aligned, and view-invariant embeddings by proposing three core components: multiview decorrelation via a Barlow Twins-style loss, vision-language contrastive alignment, and cross-modal redundancy minimization. Our framework achieves the state-of-the-art performance on multiple benchmarks. We further extend SMILE-VLM to the task of 4D micro-expression recognition (MER) to recognize the subtle affective cues. The extensive results demonstrate that SMILE-VLM not only surpasses existing unsupervised methods but also matches or exceeds supervised baselines, offering a scalable and annotation-efficient solution for expressive facial behavior understanding.