LOGO-Former: Local-Global Spatio-Temporal Transformer for Dynamic Facial Expression Recognition
This work addresses the problem of recognizing facial expressions in videos for applications like human-computer interaction, offering a novel method that improves over previous CNN and Transformer approaches by reducing computational complexity.
The paper tackles dynamic facial expression recognition in the wild by proposing LOGO-Former, a local-global spatio-temporal Transformer that captures both local and long-range dependencies in videos while balancing computational costs, achieving state-of-the-art performance on datasets like DFEW and FERV39K with improved accuracy and efficiency.
Previous methods for dynamic facial expression recognition (DFER) in the wild are mainly based on Convolutional Neural Networks (CNNs), whose local operations ignore the long-range dependencies in videos. Transformer-based methods for DFER can achieve better performances but result in higher FLOPs and computational costs. To solve these problems, the local-global spatio-temporal Transformer (LOGO-Former) is proposed to capture discriminative features within each frame and model contextual relationships among frames while balancing the complexity. Based on the priors that facial muscles move locally and facial expressions gradually change, we first restrict both the space attention and the time attention to a local window to capture local interactions among feature tokens. Furthermore, we perform the global attention by querying a token with features from each local window iteratively to obtain long-range information of the whole video sequence. In addition, we propose the compact loss regularization term to further encourage the learned features have the minimum intra-class distance and the maximum inter-class distance. Experiments on two in-the-wild dynamic facial expression datasets (i.e., DFEW and FERV39K) indicate that our method provides an effective way to make use of the spatial and temporal dependencies for DFER.