Synergistic Spotting and Recognition of Micro-Expression via Temporal State Transition
This work improves micro-expression analysis for applications like psychology and security, but it is incremental as it builds on existing deep learning methods.
The paper tackles the problem of analyzing micro-expressions in videos by addressing both spotting intervals and recognizing emotions, achieving state-of-the-art performance through a novel temporal state transition architecture and synergistic strategy.
Micro-expressions are involuntary facial movements that cannot be consciously controlled, conveying subtle cues with substantial real-world applications. The analysis of micro-expressions generally involves two main tasks: spotting micro-expression intervals in long videos and recognizing the emotions associated with these intervals. Previous deep learning methods have primarily relied on classification networks utilizing sliding windows. However, fixed window sizes and window-level hard classification introduce numerous constraints. Additionally, these methods have not fully exploited the potential of complementary pathways for spotting and recognition. In this paper, we present a novel temporal state transition architecture grounded in the state space model, which replaces conventional window-level classification with video-level regression. Furthermore, by leveraging the inherent connections between spotting and recognition tasks, we propose a synergistic strategy that enhances overall analysis performance. Extensive experiments demonstrate that our method achieves state-of-the-art performance. The codes and pre-trained models are available at https://github.com/zizheng-guo/ME-TST.