RMES: Real-Time Micro-Expression Spotting Using Phase From Riesz Pyramid
This work addresses the challenge of detecting subtle, involuntary facial expressions in real-time for applications like lie detection or psychology, representing an incremental improvement with specific computational gains.
The paper tackled the problem of real-time micro-expression spotting in videos by proposing RMES, a framework that uses phase from Riesz Pyramid for motion representation and a shallow CNN, achieving state-of-the-art performance on CAS(ME)2 and SAMM Long Videos datasets while reducing computational complexity by 77.8%.
Micro-expressions (MEs) are involuntary and subtle facial expressions that are thought to reveal feelings people are trying to hide. ME spotting detects the temporal intervals containing MEs in videos. Detecting such quick and subtle motions from long videos is difficult. Recent works leverage detailed facial motion representations, such as the optical flow, and deep learning models, leading to high computational complexity. To reduce computational complexity and achieve real-time operation, we propose RMES, a real-time ME spotting framework. We represent motion using phase computed by Riesz Pyramid, and feed this motion representation into a three-stream shallow CNN, which predicts the likelihood of each frame belonging to an ME. In comparison to optical flow, phase provides more localized motion estimates, which are essential for ME spotting, resulting in higher performance. Using phase also reduces the required computation of the ME spotting pipeline by 77.8%. Despite its relative simplicity and low computational complexity, our framework achieves state-of-the-art performance on two public datasets: CAS(ME)2 and SAMM Long Videos.