A Multi-stream Convolutional Neural Network for Micro-expression Recognition Using Optical Flow and EVM
This work addresses the problem of low recognition rates in micro-expression analysis for applications like public security and psychotherapy, representing an incremental improvement.
The paper tackles micro-expression recognition by designing a multi-stream convolutional neural network that uses optical flow and EVM to magnify subtle movements, achieving promising recognition results on CASME II and SAMM databases compared to state-of-the-art methods.
Micro-expression (ME) recognition plays a crucial role in a wide range of applications, particularly in public security and psychotherapy. Recently, traditional methods rely excessively on machine learning design and the recognition rate is not high enough for its practical application because of its short duration and low intensity. On the other hand, some methods based on deep learning also cannot get high accuracy due to problems such as the imbalance of databases. To address these problems, we design a multi-stream convolutional neural network (MSCNN) for ME recognition in this paper. Specifically, we employ EVM and optical flow to magnify and visualize subtle movement changes in MEs and extract the masks from the optical flow images. And then, we add the masks, optical flow images, and grayscale images into the MSCNN. After that, in order to overcome the imbalance of databases, we added a random over-sampler after the Dense Layer of the neural network. Finally, extensive experiments are conducted on two public ME databases: CASME II and SAMM. Compared with many recent state-of-the-art approaches, our method achieves more promising recognition results.