Shallow Triple Stream Three-dimensional CNN (STSTNet) for Micro-expression Recognition
This work addresses micro-expression recognition for applications like psychology or security, but it is incremental as it builds on existing deep learning approaches with a focus on computational efficiency.
The paper tackled micro-expression recognition by proposing a shallow triple-stream 3D CNN (STSTNet) that uses optical flow features from video frames, achieving an unweighted average recall rate of 0.7605 and F1-score of 0.7353 on a composite database.
In the recent year, state-of-the-art for facial micro-expression recognition have been significantly advanced by deep neural networks. The robustness of deep learning has yielded promising performance beyond that of traditional handcrafted approaches. Most works in literature emphasized on increasing the depth of networks and employing highly complex objective functions to learn more features. In this paper, we design a Shallow Triple Stream Three-dimensional CNN (STSTNet) that is computationally light whilst capable of extracting discriminative high level features and details of micro-expressions. The network learns from three optical flow features (i.e., optical strain, horizontal and vertical optical flow fields) computed based on the onset and apex frames of each video. Our experimental results demonstrate the effectiveness of the proposed STSTNet, which obtained an unweighted average recall rate of 0.7605 and unweighted F1-score of 0.7353 on the composite database consisting of 442 samples from the SMIC, CASME II and SAMM databases.