Temporal Stochastic Softmax for 3D CNNs: An Application in Facial Expression Recognition
This addresses the challenge of computational inefficiency and suboptimal performance in video-based facial expression recognition, though it is an incremental improvement over existing methods.
The paper tackles the problem of inefficient training of 3D CNNs for facial expression recognition in videos by proposing a softmax temporal pooling and weighted sampling strategy to select relevant clips, resulting in improved accuracy and reduced computational cost.
Training deep learning models for accurate spatiotemporal recognition of facial expressions in videos requires significant computational resources. For practical reasons, 3D Convolutional Neural Networks (3D CNNs) are usually trained with relatively short clips randomly extracted from videos. However, such uniform sampling is generally sub-optimal because equal importance is assigned to each temporal clip. In this paper, we present a strategy for efficient video-based training of 3D CNNs. It relies on softmax temporal pooling and a weighted sampling mechanism to select the most relevant training clips. The proposed softmax strategy provides several advantages: a reduced computational complexity due to efficient clip sampling, and an improved accuracy since temporal weighting focuses on more relevant clips during both training and inference. Experimental results obtained with the proposed method on several facial expression recognition benchmarks show the benefits of focusing on more informative clips in training videos. In particular, our approach improves performance and computational cost by reducing the impact of inaccurate trimming and coarse annotation of videos, and heterogeneous distribution of visual information across time.