Statistical Selection of CNN-Based Audiovisual Features for Instantaneous Estimation of Human Emotional States
This work addresses the need for more accurate real-time emotion prediction in human-computer interaction, though it is incremental as it builds on existing feature selection methods.
The paper tackled the problem of instantaneous prediction of continuous emotional states by selecting CNN-based audiovisual features using a mutual information-based process to minimize redundancy and maximize relevancy. The result showed that the proposed model achieved higher accuracy than traditional audio or video features on the RECOLA database.
Automatic prediction of continuous-level emotional state requires selection of suitable affective features to develop a regression system based on supervised machine learning. This paper investigates the performance of features statistically learned using convolutional neural networks for instantaneously predicting the continuous dimensions of emotional states. Features with minimum redundancy and maximum relevancy are chosen by using the mutual information-based selection process. The performance of frame-by-frame prediction of emotional state using the moderate length features as proposed in this paper is evaluated on spontaneous and naturalistic human-human conversation of RECOLA database. Experimental results show that the proposed model can be used for instantaneous prediction of emotional state with an accuracy higher than traditional audio or video features that are used for affective computation.