Leveraging TCN and Transformer for effective visual-audio fusion in continuous emotion recognition
This work addresses emotion recognition for human-computer interaction, but it is incremental as it builds on existing methods for a competition.
The paper tackled continuous emotion recognition by proposing a multi-modal fusion model using Temporal Convolutional Networks and Transformer to integrate visual and audio information, achieving third place in the Expression Classification challenge.
Human emotion recognition plays an important role in human-computer interaction. In this paper, we present our approach to the Valence-Arousal (VA) Estimation Challenge, Expression (Expr) Classification Challenge, and Action Unit (AU) Detection Challenge of the 5th Workshop and Competition on Affective Behavior Analysis in-the-wild (ABAW). Specifically, we propose a novel multi-modal fusion model that leverages Temporal Convolutional Networks (TCN) and Transformer to enhance the performance of continuous emotion recognition. Our model aims to effectively integrate visual and audio information for improved accuracy in recognizing emotions. Our model outperforms the baseline and ranks 3 in the Expression Classification challenge.