Speech Emotion Recognition with Co-Attention based Multi-level Acoustic Information
This work addresses the problem of improving emotion recognition from speech for applications like human-computer interaction, but it is incremental as it builds on existing methods with a novel fusion approach.
The paper tackles the challenge of extracting comprehensive audio information for Speech Emotion Recognition (SER) by proposing an end-to-end system that uses multi-level acoustic features fused with a co-attention module, achieving competitive performance on the IEMOCAP dataset with speaker-independent cross-validation.
Speech Emotion Recognition (SER) aims to help the machine to understand human's subjective emotion from only audio information. However, extracting and utilizing comprehensive in-depth audio information is still a challenging task. In this paper, we propose an end-to-end speech emotion recognition system using multi-level acoustic information with a newly designed co-attention module. We firstly extract multi-level acoustic information, including MFCC, spectrogram, and the embedded high-level acoustic information with CNN, BiLSTM and wav2vec2, respectively. Then these extracted features are treated as multimodal inputs and fused by the proposed co-attention mechanism. Experiments are carried on the IEMOCAP dataset, and our model achieves competitive performance with two different speaker-independent cross-validation strategies. Our code is available on GitHub.