A Comprehensive Study on the Effectiveness of ASR Representations for Noise-Robust Speech Emotion Recognition
This addresses the problem of handling non-stationary real-world noises in speech emotion recognition for applications like human-computer interaction, but it is incremental as it builds on existing ASR and NSER methods.
The paper tackled noise-robust speech emotion recognition by using an ASR model as a feature extractor to remove non-vocal information from noisy speech, achieving better performance than conventional noise reduction, self-supervised learning, and text-based approaches.
This paper proposes an efficient attempt to noisy speech emotion recognition (NSER). Conventional NSER approaches have proven effective in mitigating the impact of artificial noise sources, such as white Gaussian noise, but are limited to non-stationary noises in real-world environments due to their complexity and uncertainty. To overcome this limitation, we introduce a new method for NSER by adopting the automatic speech recognition (ASR) model as a noise-robust feature extractor to eliminate non-vocal information in noisy speech. We first obtain intermediate layer information from the ASR model as a feature representation for emotional speech and then apply this representation for the downstream NSER task. Our experimental results show that 1) the proposed method achieves better NSER performance compared with the conventional noise reduction method, 2) outperforms self-supervised learning approaches, and 3) even outperforms text-based approaches using ASR transcription or the ground truth transcription of noisy speech.