Multilingual Speech Emotion Recognition With Multi-Gating Mechanism and Neural Architecture Search
This addresses the problem of limited SER capabilities for low-resourced languages, representing a novel method for a known bottleneck rather than a foundational advancement.
The paper tackles speech emotion recognition for low-resourced languages by proposing a multi-domain model with a multi-gating mechanism and neural architecture search, achieving state-of-the-art accuracy improvements of 3% for German and 14.3% for French.
Speech emotion recognition (SER) classifies audio into emotion categories such as Happy, Angry, Fear, Disgust and Neutral. While Speech Emotion Recognition (SER) is a common application for popular languages, it continues to be a problem for low-resourced languages, i.e., languages with no pretrained speech-to-text recognition models. This paper firstly proposes a language-specific model that extract emotional information from multiple pre-trained speech models, and then designs a multi-domain model that simultaneously performs SER for various languages. Our multidomain model employs a multi-gating mechanism to generate unique weighted feature combination for each language, and also searches for specific neural network structure for each language through a neural architecture search module. In addition, we introduce a contrastive auxiliary loss to build more separable representations for audio data. Our experiments show that our model raises the state-of-the-art accuracy by 3% for German and 14.3% for French.