Dynamic Layer Customization for Noise Robust Speech Emotion Recognition in Heterogeneous Condition Training
This work addresses the challenge of deploying speech emotion recognition systems in real-world noisy environments, representing an incremental advance by adapting existing methods to handle heterogeneous noise conditions without requiring unlabeled test data.
The paper tackles the problem of noise robustness in speech emotion recognition by introducing heterogeneous condition training, which improves performance by dynamically routing samples to specialized encoders for known noise conditions and using domain adaptation for unseen conditions, achieving significant improvements over non-specialized approaches.
Robustness to environmental noise is important to creating automatic speech emotion recognition systems that are deployable in the real world. Prior work on noise robustness has assumed that systems would not make use of sample-by-sample training noise conditions, or that they would have access to unlabelled testing data to generalize across noise conditions. We avoid these assumptions and introduce the resulting task as heterogeneous condition training. We show that with full knowledge of the test noise conditions, we can improve performance by dynamically routing samples to specialized feature encoders for each noise condition, and with partial knowledge, we can use known noise conditions and domain adaptation algorithms to train systems that generalize well to unseen noise conditions. We then extend these improvements to the multimodal setting by dynamically routing samples to maintain temporal ordering, resulting in significant improvements over approaches that do not specialize or generalize based on noise type.