Advancing Multiple Instance Learning with Attention Modeling for Categorical Speech Emotion Recognition
This work addresses the problem of segment-level emotion extraction for speech processing researchers, but it is incremental as it builds on existing MIL and attention methods.
The paper tackled categorical speech emotion recognition from weakly labeled utterances by proposing multiple instance learning to extract segment embeddings and attention models to focus on salient parts, achieving better or competitive results on CASIA and IEMOCAP databases.
Categorical speech emotion recognition is typically performed as a sequence-to-label problem, i.e., to determine the discrete emotion label of the input utterance as a whole. One of the main challenges in practice is that most of the existing emotion corpora do not give ground truth labels for each segment; instead, we only have labels for whole utterances. To extract segment-level emotional information from such weakly labeled emotion corpora, we propose using multiple instance learning (MIL) to learn segment embeddings in a weakly supervised manner. Also, for a sufficiently long utterance, not all of the segments contain relevant emotional information. In this regard, three attention-based neural network models are then applied to the learned segment embeddings to attend the most salient part of a speech utterance. Experiments on the CASIA corpus and the IEMOCAP database show better or highly competitive results than other state-of-the-art approaches.