Speech Emotion Diarization: Which Emotion Appears When?
This work addresses the need for more precise emotion analysis in speech processing for applications like human-computer interaction, though it is incremental as it builds on existing SER methods by adding temporal granularity.
The paper tackles the problem of recognizing emotions in speech at a fine-grained level by proposing Speech Emotion Diarization (SED), a new task that identifies when specific emotions occur within utterances, and introduces the Zaion Emotion Dataset (ZED) with manually-annotated emotion boundaries to establish a benchmark, providing competitive baselines and open-source resources.
Speech Emotion Recognition (SER) typically relies on utterance-level solutions. However, emotions conveyed through speech should be considered as discrete speech events with definite temporal boundaries, rather than attributes of the entire utterance. To reflect the fine-grained nature of speech emotions, we propose a new task: Speech Emotion Diarization (SED). Just as Speaker Diarization answers the question of "Who speaks when?", Speech Emotion Diarization answers the question of "Which emotion appears when?". To facilitate the evaluation of the performance and establish a common benchmark for researchers, we introduce the Zaion Emotion Dataset (ZED), an openly accessible speech emotion dataset that includes non-acted emotions recorded in real-life conditions, along with manually-annotated boundaries of emotion segments within the utterance. We provide competitive baselines and open-source the code and the pre-trained models.