Empirical Interpretation of the Relationship Between Speech Acoustic Context and Emotion Recognition
This work addresses the problem of improving emotion recognition accuracy in speech processing for applications like human-computer interaction, though it appears incremental by bridging psycholinguistic theory with computational modeling.
The study investigated how acoustic context and phone boundaries affect speech emotion recognition (SER) by using an attention-based approach to analyze distributed emotion states over continuous time-windows. Results from cross-corpora experiments supported the benefits of this distributed approach, showing relationships between acoustic features and emotion.
Speech emotion recognition (SER) is vital for obtaining emotional intelligence and understanding the contextual meaning of speech. Variations of consonant-vowel (CV) phonemic boundaries can enrich acoustic context with linguistic cues, which impacts SER. In practice, speech emotions are treated as single labels over an acoustic segment for a given time duration. However, phone boundaries within speech are not discrete events, therefore the perceived emotion state should also be distributed over potentially continuous time-windows. This research explores the implication of acoustic context and phone boundaries on local markers for SER using an attention-based approach. The benefits of using a distributed approach to speech emotion understanding are supported by the results of cross-corpora analysis experiments. Experiments where phones and words are mapped to the attention vectors along with the fundamental frequency to observe the overlapping distributions and thereby the relationship between acoustic context and emotion. This work aims to bridge psycholinguistic theory research with computational modelling for SER.