Do Speech Emphasis Models Generalize across Languages and Emotions?
For researchers in prosody and speech emotion recognition, this work reveals limitations of monolingual emphasis models and provides a new benchmark for cross-lingual and cross-emotion generalization.
The paper introduces MMEE, a multilingual multi-emotion emphasis corpus, and benchmarks emphasis detection models across languages and emotions, finding that multilingual training improves robustness while monolingual models degrade across distant languages.
Prosodic emphasis varies across languages, emotions, and speaking styles, yet existing emphasis detection models are largely trained and evaluated on monolingual neutral read speech. We introduce MMEE (Multilingual Multi-Emotion Emphasis), a corpus of 10,000 professionally recorded expressive utterances (14.13 hours) across 7 languages and 34 emotion/style categories, with three-level perceptual labels (10 annotations per sample). We benchmark two state-of-the-art architectures under monolingual, cross-lingual, multilingual, cross-emotion, cross-dataset, and data-scale settings. Monolingual models show limited zero-shot transfer, degrading across typologically distant languages, while multilingual training substantially improves robustness. Models transfer robustly between high- and low-arousal emotions; bidirectional transfer between synthetic and perceptual benchmarks suggests shared prosodic structure; and performance stays robust even at smaller training scales.