On the Contribution of Lexical Features to Speech Emotion Recognition
This work addresses speech emotion recognition for applications like human-computer interaction, but it is incremental as it builds on existing methods by comparing lexical and acoustic features.
The paper tackled the problem of speech emotion recognition by investigating the role of lexical content from speech, showing it can achieve competitive or higher performance than acoustic models, with a lexical-based approach achieving a weighted F1-score of 51.5% compared to 49.3% for an acoustic-only pipeline on the MELD dataset.
Although paralinguistic cues are often considered the primary drivers of speech emotion recognition (SER), we investigate the role of lexical content extracted from speech and show that it can achieve competitive and in some cases higher performance compared to acoustic models. On the MELD dataset, our lexical-based approach obtains a weighted F1-score (WF1) of 51.5%, compared to 49.3% for an acoustic-only pipeline with a larger parameter count. Furthermore, we analyze different self-supervised (SSL) speech and text representations, conduct a layer-wise study of transformer-based encoders, and evaluate the effect of audio denoising.