Speech Emotion Recognition using Attention-based LSTM-Network with Residual Connection
For researchers and practitioners in speech emotion recognition, this work offers a computationally efficient alternative to large pretrained models, though the performance gain is incremental and dataset-specific.
The paper proposes ResLSTM-SA, a lightweight LSTM-based architecture with residual connections and soft attention for speech emotion recognition. On the RAVDESS dataset, it achieves a UAR of 0.6517 with only 46.8k parameters, outperforming larger models while enabling edge deployment.
Speech emotion recognition is an important component of modern human-computer interaction systems. However, many state-of-the-art approaches rely on large pretrained models with high computational and memory requirements, limiting their applicability. This paper proposes ResLSTM-SA, a lightweight architecture that integrates residual connections with soft attention within an LSTM-based framework. Evaluated on the RAVDESS dataset under strict speaker-independent partitioning, the proposed model outperforms conventional attention-based LSTM baselines and several previously reported CNN- and hybrid CNN-LSTM architectures in terms of unweighted average recall (UAR). The best-performing variant (ResLSTM-SA-h64) achieves a maximum UAR of 0.6517 with only 46.8k trainable parameters, delivering competitive accuracy with three orders of magnitude fewer parameters than large-scale self-supervised alternatives, thereby enabling efficient deployment on edge devices and real-time voice assistants. The source code is available at https://github.com/Mak-Sim/ResLSTM-SER.