Leveraged Mel spectrograms using Harmonic and Percussive Components in Speech Emotion Recognition
This work addresses emotion recognition in speech for applications like call centers and embedded devices, but it is incremental as it builds on existing CNN-VGG16 frameworks with new feature decomposition.
The paper tackled speech emotion recognition by exploring harmonic and percussive components of Mel spectrograms for the first time, achieving a test accuracy of 92.79% on the Berlin EMO-DB database, which outperforms prior CNN-VGG16 methods.
Speech Emotion Recognition (SER) affective technology enables the intelligent embedded devices to interact with sensitivity. Similarly, call centre employees recognise customers' emotions from their pitch, energy, and tone of voice so as to modify their speech for a high-quality interaction with customers. This work explores, for the first time, the effects of the harmonic and percussive components of Mel spectrograms in SER. We attempt to leverage the Mel spectrogram by decomposing distinguishable acoustic features for exploitation in our proposed architecture, which includes a novel feature map generator algorithm, a CNN-based network feature extractor and a multi-layer perceptron (MLP) classifier. This study specifically focuses on effective data augmentation techniques for building an enriched hybrid-based feature map. This process results in a function that outputs a 2D image so that it can be used as input data for a pre-trained CNN-VGG16 feature extractor. Furthermore, we also investigate other acoustic features such as MFCCs, chromagram, spectral contrast, and the tonnetz to assess our proposed framework. A test accuracy of 92.79% on the Berlin EMO-DB database is achieved. Our result is higher than previous works using CNN-VGG16.