Voice Quality and Pitch Features in Transformer-Based Speech Recognition
This work addresses the problem of improving speech recognition robustness for applications like ASR by integrating prosodic features, though it is incremental as it builds on existing Transformer architectures.
The paper tackled the underutilization of voice quality and pitch features in neural-based automatic speech recognition by incorporating them into a Transformer-based model, achieving a mean Word Error Rate relative reduction of up to 5.6% on the LibriSpeech benchmark.
Jitter and shimmer measurements have shown to be carriers of voice quality and prosodic information which enhance the performance of tasks like speaker recognition, diarization or automatic speech recognition (ASR). However, such features have been seldom used in the context of neural-based ASR, where spectral features often prevail. In this work, we study the effects of incorporating voice quality and pitch features altogether and separately to a Transformer-based ASR model, with the intuition that the attention mechanisms might exploit latent prosodic traits. For doing so, we propose separated convolutional front-ends for prosodic and spectral features, showing that this architectural choice yields better results than simple concatenation of such pitch and voice quality features to mel-spectrogram filterbanks. Furthermore, we find mean Word Error Rate relative reductions of up to 5.6% with the LibriSpeech benchmark. Such findings motivate further research on the application of prosody knowledge for increasing the robustness of Transformer-based ASR.