WhiSQA: Non-Intrusive Speech Quality Prediction Using Whisper Encoder Features
This work addresses speech quality assessment for applications like speech enhancement, but it is incremental as it builds on existing methods using foundational models.
The paper tackled the problem of non-intrusive speech quality prediction by proposing a system using features from an ASR model, achieving higher correlation with human ratings than recent approaches on all NISQA test sets and better domain adaptation than DNSMOS.
There has been significant research effort developing neural-network-based predictors of SQ in recent years. While a primary objective has been to develop non-intrusive, i.e.~reference-free, metrics to assess the performance of SE systems, recent work has also investigated the direct inference of neural SQ predictors within the loss function of downstream speech tasks. To aid in the training of SQ predictors, several large datasets of audio with corresponding human labels of quality have been created. Recent work in this area has shown that speech representations derived from large unsupervised or semi-supervised foundational speech models are useful input feature representations for neural SQ prediction. In this work, a novel and robust SQ predictor is proposed based on feature representations extracted from an ASR model, found to be a powerful input feature for the SQ prediction task. The proposed system achieves higher correlation with human MOS ratings than recent approaches on all NISQA test sets and shows significantly better domain adaption compared to the commonly used DNSMOS metric.