MBNet: MOS Prediction for Synthesized Speech with Mean-Bias Network
This work addresses the labor cost in MOS testing for synthesized speech evaluation, offering an incremental improvement over existing methods.
The paper tackles the problem of automatically predicting mean opinion scores (MOS) for synthesized speech by proposing MBNet, which uses both mean and bias subnets to better exploit individual judge scores, resulting in improvements of 2.9% and 6.7% in system-level SRCC on VCC 2018 and VCC 2016 datasets, respectively.
Mean opinion score (MOS) is a popular subjective metric to assess the quality of synthesized speech, and usually involves multiple human judges to evaluate each speech utterance. To reduce the labor cost in MOS test, multiple methods have been proposed to automatically predict MOS scores. To our knowledge, for a speech utterance, all previous works only used the average of multiple scores from different judges as the training target and discarded the score of each individual judge, which did not well exploit the precious MOS training data. In this paper, we propose MBNet, a MOS predictor with a mean subnet and a bias subnet to better utilize every judge score in MOS datasets, where the mean subnet is used to predict the mean score of each utterance similar to that in previous works, and the bias subnet to predict the bias score (the difference between the mean score and each individual judge score) and capture the personal preference of individual judges. Experiments show that compared with MOSNet baseline that only leverages mean score for training, MBNet improves the system-level spearmans rank correlation co-efficient (SRCC) by 2.9% on VCC 2018 dataset and 6.7% on VCC 2016 dataset.