Self-Supervised Speech Quality Estimation and Enhancement Using Only Clean Speech
This addresses the need for more efficient speech processing tools by reducing reliance on labeled data, though it is incremental as it builds on existing self-supervised and VQ-VAE techniques.
The paper tackles the problem of expensive label collection for speech quality estimation and enhancement by proposing VQScore, a self-supervised metric based on VQ-VAE quantization error, and a related enhancement model, both trained only on clean speech without labels, achieving competitive results with supervised baselines.
Speech quality estimation has recently undergone a paradigm shift from human-hearing expert designs to machine-learning models. However, current models rely mainly on supervised learning, which is time-consuming and expensive for label collection. To solve this problem, we propose VQScore, a self-supervised metric for evaluating speech based on the quantization error of a vector-quantized-variational autoencoder (VQ-VAE). The training of VQ-VAE relies on clean speech; hence, large quantization errors can be expected when the speech is distorted. To further improve correlation with real quality scores, domain knowledge of speech processing is incorporated into the model design. We found that the vector quantization mechanism could also be used for self-supervised speech enhancement (SE) model training. To improve the robustness of the encoder for SE, a novel self-distillation mechanism combined with adversarial training is introduced. In summary, the proposed speech quality estimation method and enhancement models require only clean speech for training without any label requirements. Experimental results show that the proposed VQScore and enhancement model are competitive with supervised baselines. The code will be released after publication.