QualiSpeech: A Speech Quality Assessment Dataset with Natural Language Reasoning and Descriptions
This addresses the need for richer, more nuanced speech quality assessment for researchers and practitioners, though it is incremental as it builds on existing auditory LLM methods with new data.
The paper tackles the problem of speech quality assessment by introducing QualiSpeech, a dataset with natural language descriptions and reasoning for 11 key aspects, and shows that finetuned auditory LLMs can reliably generate detailed descriptions of noise and distortion.
This paper explores a novel perspective to speech quality assessment by leveraging natural language descriptions, offering richer, more nuanced insights than traditional numerical scoring methods. Natural language feedback provides instructive recommendations and detailed evaluations, yet existing datasets lack the comprehensive annotations needed for this approach. To bridge this gap, we introduce QualiSpeech, a comprehensive low-level speech quality assessment dataset encompassing 11 key aspects and detailed natural language comments that include reasoning and contextual insights. Additionally, we propose the QualiSpeech Benchmark to evaluate the low-level speech understanding capabilities of auditory large language models (LLMs). Experimental results demonstrate that finetuned auditory LLMs can reliably generate detailed descriptions of noise and distortion, effectively identifying their types and temporal characteristics. The results further highlight the potential for incorporating reasoning to enhance the accuracy and reliability of quality assessments. The dataset will be released at https://huggingface.co/datasets/tsinghua-ee/QualiSpeech.