Voice Quality Dimensions as Interpretable Primitives for Speaking Style for Atypical Speech and Affect
This work addresses the need for interpretable voice quality models in speech analysis for applications like atypical speech and affect recognition, though it is incremental as it builds on pre-trained models.
The paper tackled the problem of modeling perceptual voice quality dimensions for atypical speech and affect by developing probes on a dataset of 11,184 samples from 434 speakers, achieving strong performance and generalization across categories and zero-shot validation on unseen languages and tasks.
Perceptual voice quality dimensions describe key characteristics of atypical speech and other speech modulations. Here we develop and evaluate voice quality models for seven voice and speech dimensions (intelligibility, imprecise consonants, harsh voice, naturalness, monoloudness, monopitch, and breathiness). Probes were trained on the public Speech Accessibility (SAP) project dataset with 11,184 samples from 434 speakers, using embeddings from frozen pre-trained models as features. We found that our probes had both strong performance and strong generalization across speech elicitation categories in the SAP dataset. We further validated zero-shot performance on additional datasets, encompassing unseen languages and tasks: Italian atypical speech, English atypical speech, and affective speech. The strong zero-shot performance and the interpretability of results across an array of evaluations suggests the utility of using voice quality dimensions in speaking style-related tasks.