PathBench: Speech Intelligibility Benchmark for Automatic Pathological Speech Assessment
This benchmark provides a standardized platform for researchers to compare methods for automatic speech intelligibility assessment, which is crucial for monitoring speech disorders and therapy efficacy.
The authors introduce PathBench, a unified benchmark for pathological speech assessment using public datasets, to address the fragmentation and inconsistency in existing research. They establish benchmark baselines across six datasets and introduce Dual-ASR Articulatory Precision (DArtP), which achieves the highest average correlation among reference-free methods.
Automatic speech intelligibility assessment is crucial for monitoring speech disorders and therapy efficacy. However, existing methods are difficult to compare: research is fragmented across private datasets with inconsistent protocols. We introduce PathBench, a unified benchmark for pathological speech assessment using public datasets. We compare reference-free, reference-text, and reference-audio methods across three protocols (Matched Content, Extended, and Full) representing how a linguist (controlled stimuli) versus machine learning specialist (maximum data) would approach the same data. We establish benchmark baselines across six datasets, enabling systematic evaluation of future methodological advances, and introduce Dual-ASR Articulatory Precision (DArtP), achieving the highest average correlation among reference-free methods.