Talking to Robots: A Practical Examination of Speech Foundation Models for HRI Applications
This work highlights critical ASR limitations for human-robot interaction, where errors can affect task performance, trust, and safety, but it is incremental as it benchmarks existing methods on new data.
The study evaluated four state-of-the-art ASR systems on eight datasets covering six difficulty dimensions like noise and accents, finding significant performance variations, hallucination tendencies, and biases that impact HRI.
Automatic Speech Recognition (ASR) systems in real-world settings need to handle imperfect audio, often degraded by hardware limitations or environmental noise, while accommodating diverse user groups. In human-robot interaction (HRI), these challenges intersect to create a uniquely challenging recognition environment. We evaluate four state-of-the-art ASR systems on eight publicly available datasets that capture six dimensions of difficulty: domain-specific, accented, noisy, age-variant, impaired, and spontaneous speech. Our analysis demonstrates significant variations in performance, hallucination tendencies, and inherent biases, despite similar scores on standard benchmarks. These limitations have serious implications for HRI, where recognition errors can interfere with task performance, user trust, and safety.